Developer Docs
Recording Sync Matrix
How SnapTray verifies recordings end to end with synthetic media, injected faults, decoded-output checks, A/B reports and a real-device procedure.
Recording Sync Matrix
Encoding_SyncMatrix checks recordings end to end. Synthetic frames and audio
go through the recording’s real timing pieces (SessionClock, FramePacer,
TimestampedPcmMixer) into the platform encoder and muxer, with faults
injected along the way. The written file is then decoded and checked frame by
frame. A row passes only when the decoded media is right; a state machine
that ran, or a player that opens the file, is not evidence.
Sources: tests/Encoding/SyncMatrix.{h,cpp} (harness and analysis),
tests/Encoding/tst_SyncMatrix.cpp (rows).
What is generated
- Video: each frame carries its frame ID as a 32-bit barcode (two rows of 16 cells, so long rows never wrap), a moving square, and a full white flash at fixed wall times (every 700 ms from 350 ms). A static span repeats one ID, as an unchanged screen does.
- Audio: a 40 ms pulse at every flash time, 1 kHz from the microphone and 3 kHz from system audio, so a mixed recording shows each source apart; delivered as 10 ms capture callbacks timestamped the way the capture engines do.
Faults
Source timestamps and callback arrival are controlled separately.
| Fault | Field |
|---|---|
| Capture timestamp jitter | sourceJitterMs |
| Late and bursty delivery | arrivalDelayMs, burstEvery |
| Dropped, duplicated and backward frames | dropEvery, duplicateEvery, backwardEvery |
| Static screen | staticFromMs / staticToMs |
| Early and repeated pause/resume | pauses |
| One audio source silent for a while | audioGapFromMs / audioGapToMs |
| Audio engine clock offset and drift | audioClockOffsetMs, audioDriftPpm |
| Encoder unavailable (stalls every 2 s on the simulated clock; frames wait in a model of the encoding worker’s queue and are refused past its 30-frame / 3 s budget, as the worker does) | slowEncoderMs |
What is checked (decoded media)
- Every expected frame is present (counted per occurrence, so lost repeats of a still screen count), in order, and presented at its expected time (within half a frame); no frame from inside a pause; no gap where a pause was.
- The tail is complete: last frame ID and end time.
- Unique frames are counted apart from repeats (
uniqueFps); repeated frames never make up the frame rate. - Audio is present when requested, reaches the end of the video, and every
flash has a pulse within one frame interval + 25 ms (
worstAvOffsetMs), from each enabled source on its own (told apart by pitch), except where the scenario silences that source.
The expected values come from a reference run of the same events through the
same session clock and encoder-queue model without the injected pipeline bug.
Frame pacing in the reference follows the rule FramePacer specifies, written
out separately, so a bug in FramePacer shows as missing frames instead of
shrinking the expectation with it.
Proving the checks work
detect-* rows inject a bug or a fault the pipeline does not correct, and the
analysis must report a failure: a pause that is not removed, audio shifted by
120 ms, a tail cut by 500 ms, backward timestamps reaching the encoder (it
clamps them, so frames appear at the wrong time), an audio engine whose
clock starts 150 ms late, lost repeats of a still screen, a FramePacer that
paces at half the frame rate, and, in mixed rows, only the microphone 120 ms
late or system audio missing.
Running
./scripts/run-tests.sh # includes the fast rows (about 10 s)
./scripts/run-sync-matrix.sh # fast rows, JSON reports in build/sync-matrix/<commit>
./scripts/run-sync-matrix.sh --full # adds 4K30, 4K60 and a long 1080p30 row
./scripts/run-sync-matrix.sh reports/a --full # ...into reports/a
SNAPTRAY_SYNC_LONG_S=600 ./scripts/run-sync-matrix.sh reports/b --full # a 10-minute long row
scripts/sync-matrix-compare.py reports/a reports/b # A/B across commits
A run replaces the reports in its directory: reports left from an earlier
run are removed first, so a row the new run skips (the 4K and long rows
without --full) shows as missing in a comparison instead of passing on its
old result.
Each report keeps the scenario (configuration), the expected and measured
values, failures and skipped checks, and the cost: wall and CPU time, resident
memory, file size, frames written, and frames the encoder rejected. The compare
script exits with status 1 when a passing row fails or the A/V offset grows by
more than 10 ms. A detect-* row counts as passing when its injected error
is reported (each report stores mustPass and testSucceeded), and a
scenario missing from the candidate, an empty candidate or a missing directory
is an error. Cost changes are listed, not judged, until a baseline per
machine exists. Compare runs from the same machine, build type, encoder and
power mode only.
Platforms
| Platform | Result |
|---|---|
| Linux (FFmpeg/VA-API) | Full: encode, decode and analyse |
| macOS, Windows | Skipped, not passed: decoded-media analysis uses the FFmpeg readers of the Linux build. The test binary reports an explicit skip there. |
Native buffers, HDR to SDR, mixed DPI, control-window exclusion and real device changes are not covered by mocks; they need the real-device procedure below.
Real-device procedure (macOS, Windows, Linux hardware)
- Make a reference clip with flashes and beeps:
ffmpeg -f lavfi -i "color=c=gray:s=1920x1080:r=60,drawbox=enable='lt(mod(t,1),0.05)':c=white:t=fill" \ -f lavfi -i "sine=f=1000:sample_rate=48000,volume=enable='gte(mod(t,1),0.05)':volume=0" \ -t 60 -shortest -pix_fmt yuv420p -c:v libx264 -c:a aac reference.mp4 - Play it full screen on the screen being recorded, with system audio on, and record it with SnapTray (also: pause and resume once, and once with the microphone near the speakers).
- Measure flashes and beeps in the recording:
ffmpeg -i recording.mp4 -vf "signalstats,metadata=print:key=lavfi.signalstats.YAVG" -f null - 2>&1 | grep YAVG ffmpeg -i recording.mp4 -af silencedetect=n=-30dB:d=0.02 -f null - 2>&1 | grep silence_endFlash frames have a high
YAVG; eachsilence_endis a beep onset. Their difference is the A/V offset. Record machine, OS, SnapTray version, resolution, frame rate, encoder and power mode with the numbers.
Native editor follow-up (#185, #187, #188, #189, #190)
The native editor additions need the following macOS/Windows handoff checks.
Linux exercises the shared export loop through Video_NativeEditExport, the
native PCM packet reader, overlay font persistence/UI, and the remix worker’s
seek/pause/mute/rate behavior. Those checks do not substitute for native device
or permission testing. Native CI runs the existing Video_VideoTranscoder
contract with the newly enabled capabilities.
| Check | macOS | Windows |
|---|---|---|
| Multiple cuts + 0.5×/1.5×/2× parts; audio pulses at seams | Pending device test | Pending device test |
| Crop + zoom + masks + text/CJK fonts + frame; MP4/GIF/WebP parity | Pending device test | Pending device test |
| 1080p/720p/custom width, fractional source fps, 15/30 fps, codec choices | Pending device test, including HEVC available/unavailable | Pending device test; H.264 only |
| Both sources → separate gains/mutes/noise reduction in preview and MP4; recover project | Pending device test | Pending device test |
| Seek/pause/resume/speed changes with remixed audio and an output device disconnect | Pending device test | Pending device test |
| Keys off/shortcuts/all; pause boundary; permission refusal; protected input | Input Monitoring + Secure Event Input | UIA password field + secure desktop |
Measure A/V pulse error and output metadata using the procedure above. Check that unedited saves still use the existing fast path. Check font fallback when opening a project on a computer without the chosen family. #192 (the general preview playback backend) remains separate.