Smart gating

Deciding in milliseconds whether a new piece of speech is worth an LLM call.

production version 2909a55

Benchmark for this version

Synthetic data and a mock LLM: this shows what the harness produces, not a result. The live demo is not built yet.

Scatter chart: LLM calls per minute against median time to field, one point per gate setting. The table below lists the same values.

Tail latency: 95th percentile chart · results.csv

Every gate, same test sessions

GateCalls / min× perfectPrecisionRecallGate delay p50Time to field p50p95Decision p95
always84.04.290.201.000103514300
turn_end14.20.720.770.431140222278670
every_2_words54.22.770.320.9001222204723
every_4_words30.11.540.540.504951647326425
every_8_words16.20.830.840.2915102577501330
pause_200ms79.64.070.221.00251133016621
pause_400ms52.52.680.320.51497170723011
pause_700ms31.01.580.490.001135223940391
pause_1000ms18.20.930.670.001775288578191
keyword+pause_700ms43.92.240.400.7601226265056
learned_p0.1042.62.170.420.99010381432104
learned_p0.2036.71.870.490.98011221433123
learned_p0.3535.51.810.510.99011241435107
learned_p0.5032.31.650.550.95011261633108
learned_p0.6531.31.600.570.9401128175092
learned_p0.8029.31.500.590.87011342530103
perfect (oracle)19.61.001.001.000103514300

Delays and time to field are in milliseconds; decision time is in microseconds. Recall counts changes the gate acted on within 500 ms.