VGate — audit a retrieval gate from its log

This page runs in your browser (Python on Pyodide): nothing you upload leaves your machine. It shows the log view of the VGate demonstration; the other views need the case study's per-query outputs or a GPU (see the repository).

Video walkthrough of the whole application (2.5 minutes, captions, no sound) · code, splits, preregistration log · per-query outputs

Audit

Should you gate? (planner)

Per segment: help and harm (%), the better fixed policy and the paired labels that find it, oracle headroom and null_lead (what the best threshold of a score with no signal would lead by on a log of this size, in points); the help-vs-harm AUROC a gate needs to lead that policy by one point (auroc_floor: no score gets there below it; auroc_needed: a score of the usual shape, with its 5–95% range); with signal columns, the AUROC of a benefit gate fitted on the log (out of fold), its ceiling, estimate and lead, in points.