Join/aggregation checks, denominator audit, bias and leakage risks, corrected calculation, and reproducible validation.
See sample input → recorded output
Recorded 2026-10-11. Self-evaluated sample; results in your assistant may differ.
Sample input
QUERY_OR_FORMULA: purchase event count / distinct eligible visitors. ANONYMISED_DATA: purchasing users A, A, B; 20 unique eligible visitors. CLAIM: 15% user conversion. TIME_WINDOW: same fictional day. No joins or revenue data provided.
Checked output
The 15% calculation uses 3 purchase events / 20 visitors and mixes event grain with user grain. Unique purchasing users = 2; corrected user conversion = 2 / 20 = 10%. Event count remains 3. Reproduce by counting distinct purchasing user IDs A and B and reconciling the same-period eligible population of 20. Join cardinality cannot be audited without a join/query schema. Check missing user IDs, eligibility exclusions and duplicate export rows. Selection bias and attribution remain unknown; no causal or revenue conclusion is supported.
Codex in-session sample responses with deterministic checks. The same assistant edited the instructions and produced these responses; this is self-evaluation, not an independent or cross-model benchmark. No baseline comparison, live tools or model API were used.