Glossary
Holdout experiment
A pre-registered test that compares a treated group with an untreated control to estimate an agent's real effect.
A holdout experiment keeps part of an eligible population away from an agent's treatment so the difference can be measured. One3 requires experiments to be pre-registered before any treatment is dispatched: unit of randomisation, a frozen population hash, an assignment seed with assignment recorded before exposure, treatment and control policies, one primary metric, minimum detectable effect, analysis method and planned stop rule, contamination rule, cost and margin scope, and a human approver.
Results are immutable records with an estimate, an interval and sample sizes. Repeated peeking is not allowed, and a negative or inconclusive result stays visible. Basic holdouts for two agents are planned for W2, with experiments at scale in W3.
See how One3 would run your business
Tell us about your team and the outcomes you want. We will walk you through the apps, the Digital Employees and the plan for your industry.