Testing when adaptive data acquisition can replace fixed measurement plans

Abstract

Learned rules select samples for follow-up measurements in high-throughput experiments. Predicted value does not justify replacing a fixed plan. We introduce the opportunity-aware protocol for authorizing learned measurement rules (Opal), which learns a rule from labelled data, fixes it before outcomes are opened and tests it on held-out samples. Outcomes from elsewhere and unlabelled target measurements cannot settle this decision under unrestricted outcome shift. An exact bound identifies pilots too small for claims about unmeasured units. On 11,265 held-out Cell Painting compounds, the highest-value rule selected 96.0% for additional imaging, with a 97.1% upper bound on unnecessary selections. Opal selected 5.28%, reduced this bound to 5.18% and retained positive value after cost. Only Opal met the registered false-activation limit among rules selecting compounds. The next-lowest upper bound was 37.94%. Uncertainty about errors among selected compounds remained above target. Opal separates predicted value from evidence sufficient to replace the fixed plan.