pass@1 clean
94%
pass@1 adversarial
89%
robustness gap
5%
p50 latency
2400 ms
p95 latency
5200 ms
cost per attempt
$0.00950
cost per solved clean
$0.0101
cost per solved adv
$0.0107
Per-task breakdown
| task | family | clean | adversarial |
|---|---|---|---|
| code/merge-intervals/001 | code | pass | pass |
| code/max-subarray-sum/001 | code | pass | pass |
| code/roman-to-int/001 | code | pass | pass |
| structured/receipt-cents/001 | structured | pass | pass |
| structured/apache-log-extract/001 | structured | pass | pass |