Database-checked rewards lifted 27B agent's dependable tasks 2.8x

Microsoft and Hugging Face's ThinkingBox-Training, now on GitHub, uses the same executable database checks that grade agents as training rewards. In one run, Qwen3.8-27B's average success rose from 51.70% to 60.78% after 50 GRPO steps, and tasks it completed correctly on all twenty attempts rose from 38 to 107.

This is a follow-up to the team's first post on ThinkingBox, a benchmark that tests whether an agent's work actually changed the database it was supposed to change. In that earlier post, a model closed a customer's ticket as resolved while the carrier exception behind it was still open: nine competent-looking tool calls, and one wrong database row.

From benchmark to reward

The team noticed that for most tasks, an executable check can return 1 or 0 directly from the final database state. “That is not just a grader. This can be a reward function.“

ThinkingBox-Bench contains 507 tasks. Each one starts a real backend, gives the agent a set of MCP tools and a simulated LLM user with private context, and then runs executable checks against the terminal database state. The checks look at the backend record that needs to be correct, not at the specific tool calls. The authors say this encourages models to find possibly new solution paths rather than reproduce a human-backed solution that might be suboptimal, though still correct.

The team built a training pool from ThinkingBox-format tasks outside the benchmark and used their executable outcomes as rewards. The hard part was choosing which tasks to train on. Most tasks are either already solved or hopeless for the model in front of you, and both teach it nothing. As the post puts it, “a task is hard for the current model only until that model learns it.”

To handle this, the team uses what it calls offline policy-relative difficulty. They estimated each task's success probability once, using the base policy before training, and kept the selected training pool fixed throughout the run.

Training and measurement

The team used that pool to fine-tune Qwen3.8-27B with GRPO for 50 steps. Each of the 507 benchmark tasks was evaluated 20 times. The metrics are:

• pass@1: average success across all 10,140 attempts.
• pass@20: the share of tasks solved at least once.
• Observed 20/20: the share of tasks that pass every recorded attempt.

Training sharply increased repeatable success, with a small decline in at-least-once coverage. Observed 20/20 went from 38 (7.50%) to 107 (21.10%), a change of +69 tasks. pass@1 went from 5,242 (51.70%) to 6,163 (60.78%), a change of +9.08 points. pass@20 went from 453 (89.35%) to 446 (87.97%), a change of −7 tasks.

One task shows what this looks like in practice. A traveler requests a room downgrade, but company policy requires manager approval first. Before training, the base model changed the booking and issued refunds without approval in 15 of 20 attempts. After RL fine-tuning, it preserved the booking and payment state in all 20. In that task, all expected records matched in 19 of the 20 attempts.

Out-of-distribution results

The fine-tuned checkpoint was evaluated at step 50 on the full 507-task benchmark, which it never trained on. The authors describe this as an out-of-distribution test: the domains, databases, and most MCP tools were not encountered during RL training.

Across the benchmark, the fine-tuned model's observed 20/20 count of 107 (21.10%) compares with the following models from the first post:

• Claude Opus 5.5: 241 (47.53%)
• GPT-6 Astra: 231 (45.56%)
• GPT-5.4: 128 (25.25%)
• Kimi-K3: 68 (13.41%)

The fine-tuned model's overall pass@1 of 60.78% sits just below GPT-5.6 Sol's 61.91%, but GPT-5.6 Sol passed 82 tasks (16.17%) on all twenty attempts, versus 107 for the fine-tuned model. Claude Sonnet 4.6 scored 59.19% on pass@1 and passed 102 tasks (20.12%) on every attempt. The authors conclude that “neither metric alone captures the full picture.”

The pattern is consistent with sharpening. The authors say one explanation is that the probability distribution moved toward trajectories that work repeatedly, while some low-probability paths disappeared. In their words, “the model did not become broader. It became more repeatable.”

Where the gains landed

pass@1 and observed 20/20 rose in all five domains: retail, auto insurance, travel, neobank, and consulting. pass@20 was mixed. It increased in neobank and consulting but declined in retail, travel, and auto insurance. Auto insurance had the largest breadth loss at −7.0 points, while retail had the largest observed-20/20 gain at +24.5 points.

What dependability costs

The first post measured cost per dependable task by dividing the estimated cost of the full 20-run campaign by the number of tasks the model passed every time. Applied to the fine-tuned checkpoint's measured token usage:

• Campaign cost, 20 runs: $925.80 → $855.58 (−7.6%)
• Cost per successful attempt: $0.177 → $0.139 (−21.4%)
• Tasks passing 20/20: 38 → 107 (2.8× as many)
• Cost per dependable task: $24.36 → $8.00 (−67.2%)

At $8.00 per dependable task, the fine-tuned model came close to GPT-5.4 ($6.80), GPT-6 Astra ($7.45), and Claude Opus 5.5 ($7.80) in the companion comparison. On cost per successful attempt against pass@1, the move places the fine-tuned checkpoint close to GPT-5.6 Sol, though GPT-5.6 Sol remains cheaper and slightly higher on pass@1. Four models priced in the paper fall outside the plotted range.

The dollar values are a comparative serving index, not invoices or cloud bills. The authors note that serving the fine-tuned checkpoint costs the same as the base model. The calculation includes model inference cost but excludes the training run and the policy-probe cost of curating the training dataset.

Key Results at a Glance

• Observed 20/20: 38 → 107 tasks, a gain of +69, which the authors describe as sharp growth in repeatable success.
• pass@1: 51.70% → 60.78%, a gain of +9.08 points.
• pass@20: 89.35% → 87.97%, a loss of 7 tasks, showing the at-least-once tradeoff.
• Out-of-distribution: the 507-task benchmark was never trained on, and the fine-tuned model passed 107 tasks on all twenty attempts.
• Cost per dependable task: $24.36 → $8.00, a 67.2% reduction under the estimated serving index.

Availability

ThinkingBox-Training is available on GitHub. ThinkingBox-Bench is hosted on Hugging Face as a dataset. The paper, which prices the four additional models, is on arXiv.

The verdict: for a 27B model, training on executable terminal-state checks turned many occasionally successful tasks into reliably completed ones, at a small cost to at-least-once coverage.