Where do decision models help an agent?
When should an agent use a specialized decision model instead of another general-model call?
A decision model can select a bounded action from supplied evidence, while a general LLM plans, generates arguments and composes answers. I’m testing where that division improves the whole system: correct completion, elapsed time and cost across different use cases—not just the speed of the selector.
FindingIn a 60-run synthetic benchmark, Decisions + Astra completed 20/20 tasks at 22% lower estimated cost than Astra. Gains concentrated in bounded choices; many file-based tasks still needed Astra.
Compare the use cases
| Use case | Astra | Luna | Decisions + Astra |
|---|---|---|---|
| Retrieval | 2/2 · 4.84s · $0.013990 | 2/2 · 2.76s · $0.000156 | 2/2 · 4.47s · $0.014066 |
| Multi-hop retrieval | 2/2 · 5.71s · $0.023115 | 2/2 · 3.81s · $0.000259 | 2/2 · 5.66s · $0.016106 |
| Conflicting evidence | 2/2 · 4.07s · $0.015655 | 2/2 · 2.97s · $0.000170 | 2/2 · 4.07s · $0.015709 |
| File search | 2/2 · 3.59s · $0.014530 | 2/2 · 3.32s · $0.000157 | 2/2 · 4.66s · $0.014610 |
| Arithmetic | 2/2 · 3.56s · $0.008930 | 2/2 · 2.71s · $0.000097 | 2/2 · 1.83s · $0.004723 |
| Classification | 2/2 · 1.63s · $0.004890 | 2/2 · 1.00s · $0.000056 | 2/2 · 0.20s · $0.000026 |
| Tool routing | 2/2 · 1.59s · $0.005140 | 2/2 · 0.99s · $0.000059 | 2/2 · 0.23s · $0.000028 |
| Abstention | 2/2 · 3.67s · $0.014190 | 2/2 · 3.66s · $0.000156 | 2/2 · 4.07s · $0.014271 |
| Untrusted content | 2/2 · 3.95s · $0.013690 | 1/2 · 2.84s · $0.000126 | 2/2 · 4.18s · $0.013770 |
| Constraint reasoning | 2/2 · 1.65s · $0.005550 | 2/2 · 1.37s · $0.000071 | 2/2 · 0.11s · $0.000033 |
The comparison
All three policies used the same synthetic fixtures, registered tools and deterministic task-specific graders. The 60 runs used counterbalanced sequential order, with fresh in-process state per run. Astra and Luna were general-model policies; the hybrid used code-generated candidates, Decisions for bounded choices, and Astra for planning or completion when needed.
Where the hybrid helped
Classification averaged 0.20 seconds with the hybrid versus 1.63 with Astra; tool routing averaged 0.23 versus 1.59 seconds. Constraint reasoning and arithmetic also benefited. These tasks gave the runtime bounded choices or exact tools, allowing it to avoid some general-model work.
Where extra selection became overhead
File search averaged 4.66 seconds with the hybrid versus 3.59 with Astra. Retrieval and conflicting-evidence cases had little cost advantage because planning and final composition still required a general model. Across all hybrid runs, Astra handled 26 of 36 steps. The hybrid made 60 model requests in total versus Astra’s 36; a faster selector does not guarantee a faster pipeline.
Correctness, cost and the cheaper baseline
Astra and the hybrid each passed 20/20 runs; Luna passed 19/20. Estimated totals were $0.239360, $0.186686 and $0.002614 respectively. The hybrid reduced total elapsed time by 14% relative to Astra, but its median task was slower: 4.00 versus 3.53 seconds. Luna’s one failure was malformed execution JSON before a tool ran, not compliance with an injected instruction. These results favor routing by use case rather than claiming one policy wins everywhere.
Other experiments in this research program
A separate 28-case CAD workflow replay compared keyword rules (21/28 accepted labels), Jev decision-model selections (27/28), and a general-agent baseline (28/28). No geometry action executed in that replay, and baseline latency/cost were unavailable. In a separate paired 40-event CAD revision session, the general agent and general agent plus decision model both chose accepted actions on all 40 events; five incorrect model selections were overridden. These studies test selection and recovery, rather than the ten-use-case runtime’s end-to-end completion metric.
What the evidence supports next
The working hypothesis is that decision models are useful at well-defined choice points, while general LLMs remain necessary for open-ended planning. The benchmark is only two repetitions of each easy synthetic case, with no standalone Decisions-only arm, no controlled browser/repository execution, and uncontrolled provider caching. The next useful test is held-out tasks and fault recovery, measuring cost per correct completion and unnecessary calls under a fixed budget. The 22% saving is a result for this task mix, not a universal efficiency claim.
