# Jev the Decider: JIT for Agents

> Constraining action spaces for faster, cheaper agents
>
> Markdown version of https://cleric.ai/blog/jev-the-decider-jit-for-agents for AI agents and LLMs. Site index: https://cleric.ai/llms.txt

September 25, 2026 · Michael Saah

Watching [Jev play Doom](https://www.youtube.com/watch?v=jOdkVNsVW8M) gave me an idea: what if we could run our agent like that?

When I say “our agent,” I mean the part of Cleric that investigates problems in production. Like many others, our agent is based on a “coding agent” harness, like Claude Code, Codex, Pi, etc. These harnesses use an LLM for both deciding and doing.

What do I mean by deciding and doing?

By “deciding,” I mean the act of picking the next action. With a coding agent, this is typically unstructured. Your prompts might contain a rough process to follow, and in some limited sense might be enforced through mechanisms like structured output or hooks. But there is nothing restricting the model from going off and doing something else, like generating cat pictures, instead of following your directions.

By “doing,” I mean the execution of the decision. We might decide that we need to fetch the list of Ready pods for a service. Using an LLM with general tools like “execute-bash” is quite nice for this, as you aren’t forced to maintain an entire interface with error handling for every context source. You can just give the model permission to make queries and let it figure it out.

So we can agree that using an LLM for “doing” is worth keeping, at least for now. But is it necessary to use an LLM for the deciding? It certainly grants us a lot of flexibility. If someone comes along and asks our agent for something that isn’t investigation-shaped, like generating cat pictures, an LLM will be able to adapt to that. But it’s also slow, expensive, and requires vastly more evals than having something dumber making decisions. This is why an LLM can’t play Doom, but Jev can.

So, what if we made Jev the decider, and keep an LLM for the doing? Let’s give it a shot.

## The Experiment

To use Jev as a decider, we need to give it a discrete action space tailored to the goal. Just like how in Doom you can move in four directions, jump, turn, and shoot, our agent needs to be able to do things like form hypotheses, fetch data, and synthesize the findings.

To do this, I sampled 100 production agent trajectories and constructed an action space from them. I then generated a small harness for Jev that was able to interact with our existing “analysts” as tools. Without going into too much detail, you can think of the “analysts” as ephemeral subagents used for gathering and interpreting context from particular systems.

With this basic setup, I set up a `/goal` loop to start iterating it against our offline eval suite. Baseline results were already decent—we passed about 33% of our evals right off the bat. A few structural things were ported over—like a mandatory auditor step to catch low-quality answers, and some tooling gaps were plugged with new analysts. We also added parallel evidence gathering to the initial phase of the investigation.

The results were pretty impressive:

- 87% average recall, roughly equivalent to our existing agent
- 98% cheaper (!)
- 25% faster on average

The cost comparison isn’t totally fair, since we use more expensive models in the existing agent to support functionality that isn’t captured in these specific evals. But for the core use of investigating production issues, it appears to be quite an improvement.

## Data as Prerequisite

But not so fast, don’t go throwing away your big expensive coding agent just yet.

The first step in this experiment was defining the action space. And that came from agent trajectories that were performed by an LLM. We have a lot of this data because we’ve been at this problem for a while.

If you don’t have the agent traces to draw the action space from, you have to construct it from nothing, which is a lot harder.

Which brings me back to the title: I see this Jev-decider approach as JIT for the LLM. To replicate this approach, you would:

1. Make an LLM-based harness do stuff, and record what it did. Do this many times. Score the traces, and throw away the bad examples.
2. Use the good ones to extract an action space that covers the trajectories you want to solve for.
3. Structure your harness such that Jev makes decisions in that action space based on the latest context.

Now, you might say, “Well, this isn’t really just-in-time; you had to go and do all of this work!”

The only thing you have to change, once you’ve got the harness in place, is the action space, which is basically just a list of tool names. You can do this dynamically! Like a very slow optimizing compiler.

Furthermore, this experiment offloaded only high-level decision making, but you can imagine “squeezing” more and more of the generative work into Jev-based work by increasing the resolution with which you analyze and extract common steps from your traces.

Our next step is to put this new harness style into production for online evaluation. We’re pretty proud of our evals, but there’s nothing like the real world for testing performance in the real world.

I think this is going to be an exciting direction for harness design as more Jev-like models become available.