You’re renting your institutional knowledge

I started my career at IBM Global Services. We operated on large enterprise contracts that dispatched teams to solve problems that companies couldn’t or wouldn’t solve themselves. The product sold wasn’t IBM’s software or a methodology, it was me. Me, and my ability to absorb the customer’s tribal knowledge fast enough to stop asking dumb questions.

AI agents are the new consultants.

You talk to the customer before starting a project to get as much context as possible, create documentation to ensure you have a clear scope. Then you work on delivering the project according to the scope. The real test is the user acceptance test where the customer tries out what you delivered, throws in a few edge cases of their own and tells you whether it’s good enough. Failure to pass these means the project ends early and you’re back in the office twiddling thumbs till the next project.

I remember spending hours reviewing a new pricing scheme with a head nurse who would impatiently tell me about all the different discount codes and where they did and did not apply. Outpatient has unique discount codes that don’t apply to inpatients, pain killers are billed differently depending on ICU or pharmacy. All the information lived in her head, and she got really good at flagging any violations during her testing. She would run a test order then point at the screen, “no, this discount code should NEVER appear for satellite pharmacies.” I’m next to her, ingesting context as fast as I can.

It’s the same loop modern agents are running: ingest context, get corrected, try again. Then I left and everything I’d learnt left with me.

Agents today are falling into the same trap. They learn, but they rent that knowledge back to you, because of the way they’re deployed. The knowledge that should be compounding now requires the helpful solutions engineer who comes with your AI agent contract.

Consultants get paid to absorb what you can’t write down

Nobody hires McKinsey for simple tasks like triaging alerts (I hope!). They want help with bigger challenges like migrations, new systems, or supporting an entire product line. This work requires judgment, not throughput. The more expensive the consultant, the higher up the stack they go, and high up the stack means deciding what to do, not just doing it.

The longer an engagement runs, the more your institutional knowledge concentrates in people you don’t employ. The consultant learns your environment because that’s what you’re paying them to do. The knowledge that should be a company asset becomes a personnel dependency.

Before you say it, yes, consultants aren’t the only ones renting your knowledge back to you. Your senior SRE who joined four years ago will know every undocumented quirk of the auth service. The moment she gets recruited by Stripe, she walks out with the same knowledge. The consulting trap is a special case of the bigger trap: institutional knowledge that lives in heads leaves when the head leaves.

This is what agents should fix.

Agent companies are doing it wrong by relying on “Solution engineers”

Read the job descriptions on the careers pages of the agent companies that closed a round this year. The postings for solution engineers and forward-deployed engineers all say roughly the same thing: embed with customer engineering teams, tune agent behavior in their environment. That is basically a consultant’s job description. They get sent on-site, or hold regular calls with your team, collect feedback, and customize and tune their agent. That context accumulates in the engineer, not the product. When she leaves, the agent’s performance degrades with her.

The tell is setup time. You’ll be told the agent needs a personalized setup and a few weeks to deliver results. During those weeks either the agent is learning or the engineer is. The latter is the same old consulting trap in a shiny AI-shaped box.

Self-learning agents

The best agents will work like your best hires. They get better over time with every task they complete.

In a previous post I talked about decision traces and how reviewing decisions is far more important than reviewing outputs. Moving to a decision trace model is the first step toward a self-learning agent.

A self-learning agent has two important properties:

1. Operational memory: it decides what to remember from each task so the next one goes faster and more accurately. In our case this involves learning from human feedback (“No Cleric, you need to check Redis when debugging payment svc latency!”) and its own decision traces and where it failed.
2. Verification: the ability to verify what it learnt. Did it actually help or make things worse? Memory cannot be an append-only log, it needs constant pruning because production keeps changing. This is especially true for domains like SRE where coding agents are shipping changes at machine speed, which means operational memory needs verification to keep up.

In engineering, generic AI knows how software usually works. It doesn’t know how your production works, which is how you get confident, completely wrong diagnoses. Having memory and verification lets the AI check its reasoning against what’s actually true in your environment.

A self-learning SRE agent diagnoses, opens a pull request, and watches what happens after an engineer reviews and merges it. Both outcomes are valuable. If the fix worked, remember it as a verified pattern. If it didn’t, the agent should remember why it failed for next time. We want the agent to show up at the next similar issue and say: “I’ve seen this. I bumped up the CPU last time, that didn’t work, so I’ll check memory this time.”

Remember the user acceptance test from my consulting days? That’s what verification does. Except now it happens on every task, continuously, and the knowledge stays in a system rather than in someone’s head which helps it compound.

For instance, a live sports-streaming company we work with taught Cleric what ‘live’ means to them. Cleric needs to understand that ‘live’ means a live sports game happening now. A different streaming company will likely have a different definition of what ‘live’ means. Without this context, a generalized agent will default to the average meaning of the word and likely deliver a subtly wrong answer.

The agent should produce knowledge as an asset you own, which means it’s not in a black box. The asset is open, queryable by any agent or human in your organization. This is the consulting model inverted: instead of the consultant accumulating your tribal knowledge and walking away with it, the system accumulates it and you keep it.

An agent that learns on its own gives you two things:

1. Keep up with changes from other agents. In the SRE/engineering world, coding agents are now the biggest source of change, shipping PRs at machine speed. Keeping up with changes needs to move at machine speed too.
2. The memory becomes valuable to other agents too. In the case of Cleric, our agent is building and curating memories about your production environment in real time. This means your Claude Code/Codex coding agent can ask Cleric which service owns which alert. Your Decagon/Fin/Sierra customer support agent can see if a recent production issue could be to blame for the spike in customer complaints.

Show me how your agent knows when it was wrong

The best tell for an agent that isn’t self-learning is how long the vendor needs to show results. If the answer sounds anything like “we need a few weeks,” it’s likely an open-loop learning process with a human in the middle. You’re buying consulting again, just dressed up as software.

Instead, you should see quick results in hours with deeper context within a day. You should be able to interrogate your agent with specifics only your senior SRE knows and note how well it understands your environment and its quirks.

Codified knowledge will be the next shift

One of the core tenets of SRE is to treat operations as a software engineering problem. It’s the mindset that brought us Kubernetes and Terraform. We killed thousand-line bash scripts with pages of documentation and replaced them with simpler runbooks with most of the operational logic baked into YAML running on Kubernetes. We now have a tribal knowledge problem: outdated runbooks everywhere, and the real fixes living in senior engineers’ heads.

Self-learning agents now mean we can solve this knowledge problem with software engineering using operational memory and verification. Agents like Cleric make these runbooks unnecessary and instead push us engineers to focus on creating guardrails and exception policies. Agent memory becomes a resource engineers can interrogate and modify.

My prediction: operational memory will start agent-specific (every vendor builds its own), but within a couple of years it consolidates toward a universal standard. Multiple agents will read and write to the same memory store. Your SRE agent learns that the auth service has a quirk on weekend deploys, and the next morning your coding agent and your customer support agent both know it too. The agents will be interchangeable, while the memory layer becomes the asset.

Until that standard exists, picking a vendor means picking a memory layer. If you’re in a buying conversation with an AI agent vendor, ask them about their approach to learning and memory. Make sure the knowledge is compounding for you, not for them. That single question separates the agents from the consulting firms in disguise.

When knowledge lives in software instead of people, your senior SRE can leave and the system still catches what she would have caught. That’s what we’re focused on at Cleric. Building an agent today is the easy part. Verified memory is the hard problem that we’ve chosen to focus on, instead of giving you consultants.