Harness, Loop, & Graph: A Simple Explanation of How AI Agents Are Built
The longer your products and platform are dependent upon frontier AI, the farther behind they fall. 3 layers are the start of breaking those dependencies and building a bridge to what’s next.
These 3 terms describe the evolving engineering and architecture of an AI agent. All 3 showed up over the last 6 months. The more people build agents, the more gaps get exposed and filled. To say this is moving quickly would be an understatement.
They get discussed as rival approaches, but in reality they are different parts of the same system, and there are still parts missing. In this series, I will explain all 3 emerging agentic engineering concepts and how I fill in the gaps in my implementations. What you will quickly realize is that adopting agents, integrating them into the business, operating, and technology models, is an act of creative destruction.
But that destruction must happen as a migration vs. an explosive demolition. Harnesses, loops, and graphs create a migration path, but we aren’t explaining that very well. In this article, I will cover the architecture in line with the migration it enables. I will discuss the destination throughout.
What Does An Agent Actually Do? No One Knows…But We Can Figure It Out
Let’s give an AI agent a task, fixing a bug, to provide a high-level explanation of what each component does for the agent.
The SLM or LLM only produces text. It cannot act and do things like open a file, run a program, or save a change. Something else must sit between the SLM or LLM and the rest of the world to translate the model’s text into real actions. It must translate the results back into text the model can read. That is the harness.
One attempt is rarely enough to deliver a high-quality artifact. This is true at the task and workflow levels. The agent reads some code, makes a change, runs the tests, and finds it broke something else. It needs to try again with what it just learned. The layer that keeps an agent cycling until the job is done is the loop.
Some tasks are too big for one agent cycling on its own. The context fills up with dead ends, and independent pieces of work end up waiting in line for no reason. Splitting the job across several specialized steps with defined handoffs between them is the graph.
Those components are an attempt to cover 3 questions:
What can the agent do, and see? The harness answers this.
How does it decide what to do next, and when to stop? The loop answers this.
Does one agent handle everything, or do several split the work? The graph answers this.
There is a fourth question that these approaches fail to answer:
How does the agent know what its actions will cause? That question is the subject of the last part of this series.
Failing to answer the causal questions (predictive, prescriptive, and diagnostic) is why agents remain unreliable at anything without a built-in answer key. I will argue that an answer key is exactly what agents need to do work reliably. However, I can’t ignore the fact that most businesses don’t have that answer key yet. We must start somewhere, and it is best to meet the business where it is. That’s what harnesses, loops, and graphs are doing.
However, each layer is architected as if you will never have an answer key, and that is where I want to call your attention to a massive gap. My WITA cycles (Work, Information, Transparency, Augmentation) reveal a missed opportunity. When work gets done, just the act of using technology to do it creates information about how people do work. That information, one WITA cycle at a time, creates the answer key, but only if the system can learn and has mechanisms to capture that learning.
The ability to learn is implied in harnesses, loops, and graphs, but the mechanisms are unspecified. Andrej Karpathy’s Wiki-LLM is one attempt to address this gap. Keeping everything in text files, even when they are arranged in a taxonomy, is still inefficient. There are more suitable information structures we can use to build and iteratively improve the answer key.