From LLM to Agent Harness: Understanding Agents' Past, Present and Future (Part One)

You have probably heard people discuss AI, Agents and intelligent agents, and recommend products such as Claude Code, Codex and Manus. But when pressed, many have not worked out what an Agent is, how it differs from a chatbot, or why some products based on language models only chat while others read files, change code, run commands and keep working.

I was among them. If you are curious about Agents too, this series is for you. I want to work through the subject from the beginning, learning it myself and helping you understand. Instead of starting with a pile of definitions, I will follow the path by which Agents emerged.

In this first piece, we will not rush into tools, workflows or complex engineering Harnesses. We will understand the LLM “brain”: how it generates text, why a Prompt changes output, how Context determines what it can see, and why scale can bring abilities such as reasoning, coding and translation. Then Chatbot, Prompt Engineering, Context Engineering, Agent and Harness will not be just a pile of fashionable words.

The conclusion first

As a useful engineering approximation: Agent = LLM + Harness.

The LLM, or large language model, is like a brain. Its central task is to predict the most likely next token. Like a phone keyboard suggesting a completion, but vastly more capable. On its own it has no limbs to operate files or commands and does not maintain task state inside the model over time.

Many AI products have an LLM underneath. Connecting the brain to a chat interface gives it a mouth and produces a chatbot, mainly a question-and-answer interaction.

An Agent adds, beyond the brain and mouth, limbs, permitted tools, memory, a working environment, permissions and feedback loops. It can do more than speak, and can observe results and adjust after more than one step. The LLM brain is placed inside an engineered body that can act, observe, remember, verify and remain controlled. The general term for that body is Harness: the configuration of interfaces, limbs, tools, memory and environment outside the model.

A harness is the equipment placed on a horse to let it pull and work—reins, saddle and so on. Applied to Agents, the metaphor is a way to direct the model's “wild” capabilities into controllable, reliable productive work.

This is not the strictest definition, but a useful, understandable perspective with broad industry agreement. The LLM reasons and generates; the Harness connects it to the real world; an Agent is an LLM surrounded by that Harness.

Using Claude Code, a prominent coding Agent, I see two Harness layers:

• Builder Harness: lower-level execution machinery for the LLM, such as the Agent's system prompt.
• User Harness: user-level constraints and standards helping a person use the coding Agent for a particular task under particular rules.

Claude Code is Anthropic's agentic coding tool. In this framework, it is a Harness connecting an LLM such as Claude to a repository, terminal, filesystem, tool calls and feedback. The engineered body, not the model alone, helps turn “can speak” into “can act.”

The jump from LLM to Agent was not instantaneous; the Harness did not appear fully formed. To understand how a model that continued text came to read code, modify files, run commands, inspect errors and repair them, start with the LLM.

At first, AI was a language model that could continue text

An LLM is a Transformer-based autoregressive language model: it generates one token after another. Its workflow can be summarized:

Given preceding context, calculate a probability distribution for the next Token, select one according to a strategy, append it to the context and repeat.

In short: predict the next Token from the context. Three foundational concepts are Token, Prompt and Context.

Token

A model cannot directly process raw text. A tokenizer first splits it into processing units called Tokens. A token may be a subword or character fragment: a Chinese character or word, an English word or piece of one. For example, “unbelievable” might become ["un", "believe", "able"].

Why not use individual letters or whole words? The goal is to balance vocabulary size and sequence length.

• Each character as a token: in my analogy, the units carry too little meaning. Splitting “like” into l, i, k, e forces the model to assemble whether the letters mean the verb, preposition or a noun. Individual letters have little meaning; learning many words from strokes becomes difficult.
• Each word as a token: the vocabulary must contain an enormous number of forms. The family eat, eats, eating, ate, eaten, overeat, overeating, uneaten and so on multiplies across thousands of verbs, tenses and affixes. A word-level model would treat forms as separate entries rather than sharing a core action.

Subword tokenization can reuse roots and cover possible text while balancing vocabulary and length. “Unbelievable” becomes ["un", "believe", "able"]; “eating” becomes ["eat", "ing"]. Shared pieces help express roots, tense and modifiers; thousands of pieces can cover many more word forms.

Any text given to an LLM becomes a Token sequence. Each token has an ID like a dictionary entry; predicting a token means predicting its ID. The sequence forms the beginning of context, from which the model predicts the next token.

Tokens are now a common unit for AI spending. Managing that spending means allocating a token budget. “Token economics” has also become popular, with firms and individuals reselling tokens and offering packages, some in ways I find unappealing.

Prompt

With no preceding text, the model cannot proceed; it needs initial input to calculate probabilities. That input is the Prompt. Mathematically, it is a condition for a conditional probability.

After seeing “Write a moon poem in the style of Li Bai,” the computation can be understood as:

P(next token | “Write a moon poem in the style of Li Bai”)

Training exposed it to poems and instructions; when it sees “Li Bai,” “style” and “poem,” subsequent words are more likely to resemble classical poetry with Li Bai's characteristics. A Prompt controls preceding context and thereby shifts the predicted conditional distribution. A small change changes the condition and may substantially shift output: “write a poem” and “write a seven-character quatrain” lead down different paths. This foreshadowed Prompt Engineering's popularity.

Context

When producing each new Token, the model “looks back” at preceding material. This whole context includes the Prompt, conversation history and its own output.

Beyond parameterized knowledge learned in training, a bare model has no inherent long-term memory during a conversation and no inherent network connection. Products may add memory, search, databases and tools outside the model, but the model directly processes what is placed in its context window.

This look-back capability comes from the Transformer's self-attention. Simply put, each word can “greet” other words in context and calculate attention weights indicating which matter most.

In 2017, Google introduced the Transformer in Attention Is All You Need. It moved away from recurrent neural networks and achieved stronger performance and training efficiency using attention. Almost all today's LLMs build on Transformers.

Can it see unlimited context? No. Standard full self-attention complexity depends on Token count and grows quadratically: with n tokens, this component is O(n²). Long-context systems use KV caches, chunking, sparse attention, retrieval and other engineering optimizations, but longer context remains more costly and difficult.

A system therefore sets a maximum context window. This hard length limit is a context constraint: material outside the window is invisible in the current inference, as if it did not exist.

The next figure illustrates the trend in window size; it was organized and generated with web DeepSeek, not copied from a paper.

To visualize different sizes:

1. Sticky note (≤4K): a palm-sized note you must erase and rewrite after a short chat; sufficient for very short Q&A.
2. Short story or manual (32K): a whole product manual or dozens of contract pages can fit.
3. Novel or project documents (128K–200K): a mainstream workbench able to hold the first Three-Body Problem novel or a medium project's materials, like a colleague who has read them.
4. Complete works or library (1M–10M): a desk large enough for the Three-Body trilogy or Harry Potter series. A million-token window permits analysis across complete works or a code repository; ten million resembles a small specialist library.

A crucial caveat: a larger desk does not mean everything is immediately found. A person reading a long book laid open may focus on the beginning and current page and skim details in the middle. Models likewise tend to retrieve information at the beginning and end more accurately and overlook the middle: the Lost in the Middle effect.

Context engineering is therefore not merely a bigger desk. It means arranging the most important material where the model can readily find it. Even with million-token windows, carefully designed prompts and retrieval keep key information from drowning in the rest.

When next-token prediction appeared to acquire new abilities

How can a statistical model that started with word continuation write poems, translate and code? Does it understand language or memorize it? Two keywords are Scaling Law and Emergence.

Scaling Law: a basis for scaling up

Dario Amodei, later an Anthropic co-founder, worked on speech recognition at Baidu's Silicon Valley AI lab with then-chief scientist Andrew Ng. In a later interview he recalled an early signal: more data, larger models and longer training produced significant, sustained performance improvements. This is an interview recollection, not a paper's formal conclusion, but helps explain the early context of the idea.

In 2017 Baidu Research published Deep Learning Scaling is Predictable, Empirically, reporting a predictable power-law relationship between generalization and factors such as model size and training data. This was an early industrial empirical exploration that helped prepare later work.

By 2020 Amodei had joined OpenAI as vice president of research. OpenAI researchers trained many models of different sizes and plotted their performance on logarithmic axes. A smooth line appeared:

In January 2020 OpenAI published Scaling Laws for Neural Language Models, reporting stable power-law relationships across seven orders of magnitude between language-model performance and compute, dataset size and parameters. In other words, the “alchemy” of how much improvement resources could buy became empirically predictable rather than pure guesswork: the Scaling Law.

It became a guiding principle and foundation for the LLM arms race. Amodei later left OpenAI and founded Anthropic, carrying a strong belief in the principle.

This reassured the industry: across a substantial range, adding parameters, data and GPUs could yield predictable improvements. It does not mean adding resources always improves a model, but it moved training beyond pure gambling toward a better-grounded engineering investment model. The LLM race began.

Aside: has Scaling Law reached its limit?

This has been debated since AI took off, with frequent reports of diminishing returns. Perhaps only researchers at the forefront can fully assess it. In Zhang Xiaojun's interview “An Exclusive Conversation with Shunyu Yao: Allow Me to Be a Little Crazy,” Yao offered views from frontline research—not paper conclusions:

1. Theory: no ceiling yet. Scaling Law as an empirical law remains valid and promising.
2. Practice: many bugs. Apparent walls often arise in execution and obscure real progress.
3. Future: systems matter. The benefit of lone heroes is fading; breakthroughs depend on reliable collective systems and excellent engineering approaching theoretical limits.

Yao has a theoretical physics background, worked at Anthropic and later joined Google DeepMind. His view represents an important perspective among frontline researchers: scaling has not simply failed; the question is whether engineering systems can realize theoretical potential.

Similar views appear in public comments by model-company leaders and researchers. More precisely, the field is moving from scaling pre-training alone toward scaling pre-training, post-training, inference-time compute, synthetic data and Agent systems together.

Imagine Scaling Law as an automated “super stomach” whose goal is to digest all the world's food—data on the internet—and grow stronger. Much naturally available food, public human internet data, may already have been consumed. Scaling may not be over, but people feel diminishing returns.

Scaling research mainly increases parameters—the stomach—and training data—the food. OpenAI's 2020 work encouraged expansion of parameters, data and compute, leading to models such as GPT-3 with 175 billion parameters.

DeepMind's 2022 Chinchilla research offered a different view: for a given compute budget, many models were underfed, with large stomachs and too little food. Stomach capacity and food should grow proportionally for better efficiency. Chinchilla, with 70 billion parameters, was much smaller than the 280-billion-parameter Gopher but performed better after consuming more than four times as much data.

Chinchilla Law gives a “nutrition recipe”: roughly one model parameter for 20 training Tokens to maximize performance for a particular compute budget.

New ways of eating may be beginning. If returns from new raw dishes fall, change the cooking:

• Before: gulp down more food—pre-training.
• Now: chew and reflect: spend ten times longer examining a dish's molecular structure (inference-time compute), make new dishes (synthetic data), or send several assistants around the world for recipes (Agent scaling).

The basic idea that consuming resources can increase capability has not simply failed. It is moving from gulping raw food toward processing, synthesis and reuse. In my analogy, scaling still invests resources to compress information, but resources increasingly include thinking time and synthetic creativity rather than food alone.

Emergence: when quantitative change brings qualitative change

Scaling Law describes steady performance improvement and provides a basis for sudden changes in capability. As parameter counts grew from tens of millions to tens or hundreds of billions, researchers observed behaviors apparently absent in smaller models. The models were not specifically trained for those behaviors; the goal remained next-token prediction. Yet the behaviors appeared, like water boiling at 100°C.

Researchers borrowed emergence from complexity science: quantitative change producing qualitative change. An emergent ability is one absent in small models but present in larger ones.

Emergence is not unique to AI. P. W. Anderson's “More is different” expresses the idea that when simple components interact at sufficient scale, the whole can display new macroscopic properties not directly predictable from individual behavior.

Why does emergence occur? Research into LLM emergence is active but lacks a single strict proof; specialists cannot yet explain everything. It is difficult to explain a complex-system phenomenon, but we can still examine it.

Analogies from nature:

• One H₂O molecule is not wet and does not flow; many together exhibit wetness and flow.
• A single ant acts simply, while a large colony can exhibit fungus farming or tending aphids, although dissecting one ant does not reveal an organ for those collective behaviors.

Likewise, with enough neural-network “ants” or parameters, new forms of cooperation may arise.

Discovery: GPT-3 and “scale as capability”

OpenAI's 2020 paper Language Models are Few-Shot Learners introduced GPT-3, with 175 billion parameters. Its contribution was not a new architecture, but a demonstration of “scale as capability”: without task-specific fine-tuning or parameter updates, a model given only a few examples could perform translation, Q&A, arithmetic, word rearrangement and many other NLP tasks at or above the leading models of the time.

This ability, already emerging in GPT-2, became stronger and more stable in GPT-3 and is often called in-context learning. To translate English into French, provide examples instead of retraining:

English: Hello       → French: Bonjour
English: Thank you   → French: Merci
English: Good morning → French:

The model recognizes the translation pattern and continues with Bonjour.

GPT-3 did not invent in-context learning; it systematically demonstrated its strength and generality at scale. It helped establish a “pre-training + prompting” paradigm, moving AI from a tool needing task-specific training toward an assistant that can take on new tasks through context.

GPT-2's 2019 paper had shown early signs: downstream tasks without fine-tuning in a zero-shot setting. GPT-3 increased scale to 175 billion parameters, making the ability stronger, more stable and general.

Formalizing emergence: Google's large-scale empirical study

GPT-3 brought attention to the phenomenon. Google's 2022 paper Emergent Abilities of Large Language Models examined model families including GPT-3, Gopher and LaMDA and reported that performance did not always increase linearly with scale: some tasks showed thresholds beyond which abilities absent in smaller models appeared. The authors offered a large-scale systematic empirical study.

The figure shows arithmetic, university-level multi-task language understanding and identifying a word's contextual meaning. In each, models initially performed poorly and largely independently of size, then improved sharply at a scale threshold.

Such abilities can be hard to predict by extrapolating small-model performance, suggesting further scaling might expand capability. Some tasks showed jumps around 10²² training floating-point operations, but this is not a universal switch: task, data and evaluation affect where changes appear.

What abilities emerged? Researchers have cataloged in-context learning, chain-of-thought reasoning, multi-step arithmetic, cross-language translation, coding, MMLU, theory of mind and self-calibration, among others. See Jason Wei's list of 137 emergent abilities.

1. Chain-of-Thought: thinking step by step

A small model might guess when asked: “Xiaoming has five apples, gives Xiaohong two and buys three. How many remain?” A large model may write 5 - 2 = 3, then 3 + 3 = 6, and answer six.

2. Arithmetic reasoning: from counting to calculation

A small model may barely manage 1 + 1 = 2. A larger one can handle three- or four-digit arithmetic and mixed operations. I interpret this as emerging abstraction of decimals, carrying and arithmetic rules rather than memorizing an unlimited set of problems.

3. Code generation and understanding

Given “write a Python program selecting all even numbers in a list,” a model can produce runnable code and identify bugs in a longer program. Programming is a rigorous formal language with nested logic. Small models may imitate its appearance but produce invalid or illogical code; larger models can develop representations of algorithms, loops and conditions and treat a program as a step-by-step logical machine.

4. Multilingual translation

Even without Chinese–Swahili parallel sentences, a model that learns enough Chinese, English and English–Swahili pairs may translate Chinese to Swahili directly. The explanation offered here is an unsupervised semantic intermediate layer: meanings such as happiness, walking and sun become nearby in a high-dimensional space, enabling cross-language translation.

5. Multi-task language understanding (MMLU)

MMLU is a broad test across 57 subjects, from mathematics and physics to history, law, medicine and ethics. Small models may score near guessing; larger models can behave like broadly read generalists with much higher accuracy. This suggests common-sense and reasoning abilities: using relevant background knowledge from context to select a plausible answer.

Emergence connects quantitative and qualitative change, appears as an unexpected result of large-scale training, changes application patterns and invites debate about whether jumps are real or an illusion.

Are these abilities the “intelligence” humans imagine? A more cautious distinction is observable, measurable emergent abilities, not rigorously proven emergent intelligence. The terms should not be used interchangeably.

We can still appreciate and use the phenomenon. Eighteenth-century physicists used steam engines before fully understanding the microscopic nature of thermodynamics; we may be at a comparable moment with large models.

Even without a full underlying explanation, a new application pattern appeared: the Prompt became an interface for programming the model. The barrier shifted from understanding model training to organizing language. This helped models leave the lab and planted the seeds for Prompt Engineering and chatbots such as ChatGPT.

What we have covered

We opened up the LLM “brain.” Its basic operation predicts the next Token from context. Tokens determine how it processes language, Prompts where generation begins and Context what it can see. Scaling Law and Emergence offer explanations for why a continuation model, at greater scale, begins to exhibit reasoning, translation, coding and in-context learning.

The later AI boom came as the brain entered a chat window, producing chatbots. Different questions produced different outputs, driving Prompt Engineering. People then saw that not only one Prompt but the whole organization of Context mattered: materials, history, rules, tool results and outside knowledge; what to include and how.

When an LLM began calling tools, observing results and continuing to act, the Agent problem truly appeared. The next article starts with Chatbots, Prompt Engineering and Context Engineering to trace that path.

Why spend time on the history of Agents?

I have used AI heavily since its boom and enthusiastically published some content. As someone outside the technical field, I have learned that AI is not as simple as it looks.

Using it daily does not by itself mean you know how to use it; saying “distillation” and “Skill” every day does not mean you understand it. These are the tip of the iceberg. Some people cannot explain a prompt yet call themselves experts or AI builders and open-source Skills. Do those releases have value beyond social-media promotion?

As an outsider, I do not want to be a talk-only marketing account. I want to participate in the field more seriously, use AI better and, while building some influence, help others outside the field understand AI more deeply and be less likely to be taken advantage of.

That requires more understanding of the underlying principles, so revisiting history and fundamentals matters. Once you understand them, you may find many AI-use tricks have little meaning: the core principles remain the same.

Lead author: Alchemist Zhou Lengqing

AI assistance: DeepSeek Chatbot, ChatGPT Chatbot, Codex Agent

Assistance covered: source organization, structural discussion, fact-checking, text polishing and image generation