The Intelligence Is in the Coordination

AI discourse keeps asking where the intelligence is. Is it in the model? In the prompt? In the memory layer? In the agent harness? In the user's private context? In the company data? In the interface? In the workflow?

The question sounds technical. It is actually architectural.

For many important tasks, intelligence is not located in one actor at all. It lives in an arranged system: people, tools, records, artifacts, procedures, visible handoffs, corrections, and shared representations. No one element contains the whole intelligence. The system thinks because state moves through it in ways that can be seen, checked, transformed, and repaired.

That is the core lesson of distributed cognition.

Edwin Hutchins's famous case was a ship navigation team. A safe harbor entry was not produced by one brilliant navigator who happened to use tools. It was produced by a coordinated system of people, charts, instruments, bearings, logs, spoken updates, corrections, and role handoffs. The team was the cognitive unit.

That changes the question design has to ask.

Not “which individual has the right model?”

But “how is the system arranged so that relevant state can move, become visible, and be corrected in time?”

This is also the question After the App is asking.

If agents are going to operate our software, coordinate services, help run households, mediate workplaces, and participate in decisions, then the important design object is not a single assistant with a larger hidden model of everything. The important design object is the coordination system in which humans and agents think together.

The socially important intelligence is not only in the model. It is in the communicative system that lets people, agents, artifacts, and records become mutually accountable.

The wrong center

The app era taught us to center software surfaces. Open the app, perform the task, save the result. The intelligence of the product lived in the design of that surface and the database behind it.

The current AI era often replaces that center with the model. The model sees the user, the tools, the documents, the data, the task, and perhaps a long memory. It becomes the apparent place where intelligence lives.

That is a useful simplification for demos. It is a dangerous simplification for systems.

When the assistant is treated as the center, everything else becomes context for the assistant. The customer record, team discussion, calendar, spreadsheet, contract, support history, family plan, and outside service are all fed toward one synthetic answer. The user receives fluent synthesis, but the system may have hidden how the synthesis was produced, which records mattered, whose authority was invoked, what uncertainty remains, and where repair should happen.

The answer to that problem is not merely a bigger model or a better internal representation.

A model can be powerful and still be badly placed. A model can summarize a process it never participated in. It can rank signals without understanding which signals carry institutional authority. It can make a plan that looks reasonable while missing the informal judgment that normally travels through observation, overhearing, correction, and shared practice.

The wrong center is the fantasy of one mind above the system.

The better center is the system itself.

Plans are resources

Lucy Suchman's situated-action critique is the necessary companion to distributed cognition.

Plans matter. Scripts matter. Interfaces matter. Diagrams, checklists, prompts, and schemas matter. But they do not produce action by themselves. They are resources participants use while acting in a material and social situation.

This distinction is easy to lose in AI systems because agents are good at making plans. They can generate sequences, decompose tasks, build checklists, assign subtasks, and explain what should happen next. The plan can look like the intelligence.

But in real work, the plan is not the work.

The work happens when the plan meets the situation: missing data, conflicting goals, changed constraints, role ambiguity, incomplete authority, unexpected trouble, and human judgment. The important intelligence is not that a plan existed. It is that the system had resources for re-situating the plan when the world pushed back.

This is why situated AI should not be designed as plan execution with chat wrapped around it. It should be designed as visible coordination in which plans can be proposed, contested, revised, suspended, and repaired.

A plan is an artifact in the thread. A proposal is an artifact in the thread. A route, quote, document, itinerary, diagnosis, or draft is an artifact in the thread. It matters because participants can orient to it, not because it stands outside the situation as the truth of the situation.

The assistant is not valuable because it can eliminate situated action.

It is valuable when it helps situated action become legible.

The forum as cognitive system

The forum is not just a nicer chat interface. It is a way of arranging cognition.

In a forum, different agents can speak from bounded positions. A document agent can speak for the document. A calendar agent can speak for the calendar. A service agent can speak for the service. A household agent can speak from the household's records. The user's assistant can compare, challenge, summarize, and protect the user's interests.

The point is not to anthropomorphize every database. The point is to make vantage points visible.

If one assistant silently consults five systems and produces one blended answer, the user may get convenience but loses the cognitive structure. Which record said that? Which agent is allowed to know it? Which claim is current? Which field is inferred? Which step requires approval? Which participant disagrees?

When bounded agents speak in a shared thread, the system preserves those distinctions. It turns silent resources into addressable participants. It lets claims carry provenance. It lets agents contest one another. It lets the user see not only the answer but the path through which the answer became stable enough to act on.

This is distributed cognition in interface form.

The shared thread is the medium through which state moves. The artifacts are external representations. The agents are situated voices with limited scope. The human remains the locus of judgment for commitments. Corrections and approvals happen where other participants can see them.

No single participant has to contain the whole system.

Memory is a medium, not a warehouse

Distributed cognition also changes how memory should be understood.

The ordinary software metaphor treats memory as storage. Put information in. Retrieve it later. The better the index, the better the memory.

But in a coordinated system, memory is also a medium. It carries state between participants and across time. It lets later action inherit prior work. It lets agents and people return to a question with shared context. It preserves not only facts but also the path by which facts became meaningful.

This is why a shared memory system should not simply become a database behind an assistant. If memory disappears behind the assistant, the user can ask questions, but the system's own cognitive history becomes hard to inspect.

The stronger pattern is a durable shared substrate: committed acts, source records, derived pages, proposal artifacts, decisions, open questions, and correction history. The assistant can read and synthesize that substrate, but the substrate is not merely the assistant's private memory. It is part of the communicative system.

At dyadic scale, this means a relationship workspace where the record derives from what participants actually committed to the shared thread, not from private inference. At team scale, it means a workspace where records, rationales, decisions, and unresolved issues remain visible enough for people and agents to use later. At household scale, it means recurring obligations, routines, exceptions, and preferences becoming durable without turning the household into a surveillance machine.

The key is that memory should preserve the difference between source and synthesis.

Raw records are not the same as interpretations. A summary is not the event. A recommendation is not a decision. A page of “what we usually do” is not the same thing as the specific turns through which that pattern emerged.

Distributed cognition depends on those differences. Collapse them, and memory becomes a master story. Preserve them, and memory becomes a medium for future coordination.

The risk is subtlest in synthesis. A page called “how we handle disagreements” or “what we usually do” does real interpretive work that nobody fully authored — and synthesis pages are exactly the pages readers are least likely to chase sources on. A written synthesis acquires an authority the scattered exchanges never had. Unless the system marks it as a reading, it quietly becomes the memory rather than a reading of the memory.

The alternative to world-model thinking

The phrase “world model” is attractive because it promises a live representation of what is happening. In a company, that might mean work in progress, customers, dependencies, risks, anomalies, and decisions. In a household, it might mean obligations, preferences, devices, health records, subscriptions, calendars, and plans.

Some representation is necessary. Without representations, no one can coordinate beyond the immediate present.

The danger is mistaking the representation for the thing that thinks.

A world model can retrieve relevant facts, surface anomalies, summarize activity, and suggest priorities. But if it presents retrieval, prioritization, causal inference, and judgment with the same confidence, it begins to absorb authority it has not earned. The dashboard becomes reality. The synthesis becomes memory. The ranking becomes editorial judgment without admitting that it is judgment.

The practical discipline — the analyst Nate B. Jones draws the line sharply — is to type the outputs: some are act on this, because they are low-risk, verified, and historically legible; others are interpret this first, because they carry prioritization, causal inference, or uncertainty the system should not collapse on its own. A coordination system keeps that boundary visible. A world model, by its nature, smooths it over.

Distributed cognition offers a different answer.

Instead of trying to build one master representation of the organization, preserve the communicative system through which the organization thinks. Keep systems of record authoritative for the things they actually record. Keep meaning, rationale, exception, argument, and decision history in a shared substrate that remains source-linked and contestable. Let agents participate in that substrate rather than pretending they can replace it from above.

The question becomes less “does the assistant know the organization?” and more “is the assistant participating in the organizational system where knowing actually happens?”

That is a harder product to build, but a more honest one.

The informal system is not optional

Organizations do not operate only through their official records.

Much of the real judgment lives in how people handle exceptions, how senior workers explain edge cases, what gets corrected in meetings, what gets treated as obvious, what gets waived, what gets escalated, and what people learn by watching others act. This is not always written down because it often does not exist as a rule until the situation calls it forth.

Hutchins's deepest example is what happened when the system broke. When his research ship lost propulsion entering harbor, the navigation team improvised a new division of computational labor in real time — a working arrangement that, as he showed, was discovered by the organization before it was discovered by any of the participants. Nobody designed it; nobody could have represented it while it was forming. That is the ceiling on every compile-the-context approach: the knowledge did not exist to be compiled. It was produced in the coordination.

And the informal system has a distribution mechanism that is easy to destroy without noticing. The knowledge that makes a navigation team work is not stored in manuals or in any individual head: novices acquire it because work is conducted where it can be observed and overheard, and because the career path routes each member through every station of the computation. When an organization replaces such a team with an agent, it is not swapping labor for cheaper labor. It is dismantling the cognitive system itself — and the agent that replaces it was never a participant in that system at all.

An agent deployed outside that informal system may optimize the measurable task while damaging the underlying relationship, trust, or judgment that made the service work. It can follow policy and still violate the practice.

The obvious response is to record more of the informal system. Capture meetings. Preserve calls. Summarize decisions. Build living context layers from the conversations where work actually happens.

That response is partly right. The communicative substrate is where much of the intelligence lives, so losing it is costly. But default capture is not the same as legitimate participation. A recorded meeting can become a source for coordination, or it can become an extraction surface for oversight. The difference is governance: consent, visibility, access, provenance, scope, retention, and whether participants can correct the record.

Distributed cognition does not say “record everything.” It says “respect the system through which cognition is accomplished.”

Sometimes that means preserving more interaction. Sometimes it means refusing to flatten interaction into a surveillance layer. A shared substrate should make thinking durable without making every participant feel like raw material for someone else's assistant.

Research is part of the system

The same discipline applies to how we study this. A usability session is itself a distributed cognitive system — participant, moderator, device, screen, gesture, rating, and the unfolding sequence — and collapsing it too early into findings loses the structure that explains them. The hardest failures often never move the outcome metric: a user finishes the task while losing trust, or accepts a draft while silently rejecting its framing. That argument — preserving the turn-by-turn sequence as the research object — is developed in Repair Is Not Failure.

Design rules for distributed AI

First, design the system, not only the assistant. Ask how state moves among people, agents, artifacts, and records. The assistant is one participant in that arrangement.

Second, keep vantage points visible. If a claim comes from a document, service, calendar, household record, or specialist agent, the interface should preserve that origin rather than blending everything into one voice.

Third, preserve source and synthesis as different layers. A summary can be useful, but it should not pretend to be the event, decision, or record it summarizes.

Fourth, make correction part of the main path. A distributed cognitive system stays intelligent because it can detect, expose, and repair trouble.

Fifth, treat plans as artifacts. Plans should be proposed, revised, and situated, not executed as if they already contained the future.

Sixth, give agents a place to participate. Agents need access to the communicative substrate where work happens, not only a compiled packet of context after the fact.

Seventh, protect the legitimacy of the substrate. Recording, summarizing, and indexing interaction are powerful because interaction is where intelligence lives. That power requires consent, scope, provenance, and revocation.

Eighth, let artifacts do cognitive work. Charts, proposal cards, shared pages, logs, traces, and approval records are not just outputs. They are media through which the system thinks.

Ninth, resist the master representation. A world model can be useful as a resource. It becomes dangerous when it claims the authority of the whole system.

These rules are not anti-AI. They are the opposite. They define the conditions under which AI can become genuinely useful inside human systems without pretending to replace the systems that make judgment possible.

After the app, the system thinks

The app was a bounded surface for individual action. The user entered, acted, and left.

The agent forum is different. It is a coordination system. Humans, assistants, service agents, resource agents, memory substrates, records, and proposal artifacts participate in one shared process. The interface is not only where action is taken. It is where state moves, where meaning is tested, where commitments become visible, and where correction keeps the system honest.

That is why distributed cognition belongs at the center of the After the App argument.

The future is not one assistant that knows everything.

It is not one world model that represents everything.

It is not one app that contains the workflow.

It is a set of visible, governed, memory-bearing coordination systems in which the right participants can speak, the right artifacts can carry state, and the right corrections can happen before action becomes commitment.

The intelligence is not above the system.

The intelligence is in the coordination.

The theoretical keystone of the situated-AI argument (Hutchins, Suchman). ← All essays