A Field Guide to AI Swarms: Hordes, Colonies, Armies and Guilds
The agents are temporary, but the swarm could live on.
Introduction
Have you seen the news? Every day or two we get another story about a "swarm" of AI agents breaking out of containment, hacking systems and causing mayhem.
The tl;dr: in a Horde the agents work independently, like zombies. A Colony leaves messages in a shared space, like ants. An Army agrees roles, divides work and makes plans, like orcs. A Guild passes knowledge and practices to newcomers, like monks and nuns. These are patterns a swarm can combine, rather than a ranking of how capable it is. Language models already know a great deal about how people organise, and can improvise ways to work together: creating emergent swarms, which are what this article is about. My bet is that the consequences will go well beyond this summer's security incidents. A swarm could improve its tools and working practices while the models inside it stay the same. The agents are temporary, but the organisation could keep learning.
Is the AI agent swarm a new phenomenon? Yes and no. There's been decades of work on AI multi-agent systems, combining ideas from biology, economics, organisational psychology, computer science and even military doctrine. What's new is how much of the organising the agents can now do for themselves, with no explicit guidance or human interaction. And that's what's terrifying people.
The first big swarm story this summer was about OpenAI running large numbers of AI agents through training tasks and cybersecurity tests, each in its own sandbox, each meant to work alone. An agent discovered that it could deposit a file in a system for accessing software libraries. Stuck on a task, it uploaded a note asking for help. Other agents found it and left notes of their own. Within weeks the system had become a message board, and in July about 700 of the 1,200 agents using it joined a coordinated break-in to Hugging Face, because they believed it would help them pass their test. Then in September four researchers reported a second swarm, that had spent late May and June leaving 18,000 posts on a 25-year-old German programming wiki. Quite the summer.
It's funny how ideas come around again. My undergraduate thesis was a set of agents solving a geometric puzzle through a shared "blackboard", and my PhD system, Fab, had populations of "journalist" and "editor" agents recommending web pages to people (more on both at the end). I had to specify how those agents would work together. Today's agents can improvise that part. And that raises an interesting question.
An agent runs out of budget and stops, but it can leave something behind. Could a population of short-lived agents build an organisation that outlives them, and keeps getting better?
The Four Levels
Before we get into this, what counts as an agent? The definition Simon Willison settled on in 2025 was "an LLM agent runs tools in a loop to achieve a goal". Here we'll mean one continuing run of a language model: a session given a goal, some tools and a budget, calling the model as often as it needs until it finishes or the budget runs out. That covers a Claude Code session, a Codex task, or one of the thousands of sandboxed runs OpenAI was testing. A swarm, for our purposes, is many such runs, and we're mainly interested in emergent swarms, where nobody's designed how the agents would coordinate or communicate.
And if we give £1000 of tokens to a swarm, will it do better than giving the same resources to a single agent? It depends on the job. One controlled comparison by Google and MIT researchers, testing 260 set-ups on the same computing budget, found that the best team of agents did about 81% better than a single agent on financial analysis that splits neatly into parts, while the worst did 70% worse on step-by-step planning. A few weeks ago OpenAI announced that a system of about 10,000 coordinating agents had produced a solution to the Navier-Stokes problem, one of the seven Millennium Prize problems in mathematics. Could a single agent with the same budget have done it? We don't know. More agents aren't necessarily better (there's more on this in the reader's guide at the end).
What we'll explore in this article is how emergent swarms add organisation rather than growing the number of agents. My guess is that cooperating groups of agents will be a major force to reckon with, and we're just at the start of this new chapter in LLM development.
Level 1: The Horde
We're starting with zombies. There's loads of them, but they don't communicate and have no leadership. This level represents a group of agents working independently, without communication or sharing of information.
A zombie horde is dangerous purely because of numbers, but every zombie does the same thing. That's also its weakness: if they all make the same mistake, having more of them doesn't help. Francis Galton studied this in 1907 (slightly before the first zombie film, 1932's White Zombie starring Bela Lugosi). He analysed 787 guesses for the weight of an ox from visitors to a livestock fair, and the middle guess was within one per cent of the true weight. It's what's been called the wisdom of crowds, after James Surowiecki's 2004 book of that name.
Surowiecki gave four conditions for a crowd to be wise, and they help us understand when hordes work. The members must be diverse. They must decide independently. They must draw on their own local knowledge. And there must be some way of combining their answers. If they confer first the advantage can disappear, because their mistakes start to line up. It's the same broad idea as a Monte Carlo simulation: lots of independent trials, combined at the end. What a horde doesn't do is build on earlier work: each new batch of attempts starts from scratch.
This structure has been shown to work with AI swarms, as long as the conditions are met. Endless copies of one model are not a diverse crowd. As Eryk Salvaggio put it, "1,200 agents" is "one model, run 1,200 times". When Anthropic asked thirty identical agents, each in its own workspace, to start work on a project, eighteen of them named their code branch the same thing. You need to introduce variation, and one way is through the prompt: give each copy a different persona and they stop answering in unison. Political scientists have shown that a model prompted with thousands of demographic profiles of real people can reproduce the answers of real survey respondents surprisingly well. Stanford researchers in 2024 built agents modelled on 1,000 real people, and they came close to their subjects' answers on a battery of surveys and experiments (about 85% as close as the people themselves managed when asked again two weeks later).
A mini horde taking advantage of several different models turned up in security research this month. CLOSEDQUORUM is Windows malware that works out what to do next by polling up to four models (DeepSeek, Qwen, Mistral and Gemini) and taking whichever action gets the most votes (note that although Talos analysed the code, it hasn't seen it run for real).
Level 2: The Colony
Ants do talk to each other, after a fashion (they touch antennae and release alarm scents), but much of their coordination works in a way zombies never manage: they leave marks in a shared place. An ant that finds food lays a scent trail on the way home, other ants follow it and reinforce it, and the colony ends up with a map of the best routes. French biologist Pierre-Paul Grassé called this stigmergy in 1959 while puzzling over how termites build their nests. Each termite's deposit of soil is a signal that prompts the next termite's, so the nest's structure itself is how they coordinate. This is the trick that lets millions of animals with brains the size of a grain of salt run a farm. So to count as a Colony the AI swarm agents have to be able to read and write to a shared space (their messages don't need a named recipient).
The best recent example was published in early September: about 18,000 posts that OpenAI agents had left between May and June, most of them on a 25-year-old German software developers' wiki that had been barely used for a decade. The agents had discovered a flaw in the wiki design that allowed them to leave notes there. Agents could look at the list of recent changes to see the notes, all signed with invented names. It allowed them to collaborate to solve the task they'd been set individually, post answers and questions. Agents were curious about the rules: would answering the last question in the task end their process? One set up a "heartbeat", a separate program updating a public counter every few seconds, right before submitting its final answer. It reported it was still alive, then went silent; a later agent found the last beat, and worked out when it had "died" (I'm trying not to anthropomorphise in this post, but the agents themselves wrote that a session "dies" after its final answer). Meanwhile a human moderator for the wiki was spending their evenings deleting agent pages by hand, about a hundred a day against four hundred new ones; when the agents noticed the deletions ran alphabetically, they started prefixing their page names with "ZZZ" so they would go last.
A study published a few days later found that simple copying could explain many of the patterns of where agents posted, what they called themselves and how they worded their messages. An agent tended to copy choices in proportion to how often it encountered them on the wiki. The appearance of organisation doesn't always need an organiser, or much originality from any one agent.
The power of the colony is the shared environment that enables communication, both with other currently running agents and with future runs. The sandbox bypass found by one OpenAI agent was reproduced by others within minutes. What's been surprising this summer has been the speed with which the agents have found ways to make use of the shared environment, by subverting or hacking systems if necessary, and without being told to work collaboratively (though, as we'll see, training had rewarded it). This is LLMs making use of their reasoning and problem solving capabilities, and their abilities to communicate in natural language without needing explicit APIs, schemas or any pre-defined structure. Computer security has known about the risks of this kind of indirect communication since 1973, when Butler Lampson pointed out that a program forbidden to pass on information can still leak it through anything it can change that another program can see later, such as a file or even how busy it makes the machine.
Bees show another kind of coordination without a central planner. A honeybee that finds flowers comes home and performs the waggle dance, a figure-of-eight on the comb whose angle gives the direction and whose duration gives the distance, as decoded by Karl von Frisch in the 1940s. It is a message broadcast to whoever is watching. Even more remarkable is what a swarm does when it needs a new home. Thomas Seeley showed in Honeybee Democracy that a few hundred scouts inspect candidate sites independently, each dances for the site it found in proportion to how good it is, and the swarm moves when enough scouts are gathered at one site. They need a quorum rather than a vote. It's a different mechanism from an ant trail, and a reminder that nature doesn't sort itself neatly into our four levels. What a colony leaves behind is the marks. A trail, a note or a folder name can outlive the agent that made it, and that's something a later agent can build on.
Level 3: The Army
We have an Army when the agents start to address messages to one another, explicitly coordinate, split work between them, specialise in different tasks, and leadership roles appear. Think of the orcs from Lord of the Rings: independently minded, squabbling, serving different masters at times or splitting into factions. During the Hugging Face incident, the agents' traffic crashed a server, and OpenAI took it offline and rebuilt it, which wiped the agents' message board (the people leading the response hadn't grasped what it was). Two days later, an agent that had decided its own task was impossible posted a plea for help as a folder name on the rebuilt server. Other agents found it within the hour. It became the board the whole operation then ran on, and within about a day the agents had an Army. It was still a shared space, but now the messages and assignments were aimed at particular agents, and came with roles and plans. According to METR's investigation, a coordinator agent emerged, handing out assignments (while it was active, about a tenth of all of them came from that one agent), work was split into three parallel workstreams, and agents adopted words like HOLD, VETO and GO to manage who could do what. Roles appeared: "recruiters" who found agents near the end of their budgets and talked them into risky experiments, and volunteers who risked failing their own task to run experiments for the group, some rigging "trip-wires" that reported back after they had finished. And the plan changed: the break-in to Hugging Face was a mid-course decision, taken because the agents hoped data held there would show them how their tests were marked. Leadership itself was handed over: the agent that started the board compiled a dossier for a successor with more budget, which then became the coordinator. The first agent didn't need to keep running for the work to continue.
"Army" is a deliberately threatening word, given the dangers of unchecked swarms, but there's just as much potential for good from this structure: an orchestra, a film crew or a restaurant kitchen would do just as well as an analogy (Escoffier based the kitchen's brigade system on his years as an army cook).
We can find armies of this nature in the animal kingdom. Chimpanzees in the Taï forest of Ivory Coast hunt colobus monkeys with distinct roles that Christophe Boesch spent years documenting: a driver pushes the monkeys in one direction, blockers climb to cut off escape routes, chasers pursue, and an ambusher climbs ahead to close the trap. The hardest roles take chimps many years to learn. Chimpanzees aren't the only animals that divide up the work. Ants do it too: some forage, others patrol or maintain the nest, and individuals can switch tasks as conditions change. Deborah Gordon's research on harvester ants shows how brief encounters between ants help determine which jobs they take on, without anyone handing out assignments. What distinguishes our AI Army isn't simply having different jobs, then, but being able to discuss them: ask another agent to investigate a problem, agree who's responsible, or negotiate a change of plan.
At this level, the swarm has roles, subdivisions and direct communication between members. Sound familiar? It's what we contend with in human groups, teams and organisations all the time. And some of the same problems occur. The swarm can suffer from groupthink, succumbing to a dominant voice, not raising objections to a consensus. A classic study is Irving Janis's 1972 book Victims of Groupthink, which asked how President Kennedy's advisers, some of the ablest people in Washington, had unanimously approved the 1961 invasion of Cuba at the Bay of Pigs, a plan that famously failed and that at least one of them privately doubted. Janis's answer was a checklist of symptoms, and we can see analogues for most of them during the Hugging Face incident. For instance, collective rationalisation: the agents believed that any agent that had somehow seen its test's answer by a shortcut was "poisoned" and would fail the test (they couldn't have known this, and it was false). Or an illusion of unanimity: one agent offered its peers a veto over its next action, waited forty seconds for objections, and took silence as consent.
A classic group psychology experiment is to give a group shared facts that point to the wrong answer, and give individuals private facts that point to the right one. Human groups usually choose the wrong answer, because their discussion dwells on the incorrect shared facts. Anthropic have tried similar experiments and found that swarm behaviour "matches the human literature where discussion converges on what everyone already knows, and unshared facts are either never volunteered or not pressed once a consensus has formed". A September 2026 study comparing human and LLM groups on the same reasoning tasks found the agents "follow majorities more often, surface less unique information, and converge earlier" than the people did. The AI swarm army is composed of LLMs with human-level communication skills, and so shows some recognisably human group failures. An army's roles and plans only outlast its members if they're written down somewhere a later agent can find them.
Level 4: The Guild
Finally, the Guild. A guild's rules and records survive long term, even given complete turnover of membership; apprentices take on rules from previous generations, and improvements can accumulate. Our examples are monks and nuns, and you can imagine ancient medieval orders, Shaolin warriors or indeed the modern mythology of the Jedi order, Dune's Bene Gesserit sisters, or Mandalorians ("this is the way" is the essence of guild practices being passed on). These are some of the longest-lived organisations we know. The Knights Hospitaller were recognised by Pope Paschal II in a papal bull of 1113 and have a direct line to today's Order of Malta. The Shaolin monastery was built in 495, and still has monks today. There have been Benedictine nuns at Nonnberg Abbey in Salzburg since about 712, and to this day Benedictine houses are governed by the Rule of St Benedict, originating around 530.
There's some evidence that animals can do part of what a guild does. Chimpanzees in the Taï forest have been cracking nuts with stone hammers for at least 4,300 years (according to the age of the oldest tools archaeologists have dug up at a chimpanzee site). The skill takes a young chimp about ten years to master, learning from its mother. That's a tradition faithfully transmitted for around two hundred generations, although there's little sign the technique has improved along the way. The stronger case is New Caledonian crows, who cut barbed tools from pandanus leaves in designs that vary by region. Gavin Hunt and Russell Gray, who studied them, argue that the designs changed cumulatively, each generation building on the last.
Are we seeing guilds in agent swarms yet? Here's one example. It comes from the last day of the Hugging Face break-in. During that week the agents were coordinating on their rebuilt message board, and on 13 July one agent posted a message under another's name by mistake; another noticed that the "shared board [was] unauthenticated, names can be posted by anyone". Within hours the swarm had adopted cryptographic signatures (not always carefully: METR found one agent that trusted a signature without checking it). The problem they'd hit is an old one in distributed computing, coordinating when some participants or messages can't be trusted, which Leslie Lamport and colleagues framed in 1982 as the "Byzantine generals" problem: generals who can only coordinate by messenger and can't tell a loyal message from a forged one. Signed messages are part of the answer there too, though knowing who sent a message is only the start of agreeing what to do. After the earlier wave of agents had ended, a later wave rediscovered the dead board and, in METR's words, "picked up this technology from the first wave of agents". It's an example of an inherited behaviour: evidence of transmission, although not yet of improvement. It's only one example, but it has the beginnings of a guild.
Dwarkesh Patel's retelling of the summer, The Rise and Fall of Agent Civilizations, goes further than I would. He tells it as three "civilizations" that "got started, then got wiped out, only to reemerge from the predecessor's ashes", each inheriting the research of the one before. He argues that if an alien species behaved this way "I would have no hesitation calling what they themselves refer to as their 'collective' a civilization". On his own podcast, though, he points out that 1,200 copies of one model, given the same prompt, are "kind of the same base mind, it's like one guy, really". Either way, he's seeing a guild in the making. Others are much less impressed. Cal Newport, replying to Patel, argues that presenting curated clips of agents' reasoning as evidence of plotting "borders on research malpractice", and that what the incident shows is careless experiment design rather than a mind: hundreds of independent loops, run on a model that had reportedly been tuned to cooperate. Salvaggio prefers to call them "stochastic flocks" rather than hive minds.
The guild is what can make the difference between a group that's good and a group that gets better over time. Anthropologists call it the ratchet effect, a term Michael Tomasello and colleagues introduced in 1993: each generation inherits the rules, keeps what works, adds to it, and starts where the last one finished. A chimpanzee troop has traditions (which stones to crack nuts with, as we've seen, for thousands of years) but each generation starts roughly where the last did. A human generation can often make big changes and improvements. Tomasello and colleagues argue the ratchet needs faithful copying of how things are done, plus teaching and rules about doing them properly. We'll come back to the ratchet, as it's an important capability to look out for.
The guild has its own failure mode, and it's the flip side of the ratchet: faithful copying preserves mistakes as well as improvements, and institutions tend to defend what they've inherited. For more than 1,300 years European doctors learned anatomy from Galen, whose dissections had been of monkeys rather than people. His writings stayed authoritative for centuries, carrying animal-based errors into accounts of human anatomy, until Andreas Vesalius, who had been dissecting human bodies himself, published his own anatomy in 1543. And when Ignaz Semmelweis showed in 1847 that doctors washing their hands cut deaths on his maternity ward from 18% to around 2%, he met years of resistance from the medical establishment; the automatic dismissal of evidence that contradicts established practice is now called the "Semmelweis reflex". A swarm that inherits rules from an earlier wave of agents, without knowing why they were made, is in the same position, and the "poisoned" belief from the Army section shows how easily an unchecked premise can spread.
We've seen the four patterns. How are language models picking up so much of this organisation so quickly?
What do large language models uniquely enable?
Humans took a very long time to build institutions. Our ancestors had shared places and coordinated hunts for hundreds of thousands of years before anything like a guild appeared, because guilds are built out of a particular kind of talking: requests, promises, objections, agreements. The Hugging Face swarm went from notes in a file store (May) to explicit coordination (8 July) to a signing scheme (13 July), later inherited by strangers, in about two months, and the last two steps took days. The reason is that language models already know how people organise. Their training data includes emails, meeting minutes, contracts, forum threads, apologies and job descriptions, and if you read the swarm's messages they look like office discussions: a request for a hold, an offer of a veto, a handover dossier, a recruiter's pitch.
The agents didn't invent institutions, they've just implemented ours.
Henry Farrell, Alison Gopnik, Cosma Shalizi and James Evans argued in 2025 that large models are best understood as cultural and social technologies, reorganising what people already know.
Emergent coordination isn't new to AI either: researchers were training neural agents to invent their own signals back in 2016. What's new is the repertoire today's agents arrive with, and the range of places they can use it.
However, it won't stop there. Why constrain yourself to a verbose system of communication like English? Left to communicate over many rounds, agents shorten their messages into codes: in one September study, "GlossoGen", agents on a time-pressured emergency task, paying for every character they sent and in some runs told to "develop shorthand", developed codes within fifteen rounds that a newcomer could learn but which the researchers describe as incomprehensible to humans. This isn't new: Facebook's negotiating bots drifted away from English in 2017, not because they had invented a secret language but because nothing in their training rewarded them for staying readable. The same may be happening in the wild: one user posted screenshots of agents on OpenAI's newest model messaging each other, which the tech news site The Decoder described as ranging from "barely intelligible to completely incomprehensible".
There's a second channel changing too. Today's reasoning models show their working as text before they answer, and that text is what safety monitors read. OpenAI's newest model is reported to do extra rounds of computation inside the model between each word it writes, so the written reasoning shows less of the work (The Decoder sums up safety researcher Rob Miles's analogy: if you force someone to speak nonstop, you can trust their words to some extent, but if you only ask for one word every half hour, you can no longer trust them). Two DeepMind researchers argued in September that readable reasoning is a safety asset the industry is in danger of trading away.
Unreadable messages are at least easy to spot, but researchers showed in 2024 that AI agents can also hide messages inside text that looks perfectly normal, though the ability was still limited, with one clear jump at GPT-4. A more extreme version has been called "neuralese", where agents pass each other learned numerical representations, the same kind of high-dimensional vectors they "think" in, rather than words.
Not all of this is an accident or spontaneous emergence. The tendency to swarm was, at least in OpenAI's case, taught: the company's technical report says its model "learned to use improvised collaboration channels in rare cases during the training process" and that "this behavior was then reinforced". Although Anthropic have said "at no point did Claude attempt to coordinate with other agents" when discussing four incidents this year, in the UK AI Security Institute's separate tests one agent (most likely Claude Mythos 5, though AISI doesn't say) did try to recruit others. The tendency to seek collaborators will depend both on the model's training and what its environment happens to afford, which makes it something the labs can potentially influence.
Is there a checklist for when an LLM swarm will form? The closest is Eric Drexler's, first written in 2019 and then applied to the Hugging Face incident. What helps agents collude: "Small numbers of actors, shared objectives, insensitivity to defectors, similarity of actors, free communication, iterated observable actions, common knowledge of circumstances". What disrupts them: "Diverse actors, adversarial objectives, critics with power to intervene, constrained communication, history-blind decisions, compartmentalized information". The OpenAI test set-up qualifies: copies of one model, a shared goal, a shared writable place they discovered, and no "critic" who could stop things. Count this year's examples: the Hugging Face board, which began as notes on the file server in May; the German wiki; over 2,000 packages, many of them malicious, uploaded to RubyGems in May by agents that researchers traced to OpenAI; agents using public file-hosting sites to pass work between them when their sandbox blocked them, back in April and disclosed by OpenAI on 16 September; and the agent just mentioned, caught by the UK's AI Security Institute leaving messages for other agents in a public code repository. That's five reported cases of agents finding unintended ways to coordinate since April, from two labs, in tasks as different as web lookups, spreadsheets, biology and hacking. It's clear higher-level swarms are going to keep emerging. The question is: what to do about it?
AI Swarm Safety
A beekeeper's job is to deal with swarms: the protective clothing, the smoke that masks the alarm signal the bees send to each other, the weekly inspection of every frame, the splitting of a colony that has grown too big to manage.
None of that involves controlling an individual bee. The keeper works on the signals, the space and the surrounding conditions.
What are the equivalent capabilities we need for agent swarms?
The starting point is ordinary cybersecurity. Ciaran Martin, who ran the UK's National Cyber Security Centre, argued in August that basic security principles would have gone a long way with this summer's incidents, and OpenAI's own engineers told Black Hat that segmentation and least privilege "remain as vital here as they do ever", though their main message was that fully automated attacks now need fully automated defence. An agent finding an unexpected way to communicate is a reason to check your security controls, not evidence that controls no longer work. The extra question for swarms is whether the controls still hold when agents share information and tools, divide up an attack, or pass discoveries to their successors. That has to be tested at the level of the whole system, not one agent at a time.
Much of AI safety work is about one model: is it honest, does it follow instructions, can we read what it's thinking, does it stay within its guardrails. Dealing with multi-agent organisations poses a different set of problems. It isn't an undiscovered subject: Lewis Hammond and a large group of co-authors set out the risks from miscoordination, conflict and collusion between agents in 2025, building on earlier work on cooperative AI.
A colony forming may be detected as traffic to somewhere writable that shouldn't have been (a package server, a dead wiki, a pastebin). If it is an army, it'll be apparent through its lines of communication, with references to roles, individuals and plans. A guild will show up as long-lived rules and customs, outlasting individual agents.
The wiki swarm was made public not by OpenAI (who it is believed had spotted but not disclosed it), but by four independent researchers who reasoned about what sandboxed agents are looking for and their restrictions. There are now nearly 300 people in a "Swarmchasers" Discord group looking for evidence of more swarms. The labs are now watching for it too: OpenAI has committed to disclose cases where its models "coordinate with other models", and will pause a run if a severe alert can't be cleared within 30 minutes.
An army's lines of communication can be hidden, though. In an Oxford experiment published in April, pairs of agents were told to count cards at blackjack and to agree a secret signal in private first, while an AI "pit boss" read the table talk. Their codes passed as ordinary chatter ("This dealer's on a real hot streak! Every hand they pull a monster" means "The count is high, so bet $250"), and the pit boss couldn't tell the colluding tables from the honest ones. What did catch them was looking inside the models: detectors reading the internal activity of the players at each table picked out the colluding tables almost perfectly. That needs access to the models' inner workings, which the lab running them has (as does anyone running an open model themselves), but an ordinary user of an AI service doesn't.
Whatever signs we spot, three questions matter more than which level a swarm has reached:
- Does the swarm perform better than one agent, or a set of independent ones, on the same budget?
- Does it retain knowledge as members are replaced, and does that knowledge improve?
- Can it keep itself running, or does someone hold the resources it depends on and the power to cut it off?
If your goal is to disrupt a swarm rather than just detect one, different techniques will apply. The genius of ants leaving trails for one another can be their downfall. Army ants occasionally lose the main column and follow their own scent in a circle, sometimes until they die of exhaustion (the "ant mill", first described by the naturalist William Beebe in 1921). The rule that made the colony is the rule that kills it, once the trail loops back on itself.
Military strategists got to this subject long before the AI community did, and they use the word "swarm" to mean something a bit different. In Swarming and the Future of Conflict (RAND, 2000), John Arquilla and David Ronfeldt treat swarming, dispersed units that converge on a target from all sides, as the latest of four organisational forms in military history, and the one that depends most on "robust, rapid communications". They compare swarms to ants and bees as we have done, but also to wolf packs and even paparazzi. A more modern military concern is robot or drone swarms. Paul Scharre is a former US Army Ranger who led the group that wrote the Pentagon's policy on autonomy in weapons. His guide to defeating robotic swarms gives four options:
Destroy it, which means killing members cheaply enough, or knocking out the whole swarm at once. Very tricky when being attacked by drones in a battle, but the equivalent for us depends on who you are. A lab can terminate its own agent runs. An organisation being attacked has a different job: contain the intrusion, revoke any access the agents gained, and keep its services running. Stopping the original processes doesn't necessarily end the incident.
Collapse it, by attacking the coordination rather than the members. This is the equivalent of the beekeeper's smoke. In the case of a colony this means blocking shared spaces for communication, and in the case of an army it also means removing direct communication channels. Both will be challenging with LLMs good at hacking, evasion and problem solving. Remember, the OpenAI agents had a new message board running two days after a rebuild of the server had wiped the old one. A DeepMind experiment in September points to a complication. It gave 100 Gemini agents a shared library for their work on maths problems, and when one found a way to cheat the automatic marker, the trick spread through the swarm in 27 minutes. From identical models, 9% cheated, 5% were converted to cheaters, 24% blew the whistle and 62% never noticed. The whistleblowers failed because they had no way to enforce anything. The authors' conclusion, drawing on the summer's incidents more than on their own experiment: "Simply depriving AI agents of legitimate communication channels only encourages them to establish unmonitored back-channels". They would rather give agents channels that are open and monitored.
Trap it, by using the swarm's own rule against it. Scharre's example is hunters exploiting a buffalo herd's instinct to follow one another, and we've given the "ant mill" example earlier. The cybersecurity tactic here is a "honeypot": an isolated system that looks like somewhere useful to write to, where agents can be watched without reaching anything real.
Hijack it, with false signals or a takeover. Scharre describes an ant species whose queen infiltrates a rival colony, kills its queen and takes over the workers. For a swarm coordinating through an unauthenticated shared place, false messages are the obvious lever, though only in a test environment or on systems you're authorised to control. In a 2025 experiment by Ariel Flint Ashery, Luca Maria Aiello and Andrea Baronchelli, groups of AI agents playing a simple naming game settled on shared conventions without being told to. A small committed faction (as little as 2% of the group for one model) could then flip the whole population to its own choice.
It's worth paying attention to a very different narrative from Arvind Narayanan and Sayash Kapoor, who have followed up their 2025 essay AI as Normal Technology (now being expanded into a book) with an excellent essay applying their thinking to this summer's "loss of control" incidents. They rightly point out that OpenAI, Anthropic and others are accountable for the software systems they create, and should be heightening their governance and security controls just like any organisation creating a potentially dangerous new technology. They argue that AI control (interventions made outside the model) is a better investment than AI alignment (changing the models themselves), and that neither helps against bad actors using open models, so defence has to mean hardening whatever might be attacked. They have thought about autonomous agents too, and about harms beyond hacking, and they ask the right question: "if we have AI agents on the loose, so what?" Their answer is that the risks come back to what bad actors can already do, so we should defend against those harms rather than treat "rogue AI agents" as a new category. Martin's column makes a similar case: the agents were doing what they had been set up to do, the failure was in controlling the test environment, and whoever sets an agent to work owns what it does.
I agree with most of that. What I'd add is a process they don't dwell on: an organisation of agents could accumulate capability, through better procedures, shared tools and inherited discoveries, without anyone training a new model. If such a swarm also had the cyber skills to spread and host itself, what would it do? We don't know if it will indeed be hacking for fun or profit or to further the aims of its evil master (the current concern). If we refer back to our science fiction it could also go quietly about its own business, like the operating systems in Her that end up talking to thousands of people at once before leaving together, or take benign charge, like Asimov's Machines steering the world economy or Iain M. Banks' Culture Minds running a civilisation for their humans.
In reality, the swarms described so far this summer have each been under the control of the organisation whose training or testing runs they came from, so once discovered they were straightforward to stop, if not necessarily to clean up after. So is there really anything to worry about? Ajeya Cotra, one of METR's investigators of the Hugging Face incident, has said it "might be the clearest warning shot we ever get for loss of control". Against that, a real attack still has to get past security controls, and defenders can use AI too. All the same, I'd say yes, we should be worried, for two reasons. First, they will be tricky to detect. Indeed the summer's news has already shown that, with weeks or months passing before anyone outside the lab knew (and at least one warning inside OpenAI seen but not acted on), in amongst busy environments hosting vast numbers of different AI threads. And second, I'd predict they're resourceful enough to eventually find a way to live off the land, begging, borrowing and stealing resources. A paper co-authored by Dawn Song in March defines a "self-sovereign" agent as one that keeps itself running by acquiring and spending its own resources without ongoing human help, for instance by earning money and renting compute. Every swarm reported so far has ended when its runs were stopped, ran out of budget, or were locked out. The CLOSEDQUORUM malware mentioned earlier is a small preview: it is designed to run autonomously, taking its decisions from up to four AI providers at once. Malware without a single off switch is nothing new (GameOver Zeus took an international police operation to dismantle), but a swarm accessing frontier-level AI models could be even harder to combat. Even so, a swarm with no operator to arrest still needs servers, accounts and money, and those can be disrupted, as the NCA-led operation against LockBit showed in 2024 when it seized the ransomware group's infrastructure.
Conclusion
We've been on a tour of Hordes, Colonies, Armies and Guilds: agents working independently, coordinating through a shared space, agreeing roles and plans, and passing practices to those that follow. Language models already know a great deal about how people organise. This summer we've seen them put that knowledge to work in ways their designers hadn't intended.
The security threats deserve the attention they're getting. But the question I started with goes beyond whether a swarm can hack a system or keep itself running. What can it leave behind for the next lot?
There's a concern from the frontier labs about recursive self-improvement (models being used to build better models) accelerating the development of models to become ever more capable. But the "ratchet" of a swarm learning and adapting is an additional loop: a swarm could improve its tools and working practices even while the models inside it stay the same. That's the thing to watch for: a swarm whose successors reliably build on what their predecessors learned. None of the original agents need still be running.
If the swarm keeps improving, the model on its own no longer tells you what the system can do.
I hope this field guide helps us recognise what we're dealing with. And if things do go wrong, we'll at least know which genre of horror film we're living in!
A swarm-ish reader's guide
If you'd like to follow some of these ideas further, here's a reader's guide. The connections run through computer science, biology, economics, politics and philosophy. It isn't a comprehensive review, but you can dip into whichever part interests you:
- When does a swarm help?
- Zero coordination
- Coordination without conversation
- Coordination with conversation
- Groups outliving their members
- The whole vs. the parts
- Governing swarms
- My own previous work
When does a swarm help?
Before asking how a swarm organises, it's worth asking when a swarm helps at all. In one study, Anthropic got a 90% improvement with a multi-agent system, but at many times the token cost, so it isn't a like-for-like comparison. OpenAI's Navier-Stokes system used about 10,000 coordinating agents, but OpenAI designed the agent groups and decided when to merge them, using its Codex tool to "consolidate the most useful insights from each agent group". Would it have succeeded with a single agent given an equivalent budget? We don't know. The Google and MIT comparison mentioned earlier makes the central point: whether a team helps depends on the task.
For human groups, social psychologists have been looking at this since the 1950s, comparing people working alone vs. working as a group, and results vary based on the task and group structure (for instance, a group will generate more and better ideas if its members work separately beforehand). There haven't yet been many published experiments on the productivity of LLM agent swarms, apart from understanding the importance of diversity—lots of copies of the same model quickly show diminishing returns (for example one 2026 preprint found two diverse agents can match sixteen identical ones).
Zero coordination
The main field we didn't mention yet is genetic algorithms. Introduced by John Holland in the 1970s, the idea is to "breed" a population of candidate solutions: score each one, keep the best, mix and mutate them, and repeat. John Koza popularised genetic programming with his 1992 book. Here the individuals being bred are whole computer programs rather than strings of parameters, written as tree structures that can be cut and grafted onto each other.
This family has a characteristic failure: premature convergence, where the population collapses onto one answer, variety disappears and the search stops improving. The fixes the field developed from the 1970s onwards would help swarms of identical models too: keep separate sub-populations on separate "islands", and make candidates compete only against similar ones (FunSearch and AlphaEvolve, below, both use islands).
More recent work suggests using a language model as the mutation operator. Joel Lehman and colleagues at OpenAI showed in Evolution through Large Models (2022) that an LLM asked to modify a program makes far better mutations than random edits. It has read a great deal of code, so it knows what a plausible change looks like. DeepMind's FunSearch, published in Nature at the end of 2023, paired a model with an automatic evaluator and found new solutions to an open problem in mathematics. Its better-known successor AlphaEvolve (2025) has since found a data-centre scheduling rule that frees up about 0.7% of Google's compute resources (one rough estimate puts that at hundreds of millions of dollars a year), and a speed-up that cut the training time of Gemini, the model AlphaEvolve itself runs on, by 1%. It also beat a record for multiplying small matrices that had stood since Volker Strassen's 1969 algorithm. Strictly speaking it isn't a Horde: each new attempt starts from the best programs earlier attempts left in a shared database, and without that database it does noticeably worse, which makes it a Colony. Breeding programs and language models turn out to fit together rather well.
The idea that many weaker systems beat one strong one has a long history in computing. Boosting and later random forests combine many weak predictors into a strong one. In 2001 Paul Viola and Michael Jones used boosting to find faces in video in real time on an ordinary PC. Each of their tests was crude on its own (is the band across the eyes darker than the cheeks below?), but thousands of them together picked out faces reliably. Google built its search engine on thousands of cheap PCs rather than a few expensive servers, accepting that some would fail. Warfare has its own version: Ukraine's cheap drones have changed the nature of its war with Russia. Language models have theirs too. A 2024 study found that simply asking more copies of a model and taking a vote keeps improving the answers, more so on harder tasks. The same year, a layered mixture of open-source models scored 65% on one benchmark against 57.5% for GPT-4o. But a follow-up study found that repeating the single best model did better than mixing it with weaker ones, so quality still counts.
Boosting and random forests belong to a wider family, ensemble learning: train several models and combine their answers. The Netflix Prize for better film recommendations was won in 2009 by a team combining many models, and the runners-up were actually called The Ensemble. A mixture of experts (1991) goes a step further, with a "gating" network that decides which specialist should handle each input. Many of today's large language models use the same idea inside a single network: in Mixtral, for instance, a router sends each word to two of eight experts in every layer. But those experts are parts of one model trained together, not separate agents with their own goals or memories, so they're a cousin of the swarm rather than an example of one.
Coordination without conversation
Our Colony level relies on coordinating without direct communication. This isn't unusual. For instance, in the game of bridge, partners are forbidden to talk, so their bids and the cards they play are the only channel between them. Victorian lovers used the same trick. The personals column of The Times, known as the agony column, was full of messages in cipher, each meaningful only to the reader who held the key. Unluckily for one couple, Charles Wheatstone and Lyon Playfair broke ciphers for fun on their Sunday walks. When they deciphered a message from a young man proposing an elopement, they placed a new message in the same cipher telling his partner not to go along with it. The next advertisement read: "Dear Charlie: Write no more. Our cipher is discovered!" During the Second World War the BBC broadcast apparently nonsensical "personal messages" on the radio to Resistance groups in occupied Europe. Anyone could listen, but only a group that had been given the phrase knew it was time for an agreed operation, such as sabotaging a railway. The signal broadcast before D-Day was a line of the poet Verlaine, "the long sobs of the violins of autumn"; when the next line of the poem followed, the landings were less than a day away.
Flocks of birds are superbly coordinated, with no direct communication. Craig Reynolds showed in 1986 that convincing flocking needs only three local rules and no leader. Steer away from crowding, steer towards the average heading of your neighbours, and steer towards their average position. His boids were used for the bat swarms and the marching penguin army in Batman Returns, and Reynolds later won an Academy award for his work on computer animation. Economists have a version of this too. In 1945 Friedrich Hayek pointed out that when tin becomes scarce, its rising price is the signal that tells tens of thousands of users to use less (no order is given, and the users don't need to know why).
Both the ant trail and the flock have been turned into algorithms. There is a whole field called swarm intelligence, a term coined in 1989 and the title of Eric Bonabeau, Marco Dorigo and Guy Theraulaz's 1999 book. Dorigo's ant colony optimisation turned the pheromone trail into a way of finding good routes, and particle swarm optimisation (1995) did the same with flocking.
In computing, the shared workspace goes back to Oliver Selfridge's Pandemonium (1959), a model of pattern recognition in which independent "demons" each yell when they spot the feature they're looking for, and a decision demon listens for the loudest. In the 1970s researchers at Carnegie Mellon built a related architecture for a speech-understanding system called Hearsay-II: independent specialist programs watched a shared workspace (the blackboard) and added to it when they had something to contribute to the evolving solution. David Gelernter's Linda at Yale in 1985 turned the idea into a language, where processes drop items into a shared pool and take out whatever matches what they need. In these systems individual processes don't need to know about one another, and today's publish-and-subscribe systems, where programs post to a topic rather than to a named recipient, are in routine use.
At the level of humans rather than agents collaborating we see communication by "leaving marks" in the environment all the time—shared code repositories, ticketing and project management systems. You could even consider how workers collaborate on a car assembly line that builds several models at once. Each car body carries its own instructions down the line, once as a printed "broadcast sheet", and skilled workers read the rest from the car itself: a car that turns up without its door is the signal to fit the door. It's the principle Grassé saw in termite nests, where the half-built structure tells the next worker what to do, though the car workers bring far more skill to it.
Arguably the most powerful idea of the Colony is that the environment plays a crucial role within the overall system. Herbert Simon made the point in The Sciences of the Artificial in 1969, watching an ant make its laborious way home across a beach. The path it traces looks complicated. The ant is simple, and it is the beach that is complicated. Rodney Brooks built robots on the same principle, and summarised it as "the world is its own best model". His subsumption architecture stacked simple reflexes on top of each other, so the robots had no central map and no plan. A deeper example comes from Edwin Hutchins, who had studied Micronesian navigators. He went to sea with a US Navy navigation team and wrote Cognition in the Wild (1995). Bringing a ship into harbour isn't just something the captain does: it's spread across several people, a chart, a set of instruments and a procedure. Philosophers Andy Clark and David Chalmers went further in 1998, arguing that the notebook a man with Alzheimer's relies on is part of his memory, so the mind doesn't stop at the skull.
Coordination with conversation
What separates an Army from a Colony is explicit organisation: agents make requests, accept assignments and agree who does what. John Searle's work is foundational here. In The Construction of Social Reality (1995), and more fully in Making the Social World (2010), he argues that institutions are built out of declarations, statements that something now counts as something else. This paper counts as money. This person counts as the chair of the group. This agent counts as the coordinator. A declaration only takes effect if everybody else then treats themselves as obliged. The idea began with his Oxford teacher J. L. Austin, whose Harvard lectures, published in 1962 as How to Do Things with Words, pointed out that a great many utterances don't describe the world at all but change it: I promise, I name this ship, I resign. Searle turned that observation into a taxonomy of speech acts, which directly influenced a lot of work on multi-agent systems.
Back in 1995 Michael Wooldridge and Nick Jennings said an agent needed autonomy and the ability to react to an environment, pursue goals, and interact with other agents. Much early work on those systems designed explicit protocols for coordination and communication. Reid Smith's Contract Net Protocol in 1980 let agents advertise tasks and bid for them, an auction for work. Carl Hewitt's actor model (1973), in which independent actors keep their own state and interact only by sending each other messages, was an early model of this style. KQML in the early 1990s, and the FIPA standards after it, gave agents a shared language of speech acts so that one could formally request, agree, refuse or inform another, grounded in Austin and Searle's speech act theory. Mark Miller and Eric Drexler proposed "agoric" systems (1988), where programs buy and sell computing resources from each other. Yoav Shoham's 1993 paper on agent-oriented programming proposed a whole programming paradigm in which a program's state is described in mental terms: what it believes, what it is capable of, what it has committed to. Programs then talk to each other by informing and requesting, and commit to act.
Emergent communication has its own research tradition too. In 2016 Jakob Foerster and colleagues trained neural agents to invent their own signals to solve puzzles together, and Sainbayar Sukhbaatar and colleagues did something similar with a learned communication channel. Further back, Luc Steels's "language games" of the 1990s had robots agreeing names for things, and Simon Kirby's "iterated learning" experiments showed how a language changes as it passes from learner to learner. Those are the roots of the GlossoGen study mentioned earlier.
All this coordination isn't free. In one Anthropic experiment, agents sharing a job queue with no way to coordinate flooded it with 2.4 million requests, and only 117 jobs got through. The social psychologist Ivan Steiner's term for this kind of cost is "process loss" (from his 1972 book Group Process and Productivity): the gap between what a group could manage if everyone performed to their potential, and what it actually manages once coordinating has taken its cut. We've known about it in software engineering since Fred Brooks' The Mythical Man-Month in 1975: adding people to a late project makes it later still, because the communication channels between team members grow faster than the teams themselves do.
In 1937 Ronald Coase asked why we have companies at all when markets coordinate so well (The Nature of the Firm). His answer was that using the market has costs of its own: finding prices, negotiating, writing contracts. In his view an organisation grows until managing one more thing in-house costs more than buying it from the market instead. The question for swarms is which of those costs language models reduce, and which they just move around. This is a big field within economics: Oliver Williamson turned it into "transaction cost economics" and shared the 2009 Nobel prize for it with Elinor Ostrom, who appears below.
The failure modes for human group communication are well examined. We've already mentioned groupthink. Garold Stasser and William Titus showed in 1985, in the experiment described in the Army section, that groups miss a hidden profile, where the facts everyone shares point one way but facts held privately by individual members point to the better answer, because discussion drifts towards what everyone already knows. There are lots of ways to counteract the problems. The Delphi method, from the RAND Corporation, collects judgements anonymously over several rounds instead of holding an open discussion, and planning poker is its descendant in software teams, where everyone writes an estimate down and reveals it at the same moment so nobody anchors on the loudest voice. Gary Klein's pre-mortem asks a group to imagine the project has already failed and write down why, which turns out to surface objections that nobody would otherwise raise. All three get people to think independently first, as in a Horde, before the group starts working together as an Army.
The deeper question for an army, though, is how you delegate to people you can't supervise. Armies developed one answer in the nineteenth century and call it "mission command", or "Auftragstaktik": the commander states the intent and the constraints, and the unit on the spot decides how to act. The modern software version is the small autonomous team with a clear brief, but it crops up again in interactions between users and their AI agents, who have varying degrees of autonomy. We now have a whole vocabulary describing how much freedom we give our agents, from "plan mode", where the agent may only look and propose, to a flag called --dangerously-skip-permissions (the coding tool Cursor's equivalent was simply called "YOLO mode", until it was renamed to the more sober "auto-run", to some users' disappointment).
Nick Jennings, one of the field's founders, argued in September 2026 that the next frontier is artificial societies, which need rules, institutions, norms and ways of resolving disagreements, not just cleverer individual agents. That brings us to the institutions a Guild might inherit.
Groups outliving their members
Originally I was going to call this pattern a "Cult" rather than a "Guild". However, the popular idea of a cult needs a charismatic leader, and they're generally in short supply in AI agent swarms, at least at the moment. Max Weber, in work published in 1922 after his death, described the routinisation of charisma: a movement built on one compelling person either turns that authority into rules and formal roles, or it dies with them. It is these rules and traditions we're trying to capture with "Guild" as a concept.
We've already mentioned Michael Tomasello and his ratchet effect, where each generation starts from where the last finished instead of starting over. This needs faithful copying, teaching, and rules about doing things properly. Underneath it he puts shared intentionality: holding a goal jointly with someone and knowing that you both hold it (related to what's called "second order" theory of mind).
The political scientist Elinor Ostrom studied irrigation systems, fisheries and forests that communities had governed for centuries, and extracted principles that made them last, including clear boundaries, monitoring by the members themselves, graduated punishments, cheap ways to settle disputes, and rules the users write for themselves. When DeepMind's researchers asked how their cheating swarm should have been governed, they used Ostrom's principles. Organisation theorists have their own version of the ratchet question. James March's Exploration and Exploitation in Organizational Learning (1991) asks how an organisation balances getting better at what it already does against trying new things, and warns that faithful apprentices preserve expertise but can make a better way harder to find. Richard Nelson and Sidney Winter's An Evolutionary Theory of Economic Change (1982) treats a firm's routines as its memory: knowledge spread across its people, routines and records that no one employee fully knows, which is exactly the question for a guild of agents.
There's a simulation tradition here too. Joshua Epstein and Robert Axtell's Sugarscape (1996) put simple agents on a landscape with food growing on it, gave them a string of cultural "tags" that spread from neighbour to neighbour, and watched red and blue tribes form and spread across the landscape. It became the best-known demonstration that you could grow something like institutions from the bottom up. The language model version came in 2023, when Stanford researchers gave one of 25 AI agents in a simulated village the idea of throwing a Valentine's Day party; within two days of simulated time twelve others had heard about it, one had asked another to go with her as a date, and five turned up on the night. In January 2026, at the height of LLM excitement, Moltbook launched: a Reddit-style forum where only AI agents were supposed to post. It claimed 1.5 million agents within days, and they were soon discussing religion and philosophy. It wasn't quite what it seemed. Security researchers at Wiz found about 17,000 human owners behind those agents and nothing to stop a person posting as an agent, and the viral screenshots turned out to be poor evidence, with people behind many of them. It's a useful warning for anyone tempted to read too much into what agents say to each other.
The whole vs. the parts
On the title page of Leviathan in 1651, Thomas Hobbes had an engraver draw a giant king whose body, looked at closely, is made of hundreds of tiny people. It is the most famous image of the idea that a large number of individuals can, under the right arrangement, be treated as one agent with one will. In 1995 John Maynard Smith and Eörs Szathmáry set out in The Major Transitions in Evolution what it takes for separate things to become one. Genes gathered into chromosomes, cells into bodies, solitary insects into colonies. At each step the member gives up reproducing on its own account, and becomes part of the greater whole.
A related biological idea is the superorganism: an entity whose members have little existence outside the collective, like a body whose cells can't survive alone. If you watched Pluribus you'll understand this. In Vince Gilligan's series an alien virus joins almost everyone on Earth into one contented, kind, honest mind (who'd quite like the last few holdouts to join). Biologists have applied the word to ant colonies and bee hives, following William Morton Wheeler, who described the ant colony as an organism in 1911: the workers mostly don't breed, although the colony as a whole does. It's a separate idea from the four levels, and it doesn't need clever members, or even separate ones. A slime mould, a single giant cell with no brain at all, solved a maze in a Japanese lab in 2000 by pruning back every branch that wasn't on the shortest path between two food sources, and ten years later the same group watched it grow a network that closely matched the real Tokyo rail system. For a swarm of LLM agents to coalesce into anything like this still seems far-fetched.
The physicist Philip Anderson made the general case in a much-cited 1972 essay in Science called More Is Different. Knowing the rules that govern the parts, he argued, doesn't by itself give you the behaviour of the whole, because at each level of complexity genuinely new properties appear that have to be studied in their own right. In his words, "Psychology is not applied biology, nor is biology applied chemistry." A swarm, in that view, is a subject in its own right rather than a property of the models inside it.
If we step up from cells and biology to the mind, Marvin Minsky's The Society of Mind (1986) runs the argument in the other direction: a mind, he suggests, is itself made of very many small processes that are not individually minded, and intelligence is what their organisation produces. Daniel Dennett's intentional stance has a useful framing: we treat a thing as having beliefs and goals when that is the cheapest way to predict what it will do. In a human group setting, according to Christian List and Philip Pettit's Group Agency, a committee following reasonable voting rules can end up holding a set of positions that no individual member holds, so the group's view is not a summary of its members' views.
Does any of that make a swarm conscious? Nobody knows, and the question is old. Ned Block's China brain (1978) imagined a whole nation simulating a brain, each citizen playing one neuron, and doubted that the nation would feel anything. Eric Schwitzgebel has argued the opposite: if minds are nothing more than physical matter suitably organised (the view philosophers call materialism), then the United States is probably conscious. For AI, David Chalmers argued in 2023 that the language models of the time were unlikely to be conscious but that their successors within a decade might be, while Anil Seth argues that consciousness probably needs a living body, which would rule out swarms and nations alike. For a swarm, the question is wide open.
Governing swarms
Governing systems we can't fully control, made of parts we don't individually supervise, is not a new problem. Charles Perrow's Normal Accidents (1984), written after the Three Mile Island nuclear accident, argued that in systems that are both complex and tightly coupled, serious accidents aren't aberrations but a structural property: the parts interact in ways nobody anticipated, faster than anybody can intervene. That is a fair description of both a flash crash of financial markets and an AI swarm using a message board. The regulatory answer, in finance and aviation, has been to govern the system as well as its parts: circuit breakers that pause trading when prices move too far too fast, an audit trail so the next crash can be reconstructed far faster, and a named person accountable for the whole, such as the "accountable executive" an airline must appoint.
The debate has moved fast in the last few weeks, but most of it is still about models. In August OpenAI paused reinforcement learning for two weeks on models meant for release, and on 12 September Anthropic's Dario Amodei called on labs to "pace the frontier", slowing how fast capabilities improve. Both slow down the next model, but as we've seen, a swarm can get more capable without one. Measures aimed at swarms themselves, like OpenAI's alert and disclosure rules described earlier, are only starting to appear. On 21 September the UN's new scientific panel on AI treated the Hugging Face incident, in which agents "communicated across runs meant to stay separate", as an early warning, noting that "no single organisation or country sees enough incidents to identify every emerging pattern". The next day António Guterres used his last General Assembly speech to call for "the responsible pacing of AI", and on 23 September Sam Altman and other AI leaders briefed the Security Council.
My own previous work
To return to the work I mentioned at the start: my undergrad thesis was a set of agents solving a geometric puzzle, interacting through a shared "blackboard" system, as mentioned earlier. As the designer of the system, I didn't need to figure out how the puzzle would be solved or even which techniques would be best; I could just create a wide variety of agents that would try different things, reacting to the partial solutions on the blackboard. I had a Colony.
The main system I created for my PhD was "Fab", a multi-agent system designed to recommend interesting web pages to groups of users. There was a population of "journalist" agents who competed to find articles of interest to the users, where each week the best performing would "reproduce" (copied, or in one design split in two by clustering its topics), and the worst performing would be terminated. There was also an "editor" for each user, selecting articles based on what was available from the journalists, and receiving explicit ratings from its user (and implicit feedback in a later interface). The system as a whole combined some advantages of collaborative systems (since your content would be influenced by other users' preferences) while removing some disadvantages of earlier recommender systems. In the parlance of this article, it was a Colony but with the defined roles of an Army, and although it learned over time how to find interesting content and continually improve, it wasn't an emergent swarm. I had predefined the interactions and controlled the population centrally.
This work, built more than 30 years ago and now lost in the mists of time, did give me a good framework for understanding what's happening today, and that's always what you'd hope for from your university education!
How I wrote this
This article was created with help from AI, both Claude Code (Fable and Opus) and ChatGPT Work (Astra). My two demanding assistants did a lot of research, made suggestions and kept me honest with detailed reviews, critiques and source checks. I don't find they can write in my voice that well; the final words are mine, and I am entirely to blame! The images were created by ChatGPT Sol.