AWS Parks 75,000 Idle Agents on a Raspberry Pi

By Daniel Rao | October 24, 2026

Here is the dirty secret of the agent boom: most of your expensive AI workforce is doing nothing. It sits there, process held open, memory reserved, meter running, waiting for a build, a webhook, or a human approval tomorrow morning. AWS has just open-sourced a runtime that refuses to pay for that idleness, and its headline number is gloriously absurd: 75,000 concurrent agents fitted into the 8 GB of a Raspberry Pi 5.

The runtime is called Infinity. It arrived on October 23, 2026, from AWS and UC Berkeley as part of the Hydro project, under the Apache 2.0 licence, as an early research release rather than a polished product. But the idea at its centre is elegant, and it attacks a cost every team running agents at scale already feels: waiting is not free when your architecture holds a process open for every waiting agent.

Anyone who has watched a terminal sit blinking for twenty minutes while a build crawls knows the feeling. Multiply that terminal by ten thousand, and pay rent on every one while it waits. That is what most agent harnesses do today, and Infinity is AWS saying: stop.

Photo: Rear of a server rack at NERSC. Credit: NERSC, CC0. This shows data centre hardware of the kind Infinity is designed to relieve, not an AWS facility.

The bottleneck was never the model

For two years the conversation about agents has been dominated by models: bigger context windows, better tool use, sharper reasoning. All of that matters, but it misses the plumbing problem. A coding agent may start a build that runs for twenty minutes; a triage agent may wait hours for the next pull request; an assistant may wait until morning for an approval. In the conventional harness, the loop is blocking: call the model, invoke a tool, wait, repeat. While the agent waits, it holds a process and its memory, and cannot respond to events that arrive in the meantime.

Infinity borrows an old trick from programming languages. In Rust, JavaScript and Python, async/await suspends a function, lets the thread do other work, and resumes it where it stopped. Infinity treats an agent the same way, because when an agent calls a tool, its conversation history already holds everything needed to continue once the result arrives.

In practice, Infinity executes each agent as a sequence of bounded slices. In a slice, the runtime loads the agent's history, runs one model completion, persists the output, and dispatches at most one tool call. Then the slice ends. Tool calls are the await points: a slice never blocks for a tool, a timer, or any external operation. When the tool result arrives on a shared input queue, the runtime resumes the agent from its saved state.

We checked the live AWS announcement and the Infinity documentation site on October 24, and both describe this suspension model, with the same figures.

What 75,000 agents in 8 GB actually means

The Raspberry Pi number deserves unpacking. AWS did not run 75,000 agents doing useful work simultaneously on a Pi. It ran a benchmark: waves of agents backed by a simulated model, each running twenty turns, 80,000 agents for 3.2 million completions on a single thread of an AMD EPYC processor. Resident memory grew linearly, at about 100 KB per idle agent. At that rate, about 75,000 agents fit in 8 GB. The Infinity documentation site puts it at about 103 KB after twenty tool-calling turns.

That distinction is the whole point. Most of that memory is conversation history, because an idle agent has no task, no stack and no open connection. A suspended agent, in Infinity's framing, is pure data. It costs roughly as much as its history, and nothing else. Compare that with a harness that keeps a process, a stack and a connection warm for every waiting agent, and the arithmetic of a 10,000-agent fleet changes from frightening to merely interesting.

There is a catch: the benchmark used a simulated model, not a frontier model answering real questions. The 100 KB figure measures the runtime's overhead, not the cost of intelligence. Nobody should read the Pi number as a promise that a fleet of frontier agents can run on a hobby board. What it does promise is that the runtime itself will stop being the thing that makes large fleets unaffordable — a narrower claim, and a more believable one.

Infinity organises agents into an agent system: a pool of threads sharing stores, tools and a model, organised much like an actor system. Each thread has its own input queue and threads communicate only by message. Messages for the same thread are processed in order, one slice at a time; slices for different threads run in parallel. When a thread has nothing queued, its task exits, and a new one starts on the next message. Nothing lingers.

Serverless, finally, without the fudge

This is where the design pays off. Because a slice never waits and loads all state from storage, Infinity runs on serverless platforms: in step mode, the platform calls a step per batch of delivered messages, and nothing is retained in memory between calls.

AWS's reference deployment uses an SQS FIFO queue, one message group per thread. Each delivery invokes a Lambda function that runs one step and exits. Histories live in Aurora DSQL, deduplication state in DynamoDB, and sleep tools use SQS delays and EventBridge Scheduler. CDK constructs provision these resources.

AWS writes: "When Infinity suspends an agent during a tool call, the serverless function can end. So even if a tool call takes days, nothing needs to run while the agent waits." That inverts the usual serverless compromise: traditional runtimes that block on tool calls must keep each invocation open while tools run. In Infinity, a waiting agent has no running invocation at all, so an agent waiting on a three-day CI pipeline "uses no compute until the pipeline reports its result".

Making that safe takes quiet engineering: each input carries a stable ID, redelivered messages are recognised and skipped, and streamed output is committed only when a turn completes. FIFO message groups serialise each thread's slices at the queue, so no distributed locks are needed.

The clever bit: events disguised as tool calls

Suspending agents solves waiting, but creates a second problem: how does a sleeping agent learn that something happened? A long command produces output over time; a repository subscription should produce an event per pull request. One tool call can therefore produce a sequence of results, and the runtime must present each to the model in a form it can reason about.

Model APIs, inconveniently, provide no way to represent an asynchronous event. Each tool call must be answered by exactly one result. AWS says it rejected the obvious workarounds: a second result violates the one-result rule, a user message misattributes the event to the user, and a system-prompt note loses the event's place in the history's order.

Infinity's answer is synthetic tool calls. When an event arrives, the runtime adds an assistant tool call the model did not actually make, followed by a tool result containing the event, under the dedicated name receive_event__injected. The __injected suffix marks it as runtime-created; the runtime never offers that name to the model as a tool. It is protocol judo, and arguably the most interesting idea in the release: instead of waiting for model APIs to grow an event primitive, Infinity smuggles events through the primitive that already exists.

A tool produces events by sending messages to the agent's input queue, marking its first result as the start of a subscription. The agent can go idle between events, and each event resumes it — in a local process or via a new Lambda invocation — so a subscription keeps nothing running while it waits.

Agents talking to agents, without a second system

The same mechanism carries messages between parent and child agents, so Infinity needs no separate messaging system. Each agent has a built-in spawn_thread tool, creating a child thread with its own input queue. The call returns at once, the parent continues, and reports arrive later as events on that call. When the child calls report_to_parent, the runtime tags the message as an event for the original spawn; a final report arrives the same way via close_thread. Each report refers to its spawn, so the parent can tell which child sent it.

Then there is the context-window problem. A hundred subscription events would take up much of a single agent's context window. Infinity's default is to process each event in a separate child thread that inherits the parent's context, acts, and reports back — or closes without a report if irrelevant, leaving the parent's history unchanged. A tool can instead mark an event as associative, adding it directly to the subscribing thread's history.

This is Infinity's bet on how agent systems will actually be built: not one giant agent with an ever-growing context, but a tree of small threads, most asleep, most cheap, waking only when they have something to say. For a fuller grounding, see our explainer on what AI agents actually do.

Photo: A server room photographed in 2002. Credit: sylvar via Flickr / Wikimedia Commons, CC BY (attribution required). Not an AWS data centre.

Our view: this is infrastructure, and infrastructure wins quietly

Here is where we stop summarising and say what we think. Infinity will not go viral, and no chief executive will mention it on an earnings call. That is precisely why it matters. The agent era, if it arrives, will not arrive as a single brilliant model. It will arrive as a thousand unglamorous decisions about what happens while agents wait, how they are woken, how they talk to each other, and who pays for the silence between events. AWS has just given away, under a permissive licence, a credible answer to all four at once.

The honest counter-argument is that Infinity is early, and AWS says so itself: "an early research release" shared "to get early feedback". Its open questions are not footnotes: how do models trained mostly on synchronous traces reason about interleaved events? How should a scheduler prioritise user-facing threads over background handlers? How should compaction work for large conversation trees? And how does a runtime establish event provenance when agents exposed to the internet may receive inauthentic payloads? Any team adopting Infinity today is signing up to live with those problems.

There is also a competitive shadow worth naming. Infinity should not be confused with AWS's managed AgentCore offering, which serves a different, production-managed need. Infinity's pitch is openness and control; a managed runtime's pitch is that you do not have to think about any of this. Many teams will choose managed. But for teams that want fleets of long-lived, event-driven agents without renting a small data centre to keep them asleep, Infinity just moved the frontier of what is cheap.

Consider AWS's own cases: a coding agent waiting on a twenty-minute build, a triage agent waiting hours for a pull request, an assistant waiting overnight for an approval. Scale those to fleets watching repositories, inboxes and tickets, most idle at any moment. The old architecture taxes all of them, all the time. Infinity taxes almost none of them, almost all of the time. That is a different economics, and economics decides which architectures survive. The parallel is worth noting: while runtimes get cheaper, the computers agents run on are becoming a business of their own, as we covered when Daytona hit a $1 billion valuation renting computers to AI agents.

What to watch next

First, watch whether anyone outside AWS builds something serious on Infinity in the next few months: an open-source runtime lives or dies on its second adopter. Second, watch event authenticity, flagged by AWS as open work. Third, watch how the synthetic tool call idea travels — adopted elsewhere, or replaced by a native event primitive, either outcome validates the problem identified.

For developers, start small: the Infinity documentation site offers a quickstart that builds a simple agent in a few minutes, and the same code runs locally or on Lambda. That claim — write once, run asleep anywhere — is the one to test first. For everyone else, the takeaway is stranger: the future of AI agents may be decided not by the smartest model, but by the cheapest way to let thousands of not-very-busy agents wait. Where products like Anthropic's Claude Duet, which promises not to touch your files without asking, compete on trust, Infinity competes on cost and scale. Both bets can be right. But only one lets you park 75,000 agents on a Raspberry Pi and forget them until something happens.

Update, Oct. 24: what the docs add — Rust, a working demo, and the benchmark's fine print

One day on, there is still no independent coverage to add — the story remains the AWS announcement and its relay — but the Infinity documentation site, checked this run, fills three gaps in the first version of this article. First, the language: Infinity is a Rust framework, in the docs' own words "for building massively concurrent agentic systems." That matters because the suspension trick at its heart is an async-executor idea, and Rust is where that idea is most at home. Second, a concrete demo the announcement never gave: the docs walk through an agent that runs "cargo test --workspace," returns immediately, and streams each chunk of test output back as an event while it drafts release notes in parallel. The test run fails, the agent fixes the failing refunds test, reruns it, and reports green — with 18 minutes of idle time in the middle at zero compute, because nothing was running while the tests ran. That is the whole thesis in one terminal session. Third, the benchmark's fine print, stated plainly on the docs homepage: the 80,000-agents-in-8.4 GB figure (about 103 KB per agent after twenty tool-calling turns, 3.2 million completions) was measured on a single thread with in-memory stores. The serverless deployment this article describes swaps those stores for Aurora DSQL and DynamoDB, so the Pi number is a statement about the runtime's overhead, not a promise about a production bill — exactly the distinction this article drew, now with the docs' own measurement conditions behind it. The docs' getting-started example, incidentally, wires up a Claude Sonnet 4.5 model, a useful reminder that Infinity is a runtime, not a model: the intelligence is whatever provider you plug in. (Language, demo, and benchmark conditions per infinity.hydro.run, fetched Oct. 24; no second-day independent coverage found this run — do NOT present the docs as third-party validation.)

Frequently asked questions

Can 75,000 frontier agents really run on a Raspberry Pi?

No — and this article never claimed they could. The 75,000 figure counts idle agents, each costing roughly its conversation history (about 103 KB after twenty turns), in a benchmark that used a simulated model, not a frontier model answering real questions. The runtime's overhead is what fits on the Pi; the cost of intelligence is separate, and a fleet of frontier agents still needs real model inference behind it.

Is Infinity the same as AWS AgentCore?

No. AgentCore is AWS's managed, production runtime — the pitch is that you do not have to think about any of this. Infinity is the open-source, self-run alternative under the Apache 2.0 licence: you get openness and control, and you take on the open questions (event authenticity, scheduling, compaction) yourself. Teams that want a managed service will choose AgentCore; teams that want fleets of long-lived, event-driven agents without renting a small data centre to keep them asleep will look at Infinity.

Daniel Rao covers AI infrastructure and developer tools for TechPulse USA, and still thinks the most interesting number in any AI announcement is the one on the invoice.

Sources: AWS Open Source Blog, "Introducing Infinity: a runtime for extreme agent concurrency" (October 23, 2026); Infinity documentation at infinity.hydro.run; Dosa, "Today in AI — October 24, 2026"; AWS News Feed relay (date and summary cross-check).