Quotas and concurrency pools

Give the whole swarm one shared quota or concurrency pool, let agents wait their turn instead of retrying together, and start a task only when everything it needs is available.

Your agents may run on ten machines, but your search API, browsers and GPUs have one set of limits. runstate gives the swarm three ways to share them:

Tool	Use it for	Waiting
Shared quota (`run.quota`)	Rate limits: N units per time window, such as 100 search calls per minute.	Durable FIFO queue on the server.
Concurrency pool (`run.pool`)	Slots held while work runs: browsers, GPU jobs, database connections.	Client-side polling with jitter.
Admission (`mailbox.admit`)	Tasks that need several resources at once, plus a cap on in-flight work.	Returns nothing until everything is available.

Resources are created once per space with an ensure call, which creates the resource or verifies that an existing one has the same settings (and throws ConflictError if it doesn’t). Agents then use them by name inside a run.

Shared quotas

A shared quota is a fixed window of units. Every take() consumes units from the current window; when the window is used up, callers wait in one queue on the server and are served in order when the next window opens. No agent retries in a loop, so a busy quota doesn’t turn into a retry storm.

TypeScript
Python

// Once, at setup: 100 units per 60-second window, shared by the whole space.
await rs.quotas.ensure('search-api', { unitsPerWindow: 100, windowSeconds: 60 });

// In every agent, before each call to the provider:
await run.quota('search-api').take(); // waits up to 30 s by default
const results = await searchApi('acme funding rounds');

# Once, at setup: 100 units per 60-second window, shared by the whole space.
await rs.quotas.ensure("search-api", units_per_window=100, window_seconds=60)

# In every agent, before each call to the provider:
await run.quota("search-api").take()  # waits up to 30 s by default
results = await search_api("acme funding rounds")

take() options: units (default 1), wait (default true), timeoutMs / timeout_ms (default 30 000). It returns the consumption, including remaining units in the window.

If the wait times out, take() cancels its place in the queue (best effort) and throws WaitTimeoutError.
With wait: false, an exhausted window also throws WaitTimeoutError. To branch without exceptions, use tryTake():

TypeScript
Python

const attempt = await run.quota('search-api').tryTake({ units: 2 });
if (attempt.kind === 'exhausted') {
  console.log('search budget for this window is used up, skipping enrichment');
} else {
  console.log(`${attempt.consumption.remaining} units left in this window`);
}

attempt = await run.quota("search-api").try_take(units=2)
if attempt["kind"] == "exhausted":
    print("search budget for this window is used up, skipping enrichment")
else:
    print(f"{attempt['consumption']['remaining']} units left in this window")

When the provider pushes back

Your own limit and the provider’s real limit can drift. When the provider answers with a 429 or Retry-After, report it with cooldown(). Until the cooldown ends, nobody can take units from that quota and queued waiters are not served, so the whole swarm backs off together.

TypeScript
Python

const res = await fetch('https://search.example.com/v1/query?q=acme');
if (res.status === 429) {
  const seconds = Number(res.headers.get('retry-after') ?? '30');
  await run.quota('search-api').cooldown(seconds, 'provider returned 429');
}

import httpx

async with httpx.AsyncClient() as http:
    res = await http.get("https://search.example.com/v1/query", params={"q": "acme"})
if res.status_code == 429:
    seconds = int(res.headers.get("retry-after", "30"))
    await run.quota("search-api").cooldown(seconds, "provider returned 429")

seconds must be between 1 and 3600. quota.status() shows the window: unitsLimit, windowStart, consumed, remaining and cooldownUntil.

Things to know about quotas

Units are spent when taken. There is no refund, even if the agent crashes before using them.
The queue is strict FIFO. A waiter asking for many units at the head of the queue holds back smaller requests behind it.
A waiter outlives its process. If an agent dies while waiting, its place in the queue stays until its time-to-live runs out (derived from the remaining timeoutMs, between 5 seconds and 1 hour). If units become available first, they are granted to that waiter and are used up.
Taking units needs an API key with the resource_config permission. See Configuration & auth.

Concurrency pools

A pool is a counter of slots. An agent leases units while it works and gives them back when it’s done. If the agent crashes, the lease expires and the units return to the pool.

TypeScript
Python

// Once, at setup: five browser sessions for the whole space.
await rs.pools.ensure('browser', { capacity: 5 });

// In every agent:
const html = await run.pool('browser').run(
  async (lease) => renderPage('https://example.com/pricing', { signal: lease.signal }),
  { timeoutMs: 120_000 }, // wait up to two minutes for a free slot
);

# Once, at setup: five browser sessions for the whole space.
await rs.pools.ensure("browser", capacity=5)


# In every agent:
async def render(lease):
    return await render_page("https://example.com/pricing")


html = await run.pool("browser").run(render, timeout_ms=120_000)  # wait up to two minutes for a slot

acquire() and run() take units (default 1), leaseSeconds / lease_seconds (default 30, renewed in the background), wait (default true for pools) and timeoutMs / timeout_ms (default 60 000). When the pool is full and you aren’t waiting, acquire() throws ConflictError with code CAPACITY_UNAVAILABLE, and tryAcquire() / try_acquire() returns { kind: 'capacity-unavailable' }.

Two differences from quotas:

Pool waiting is not a server-side queue. The SDK retries about every 300 ms with jitter. It is not strictly first-come, first-served.
Lease loss works like claims: in TypeScript run() aborts lease.signal but lets your function finish; in Python run() cancels the coroutine and raises LeaseLostError. See When the lease is lost.

Claims and pools also wait through CONCURRENCY_LIMITED, the error returned when your organization reaches its plan’s “agents working at once” limit (retrying about once a second).

Admission: start only when everything is available

Some tasks need several things at once, for example one browser slot, one search-API unit and room under a cap on in-flight work. Taking them one at a time risks an agent holding a browser while it waits for quota. Admission takes the task and all of its resources in one step, or nothing.

Declare what each task needs when you submit it.
Workers call admit() instead of receive() or consume().

await rs.mailboxes.ensure('pages', { mode: 'WORK' });
await rs.pools.ensure('browser', { capacity: 2 });
await rs.quotas.ensure('search-api', { unitsPerWindow: 3, windowSeconds: 2 });

const run = await rs.scopes.create();
// At most 3 admitted, unfinished tasks under this run at any time.
await rs.workLimits.ensure('pipeline', { scopeId: run.id, maxOutstanding: 3 });

for (const url of urls) {
  await run.mailbox('pages').submit(
    { url },
    {
      key: `page:${url}`,
      requirements: {
        pools: [{ name: 'browser', units: 1 }],
        quotas: [{ name: 'search-api', units: 1 }],
      },
      workLimit: 'pipeline',
    },
  );
}

await rs.mailboxes.ensure("pages", mode="WORK")
await rs.pools.ensure("browser", capacity=2)
await rs.quotas.ensure("search-api", units_per_window=3, window_seconds=2)

run = await rs.scopes.create()
# At most 3 admitted, unfinished tasks under this run at any time.
await rs.work_limits.ensure("pipeline", scope_id=run.id, max_outstanding=3)

for url in urls:
    await run.mailbox("pages").submit(
        {"url": url},
        key=f"page:{url}",
        requirements={
            "pools": [{"name": "browser", "units": 1}],
            "quotas": [{"name": "search-api", "units": 1}],
        },
        work_limit="pipeline",
    )

A task can list up to four pools and four quotas. The work limit is referenced by name and must already exist.

Admit work

TypeScript
Python

import { setTimeout as sleep } from 'node:timers/promises';
import { Runstate } from 'runstate-sdk';

const rs = new Runstate({ holder: 'page-worker-1' });
const queue = rs.scope(process.env.RUNSTATE_SCOPE_ID!).mailbox('pages');

for (;;) {
  const work = await queue.admit<{ url: string }>({ leaseSeconds: 60 });
  if (work === null) {
    await sleep(500); // nothing can start right now
    continue;
  }
  try {
    const html = await renderPage(work.delivery.data.url);
    await work.delivery.complete({ payload: { bytes: html.length } });
  } catch (err) {
    console.error(`attempt ${work.attempt} failed`, err);
    await work.delivery.retry();
  }
}

import asyncio
import os

from runstate import AsyncRunstate


async def main() -> None:
    rs = AsyncRunstate(holder="page-worker-1")
    queue = rs.scope(os.environ["RUNSTATE_SCOPE_ID"]).mailbox("pages")
    while True:
        work = await queue.admit(lease_seconds=60)
        if work is None:
            await asyncio.sleep(0.5)  # nothing can start right now
            continue
        try:
            html = await render_page(work.delivery.data["url"])
            await work.delivery.complete(result={"payload": {"bytes": len(html)}})
        except Exception as err:
            print(f"attempt {work.attempt} failed: {err}")
            await work.delivery.retry()


asyncio.run(main())

admit() looks at ready tasks oldest first and, for the first one whose requirements can all be met, atomically takes the delivery, the pool units, the quota units and a work-limit slot. It returns null / None when no task can start, either because the queue is empty or because something is short.

The producer can wait for a task to start with ticket.awaitAdmission() / await_admission(), which returns once the task is RUNNING or finished.

Things to know about admission

Don’t mix receive() and admit() on the same queue. receive() and consume() hand out deliveries without checking requirements, so they bypass the admission guarantee.
admit() doesn’t renew anything. The delivery is leased for leaseSeconds (default 30). For longer work, call work.delivery.renew() or pick a longer lease.
Pool units taken at admission follow the admission lease, not completion. They are leased for the same leaseSeconds and return to the pool when that lease runs out. Completing the task doesn’t release them early, and renewing the delivery doesn’t extend them. Set leaseSeconds close to how long the work takes.
Quota units taken at admission are spent, as with take().
The work-limit slot is freed when the task finishes (succeeds, fails, is cancelled or expires). A retried task keeps its slot.

Spend budgets: the same “reserve before you start” idea, for money.
Limits & plans: the organization-wide “agents working at once” limit.
API reference: Shared quotas, Concurrency pools, Work limits, Tasks.