System requirements

Agoranomos runs on any machine with Docker that has the resources below (ADR-0010, and ADR-0022 for the memory). ./agoranomos preflight checks them before every start (Command line).

Docker runs as root, the default of Docker Engine: rootless Docker and userns-remap, which map the users of containers to other users of the host, are not supported with the backups, which would belong to another user (Configuration).

On macOS, Docker Desktop sets the memory, CPUs and disk of its virtual machine in Settings → Resources → Advanced. Windows is supported through WSL2: run the commands of Agoranomos in a WSL2 terminal, with the checkout in the Linux file system of WSL, not under /mnt/c, where by default chmod cannot make .env readable only by its owner (Run Agoranomos locally). With WSL2, Docker Desktop leaves the memory and the CPUs to WSL, which gives its virtual machine half of the memory of Windows and all of its logical processors by default: memory= and processors= in the [wsl2] section of .wslconfig, in the Windows user folder, change them, and wsl --shutdown, run in PowerShell, restarts WSL with them. .wslconfig does not exist by default: create it (as of October 2026, Docker's and Microsoft's documentation). Docker reports a little less memory than its machine has, so preflight counts 5,632 MiB as 6 GB, 7,680 MiB as 8 GB and 11,776 MiB as 12 GB.

The CeleryExecutor variant needs more: its broker and its worker come on top of the services of the default variant, and 12 GB of memory for Docker is recommended for it, which preflight checks (ADR-0022); the minimum stays 6 GB.

Memory limits

Every service has a memory limit, with no swap (Configuration). On the recommended memory, 8 GB or 12 GB for the CeleryExecutor variant, a limit keeps a service that runs short of memory from taking it from the others: the kernel stops a process of that service when the service reaches its limit, and the other services go on. On the minimum of 6 GB the limits do less, as the end of this section says.

The limits are set from measurements made on 8 and 10 October 2026, those of warehouse-db and airflow-triggerer from measurements up to 19 October 2026, and that of airflow-dag-processor from measurements up to 21 October 2026, as follows:

• A service that runs all the time gets 1.25 times its highest peak, rounded up to a multiple of 128 MiB, and at least 256 MiB.
• A one-shot setup job gets 1.5 times its highest peak, rounded up to a multiple of 128 MiB, and at least 128 MiB: it runs for seconds, and a sample can miss the moment of its peak.
• The peaks are the highest of every measurement, with the limits or without them: a peak under a limit is never higher than without it, so this can only raise a limit. They come from measurements on a Mac (arm64) at the first start, during the stack tests and idle, and in CI on amd64 and arm64 during the stack tests, each in both variants. The one-shot jobs are measured at the first start, and in CI at the starts of the restores of the backups. db-init mostly ends before a sample catches it, and gets the minimum.
• Metabase gives half of its limit to the heap of Java (-XX:MaxRAMPercentage=50). The other half holds what Java needs besides the heap, about 0.5 GiB for Metabase 0.64.1. Its limit is at least 2,048 MiB.

A restore runs inside the memory limit of the database that it restores (restore). With the demo data, whose dumps are 1.3 MB, the restore of metabase into the running stack took platform-db to 154 MiB and warehouse-db to 50 MiB, and the restore of all three databases on empty volumes took them to 138 MiB and 49 MiB (Mac, 11 October 2026). In CI on amd64 on 11 October 2026, the restore of Metabase into the running stack took platform-db to 251 MiB and warehouse-db to 61 MiB, and the restore on empty volumes took them to 134 MiB and 54 MiB; on the Hetzner CAX21 on the same day, 227 MiB and 52 MiB, and 131 MiB and 48 MiB. The restore of all three databases on empty volumes took 91 seconds there. All of them are below the limits of 512 MiB and 384 MiB. A restore of a warehouse of several GB takes more memory in warehouse-db: raise WAREHOUSE_DB_MEMORY before it.

warehouse-db compiles no query with JIT: db-init sets jit = off on the database warehouse on every start. The tests of the intermediate models of dbt read views over views, and with JIT the code compiled for each of them took more memory than the test itself. On the Mac on 16 October 2026, with the history of the demo platforms, the six tests of int_fx_rates__filled, four at a time as dbt runs them, took warehouse-db to about 510 MiB with JIT, past its limit of 256 MiB, and to 160 MiB without it. Without JIT, a whole run of the transform of the model, 82 of 82 tasks in 140 seconds, still took it to 239 MiB, close to that limit. Its limit is 384 MiB since 19 October 2026, by the formula above: the stack tests in CI, with a task of dbt build for each node, took it to 305 MiB of it the same day, which the formula keeps at 384 MiB.

Each variant has a budget for the sum of its limits, the one-shot jobs included, so that on the recommended memory every service at its limit at the same time fits what Docker reports (ADR-0022): 7,680 MiB, 8 GB, for the default variant, and 11,776 MiB, 12 GB, for the CeleryExecutor variant. Today the limits add up to 7,296 MiB in the default variant in local mode, with the 384 MiB of the container ai, which leaves 384 MiB, and to 8,192 MiB in the CeleryExecutor variant. Server mode runs no container ai and adds the proxy and the backups, 256 MiB each: 7,424 MiB in the default variant, which leaves 256 MiB, and 8,320 MiB in the CeleryExecutor variant. A unit test checks the sum for the defaults of both variants, in local and in server mode.

On the minimum of 6 GB, 5,632 MiB as Docker reports it, the limits do not keep the services apart: in local mode, the limits of the services of the default variant that run all the time add up to 6,272 MiB with the container ai, more than that, so services that reach their limits at the same time can still run the whole machine out of memory. What holds there is that the highest measured peaks of the services of the default variant add up to at most 5,068 MiB with the 266 MiB of the container ai, which fits. In server mode, with the proxy and the backup service, the limits add up to 6,400 MiB and the highest peaks, with the 77 MiB of the proxy and the 66 MiB of the backup service, to at most 4,945 MiB, which fits as well. For the CeleryExecutor variant, 12 GB of memory for Docker is recommended, and 6 GB stays the minimum, as for the default variant.

Measured memory

These tables were measured on 8 October 2026, with Airflow 3.3.2, Metabase 0.64.1, PostgreSQL 18.6 and Valkey 9.1.2 and with the default limits, on a Mac with Apple Silicon (arm64) in Docker Desktop 4.93.0 (Docker Engine 29.8.1, Docker Compose 5.5.1), with 7,935 MiB of memory and 4 CPUs as Docker reports them. Each variant started from empty volumes (first start), then ran the stack tests, then stood idle for 10 minutes. Every peak stays below its limit, and no process was killed and no container restarted.

These tables were measured with the limits of 8 October 2026; the limits of the tasks and of the Celery scheduler have grown since, as the table above says.

The peaks of airflow-dag-processor and airflow-triggerer in these tables, and in those of 11 October 2026 below, include the health checks of the two services of that time, which ran airflow jobs check every 30 seconds: it starts Airflow in the container, which took about 146 MiB and more than a second of CPU each time. Since 12 October 2026 the check reads the metadata database without Airflow, in about 25 MiB and 0.03 seconds of CPU. On the same Mac, idle for 10 minutes before and after that change, the peak of airflow-dag-processor went from 423 to 388 MiB and that of airflow-triggerer from 440 to 335 MiB. The limits stay as they are until new measurements in CI. Those of airflow-triggerer came: in the 54 tables of the 12 runs of the Stack workflow on main from 12 to 19 October 2026, on amd64 and arm64, in both variants and in local and server mode, its peak was 306 to 344 MiB, and 372 MiB since its start, so its limit is 512 MiB since 19 October 2026, which makes room for that of warehouse-db. Those of airflow-dag-processor came on 21 October 2026: in the 93 tables of the 32 runs of the Stack workflow from 13 to 21 October 2026, on amd64 and arm64, in both variants and in local and server mode, it peaked at 416 MiB at most while it still ran dbt ls, and at 379 MiB since it renders the transform DAG from the manifest of dbt, in a restore of the backups on empty volumes, and 368 MiB in the stack tests. No process was stopped at a limit and no container restarted in any of the tables. Its limit is 512 MiB since 21 October 2026; with its page cache, it reached 507 MiB since its start, which the kernel takes back before it stops a process.

The same holds for airflow-worker with the CeleryExecutor. In v0.1.5 and before, its health check ran celery inspect ping with the Celery app of Airflow every 30 seconds, which starts Airflow in the container too. The memory of a health check counts toward the limit of its container, so the peaks of airflow-worker on this page can include one that ran during the tasks. The check now pings the worker through the broker with Celery alone, without Airflow, and still fails when the worker does not answer within 5 seconds. On the same Mac on 14 October 2026, with the CeleryExecutor:

• In 10 runs of each in the worker, the old check took 1.22 to 1.41 seconds of CPU, reached 144 MiB, and raised the memory of the container by 120 MiB while it ran. The new check took 0.16 to 0.20 seconds of CPU, reached 39 MiB, and raised it by 27 MiB.
• Idle for 10 minutes, the worker used 4.6 % of a CPU with the old check and 1.0 % with the new one.
• Over the stack tests, the worker peaked at 918 MiB with the old check, and at 949 and 896 MiB in two runs with the new one. These peaks come from the tasks. A check adds to them only when it runs at the same moment as the peak.

On 13 October 2026 the transform DAG moved from dbt ls to a manifest of the dbt project, and its tasks of dbt to a pool of two (ADR-0027). On the same Mac, with the demo project of two models, a stack of v0.1.5 moved to that change and then ran the stack tests, in the default variant and with the CeleryExecutor. airflow-dag-processor no longer starts dbt ls: it parsed the transform DAG in 0.08 to 0.27 seconds, where dbt ls had taken 1.3 to 2.8, and, idle for 10 minutes, peaked at 242 MiB in the default variant and 330 MiB with the CeleryExecutor, against the 388 MiB of 12 October 2026 above; its highest peak was 342 MiB, in the stack tests with the CeleryExecutor. A successful run of the transform took 8 to 13 seconds instead of 17, with a median task of dbt of 1.7 to 2.6 seconds instead of 3.3; the four tasks of the demo form one chain, so one ran at a time. In the stack tests, the container of the tasks peaked at 617 MiB (airflow-scheduler, default variant) and 883 MiB (airflow-worker, CeleryExecutor); across all these measurements, airflow-triggerer peaked at 341 MiB and warehouse-db at 78 MiB. metadata-export, which now also copies the manifest, peaked at 114 to 117 MiB, and at 131 MiB with its page cache. Every peak is below its limit, and the limits stay as they are.

CI measures the stack in the same way in every run of the stack tests, on the GitHub-hosted runners ubuntu-24.04 (amd64) and, every week and in a manual run, ubuntu-24.04-arm (arm64), and adds the table to the summary of each job of the Stack workflow. As of October 2026, the tables with the limits on this page come from the Mac; the CI peaks in the table of the limits above come from runs without limits, except those of airflow-triggerer, the tasks and the Celery scheduler, which come from runs with the limits of October 2026; those of warehouse-db and airflow-triggerer come from runs with their limits of 19 October 2026, and those of 13 to 21 October 2026 from runs with the limits of their time.

"Since start" counts the page cache too, most of which the kernel takes back before it stops a process, so it can come close to a limit without harm. Without limits, platform-db reached 389 MiB since its start, most of it page cache, which was more than its earlier limit of 384 MiB; with the limit, the kernel keeps less page cache for it. With that limit, the stack tests of 11 October 2026 showed 384 MiB since the start and a peak of 321 MiB, which is why the limit is 512 MiB now.

Mac arm64, local, first start (aarch64)

Mac arm64, local, the stack tests (aarch64)

Mac arm64, local, idle (aarch64)

Mac arm64, celery, first start (aarch64)

Mac arm64, celery, the stack tests (aarch64)

Mac arm64, celery, idle (aarch64)

On 11 October 2026 the stack also ran in server mode on a Hetzner Cloud CAX21, the reference server of the project (arm64, 4 vCPUs, 8 GB; Ubuntu 26.04, Docker Engine 29.9.0, Docker Compose 5.6.0, 7,720 MiB of memory as Docker reports it, no swap), set up as Deploy on a server says, with a certificate of Let's Encrypt and the default limits of that day. It ran the stack tests, with the memory report around them: the pytest summary of that run was lost, and a repeat run passed (519 passed and 9 skipped, the skips being the CeleryExecutor variant and the opt-in restore tests). It then stood idle for 10 minutes. Every peak stays below its limit, and no process was killed and no container restarted.

Hetzner CAX21 arm64, server mode, the stack tests (aarch64)

Hetzner CAX21 arm64, server mode, idle (aarch64)

When a service reaches its limit

A task that writes 4 GiB of memory, more than AIRFLOW_TASKS_MEMORY, ran on the same Mac with the default limits, on 8 October 2026:

• With the CeleryExecutor, the kernel stopped the task process in airflow-worker. The task failed within a second, and its log in Airflow says Process terminated by signal. Likely out of memory error (OOM)., with the signal -9 (SIGKILL). The worker went on.
• With the LocalExecutor, the same happened in airflow-scheduler, in each of five runs: the task failed within a second, and the scheduler went on.
• Each time, the other services went on, and agoranomos_selfcheck succeeded right after.

The kernel stopped the task first because Agoranomos makes each task process the first that the kernel stops at the memory limit, on a host with Yama (below). A plugin of Airflow in the image, airflow/plugins/agoranomos_oom.py, writes 1000, the highest value, to /proc/self/oom_score_adj of each task process as the task starts, and the child processes of the task, such as dbt, inherit it. The scheduler, the worker and the other services keep 0, except two children of the scheduler that go between the tasks and the scheduler (below). Without the plugin, the kernel stops the process that uses the most memory, and with the LocalExecutor that can be the scheduler itself, as Airflow's documentation warns (ADR-0015): in one of two such runs before the plugin, it did. Docker restarted the container, and the task stayed running in Airflow until the new scheduler reset it as orphaned five minutes later and ran it again, as a second try although the task had no retries.

The plugin works only on a host whose kernel restricts ptrace with Yama, such as Ubuntu, which is the usual case on a server. Airflow marks the process of each task so that the other processes of its user cannot read its memory, and for such a process the kernel gives the files of the process in /proc to root. So for the write the plugin lifts that mark for the time of two system calls and puts it back, before any code of the DAG runs, and only where Yama keeps the other processes of the user out: /proc/sys/kernel/yama/ptrace_scope reads 1, 2 or 3, and a process that is not an ancestor of the task cannot open its memory in the meantime. Docker Desktop on macOS has no Yama: its kernel has no such file (checked on 9 October 2026), and the same may hold for Docker Desktop on Windows. On a host without Yama, the plugin leaves the task as it is and writes one line to the log of the task, which says that the task keeps its place at the memory limit. There the kernel can still stop the scheduler, as it did before the plugin: write tasks that are safe to run twice, and run heavy tasks on the CeleryExecutor (Run the tasks on a separate worker).

The five runs with the LocalExecutor above were made on a Mac, with a version of the plugin that lifted the mark on every host, before it was limited to hosts with Yama. On that Mac the plugin now leaves the task as it is. On 11 October 2026 the runs were repeated on an Ubuntu server, where the plugin acts: a Hetzner CAX21 (arm64, 8 GB) with Ubuntu 26.04 and Linux 7.0, Docker Engine 29.9.0 and Docker Compose 5.6.0, in server mode with the default limits. The kernel stopped the task each time, but in 5 of 39 runs it then stopped the scheduler too, a few milliseconds later, and Docker restarted the container; the task ran again when the new scheduler reset it as orphaned, five minutes later. With AIRFLOW_PARALLELISM=2, it happened in 2 of 10 of these runs.

The cause is in the kernel. Since Linux 7.0 (commit a72f73c4dd9b), the kernel no longer sees a process that it has stopped once the process starts to exit. So while the task still frees its memory, which took up to a tenth of a second in these runs, the container stays at its limit, and a process that asks for memory then makes the kernel stop a second one: the largest of those that keep 0, which is the scheduler. In the runs where the log of the kernel tells which process it was, it was an airflow command that docker compose exec had just started in the container, but any process of the container can be one.

So on a host with Yama, the plugin also ranks two children of the scheduler between the tasks and the scheduler: serve-logs at 800 and the health check of the scheduler at 700. The kernel stops them before the scheduler, and the stack works without them: the API server reads the log of a finished task from the volume of the logs that the Airflow containers share, and asks serve-logs only for the log of a task that runs. Without the health check, ./agoranomos status shows airflow-scheduler as unhealthy, and the tasks go on, but the next ./agoranomos up stops on it, naming the scheduler: up waits for a healthy scheduler and does not restart a container that runs. The scheduler starts each of them once, so they cover the first two such rounds in its life, and a third one restarts the scheduler, as before. To get them back, run docker compose restart airflow-scheduler in the checkout when no task runs, since a restart of the scheduler runs its tasks again. The LocalExecutor workers keep 0, like the scheduler: an idle worker that the kernel stops while it holds the lock of the queue from which the workers take their tasks leaves the other workers waiting for that lock, and no task runs until the scheduler restarts, as a run on the same server showed. In 48 runs with these ranks, the kernel stopped a second process 5 times: serve-logs 3 times, the health check once, and the scheduler once, after both had been stopped. agoranomos_selfcheck succeeded after each series of runs.

The plugin does not help when the scheduler itself goes over its limit, or restarts for another reason. As of Airflow 3.3.2 (October 2026), a restart of the scheduler runs again every task that was running in it, whatever its retries, so every task has to be safe to run twice, as the tasks of the demo are. With the CeleryExecutor, the scheduler has a container of its own, so a task at its limit cannot stop it.

The memory report shows such a kill in the column OOM kills and a restart in Restarts, docker inspect of the container says "OOMKilled": true, and the stack test test_no_service_ran_out_of_memory_or_restarted fails.

When it happens to a service that runs all the time, raise its limit in .env, AIRFLOW_TASKS_MEMORY for a task, and run ./agoranomos up (Configuration). With the LocalExecutor, up to AIRFLOW_PARALLELISM tasks run in the scheduler at the same time, 4 by default, so raise AIRFLOW_TASKS_MEMORY with it, and for pipelines heavier than the demo. Each limit that you raise needs memory on the machine as well (Memory limits).

The one-shot setup jobs have their limits fixed in compose.yaml, with no setting in .env. A job that reaches its limit usually exits with the code 137, and its log ends without an error of its own, or is empty. A job that does its work in a child process can exit with another code and say so in its log instead: metadata-export, for example, reports that dbt ls exited with -9, the signal that the kernel sends at the limit. When other services wait for the job, such as for airflow-init, docker compose up stops first and names it: service "airflow-init" didn't complete successfully: exit 137. Otherwise up names it and adds that exit code 137 means that it was killed, most often because it reached its memory limit (Command line). ./agoranomos status shows such a job as Exited (137), and docker inspect says "OOMKilled": true. Then raise its mem_limit and memswap_limit in compose.yaml, both to the same value, run ./agoranomos up, and report it in an issue, so that its limit grows in Agoranomos too.

How to measure

scripts/memory_report.py samples every container of the stack while a command runs, about twice a second, and prints a table: the limit of each service, its highest sample (the memory of the container without the page cache that the kernel can take back), its highest use since the container started (from its cgroup, page cache included), and how many of its processes the kernel killed at the limit. In the root of the checkout, with the stack running:

uv run python scripts/memory_report.py --title "Local, the stack tests" -- uv run pytest tests/stack
uv run python scripts/memory_report.py --title "Local, idle" --seconds 600

CI does the same in every run of the stack tests, and adds the table to the summary of the job.

In the table, n/a means that there is nothing to read: a one-shot job that has exited, or a machine without cgroup v2. unread means that docker inspect or docker exec failed for that container, for example because it stopped while the table was made, as a container that the kernel stopped at its limit can: the rest of the table still comes out, and when the limit of a running service is unread, so is the total of the limits. A line of docker stats that cannot be read is left out, and a note under the table says how many were, such as 1 line of docker stats could not be read and was left out. The peaks then come from fewer samples, so they can be a little lower than the real ones.