Out of YAML
I have used Ansible since around 2013. The setup this post is about is much younger: I started it in late summer 2025, built with coding agents, and it grew to 248 roles, 229 playbooks and a bit over 200,000 lines of YAML and Jinja templates in a little over a year. All 177 services I run now deploy with pyinfra by default, a deploy tool where the deploys are ordinary Python. Coding agents did the porting, just as they had built the Ansible setup in the first place.
Every deploy is switched, and Ansible is gone. What I want to write down is why I wanted to switch at all, how the two tools differ, what was easy and what was hard, and what actually got better. The short answer: the tests went from two hours to minutes, deploys to my servers take about half as long when little changes, and the code did not get smaller.
The short version of the results:
- Tests: serially, a bit over two hours for Ansible against about 17 minutes for pyinfra. With both suites parallelised, about 34 minutes against under two and a half. The two sides ran on different Macs, so the ratios are rough.
- Deploys: against my servers on the internet, about half the time when nothing changes, and a fifth to a third less with a code change. On the local network, a redeploy with nothing to do goes from half a minute or more to under ten seconds.
- Code: not smaller; by line count about 15 % larger. In exchange it is type-checked, its configuration is validated before any SSH connection, and it can be tested without a host.
Why I wanted out
The pain was the tests. They ran all the time, which meant Docker ran all the time — on a Mac, inside a Colima VM — and my machines were slow because of it. You don't need a benchmark to notice that; I measured it only later, for this post.
The numbers matched the feeling. The Ansible roles are tested with Molecule, Ansible's role-testing tool, which starts a container per scenario. The complete suite — the unit-level targets behind just test (I use just as the command runner) plus all 56 Molecule scenarios — took 2 hours and 13 minutes, one target after another, on a Mac Studio (M4 Max, 16 cores). 69 % of that time was spent in Docker.
That is not a test suite you run before every change. And slow feedback compounds: every new service needed a new role, every role its Molecule scenario, and each one made the next change a little more expensive. It was the slope.
The second reason was the YAML itself. "Dislike of YAML soup" is literally what I wrote into the spec for the migration. Roughly half of the task volume in my Ansible roles was assert, set_fact, stat, debug and fail — input validation and data plumbing written as a list of tasks, evaluated on the remote host, in the middle of a run.
The third reason is the agents. They work about as well as their feedback loop; I wrote about that in Cheap Oracles. An oracle is anything that tells an agent whether an attempt worked, and a two-hour test suite is an expensive one. Tests that run in-process, without a host or a container, are a cheap one.
There was also little lock-in: only three collections from Galaxy, Ansible's package index, were in use, and secrets already lived in SOPS with age keys, not in ansible-vault.
How pyinfra differs from Ansible
Both tools are agentless: nothing runs permanently on the servers, and everything is pushed over SSH from the machine you deploy from. The difference is what gets pushed.
Ansible ships Python. Every task is a module, and for every task Ansible packs the module and its helper code into a small zip payload (AnsiballZ), sends it to the host and runs it there with the host's Python interpreter. The result comes back as JSON. Facts — which OS, which packages, which users — are gathered the same way, by a Python module on the host. With pipelining, which my configuration had switched on, the payload goes over stdin instead of being copied to a temporary file first, but every task still starts a Python process on the host and does its own check of whether there is anything to do.
pyinfra ships shell commands. The deploy code runs only on my machine. A fact is a shell command like uname -n whose output pyinfra parses locally. An operation compares the state I want with those facts and produces only the shell commands needed to close the gap. That happens in two phases: first pyinfra reads the facts and works out what would change — which you can also run on its own, as a dry run that prints the plan — then it executes. The host needs a shell and the usual tools, but no Python.
Shipping shell commands instead of Python is where pyinfra's speed comes from, when it is faster:
- No interpreter per task. Ansible starts Python on the host for every task, including the ones that end up changing nothing. pyinfra starts nothing on the host beyond the shell commands themselves.
- Nothing to do means nothing sent. An operation whose facts already match produces no command at all. In Ansible, finding out that there is nothing to do is itself a module run on the host.
- Less unconditional work in my deploys. This one is not pyinfra's merit but the port's: the old roles restarted Traefik and re-ran recursive
chowns on every deploy, and the ported deploys don't.
And where it isn't: every fact and every command is still a round trip over SSH. On a fast network that costs nothing; over the internet, latency per round trip replaces Ansible's cost per task. And real work — installing dependencies, migrating a database, building a frontend — takes as long as it takes, whichever tool asks for it. The numbers further down show both.
What about roles? pyinfra has no role format of its own. The equivalent is a plain Python function decorated with @deploy that calls operations and other deploys; you share them as Python packages instead of through Galaxy. The rest of a role maps onto ordinary code: defaults become default values on a typed config class, handlers become operations with a condition like _if=unit.did_change, and templates stay Jinja. In my port, a service's separate deploy, backup, restore and remove roles became one Python package with one config class.
A third approach keeps Python on the host but stops paying for it per task: Mitogen for Ansible replaces Ansible's module runtime with calls to a persistent Python interpreter tunnelled over SSH, the same idea as execnet, which pytest-xdist uses to run tests on other processes and hosts. I wanted no Python on the hosts at all, so plain shell commands were the better trade for me.
Pros and cons
What I gained:
- Python — not pyinfra's feature, but what it makes possible: typed configuration, validated before the first SSH connection and checked by
mypy --strict, instead ofasserttasks on the host. - Tests without a host, through pyinfra's fake connector — a stand-in for SSH that records the commands instead of sending them.
- A dry run before every deploy that shows what would change.
- No Python on the hosts, and very little per-task overhead.
What it costs:
- A smaller ecosystem. No Galaxy full of roles; pyinfra's
files.rsyncis still alpha, so I wrote my ownsync_tree, plus a thin layer of helpers, capped at 1,500 lines so it doesn't turn into a framework. - Latency, which adds up over the internet.
- More code, and two engines side by side during the migration.
- Tests that check less. Molecule runs real roles in real containers. The pyinfra tests check what would be sent to a host; real behaviour is checked by live deploys and smoke tests.
How the port worked
The architecture has three parts. The typed deploys live in a public repository, ops-deploy, with no host names and no secrets in it. Inventory, encrypted secrets and the registry of which service runs where live in a private repository. The old Ansible roles were in ops-library, also public and now archived.
Both engines read the same inventory file, in Ansible's YAML format. A small Python loader reproduces Ansible's variable precedence and refuses anything it cannot reproduce exactly. During the migration, every service had one line in its metadata, engine: pyinfra, that decided which engine just deploy used. Rolling back meant deleting that line.
It started with a pilot: two of my own open-source projects, nyxmon and fastdeploy, against go/no-go criteria written down beforehand — at least 30 % faster, a second run with no changes, no secret ever written in plaintext, and one gut-feeling question: would I rather write the next service this way? Yes, the same day.
After that, each service went through the same chain: port it, add it to the service registry, run the dry run against the real host, deploy live, deploy live a second time and require that nothing changes, and only then flip the engine line. Behind that ran a safety net: smoke tests on a local VM that deployed with Ansible, then pyinfra, then Ansible again; byte-for-byte comparisons of rendered templates; and hashes of every Ansible role, so a change on the old side fails a test until it is ported to the new side.
Coding agents did the porting, one service at a time, each change reviewed by a second model before it was merged. No agent ran anything against production without my approval. I could never have done all of this alone.
What was easy
The bulk. Small Django apps were nearly mechanical: a dozen of them went live on pyinfra on the second day. The newest services never got an Ansible role at all.
The templates. About 42,000 lines of Jinja came over almost unchanged. The YAML around them was what needed rewriting.
Validation. Here is what the YAML soup looks like in practice — the Ansible version of checking a few of nyxmon's settings:
- name: Validate Nyxmon reliability settings
assert:
that:
- nyxmon_notify_consecutive_failures | int >= 1
- nyxmon_notify_consecutive_failures | int <= 100
# …
- nyxmon_check_batch_size | int >= 1
- nyxmon_check_batch_size | int <= 100
- >-
nyxmon_processing_lease_seconds | int
>= ((nyxmon_check_batch_size | int) + 5) * 60
fail_msg: >-
Nyxmon notification thresholds are out of range. …
A second task before it loops over the same variables just to check that they are whole numbers. The pyinfra version:
Count = Annotated[int, Field(ge=1, le=100)]
class NyxmonConfig(BaseServiceConfig):
notify_consecutive_failures: Count = 2
check_batch_size: Count = 5
processing_lease_seconds: int = Field(default=900, ge=30)
# …
@model_validator(mode="after")
def _lease_covers_a_batch(self) -> Self:
if self.processing_lease_seconds < (self.check_batch_size + 5) * 60:
raise ValueError("processing_lease_seconds must be at least "
"five minutes plus one per claimed check")
return self
Validation that was a task is now a type. Secrets follow the same idea: a config field references a secret, the config is validated before anything connects, and the value travels over stdin into a 0600 file. In Ansible that discipline depended on remembering no_log on every task.
What was hard
The risky services. Mail, DNS, Traefik and storage took the longest and got the most care. A switch stops when the dry run finds something unexpected, and it did: an out-of-band apt dist-upgrade had changed a package's configuration on one host.
Shell differences. The first live deploy of a small Django app failed while running its database migrations. The login shell on that host is fish, and fish turns \\ inside single quotes into \. The smoke-test VM used bash, so the smoke test passed. With pyinfra sending shell commands, the shell on the host suddenly matters.
The old roles had bugs. The pilot alone found four. One of them made every deploy restart Traefik, because a bcrypt hash in its configuration was regenerated on every run.
Agents and credentials. Twice on one day, an agent's "filtered" read of a database table still put a credential into its transcript, and I had to rotate it. The rule since then: never SELECT * on a table that holds credentials.
Tests that ran nowhere. Making the test suites fast (more on that below) made me look at which tests were actually running. About 370 tests that compare pyinfra's rendered output with Ansible's were not part of any standard run. Seventeen test files for the Home Assistant deploy were red and not part of any full run either. And the private repository has no CI; the local just test is the only check. A cheap oracle only helps if you ask it everything.
What actually got better
Tests. This is the number the migration was about.
Serially, the Ansible suite took 2 hours 13 minutes on the Mac Studio. But Molecule scenarios can run in parallel too, and comparing a parallel suite against a serial one would have been cheap. So I parallelised the Ansible side as well: just test with eight jobs, Molecule with four scenarios at once and SSH pipelining. That brought it down to about 34 minutes — Molecule alone is 21 of them — measured while other jobs were loading the machine.
pyinfra's roughly 17,500 tests take about 17 minutes serially and under two and a half minutes on nine workers, measured one suite at a time on an idle M5 Max. Part of that came after the migration: work on the tests themselves (a shared stub cache, a bytecode cache, fixes to slow tests), and retiring the tests that compared pyinfra's output with Ansible's. The two sides ran on different machines, the pyinfra side on a newer one, so don't read the ratio too precisely. The order of magnitude holds: hours against minutes.
Deploys. "About 80 % faster" is what my own pilot had said. It depends mostly on where the server is.
On the local network, it's true: in the pilot, nyxmon and fastdeploy went from 31 and 65 seconds to 7 and 9 (part of fastdeploy's 65 seconds were two Traefik restarts the old role didn't need). Against my real servers on the internet, it's not. For my homepage and python-podcast, which run the same way on the same host, a deploy with nothing to change takes 26 to 30 seconds with pyinfra. The last Ansible deploys of python-podcast I have timings for — single runs from earlier this year — took 52 to 59. About half the time; 80 % faster would be a fifth. A one-line code change sets off the whole chain — dependencies, migrations, static files, search index, restarts — and takes 41 seconds for python-podcast, a fifth to a third less than those Ansible runs, and 65 seconds for the homepage, whose chain is longer.
That's the architecture difference from above, in numbers. In a no-op deploy of the homepage, about ten seconds go into gathering facts, and the 57 operations take about 0.3 seconds each — a handful of round trips at roughly 50 milliseconds — which adds up to another 18.
Safety. The dry run earned its keep even after the migration. While I was measuring deploy times, it showed that deploying from my local checkout would have rolled production's dependencies back, because the code running there was newer than mine. Nothing was deployed.
Not better, not worse: the nightly backups. They are scripts on the hosts; the deploy tool only installs them, and they take the same time as before.
Code size. It did not shrink. Counting lines of code without blanks and comments (the 200,000 at the top includes them), the Ansible core is about 171,000 lines (129,000 YAML, 42,000 Jinja), the pyinfra side about 198,000 (154,000 Python, 44,000 templates): roughly 15 % more. The scope isn't identical — the pyinfra side includes services that never had an Ansible role, helpers a Galaxy collection would have provided, and the machinery that let both engines run side by side — and typed Python is simply longer than a YAML list. The pyinfra side also has about 199,000 lines of tests, as much test code as deploy code.
The last stretch
Switching every deploy was not the same as getting rid of Ansible. With every service on pyinfra, Ansible still ran in three places: Home Assistant rendered its configuration through Ansible's templating engine, deploy-all picked its hosts through Ansible's inventory manager, and 18 pyinfra modules read variables out of playbook files. The agents took care of all three. Home Assistant took far less than the three to five days I had estimated, because a complete native path already existed in the pyinfra code; its rendered files are byte-identical to the recorded outputs. The 26 remaining operator playbooks — a Traefik transaction, Vaultwarden controls, SSH keys, macOS agents, disk maintenance and the like — were ported too.
Then the public deploy code dropped Ansible entirely: its integration tests no longer start it, and the tests that compared pyinfra's output with Ansible's were replaced by golden files — recorded copies of what each deploy writes to the host.
The last piece was MinIO, which I retired rather than ported. Its buckets had already moved to RustFS and nothing used it any more, so I switched it off and deleted its data. Its seven playbooks went with it, along with the tooling that only existed for them: a syntax test, a collections install, Ansible in CI. The status page that counts Ansible dependencies went from 88 to zero, and ops-library is archived.
Ported and tested isn't the same as verified live, though. Read-only checks against the real hosts ran for some of the new ports — Vaultwarden, the Traefik status, the container CA, root SSH keys and the macOS agents — and came out as expected. The rest are still waiting for my approval.
The long tail wasn't services; it was plumbing.
What I learned
Test speed was the goal; deploy speed was a bonus. Shaving 10 to 30 seconds off a deploy is nice. Turning a two-hour test suite into minutes changes what you are willing to try, and what an agent can do with the codebase.
The safety net mattered more than the speed. A mandatory second live run, a dry run before every deploy, rollback by deleting one line while both engines existed, and no live run without my approval.
Porting is a code review of the old code. And parallelising the tests was a code review of the tests.
Compare like with like. Parallelising the Ansible suite took it from over two hours to 34 minutes. Comparing pyinfra against the serial number would have flattered it about fourfold.
The agents moved every deploy to pyinfra, and then everything that was still hiding behind it. Ansible is gone. The inventory is still YAML, so not entirely out of YAML — but nothing reads it with Ansible any more.