Undo a bad deploy, data included: Rewind and Replay

An agent can write a pull request in four minutes. Reading it properly takes me forty. Multiply that by a day of agent work and the maths stops working: either the agent waits for me, or I start approving diffs I've skimmed.

Most people I talk to are doing the second one and feeling slightly sick about it. I don't think the answer is to slow the agent down. I think the answer is to make shipping safe even when nobody read every line, the same way a guardrail doesn't make you drive slower, it makes a fast bend survivable.

So we built two things, and they're live on velixir today.

Why isn't a rollback enough?

Every host has a rollback button, and it works right up until the bad release touched your data. A migration that dropped a column. A background job that rewrote ten thousand rows with the wrong currency. A cache now full of objects in the new shape. Roll the code back and the old code starts reading data it doesn't understand.

What you actually want is the whole app as it was at 09:41: the code that was live, the env vars it ran with, and the database as it was that second. That's Rewind.

How do I roll back the database with the deploy?

Open your app's Rewind tab and drag back to a moment. Every part of the app gets its own switch, and next to each one you see exactly what it would come back as:

Switch off whatever you want to keep as it is. Rewinding the code and config but not the database is a perfectly sensible thing to want.

Then press Preview this rewind. velixir builds a copy of the app as it was, with its own copies of the data, on its own address. Nothing on the live app changes while you look around. It takes a few minutes, mostly while Postgres replays its history up to the second you picked.

When you're happy, Apply. The live app is pointed at the copies and the release from then is deployed. Your original databases aren't overwritten: they keep running, with everything written since 09:41 still in them, so an applied rewind can be undone with one button, and any orders that came in after the moment are there to copy across.

How far back you can go depends on what you're rewinding. Code and config go back 28 days. A database goes back as far as its plan keeps backups: 7 days on Starter, 14 on Standard, 28 on Pro and Scale.

How do I test a release against real traffic?

Rewind is for when something got through. Replay is for catching it first.

Pick a release that isn't live yet, say the one your agent just pushed to a staging slot, and press Replay. velixir makes two copies of your app from the same moment, one running what's live and one running the new release, each with its own copy of the data. Then it sends both your most requested pages from the last day, plus a sample of the rarer ones, and compares every answer.

The live copy answers everything twice. Anything that changes between those two answers on its own, a timestamp or a request id, is ignored when it compares against the new release. What's left is real: a page that now returns a 500, a JSON field that changed, a route whose 95th percentile got 50% slower. You get a verdict of clean, review or regressions, with the requests that changed.

From a terminal or a CI job:

velixir deploy --slot staging
velixir replay v43 --wait

--wait exits with code 1 on regressions, so the pipeline stops before promoting.

Can my AI agent use it?

That's the point. The velixir MCP server has replay_release and replay_status, so an agent can test its own release, read the report and fix what broke before you ever see the diff. It also has rewind_plan and rewind, and the tools tell the agent to ask you before applying anything to the live app.

You review the verdict and the handful of requests that changed, not four thousand lines.

Is it safe to copy production like this?

The copies have your env vars, live API keys included, which is exactly why they can't reach the internet. A preview, a replay copy or a fork reaches its own data and nothing else, so a background job inside one can't charge a card or email a customer. Replayed requests are only GET and HEAD, sent without cookies or sign-in headers, and they never touch your live data.

What it won't do

Rewind restores your app, not the world around it. An email that went out, a payment that was taken or a webhook another service received stays done.

Replay only replays reads, so a bug that only shows up when someone submits a form needs a test of its own. Pages behind a login answer as they would to a signed-out visitor. And it can only read as far back as your access logs go: a day on the Free plan, a week on paid plans.

The copies are billed by the hour like any app while they run, which for a replay is usually under an hour. Replay itself costs nothing extra. Because a copy is an app on your plan, a Free app needs to move to a paid plan before you can rewind or replay it.

Where do I start?

If you already run an app on velixir, the Rewind tab is there now. The Rewind + Replay page has an interactive demo, and the docs cover Rewind, Replay and forks. If you don't, deploy something and let your agent loose on it. That's what it's for.