Most database changes that cause outages do so because the schema and the application changed together. For a moment during deployment, old code is running against a new schema, or new code against an old one, and one of those combinations does not work. The technique that removes this entire class of failure is to make every change backwards compatible and to deploy it in stages, so that at no point is any running version of the code incompatible with the schema it sees.
The pattern is expand, migrate, contract. To rename a column: add the new one, deploy code that writes to both and reads from the old, backfill the new one, deploy code that reads from the new, then - a release or two later, once you are confident - drop the old. Four deployments to rename a column, which sounds absurd until you compare it with the maintenance window and the rollback that has no path back.
Adding a column is safe if it is nullable or has a default, on any modern Postgres. Adding a NOT NULL column without a default is not, because it rewrites the table and holds a lock while doing so. Adding an index is safe only if built concurrently; the ordinary form blocks writes for the duration, which on a large table means an outage nobody planned. These specifics vary by engine and version, and it is worth knowing them for the one you run rather than trusting a general rule.
Backfills need to run in batches with a pause between them, not as a single statement. One UPDATE across ten million rows holds locks, bloats the write-ahead log, and can take the database down through replication lag even though the statement itself is valid. Batches of a few thousand with a short sleep are slower in elapsed time and invisible to users, which is the trade you want.
The step teams skip is the contract phase. Dropping the old column, the old table, the compatibility code. It never feels urgent, so it accumulates, and eighteen months later the schema is full of columns nobody can explain and every developer writes to both because they are not sure which one is real. Put the cleanup in the backlog with a date attached at the moment you write the expand step, or it will not happen.
Test the whole sequence against a copy of production data, not a small development database. The behaviours that hurt - lock duration, replication lag, backfill time - are all proportional to size, and a migration that takes two seconds on a thousand rows can take forty minutes on forty million. A restored production snapshot is the only honest rehearsal, and it is the cheapest insurance available on this kind of change.
