Why We Didn't Build a Database Fallback
Every service that stores something eventually has the same conversation. Uptime matters, the database is the scary single point of failure, so: add a second one. Write to both. If the first goes down, fall back to the second.
We had that conversation about ResetKit recently and decided against it. Here's the reasoning, because the decision is less obvious than it sounds and the failure mode it introduces is worse than the one it prevents.
The appeal
The pitch is clean. You have one database. If it disappears, you are down. Two databases means one can disappear and you keep serving. It maps onto how we think about stateless services, where running a second copy really does just make you more available.
The problem is that a database is not a stateless service. The hard part was never "run another copy." The hard part is what happens to writes during the window where the two copies disagree, and how they reconcile afterward.
Where it breaks for auth specifically
Most of what an auth service does is write. Create a reset session. Consume a token. Record that a code was used. Almost none of the interesting operations are reads you could serve from a stale replica.
And several of those writes are not just data. They are guarantees.
A password reset token is single use. That is enforced by a flag on the record: once consumed, it cannot be consumed again. A payment record is idempotent because a uniqueness constraint refuses a second row with the same transaction identifier. Those constraints are the security property. They are the difference between a reset link and a reusable key to someone's account.
Now split that across two stores with asynchronous replication between them.
A token gets consumed on the primary. Replication lags, as replication does. During that window the secondary still believes the token is unused. If anything reads from the secondary, or if you fail over mid-window, that token is live again. You have not built redundancy. You have built a replay window, on the exact code path where a replay is an account takeover.
The uniqueness constraint fails in the same shape. Unique in store A and unique in store B does not add up to unique across the system. Two writes racing during a failover can land one in each, and both look perfectly legal from where they sit.
This is why the dual-write version of this idea is so much harder than it first appears, and why teams who attempt it often stall partway. It is not a stamina problem. Keeping two stores in agreement under partial failure is a genuinely hard distributed systems problem, and the usual resolution strategies ("last write wins," "reconcile later") are exactly the strategies that a single-use token cannot tolerate.
What most people already have
Here is the part that tends to get missed. If you are on a managed database at any serious tier, you are almost certainly already running a replica set: multiple nodes, with automatic leader election when one fails.
More importantly, modern drivers retry. When the primary goes away, the driver notices, waits for the new leader, and retries the in-flight operation against it. From the application's point of view a failover is a slightly slow request, not an error.
That is automatic database failover. It is already there. It is invisible precisely because it works, which is why people reach for a second database without realizing they are trying to build something they are already paying for.
The single node you are imagining, the one whose death takes you offline, usually does not exist.
What we did instead
Three things, none of which involve a second database.
We made the existing failover explicit. The retry options that make failover transparent are now set deliberately in code, with a comment explaining that they are the failover mechanism and not a tuning knob. A future engineer reading that file should not have to rediscover this.
We made failure fast instead of slow. Our socket timeout was long enough that a stuck operation would hold a request for most of a minute. Every query in this service should complete in well under a tenth of a second, so a socket still open after a few seconds is not slow, it is stuck. Cutting that timeout means a bad moment surfaces as a quick, clean error instead of a pile of hung connections. Counterintuitively, the resilience work made the service faster under stress rather than slower.
We invested in knowing, not failing over. The outages we have actually had were not database failures. They were an expired billing state that paused deployments, and a missing environment variable that took the process down at boot. A second database would have helped with neither. What would have helped is finding out immediately instead of by checking. So the effort went into health checks that report real dependency status on demand, and alerting on the conditions that actually bit us.
The general version
Reach for the failure you have had, not the one that sounds scariest.
"The database might die" is vivid and it is the one people design around. But if you look at an actual incident log, most outages are configuration, credentials, billing, deploys, and dependencies that failed in ways nobody modeled. Those are not solved by redundancy. They are solved by fast detection and boring, well-understood recovery.
And when you are considering redundancy for a stateful system, ask what breaks when the copies disagree, because they will. If the answer is "some data is briefly stale," that is often fine. If the answer is "a single-use credential becomes reusable," you have not added reliability. You have added a vulnerability with a reliability story attached.