The previous article in this series introduced the four architectural cache patterns and showed how each one addresses a different problem within the consistency and availability spectrum the CAP theorem describes. Cache-aside protects the database on reads. Write-through maintains coherence on writes. Write-behind absorbs write spikes at the cost of accepting loss risk. Each pattern has its trade-offs, and the choice between them depends on the data type and the system's requirements.
But there is something all patterns have in common that rarely surfaces in the initial technical discussion. Each of them adds complexity to the system, and complexity has a cost. That cost is the subject of this article.
When a team decides to add cache, the focus is almost always on the performance gain. The database is slow. Redis will fix it. Latency numbers improve in the next deploy. The story ends in success. What that narrative omits is what happens afterward, when the system has been in production for six months, the team that made the decision has grown, new developers joined without the original context, the data chosen for caching changed in nature, and the first consistency incident comes knocking.
The financial cost that shows up on the bill
Starting with the financial cost is useful because it is the most concrete and the easiest to ignore at decision time. When a team evaluates whether to add Redis, the natural thought is to compare the cost of Redis against the cost of a larger database. That is the wrong calculation.
The cost of Redis in production is not the cost of a single instance. It is the cost of a system operated at the minimum reliability standards that production demands. An isolated Redis instance that fails leaves the system without cache. In some patterns, as seen in the previous article, that means all requests go directly to the database, which may not handle the load. In others, such as write-behind, it means data not yet persisted is lost.
For this reason, cache in production means Redis in a high-availability configuration. That typically involves a primary node and at least one replica node, with Redis Sentinel monitoring cluster health and automatically promoting the replica if the primary fails. In cloud environments, this configuration has a specific cost. A 13 GB ElastiCache for Redis instance on AWS with Multi-AZ mode enabled, the reasonable minimum for a production service with high availability, cost around $340 per month in the us-east-1 region as of December 2025, according to the AWS pricing calculator. For 26 GB, the cost rose to approximately $660 per month. Exact figures change over time, but the order of magnitude tends to be stable: managed infrastructure with high availability operates in the hundreds of dollars per month, not tens nor thousands, given the fixed costs of replication, failover, and operation. That is before accounting for data transfer, snapshots, and the cost of larger instances as volume grows.
These numbers are not prohibitive for most companies, but they need to be accounted for and justified. When the FinOps team reviews the infrastructure bill and finds a Redis line that did not exist the previous quarter, the questions will come: what is this solving? What is the ROI? When was it added and who approved it? In companies with more mature cloud cost management processes, any new infrastructure resource should have a documented cost-benefit justification from the start.
The operational cost that shows up later
Beyond the direct financial cost, there is the operational cost of keeping Redis running. Redis is a stable system, but like any infrastructure component, it requires continuous attention.
Memory is Redis's most critical resource. Unlike a relational database, which can use disk when memory fills up, Redis is a purely in-memory system by default. When the memory configured for Redis is exhausted, Redis must decide between rejecting new writes or removing existing data to make room. This policy is configurable through the maxmemory-policy parameter. The most common options are allkeys-lru, which removes the least recently used data among all existing data, and volatile-lru, which removes only data with a TTL set, using the same policy.
Neither option is neutral. If Redis starts removing data due to memory pressure, the hit ratio drops. If the hit ratio drops, more requests reach the database. If the database was not sized to absorb that additional volume, latency rises. The system starts degrading for a reason that rarely surfaces on dashboards directly, because the process is gradual and comes from the interaction between two components.
To prevent this, it is necessary to actively monitor Redis memory usage, the eviction rate, and the hit ratio. When Redis starts evicting data frequently, it signals that either memory is insufficient for the current data volume, or the TTL is too long and data is accumulating beyond what it should. Both cases require intervention, and any intervention has an engineering cost.
Versions and upgrades are another vector of operational cost. Redis releases new versions periodically, with behavioral changes that can affect applications. Migrating from one version to another in production, especially with critical data in cache, requires planning, testing, and a maintenance window or a zero-downtime migration strategy. In systems using less common Redis commands, semantic changes between versions can cause silent bugs that only surface under specific load conditions.
The cognitive cost that shows up when someone needs to understand the system
The least discussed cost, and possibly the most lasting, is the cognitive cost. Every component added to a system increases the number of possible states the system can assume. A system without cache has one possible state: what is in the database. A system with cache has at least three relevant states for any given piece of data: the data is correct in the database and correct in the cache, the data was updated in the database but the cache still holds the old version, or the data is not in the cache and the next request goes to the database.
This increase in states makes the system harder to reason about, especially for people who joined the team after cache was added. When a bug appears and the data a user sees differs from what is in the database, the first question is always "is this a cache problem?" Finding the answer requires adequate instrumentation, access to logs from both systems, and knowledge of the TTLs and invalidation patterns in use. In systems with multiple cache layers — in-memory local cache in the application, distributed Redis, and CDN for static content — the diagnosis becomes even more complex.
Phil Karlton, an engineer at Netscape, captured this cognitive cost in what is probably the most cited phrase in software engineering:
"There are only two hard things in Computer Science: cache invalidation and naming things."
— Phil Karlton, as cited by Martin Fowler in TwoHardThings, martinfowler.com
The phrase is ironic in form but accurate in substance. Cache invalidation is hard not because it demands complex algorithms. The problem is one of coordination. How to guarantee that when a piece of data changes in one place, every copy of that data in every other place is updated or removed correctly, completely, and at the right time.
Why invalidation is the hardest problem
To understand why invalidation is hard, it helps to understand what happens when it fails. When the cache holds stale data, the system serves an incorrect version of reality to users. In e-commerce systems, this can mean showing a price that has already been updated, creating expectations the system cannot honor. In inventory systems, it can mean allowing a user to add to their cart a product that is already out of stock. In financial systems, it can mean showing a balance different from the actual one, with legal and reputational consequences.
The technical difficulty of invalidation comes from a constitutive asymmetry. Writing a piece of data to a database and to a cache at the same time is simple. Guaranteeing that when that data changes in the database, every cache instance holding a copy of it is notified and updated, even in the presence of network failures, process failures, node restarts, and race conditions between concurrent operations, is of a completely different order of difficulty.
In the post Cache Made Consistent, published on Meta's engineering blog, the team describes the work they carried out to improve cache consistency in the TAO and Memcache systems. The scale of the problem can be calibrated by a specific number the post presents.
"Over the years, we have improved TAO cache consistency by one measure, from 99.9999 percent (six nines) to 99.99999999 percent (ten nines)."
— Meta Engineering Blog, Cache Made Consistent, engineering.fb.com
To put that number in concrete terms, TAO serves more than one quadrillion requests per day. At six nines of consistency, fewer than one in a million writes would result in inconsistency. At TAO's volume, that still means billions of inconsistencies per day in absolute terms. Reaching ten nines, fewer than one in ten billion writes, required years of dedicated engineering, the development of a purpose-built observability system called Polaris, and a cache mutation tracking library embedded in every cache server in the company. Meta had an entire team dedicated exclusively to this problem.
This does not mean every system needs that level of sophistication. It means that cache at scale exposes a problem with non-intuitive engineering depth, and that underestimating that depth leads to systems with inconsistencies that surface intermittently and are difficult to diagnose.
Cache stampede as a specific failure mode
One of the most documented failure modes in systems with cache is the cache stampede, also known as the thundering herd problem. Understanding how it happens helps explain why cache can be the cause of an incident rather than the solution.
The stampede occurs when a popular cache key expires and multiple requests arrive simultaneously before any of them can rebuild the cache entry. Each request detects a cache miss. Each request goes to the database to fetch the data. Each request stores the result in cache. The database receives a volume of identical requests equal to the number of clients that arrived in the interval between the key's expiration and the first successful rebuild.
sequenceDiagram
participant R1 as Request 1
participant R2 as Request 2
participant R3 as Request 3..N
participant C as Cache
participant DB as Database
Note over C: Popular key expires (TTL = 0)
R1->>C: GET product:popular
R2->>C: GET product:popular
R3->>C: GET product:popular
C-->>R1: nil (miss)
C-->>R2: nil (miss)
C-->>R3: nil (miss)
R1->>DB: SELECT ... (expensive query)
R2->>DB: SELECT ... (expensive query)
R3->>DB: SELECT ... (expensive query)
Note over DB: Overloaded by N simultaneous queriesFor a key receiving 10,000 requests per second, a TTL expiration can mean 10,000 queries hitting the database in the same second. If that query takes 200 milliseconds to answer, the database receives 2,000 simultaneous queries instead of the few dozen it normally handled for that data. The database enters resource contention. Latency rises. The system degrades at exactly the point where cache was supposed to be protecting it.
Facebook described the problem in detail in the Memcache paper presented at USENIX NSDI in 2013. The solution the team developed was a mechanism called lease: a token distributed by the cache layer to control which client had permission to rebuild an expired key while others waited or received a slightly stale value. At Facebook's scale, a single expired popular key generated tens of thousands of simultaneous requests to the database, making control over who rebuilds the cache as important as the cache itself.
"We grant a lease to a client to set data back into the cache when that client experiences a cache miss. [...] With the lease, a client can detect that the value it is about to set has been invalidated by a subsequent write."
— Rajesh Nishtala et al., Scaling Memcache at Facebook, USENIX NSDI, 2013
The scenario described so far, a single popular key expiring and attracting a sudden volume of requests, is the most direct form of the problem. There is a more severe scenario that appears in systems with multiple types of cached data. Even with distinct TTLs across different key types, creation cycles and update rhythms can cause multiple expirations to concentrate within a very short time window. When that happens, each expiring data type generates its own group of simultaneous requests to the database. The effect is multiplicative. The database, which previously absorbed the impact of a single amplified key, now absorbs the impact of multiple key types amplified at the same time.
This pattern manifests most intensely during deploys that flush the cache and force a complete repopulation, during Redis failures followed by recovery, and in systems with periodic processing cycles that write to cache in bursts. In each of these contexts, a large volume of entries is created in a short interval, increasing the probability that expirations of different types overlap in subsequent cycles. The problem, which seemed to be about a popular key, turns out to be about the rhythm at which the cache was built.
Four approaches dominate the literature on the problem. Single flight ensures that when multiple requests arrive simultaneously for the same missing key, only one of them goes to the database while the others suspend and wait for the first request's result. When that request returns, all waiting callers receive the same value without having generated additional queries. The most cited implementation is the singleflight package from Go's extended standard library. The advantage over a distributed lock is that single flight operates entirely within the process, requiring no external component and carrying no liveness problem if the process dies during the rebuild. The limitation is symmetric. It only protects within a single instance. In a service running ten horizontal instances, each instance individually will not query the database more than once per missing key, but the database can still receive ten simultaneous queries, one per instance. The lease mechanism described by Facebook in the Memcache paper is, in essence, a distributed version of this same pattern. The coordination that single flight performs in memory within a process, the lease performs via tokens across distinct processes. The distributed lock ensures that only one request rebuilds the cache while others wait or receive a stale value, but shifts the problem to another dimension. If the process that acquired the lock dies before completing the rebuild, the system needs an external liveness mechanism to release it and allow another request to take over. This adds an operational component to the system that the lock itself does not resolve. TTL jitter distributes expirations over time by adding randomness to the TTL value at creation, diluting the spike rather than eliminating it. Effectiveness depends on the randomness interval being proportional to key volume — an interval calibrated for a given volume may be insufficient if the system grows. Probabilistic early expiration, described by Andrea Vattani, Flavio Chierichetti, and Kedar Dhamdhere at the VLDB Conference in 2015, recomputes the data before it expires with a probability that grows as the TTL approaches zero, preventing the cache from reaching a miss. The implicit condition is that the recomputation cost be predictable and low. For data with expensive computation, this approach can trade a periodic stampede for continuous background degradation. The pattern across all four cases is that the complexity does not disappear — it shifts to a different dimension of the system.
The memory cost and what happens when it runs out
Redis keeps all data in memory. That is what makes it fast, and it is also what makes capacity planning non-trivial.
In relational databases, when data grows beyond available memory, the database starts using disk for less-accessed pages. Performance degrades, but the system continues functioning. In Redis, when data exceeds the configured memory limit, Redis executes the defined eviction policy. If the policy is noeviction, new writes are rejected with an error. If the policy is allkeys-lru, data the system still considers valid and useful may be removed to make room for more recent data.
The problem is that memory pressure in Redis is rarely predictable with precision. Data has variable sizes. Different TTLs mean different amounts of data expire at different moments. Traffic spikes cause more data to be written to cache in less time. Memory fragmentation internal to Redis, a phenomenon where the memory allocated by the system for Redis is larger than what Redis reports using, can effectively reduce available capacity by 10 to 30 percent in workloads with many writes and deletions, as described in the official Redis documentation on memory optimization.
Correctly sizing Redis memory requires knowledge of the access pattern, the average size of stored data, the expected growth rate, and a safety margin that accounts for fragmentation and spikes. This exercise requires data that does not exist before the system is in production, creating a situation where the first weeks of a newly deployed Redis instance demand active monitoring and possible configuration adjustments.
Accidental complexity and the cost of debugging
The distinction between essential complexity in a system, the kind that comes from the problem it solves, and accidental complexity, the kind that comes from implementation choices, is the right lens for understanding what cache adds. The complexity that cache introduces — invalidation, stampede, memory pressure — serves a performance problem that arises from implementation choices, not the business problem the system exists to solve. It is accidental in the technical sense of the term, since it exists because we chose this solution, not because the problem demands it.
This accidental complexity manifests in specific ways in day-to-day development. When a developer fixes a business logic bug and deploys the fix, the fix may not appear immediately for all users if the affected data is still in cache with the old logic. This creates a class of bugs that only occur in production, that disappear after a few minutes or after a manual cache flush, and that are difficult to reproduce in development environments where cache is often not configured the same way as production.
In automated test systems, cache must be managed between tests to ensure that one test does not see data left by a previous one. This requires tests to clean the cache before or after running, adding complexity to the test environment configuration and potentially making tests slower if flushing the cache is an expensive operation.
In deploys, when a data format change is made — for example, a field is renamed or a new required field is added to an object stored in cache — the cache may contain objects in the old format that the new version of the code cannot deserialize correctly. This type of problem can cause errors in production immediately after a deploy, even if tests passed, because the tests did not exercise behavior with old-format data already in cache.
This set of problems requires the team to think about cache not only at the moment of addition, but as a permanent dimension of feature development. Any data model change must consider what happens to data already in cache. Any new query that may be cached must consider the invalidation patterns. Any bug fix must consider whether cache could be masking or propagating the bug.
The accumulated cost and the conscious decision
All these costs — financial, operational, and cognitive — exist in varying degrees depending on the system and how cache was implemented. A well-designed cache, with clear invalidation strategies, adequate monitoring, and accessible documentation, has manageable costs. A cache added hastily in response to an incident, without a defined invalidation strategy, without monitoring, and without documentation, tends to accumulate technical debt that surfaces unexpectedly months later.
The decision to add cache is sound when five questions can be answered before implementation. Is the bottleneck measurably in the data layer? Cache does not solve latency from business logic, serialization, or network, and adding it to a system where the bottleneck lies in another layer only masks the diagnosis. Does the read-write ratio of the data justify the chosen architectural pattern? Data with a high write rate makes any invalidation strategy expensive, and arriving at that number requires measurement, not assumption. Can the acceptable staleness window be defined before choosing the TTL? Without that number, TTL becomes a guess, and for data that affects financial or inventory decisions, answering this question often ends the discussion before it begins. Can the invalidation rules be stated explicitly by the team? If it is not possible to describe in a way that any team member reaches the same answer when a cache entry should be removed, the system will accumulate the intermittent inconsistencies described in this chapter. Is there prior agreement on what constitutes an unsatisfactory hit ratio in numbers, and on who acts when that threshold is reached? Without that agreement, the eviction rate can climb for weeks before anyone recognizes the symptom.
If any of these questions lacks an answer before implementation, the costs described in this chapter materialize before the benefit that motivated the decision.
Treating cache as an architectural decision with lasting consequences, rather than a tactical optimization that can be added and forgotten, changes what the team monitors, documents, and reviews in every development cycle. Redis is a solid tool. The costs described here come from what happens when the technology is added without understanding what is being operated.
The invalidation problem, which structures all other costs described here, is the subject of the next article. Fixed and dynamic TTL, active versus passive invalidation, key versioning, stale-while-revalidate, and the HTTP caching RFCs that attempted to standardize how this problem should be handled at the protocol layer. What Meta spent years solving in TAO and what the web protocol community tried to solve in the RFCs are two sides of the same problem, and Phil Karlton's phrase will make more sense once we understand why invalidation is hard from a technical standpoint, not only from a cognitive one.
