dotnet · performance  ·  04 Aug 2026  ·  5 min

The cache stampede you haven't met yet

Khaled Gadhami
A hot key expires: IMemoryCache runs 50 identical DB queries; HybridCache runs one, 49 share the flight
COMPANION REPO · LIVE FROM GITHUB

Here is a caching setup that passes every code review:

var value = await cache.GetOrCreateAsync("report", async entry =>
{
    entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
    return await LoadReportAsync();   // ~1 s against the database
});

It works in dev. It works in staging. It works in production right up to the moment the key expires during a busy minute — and then fifty concurrent requests all find the cache empty, and all fifty run the factory. Your database absorbs fifty identical one-second queries to produce one value. That's a cache stampede, and IMemoryCache does nothing to stop it: GetOrCreateAsync never promised to run the factory once, only to cache whichever result finishes last.

Most teams meet this for the first time in an incident review. The cache was "working". The database fell over anyway.

THE SAME 10 REQUESTS · ONE EXPIRED KEY
IMemoryCache.GetOrCreateAsync
db calls0
10 identical one-second queries, for one value
HybridCache.GetOrCreateAsync
single flight
db calls0
1 query — the other 9 share the flight

What HybridCache changes

HybridCache's abstract type ships in the ASP.NET Core shared framework (Microsoft.Extensions.Caching.Abstractions), but the concrete implementation and the AddHybridCache() registration still come from their own NuGet package, Microsoft.Extensions.Caching.Hybrid — stable since 9.3, after a run of .NET 9 previews. Skip the package and AddHybridCache() won't compile, even on .NET 10 (CS1061). Install it, and the rest is the same mental model, one registration:

builder.Services.AddHybridCache();
var value = await cache.GetOrCreateAsync("report", async token =>
    await LoadReportAsync(token), cancellationToken: ct);

The difference is in the guarantee: concurrent callers asking for the same cold key share one flight. One factory execution runs; the other forty-nine callers wait for its result instead of dogpiling the database. You can watch this happen in the demo repo — both endpoints count their database calls, and a parallel burst against the classic endpoint racks up the counter while the hybrid one stays at 1.

Two more things you get without extra code, because they're the library's reason to exist:

Two caveats worth having up front. Single-flight coordination is per instance: a cold-key burst that lands on two different app instances at once still runs the factory twice, once on each. L2 only shares the winning result afterward; it doesn't coordinate the race itself. And on the default path, every entry round-trips HybridCache's own serializer whether you register an L2 or not — it handles string and byte[] natively and falls back to System.Text.Json for everything else. Entries over the 1 MB default MaximumPayloadBytes are logged as an error and never written to L2; L1 still keeps them, so callers on the same instance never notice, and that error log is the only signal you get locally. Across instances it shows up as the per-instance gap above, permanently, for that one key: nothing ever reaches L2 to share.

Where the old pair still bites

If you're on the classic IMemoryCache + Redis combination and it works, this isn't a rewrite order. It's a checklist item: find the keys that are expensive to rebuild AND requested concurrently — report endpoints, dashboard aggregates, reference data at 9 am. Those are the stampede candidates, and they're the ones worth moving first. A key that's cheap to rebuild or rarely contended gains little.

And one honest caveat: none of this makes your factory idempotent for free. Single-flight protects you within the guarantee the library gives, not across instances, and it's not a distributed lock. Design the factory to be safe to re-run, same as before.

The demo is two endpoints and a counter. Run it, fire a burst of parallel requests at each, and read the numbers.