Varnish is Caching Pages and ignoring "Cache-control: no-cache" header

TL;DR; - Before V4.0.0 Varnish Cache ignored the “Cache-control: no-cache” header by default, and you’ll need to handle this yourself in your VCL.

I’ve been around this HTTP thing as a developer and ops guy for a while. Most of the time, things tend to work how I expect.

Back in Jan 1997, the specification for HTTP/1.1 was laid out and included this little gem:

The following Cache-Control response directives allow an origin server to override the default cachability of a response: …

no-cache Indicates that all or part of the response message MUST NOT be cached anywhere. This allows an origin server to prevent caching even by caches that have been configured to return stale responses to client requests.

https://www.ietf.org/rfc/rfc2068.txt

Well behaving web sites will implement this for pages that should never be kept by any cache (browser cache, corporate/ISP proxy or reverse proxy/CDN cache).

Varnish has the non-intuitive and unsafe default of not observing this header. By default Varnish will most probably act unexpectedly on a well behaved site.

You’ll need to handle it yourself with something like:

sub vcl_fetch {
    if (req.http.Cache-Control ~ "(private|no-cache|no-store)" || req.http.Pragma == "no-cache") { return (pass); }
}

The reason? Opinionated creators.

Varnish Cache is not a cache in the RFC2616 sense. It more of an extension of the web-server, which is probably best thought of as “a webserver that picks up its contents with HTTP”.

Therefore, a lot of what RFC2616 has to say about caches do not apply to Varnish Cache.

The key distinction is that a RFC2616 cache is not under the content providers control, so it must follow whatever intructions he gives in the HTTP headers, whereas Varnish Cache is under his control and he therefore has other means of instruction (notably VCL).

Ref: https://www.varnish-cache.org/lists/pipermail/varnish-misc/2007-November/001173.html

For me, it’s a little like saying, bus drivers aren’t in control of buses they must drive, so buses should act safely and have the brake pedal control the brakes. But a car you personally possess and run, that’s yours - so why connect the brake pedal to the brakes by default? You can do that yourself with only 1 hose…

I’m not going to rage about this artifact. Opinionated creators give the direction that form the products which could not exist as the output of some watered-down “none of us is as stupid as all of us” by committee approach.

Luckily, this was revisited in v4.0.0 - https://github.com/varnishcache/varnish-cache/commit/81006eafd6d4cd6f9481740a1d172e316a184d05. So be aware of which version you’re working in.

For those who encounter it, I think the situation is best summed up by timbunce’s comment:

For the record, everyone I’ve mentioned this to has responded with statements of surprise (ranging from “Wow” to “Oh shit!").

https://www.varnish-cache.org/trac/ticket/1124#comment:1 timbunce

UPDATED: A big thanks here to @denisb for pointing out that we hadn’t mentioned affected versions or that it had been fixed for some time now in v4.