[Varnish] Beware of Vary header

Vary is one of the most powerful HTTP response headers. Used correctly it’s possible to do wonderful things with it. Unfortunately most people used it so badly, that today most CDN don’t use it anymore and simply avoid caching if it used for anything else than HTTP compression.

Using it for HTTP compression

Most people use the Vary header exclusively with the Accept-Encoding value in order to cache several version of the same URI depending on the Accept-Encoding field value. Usual values for Accept-Encoding are gzip, deflate and more rarely sdch, but many more exist. Also keep in mind that any combination of these individual values is valid !

To keep the number of cache version at minimum, you can use the following VCL snippet to normalize the Accept-Encoding header, only keeping gzip and deflate as acceptable values:

if (req.http.Accept-Encoding) {
   if (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
   } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
   } else {
      remove req.http.Accept-Encoding;
   }
}

Using it with other field

It possible to specify any other field into the Vary value and also multiple fields by separating their names with a comma. But i don’t recommend this setup because of how CDN treat it.

Anyways, if you want to use Vary correctly you must always keep in mind these two cardinal rules: never use the raw values of the indicated field and never choose a field with too much variant.

For example the following fields should NEVER be used into a Vary value:

  • Referer: because that utterly stupid
  • Cookie: because that probably the most unique request header ever
  • User-Agent: because with more then 8000 usual variants, you pretty to sure to never use your cache anymore