Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine
This tutorial has a new version, check it out!

Versioning to Bust Cache

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $6.00

We've got a real nice setup here! With gulp running, if we update any Sass files, this main.css gets regenerated. But there's just one problem: when we deploy an updated main.css, how can we bust browser cache so our visitors see the new stuff?

To solve this easily, we could go into the template, add a ?v= to the end, then manually update this on each deploy. Of course, I'll definitely forget to do this, so let's find a better way with Gulp.

Introducing gulp-rev

Search for the plugin gulp-rev, as in "revision". Open up its docs. This plugin does one thing: you point it at a file - like unicorn.css - and it changes the name, adding a hash on the end. That hash is based on the contents, so it'll change whenever the file changes.

So let's think about this: if we can somehow make our template automatically point to whatever the latest hashed filename is, we've got instant cache-busting. Every time we deploy with an update, your CSS file will have a new name.

I want to do that, so copy the install line and get that downloading:

npm install --save-dev gulp-rev

Now, head to gulpfile.js. Remember, we're using gulp-load-plugins, so we don't need the require line. In addStyle, add the new pipe() right before the sourcemaps are dumped so that both the CSS file and its map are renamed. Inside, use plugins.rev():

73 lines gulpfile.js
... lines 1 - 13
gulp.src(paths)
... lines 15 - 19
.pipe(plugins.rev())
... lines 21 - 22
};
... lines 24 - 73

Ok, let's see what this does. I'll clean out web/css:

rm -rf web/css/*

Now run gulp:

gulp

Go check out that directory. Instead of main.css and dinosaur.css, we have main-50d83f6c.css and dinosaur-32046959.css And the maps also got renamed - so our browser will still find them.

But you probably also see the problem: the site is broken! We're still including the old main.css file in our layout.

Dumping the rev-manifest.json File

We can't just update base.html.twig to use the new hashed name because it would re-break every time we changed the file. What we need is a map that says: "Hey, main.css is actually called main-50d83f6c.css." If we had that, we could use it inside our PHP code to rewrite the main.css in the base template to hashed version automatically. When the hashed name updates, the map would update, and so would our code.

And of course, the gulp-rev people thought of this! They call that map a "manifest". To get gulp-rev to create that for us, we need to ask it really nicely. At the end, add another pipe() to plugins.rev.manifest() and tell that where we want the manifest. Let's put it next to our assets at app/Resources/assets/rev-manifest.json:

76 lines gulpfile.js
... lines 1 - 12
app.addStyle = function(paths, outputFilename) {
gulp.src(paths)
... lines 15 - 22
// write the rev-manifest.json file for gulp-rev
.pipe(plugins.rev.manifest('app/Resources/assets/rev-manifest.json'))
... line 25
};
... lines 27 - 76

As you'll see, this file doesn't need to be publicly accessible - our PHP code just needs to be able to read it.

There's one more interesting step: pipe() this into gulp.dest('.'):

76 lines gulpfile.js
... lines 1 - 13
gulp.src(paths)
... lines 15 - 22
// write the rev-manifest.json file for gulp-rev
.pipe(plugins.rev.manifest('app/Resources/assets/rev-manifest.json'))
.pipe(gulp.dest('.'));
... lines 26 - 76

What?

What do Multiple dest()'s mean?

So far, we've always had one gulp.src() at the top and one gulp.dest() at the bottom, but you can have more. Our first gulp.dest() writes the CSS file. But once we pipe to plugins.rev.manifest(), the Gulp stream changes. Instead of being the CSS file, the manifest is now being passed through the pipes. So the last gulp.dest() just writes that file relative to the root directory.

Let me show you. Stop gulp and restart:

gulp

And there's our rev-manifest.json file:

{
    "main.css": "main-50d83f6c.css"
}

It holds the map from main.css to its actual filename right now. It is missing dinosaur.css, but we'll fix that in a second.

Fixing the manifest base directory

But there's another problem I want to tackle first. In a second, we're going to put JavaScript paths into the manifest too. So I really need this to have the full public path - css/main.css - instead of just the filename.

So why does it just say main.css? Because when we call addStyle(), we pass in only main.css. This is passed to concat() and that becomes the path that's used by gulp-rev.

The fix is easy! Inside concat(), update it to css/ then the filename. That changes the filename that's inside the Gulp stream. To keep the file in the same spot, just take the css/ out of the gulp.dest() call:

76 lines gulpfile.js
... lines 1 - 12
app.addStyle = function(paths, outputFilename) {
gulp.src(paths)
... lines 15 - 17
.pipe(plugins.concat('css/'+outputFilename))
... lines 19 - 21
.pipe(gulp.dest('web'))
... lines 23 - 25
};
... lines 27 - 76

So nice: those two pipes work together to put the file in the same spot. Run gulp again:

gulp

Now, rev-manifest.json has the css/ prefix we need:

{
    "css/main.css": "css/main-50d83f6c.css"
}

Merging Manifests

So why the heck doesn't my dinosaur.css show up here? The addStyle() function is called twice: once for main.css and once for dinosaur.css. But the second time, since the manifest file is already there, it does nothing. Unless, you pass an option called merge and set it to true:

78 lines gulpfile.js
... lines 1 - 13
gulp.src(paths)
... lines 15 - 22
// write the rev-manifest.json file for gulp-rev
.pipe(plugins.rev.manifest('app/Resources/assets/rev-manifest.json', {
merge: true
}))
... lines 27 - 78

Let's see if this fixed it! Re-run gulp:

gulp

Yes! The hard part is done - this is a perfect manifest file:

{
    "css/main.css": "css/main-50d83f6c.css",
    "css/dinosaur.css": "css/dinosaur-32046959.css"
}

Phew! We're in the homestretch - the Gulp stuff is done. The only thing left is to make our PHP use the manifest file.

Since I'm in Twig, I'm going to invent a new filter called asset_version:

... lines 1 - 9
<link rel="stylesheet" href="{{ asset('css/main.css'|asset_version) }}"/>
... lines 11 - 48

Let's make it do something! I already created an empty Twig extension file to get us started:

<?php
namespace AppBundle\Twig;
class AssetVersionExtension extends \Twig_Extension
{
private $appDir;
public function __construct($appDir)
{
$this->appDir = $appDir;
}
public function getFilters()
{
return array(
);
}
public function getName()
{
return 'asset_version';
}
}

And I told Twig about this in my app/config/services.yml file:

... lines 1 - 5
services:
twig_asset_version_extension:
class: AppBundle\Twig\AssetVersionExtension
arguments: ["%kernel.root_dir%"]
tags:
- { name: twig.extension }

So, this Twig extension is ready to go! All we need to do is register this asset_version filter, which I'll do inside getFilters() with new \Twig_SimpleFilter('asset_version', ...) and we'll have it call a method in this class called getAssetVersion:

... lines 1 - 13
public function getFilters()
{
return array(
new \Twig_SimpleFilter('asset_version', array($this, 'getAssetVersion')),
);
}
... lines 20 - 41

Below, I'll add that function. It'll be passed the $filename that we're trying to version. So for us, css/main.css.

Ok, our job is simple: open up rev-manifest.json, find the path, then return its versioned filename value. The path to that file is $this->appDir - I've already setup that property to point to the app/ directory - then /Resources/assets/rev-manifest.json:

... lines 1 - 20
public function getAssetVersion($filename)
{
$manifestPath = $this->appDir.'/Resources/assets/rev-manifest.json';
... lines 24 - 33
}
... lines 35 - 41

With the power of TV, I'll magically add the next few lines. First, throw a clear exception if the file is missing. Next, open it up, decode the JSON, and set the map to an $assets variable. Since the manifest file has the original filename as the key, let's throw one more exception if the file isn't in the map. I want to know when I mess up. And finally, return that mapped value!

... lines 1 - 20
public function getAssetVersion($filename)
{
$manifestPath = $this->appDir.'/Resources/assets/rev-manifest.json';
if (!file_exists($manifestPath)) {
throw new \Exception(sprintf('Cannot find manifest file: "%s"', $manifestPath));
}
$paths = json_decode(file_get_contents($manifestPath), true);
if (!isset($paths[$filename])) {
throw new \Exception(sprintf('There is no file "%s" in the version manifest!', $filename));
}
return $paths[$filename];
}
... lines 35 - 41

So we give it css/main.css and it gives us the hashed filename.

Let's give it a shot! Take a deep breath and refresh. Victory! Our beautiful site is back - the hashed filename shows up in the source.

Ok ok, let's play with it. Open layout.scss and give everything a red background. The Gulp watch robots are working, so I immediately see a brand new hashed main.css file in web/css. But will our layout automatically update to the new filename? Refresh to find out. Yes! The new CSS filename pops up and the site has this subtle red background.

Go back and undo that change. Things go right back to green. Oh, and we do have one other CSS file on the dino show page. It should be giving us a little more space below the T-Rex, but it's 404'ing. We need to make it use the versioned filename.

So, open up show.html.twig and give it the asset_version filter:

... lines 1 - 5
<link rel="stylesheet" href="{{ asset('css/dinosaur.css'|asset_version) }}"/>
... lines 7 - 23

Refresh - perfect! No 404 error, and our button can get a little breathing room from the T-Rex. It took a little setup, but congrats - you've got automatic cache-busting.

Tip

You can make the getAssetVersion() function more efficient by following the advice in this comment.

Tip

In Symfony, it's also possible to avoid needing to use the filter by leveraging a cool thing called "version strategies". Check out the details posted by a helpful user here.

Don't commit the manifest

But should we commit the rev-manifest.json file to source control? I'd say no: it's generated automatically by Gulp. So, finish things off by adding it to your .gitignore file:

18 lines .gitignore
... lines 1 - 16
/app/Resources/assets/rev-manifest.json

Leave a comment!

27
Login or Register to join the conversation

Hello Ryan, I am struggling to set this service with Symfony 3.3 but I get an AutowiringFailedException because "argument "$appDir" of method "__construct()" must have a type-hint or be given a value explicitly."

can you give me some help?

thank you very much

3 Reply

Hey chieroz

In order to the Autowiring to work correctly, it needs you to type-hint every parameter, for parameter that can't be type-hinted (like strings, booleans, etc), you have to explicitly specify them, like this:


services:
    my_service:
        class: AppBundle\Service\MyService.php
        autowire: true
        arguments:
            2: '%some_value%'

That "2" is the number of the argument's position in the constructor, counting from 1 to N, from left to right

Cheers!

Reply
Default user avatar
Default user avatar Dan Reinders | MolloKhan | posted 5 years ago | edited

I got mine to work with:


services:
    AppBundle\Twig\AssetVersionExtension:
        arguments:
            0: '%kernel.root_dir%'
Reply

Hey Dan Reinders

Yeah, your code should work because the parameter you want to inject is in the "0" position. Mine is only an example of how it works :)

Cheers!

Reply
Default user avatar
Default user avatar ElHornair | posted 5 years ago | edited

Hi Ryan

If you have a couple of files in the rev-manifest, you will be doing unnecessary file system reads. Because the Twig extension is a singleton, you can load the paths once and save them in a class variable. Here is the adapted code:


    private $paths = [];

    public function getAssetVersion($filename)
    {
        if (count($this->paths) === 0) {
            $manifestPath = $this->appDir.'/Resources/assets/rev-manifest.json';

            if (!file_exists($manifestPath)) {
                throw new \Exception(sprintf('Cannot find manifest file: "%s"', $manifestPath));
            }

            $this->paths = json_decode(file_get_contents($manifestPath), true);
        }

        if (!isset($this->paths[$filename])) {
            throw new \Exception(sprintf('There is no file "%s" in the version manifest!', $filename));
        }

        return $this->paths[$filename];
    }

PS: I'd be happy to add this as a PR to the above tutorial (which is great btw, thanks a lot!) but I couldn't find this on Github. Any hints?

Reply

Yo ElHornair!

Yea, you're totally right - my algorithm could easily be made more efficient :). This *is* on GitHub (https://github.com/knpunive..., but to actually update it is a little complex - we use a proprietary "diff" system to manage changes that happen *during* the tutorial. BUT, actually, I think we shouldn't change the file anyways - I like to have the code blocks and code download match the video. But, what we *could* do is add a note to the script in *just* the right spot that points over here to your comment. Best of both worlds :). If you agree and want to get your proper props on the GitHub repo, you could edit the script here: https://github.com/knpunive.... We have a special ***TIP syntax you can see - an example is in this script: https://raw.githubuserconte...

Thanks for the comment!

Reply
Default user avatar

Alright, will try to do that.
Thanks for wrapping my code btw, didn't know you can do that in Disqus ;)

Reply

My pleasure - it ends up looking nice in Disqus, but it's got a weird, hidden syntax!

1 Reply
Default user avatar
Default user avatar irozgar | posted 5 years ago

Hi Ryan,

Thanks for your great tutorials. When I ended this video I asked myself if there was a better way than adding the asset_version filter in every versioned asset (I know that I'd forget writting it more than once) so I started looking of other ways of doing it and I learned
about the VersionStrategyInterface from the Asset Component. After a few hours I've made a simple VersionStrategy to use with assets
versioned with gulp-rev. With it the filter is no needed anymore.

Here is a gist with the code
https://gist.github.com/iro...

And a Symfony Bundle for Symfony >= 2.7 based on the gist
https://github.com/irozgar/...

Reply

Ahhh, this is awesome! What a cool way to handle this - you're absolutely right that the VersionStrategyInterface is *meant* for this kind of stuff :). Thanks for sharing! If you want, you could also add a tip in the script about this here https://github.com/knpunive... (there's already an example of a tip). I think this would be a cool thing to point out.

Cheers!

Reply
Default user avatar

I made a PR

62 Reply
Sergiu P. Avatar
Sergiu P. Avatar Sergiu P. | posted 5 years ago | edited

What's wrong with this?


{% set timestamp = (app.environment == 'dev' ? '?'~"now"|date('U') : '') %}
<link rel="stylesheet" href="{{ asset('css/app.css') ~ timestamp }}">
Reply

Hey Sergiu,

It depends on what you're trying to achieve with it? :) As I see this solution doesn't bust cache on prod when you make some changes to it.

Cheers!

Reply
Default user avatar
Default user avatar James Pike | posted 5 years ago

Why use this cache buster 'gulp-rev' when you can just do a hard reload of the browser when you upload the new content? Doesn't the browser then cache the new content and that then overrides the old cached content?

Reply

Hey James,

Yeah, you can do hard reload of your browser, but that's a user side! I mean, you need to tell all your users that they should do hard reload which is not probably appropriable for us ;) That's why we should care about these things

Cheers!

Reply
Default user avatar
Default user avatar James Pike | Victor | posted 5 years ago

Thanks Victor, i have a site hosted and i changed some styles locally then pushed them up. When i did a hard reload using Chrome the styles updated, then i checked on Firefox trying to pretend i was a random end user and the changes were already there with no hard reload required.

Reply

Hm, that's interesting... maybe because you're on the same machine, difficult to say, but this should not work so :) If you want an experiment, use 2 laptops at least, but I'd not hope hard reload in a browser will fix the problem for all browsers ;)

Cheers!

1 Reply
Default user avatar
Default user avatar James Pike | Victor | posted 5 years ago

Yeah it works on Edge and Opera also, like you say, to run a proper test i'll use my desktop and tablet, 2 different machines...thanks, by the way, do you have to cache bust for the html also or is it just for styles and js?

Reply

Usually it is only for any external resource (like CSS, JS, images, fonts, etc), but if you are using "esi_tags" or any other type of cache for your html, then you would have to refresh it as well (The most probably is that you don't have to worry about that)

Cheers!

Reply
Default user avatar
Default user avatar Marcus Stöhr | posted 5 years ago

Hey Ryan,

thank you for this great tutorial. The version busting strategy is really cool but I have a really hard time to get it working for all my styles and scripts.
I have set it up like you showed in the upcoming chapters using the pipeline. However, the manifest-file doesn't contain all my files but either only one or two of them. It also mixes styles and scripts.

Whats going on here?

Reply

Hey Marcus!

Yes, I know the problem well! Have you gone through the last 2 chapters in this tutorial (especially https://knpuniversity.com/s.... We talk about this exact issue. When recording this chapter - the manifest issue hadn't reared it's head - so we had to fix it in a later chapter.

Let me know if that helps - and happy you're enjoying things (you'll enjoy them more when cache busting works)!

Reply
Default user avatar
Default user avatar Marcus Stöhr | weaverryan | posted 5 years ago

Hi Ryan.

I just checked, double-checked, and in fact triple-checked the chapter you mentioned and my gulpfile.js contains all of this. However, it doesn't work and this is driving me nuts.

Any hint what else I can do?

1 Reply

Hey Marcus!

Hmm - could you throw up a simple example repository on GitHub? Perhaps something changed with some version of these libraries that's causing the problems. This was indeed the trickiest part of this whole tutorial :).

Cheers!

Reply
Default user avatar
Default user avatar s.molinari | posted 5 years ago

Hey Ryan,

What if the names of files are important to the actual execution of the JS app? Cache busting would break the app. Is there a way to "insert" the new versions smartly within the code too?

Scott

Reply

Hey again Scott!

Hmm, what's your use-case? There is a plugin I *almost* talked about, which is useful if you're moving files from one directory to another: https://www.npmjs.com/packa.... And one exists for this and gulp-rev (https://www.npmjs.com/packa... but I haven't tried it yet! Let me know if it fits for you (and what you're use-case in - I'm curious).

Cheers!

Reply
Default user avatar

I am playing with a JS framework called Aurelia (the one I mentioned in my other post)

http://aurelia.io/

and I noticed it wouldn't "correct" when changing files, because of the cache in my browser. Obviously I could have the browser load every file again on every call to develop, but cache busting as described here is the better answer. We'll need it anyway to fix bugs in production later, right? So I actually built in the steps you explained in this tutorial into the Aurelia demo app under "Get Started". However, because Aurelia (similar to Angular?) uses data binding with custom tag attributes as built in directives like <body aurelia-app="">, which also formulates the name of the file, cache busting busts the application too. :-(

I found another cache busting module, which partially does something like what is needed I believe.

https://github.com/hollandb...

However, in Aurelia, it doesn't use complete file names in the router.

{ route: ['','welcome'], moduleId: './welcome', nav: true, title:'Welcome' },
{ route: 'flickr', moduleId: './flickr', nav: true },
{ route: 'child-router', moduleId: './child-router', nav: true, title:'Child Router' }

I highly doubt this would work.

{ route: ['','welcome'], moduleId: './welcome#grunt-cache-bust', nav: true, title:'Welcome' },
{ route: 'flickr', moduleId: './flickr#grunt-cache-bust', nav: true },
{ route: 'child-router', moduleId: './child-router#grunt-cache-bust', nav: true, title:'Child Router' }

However, as I said "partially does what is needed", the initial call for app.js through the custom tag attribute is still not resolved.

Scott

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": ">=5.3.3",
        "symfony/symfony": "2.6.*", // v2.6.4
        "doctrine/orm": "~2.2,>=2.2.3", // v2.4.6
        "doctrine/doctrine-bundle": "~1.2", // v1.2.0
        "twig/extensions": "~1.0", // v1.2.0
        "symfony/assetic-bundle": "~2.3", // v2.5.0
        "symfony/swiftmailer-bundle": "~2.3", // v2.3.7
        "symfony/monolog-bundle": "~2.4", // v2.6.1
        "sensio/distribution-bundle": "~3.0", // v3.0.9
        "sensio/framework-extra-bundle": "~3.0", // v3.0.3
        "incenteev/composer-parameter-handler": "~2.0", // v2.1.0
        "hautelook/alice-bundle": "~0.2" // 0.2
    },
    "require-dev": {
        "sensio/generator-bundle": "~2.3" // v2.4.0
    }
}
userVoice