Configuring DoctrineCacheBundle

Hey Br!

Yea, nice catch on that! This represents a difference in philosophy between Symfony 3 and Symfony 4. In Symfony 3, we often used the container directly to fetch services - e.g. $this->container->get('doctrine_cache.providers.my_markdown_cache'). In Symfony 4, that is discouraged, and can sometimes cause this error. Behind the scenes, each service is either "public" or "private". The only difference is that a private service *cannot* be fetched with the $container->get() method. In Symfony 4, more and more services are private. Here's some more information about that: https://knpuniversity.com/s...

In this case, when Doctrine creates your cache service, it makes it private. And, there's not actually any way to change that... but that's ok! What you need to *change* is how you *fetch* the service. Instead of using $container->get(), you should use dependency injection. We actually talk quite a lot about this in our Symfony 4 tutorial - https://knpuniversity.com/s... - which (at least the services part) is relevant to Symfony 3.4, as it teaches you all about this newer way of coding. In your cache, I would probably use "bind" (https://knpuniversity.com/s... to bind $markdownCache to the service "doctrine_cache.providers.my_markdown_cache" so that Symfony autowires any and all $markdownCache arguments and passes you this service. This will make the deprecation go away.

If you have any other questions, let us know! This is a tricky thing to understand as we transition from the traditional way of fetching services to this new way. But once you get it, it's *super* fun :).

Cheers!

Hello guys,
Got a deprecated information in Symfony 3.4:
"User Deprecated: The "doctrine_cache.providers.my_markdown_cache" service is private, getting it from the container is deprecated since Symfony 3.2 and will fail in 4.0. You should either make the service public, or stop using the container directly and use dependency injection instead."
How can I fix that? I was trying to set that in services.yml or config.yml but probably I did something wrong :)
Cheers!

Wow, really? I'm wondering how can it confuse PhpStorm

Cheers!

I know this is an old question, but this worked for me:

bin/console doctrine:cache:clear-metadata

Hey Xav,

Let me clarify it a bit to see if I understand you right :)

For now you generate a page which require to send about 450+ HTTP requests to some API endpoints and then you cache received data in an XML list, i.e. you convert all the collected data during the generating to an XML list. And then you just iterate over that XML list with PHP to render an HTML page, right? And from time to time you need to regenerate this XML file due to some related changes in your useful list of articles? Does it sound correct?

If so, then yes, you can replace your XML level with Symfony Cache Component. Unfortunately, we do not have a separate screencast about this component, but it's not difficult to start. Out of the box Symfony provide "cache.app" service, and it stores cache in filesystem by default, but you can choose different adapter like Redis, Memcache, etc. in config.yml. Also, check their docs how to use it: https://symfony.com/doc/cur... and take a look at cache invalidation: https://symfony.com/doc/cur... - I suppose you want to control when to update the data in cache.

Cheers!

Hi KnpU,
My app has a page with 450+ requests to create a very useful list of articles. This list doesn't evolve much : few more articles every week, and sometimes an edit on an article which is relevant for the list. I am working on a XML list, generated and updated with PHP, but I thought that maybe Symfony 3.2 could do all that for me ? With the Cache Component ? If yes, could you please help me and point me in the right direction to solve this ?
Thank you.

haha, yeah, it's because we think faster than we type!
I'm glad you could fix your problem :)

Cheers!

Aren't mistakes silly? I had silliest one I was not caching anything :) The problem has been solved. Thank you!!

Interesting, make sure that your "cache_type" variable is set to "file_system"
also remember if your path starts with "slash", then it will be an absolute path instead of a relative one

One more thing, maybe it would be silly to you, but just in case, are you saving the cache key ?

sleep(1); // fake how slow this could be
$funFact = $this->get('markdown.parser')
                ->transform($funFact);

$cache->save($key, $funFact);

Cheers!

They are all indenting correctly.

Hey Diego,
Thanks for helping me, my config file looks exact same as yours. Actually was looking. Then end of the tutorials it looks like this and it is still not working.
parameters:
locale: en
cache_type: file_system

doctrine_cache:
providers:
my_markdown_cache:
type: %cache_type%
file_system:
directory: %kernel.cache_dir%/markdown_cache

Hey Fatih!

My first guess is you might have something wrong in your configuration, can you show me how looks like your config.yml file ?
It should be like this:

doctrine_cache:
    providers:
        my_markdown_cache:
            type: file_system
            file_system:
                directory: /tmp/doctrine_cache

Have a nice day!

Here is my first trouble :) I have Symfony 3.2.8 version. Everytime i refresh the page it is loading over 1000ms and it is not caching. there is no file under var/cache/doctrine/cache/file_system?

Hey there Ahmed EBEN HASSINE!

Oh man, that is WAY too slow of a load. But, let's get to that in a second :). To your questions!

1) We're adding caching simply to make our app faster. Parsing markdown can be a little bit slow, so we don't want to do this work on every page load. Next, why are we using DoctrineCacheBundle? Well, Symfony doesn't come with a caching library (actually, this is not true anymore - Symfony *does* have a cache component since 3.1, but it wasn't available when we recorded this tutorial). But, the Doctrine library *does* have a caching library. The DoctrineCacheBundle integrates the Doctrine cache library into Symfony. We could have also used any other caching libraries (including the new Symfony Cache library).

2) Now, about your app performance! First, your problem has nothing to do with caching :)... I think something is misconfigured with your system! You should *not* need to clear your cache (ever) in dev mode and your page loads should NEVER take longer than a few seconds (and that would be rare, only when Symfony needs to reload some of its internal cache). A 12 second reload is a clear sign of a problem. So: are you using a virtual machine? Are you using Windows? Tell me about your setup, and I'll see if we an identify the problem. If you're using a VM, then we will need to do some work (see https://www.dev-metal.com/l....

3) Ah, GREAT question! Symfony *does* cache quite a few things automatically. Specifically, you are noticing the Twig cache. This is a cache of the *parsing* process of the Twig language, it does *not* mean that your Twig template HTML is cached. Let me give you an example: when you refresh the first time (with a clear cache directory), Symfony parses the Twig language and caches this (into the location you mentioned). It then executes that template and outputs the HTML. When you refresh again, it does NOT need to re-parse the Twig language, as it can use the cached version of the template. BUT, it still *does* execute that template and return its HTML. In other words, if you have logic in your Twig template that takes a lot of time (e.g. parsing markdown to HTML), this is not cached: that's up to you!

Let me know if this clears it up! Basically, Symfony caches internal things that it does... but it's up to you to cache (if you want to) any work that *you* choose to do (e.g. parsing markdown to HTML, database queries, API calls, etc).

Cheers!

Hi ^_^
First when i work in dev mode , i always clean my cache because sometimes the changes does not work ,besides that my application take sometimes more then 12 sec to reload one page, and that's is so exhausted....
1) why we use caching system in general and "doctrineCacheBundle" in specific ?
2) how to made my app perform more fast when i try to load my pages ?
3)do we really need to implement doctrineCacheBundle to cach the markdown ,because i think symfony implement a cache system (the twig files in var\cache\dev\twig)

Thank you very much :)

Yo Denis!

Awesome! Glad it's useful :). About your questions:

1 & 2) I definitely recommend using Redis - it certainly seems like the industry leader right now. However, I'm not super performance obsessed - I mean, I have also used (and like) Memcache, or even apcu (except that you can't pool it across multiple servers). These are all SUPER fast - saving you probably 99.9% compared to not caching. So, any caching is good - I don't worry too much about the exact caching software - use what you know best and what's available :).

3) Ah yes! So, there are MANY ways to cache of course :). You are caching the queries, which I would consider more or less the lowest-level, most granular way to cache. I don't typically do this. Instead, I cache in one of two higher-level ways:

A) Often, I'll add caching to a service - exactly like we've done here - so that the output of that service call is cached. If it makes 1 database query, awesome! Maybe it makes 10 queries, even awesomer :). This is just a bit easier for me to wrap my head around, especially when it comes to knowing when that cache needs to be invalidated. It's really more-or-less the same as caching the queries... it's just that I center my caching logic around my "business model setup" instead of specific queries.

B) Varnish! Or, more generically, HTTP caching. We *also* do this, and this will always give you the MAX performance. Honestly, we don't do this on nearly as many pages as we should (it's a todo). One place we use it is here: http://knpuniversity.com/blog and you can easily see how this page is MUCH faster than almost any others on the site, despite loading up a lot of blog posts :).

ANYWAYS, depending on your app, HTTP caching may make more sense - it'll certainly be more performant. For our blogs, we could have cached the queries to fetch the blog results... but instead... we just essentially cache the final HTML, which is MUCH faster and automatically encompasses the markdown processing step. And actually, we don't use Varnish - we're using the built-in Symfony HTTP cache... simply because it's much simpler (and we're a small team). Varnish is better... but we're still getting fast page loads without it.

What's the draw-back to HTTP caching? Probably just the learning curve - the docs on Symfony.com about it are only "ok" (I've wrote much of them, so this at least a lot my fault) and we don't have a screencast here about it yet (I would LOVE to do it, but there's not as much demand as for other courses). When you get into it, the trick is to organize your page into little "chunks" (ESI) that are either cacheable or not cachable. For example, on the blog page, the entire left, main content area is one chunk that we cache. Then, the REST of the page we do *not* cache (because, for example, the upper right navigation area changes when you are logged in). But, you could do this differently: we could have cached the ENTIRE page... but rendered the navigation through an un-cached ESI blog. In case you haven't seen it this presentation by Fabien is still my favorite for diving into HTTP caching stuff: http://www.slideshare.net/f...

So yes, HTTP caching is kind of the end-all-be-all for performance... but it also depends on your app. If your app is basically data entry and everything is dynamic and always changing, it's going to be hard to cache large chunks of your page. But if you have more of a content-based site (like ours), then HTTP caching is by far the best way.

Phew! So, not a boring question at all! If you have more, let me know - this was kind of an initial brain-dump :).

Cheers!

Hi Ryan,
First I had learned a lot of you, thanks :)

In this tutorials you say a little bit of caching.
So I have three questions about caching if you can help me:

1. I think the most time to render our page is lost in database queries
Do yor recommend to use Redis. We cache our database query result in redis,
for the first visitor of the page. And for every other visitor we take
the result from redis, so we speed up our application.

2. Do you use Redis or something else?

3. I read about Varnish, if I am right Varnish cache the entire page
Does it than make sense to use redis?
Sorry to be a bore, but I'd like to know how you are doing the chaching to speed up your app.

Cheers!

Ah, welcome to Symfony! And thanks for the nice words! Symfony can also be complex at times... but it doesn't *need* to be complex, and we try to teach it in a pragmatic way :). Cheers!

Just switching from ZF to Symfony. Too much factories and factories that start factories and configs for every line of code.
I`ve search a lot for a good symfony tutorial, i just wanted to say: this symfony tutorial for new ones is like phpstorm for php developers.

Hey Noby!

Yes and no :). A lot of the work we did was initial setup. So if you wanted to cache in a different route/controller, you wouldn't need to do much work at all: (A) add a new entry in config.yml under "doctrine_cache" and "providers" then (B) fetch that service in a controller and use it to cache. Not too much work.

But also, you likely won't need to cache very often. Of course, it depends on your application, but, for example, the vast majority of the routes/controllers on our site don't have any caching: it's just fast enough.

Cheers!

Hi,
Ok, a bit complicated. but, it is working now.
Do I have to do this process for every route page/function to have cache working?

Kind regards,
Noby

Hey Marco,

Something wrong with the "markdown.parser" service definition in services.yml - please, double check its spelling and try to clear the cache. If it doesn't help you - please, show your "markdown.parser" service definition here.

Cheers!

Hey I've fixed the typo but now it says You have requested a non-existent service "markdown.parser"

Yo Marco,

I bet I know the problem. You made a type in this service name: it should be "markdown.parser" instead of "markdown.paser" ;)

Cheers!

Hey! I get the error: You have requested a non-existent service "markdown.paser". Ive done everything in the video I dont know what is wrong please help :)

Yes, it does. It seems like I had have quite a vague notion about Twig cache files.

And now I also understand why some people think Twig is too heavy. :)

Yo Maksym!

Great question :). You're right that the Twig templates are cached to PHP files. But, the *output* of the templates is not cached. What I mean is: the Twig language is parsed and a PHP file is built in the cache. This cache is just a translation of the Twig language into the PHP language. If you looked at the cached Twig file (which you can do by browsing in var/cache/dev/twig), you'll find that it includes echo statements for variables that will be passed in at runtime. This cached file is used to render different content based on the different variables that you pass into it.

This also means that your controller code is *always* executed (since it needs to prepare the variables for the Twig template). And in this case, your controller code is actually the code that will be slow (as it parses the markdown). That's why we add that caching layer.

Does that make sense? There *are* of course ways to cache the *output* of your page (what you'd normally think of as some sort of "page caching"). We don't have a tutorial on this yet, but it's one of the coolest features of Symfony: http://symfony.com/doc/curr...

Cheers!

But Twig caches its templates, doesn't it? So, do we really need another cache for markdown service?

Ah, good catch! For Windows users it makes sense since windows has a bit different file system, so it doesn't have "/tmp/" dir as Mac/Linux does. Probably, you could use C:\Users\<username>\AppData\Local\Temp instead, but I'd recommend to use the local Symfony cache dir, i.e. "%kernel.root_dir%/cache/doctrine_cache" rather.

Cheers!

I'm using Windows and I had a similar problem. I thought directory tmp/doctrine_cache didn't exist but it did, just in other location.
It should be created in d:/xampp/htdocs/my_project/ but instead I found it directly in d:/. I don't know why. I googled it and I found I can use %kernel.root_dir%, so I changed my configuration from:

directory: /tmp/doctrine_cache

to

directory: %kernel.root_dir%/../tmp/doctrine_cache

and it worked.

Later I found out that these "kernel" parameters are described in next chapters of this course, :)

Hey 3amprogrammer ,

You're right, Doctrine relates to the database, but also it relates to the data in common. Doctrine has a nice doctrine/cache library which helps to cache data, i.e. you could easily cache data which you fetch from a database, which in turn reduce a database server loading.

Cheers!

I have a silly question. Why caching markdown has anything to do with doctrine? I tough doctrine is related to database? That confuses me a bit.

Hey dankinchen ,

If folder never appear after it was deleted - probably you have permission problem. To check it - use `ls -la` command.

Yes, you're right. The "Project" mode do not show you excluded directories. Excluded files are ignored by PhpStorm while searching, parsing, watching etc.

Cheers!

I cannot get cache to work for me either. There were files in there but when I deleted the doctrine directory it never came back. I mapped $key to twig and dumped it and sure enough there was an md5 hash in there. You know another thing I cannot navigate to "dir var/cache/" I can only see the contents of var. When I switch between project files and project I get different content? Is this because in an earlier video you had us all exclude cache directory in phpstorm?

Thanks for getting back to me. After deleting the directory and refreshing, it seemed to work :)

Thanks a lot!

Hi Nathaniel!

Ah, thanks for the nice words! I'm very happy about this :).

About your question, I have a few ideas:

1) Obviously, make sure that you configure the "directory" key in config.yml set to /tmp/doctrine_cache. You said you followed word-for-word - but this was the most obvious thing to check, so let's start there!

2) If you delete the var/cache/dev/doctrine... directory and then refresh, is it re-created? Or does this directory just remain because we were using it earlier?

It's possible that there's some weird bug in the tutorial - I'm happy to help you dig into what's going on :).

Cheers!

Hi weaverryan! Your tutorials are excellent w.r.t. Symfony! I just have one quick question regarding this one though. After coding along word for word, I find that the last directory: tmp/doctrine_cache doesn't exist for me when I try to ls into it, whereas the var/cache/dev/doctrine./cache/file_system/60 does exist. Any suggestion as to why this could be?
Thanks in advance!

Hey Karlola!

Hmm, let's see if we can figure this out. First, *after* this chapter, we start getting fancier and only configuring the cache to work on one environment versus another. So, make sure that you don't have any of that code yet - else the cause could be something entirely different :).

If you're seeing no files, then we know that the cache isn't actually saving. In GenusController, make sure you have the $cache->save($key, $funFact); line in there. I might also var_dump($key); to make sure this is a real value (if, for some crazy reason this is a blank value, it might not be able to cache).

Let me know what you find out! Definitely focus on why the cache is not *saving*.

Cheers!

I've been coding along word-for-word, yet something is messed up with my caching: both the long path `var/cache/dev/doctrine/cache/file_system/` and our custom path `/tmp/doctrine_cache` contain no files or subfolders. The site loads with no explicit errors but also keeps sleeping for the ordered 1 second with every refresh. Where should I look for hints?

Cheers! And yea, I think Doctrine is much harder than Symfony :). Doctrine gets tricky with relationships, and it really just comes down to getting the configuration correct, then everything works nicely. Doctrine is also more "work" (more writing code) than something like Propel - another reason you sometimes hear bad things about it :).

Keep up the good work!

Yeah, I'm 100%(notreally) new to "Symfony" and Doctrine, I mean, not really that new to symfony itself, as I've been using a custom implementation of Silex from a friend for about 2 years, the learning curve into symfony, thats supposedly step, is not being so much as I expected it, surely because of my Silex experience.

Doctrine on the other hand, its a whole new world for me, as the Silex implementation that I've been using was based on propel1 (yes propel1 not propel2), and well I was worried that using Dogtrine*cough* was going to be a really tough act, but seeing how it works, is just neat, its still a bit black-magic-ish, but it matters not, just been reading about it for a day and half haha.

again Thanks for taking the time to reply.

Haha, yep - you totally answered your own question :). But to add some more, there end up being 3 possibilities (at least):

1) Add a different doctrine_cache in config.yml, config_dev.yml, config_prod.yml, etc, with different directories
2) Only configure in config.yml, but set the directory to something that includes `%kernel.environment%` - that's the environment name. Or, `%kernel.cache_dir%` - which is the path to the cache directory, which already differs by environment
3) Only configure in config.yml, but set directory to a parameter - like `%my_cache_dir%`. Then, set `my_cache_dir` under the parameters key in the other files (you could set a default in config.yml, then override in other config files).

The big thing is to understand how the config files load each other (which you clearly do!). Then, you start to see a lot of possibilities :).

Cheers!

ok, so here I'm guessing, if i want to have cache for dev in one place, and cache for prod in another (just for the heck of it, lets just go with the argument of wanting to have ALOT of cache because i run dev, test, prod on the same directory, bleh bleh haha)

I would need to configure the cache (doctrine_cache) on config_prod/test/dev.yml ? and assign it whatever folder i want (lets just say .var/cache/{env}

where {env} is the environment (prod/test/dev) and it should work?

checking the conf files, i see test imports dev, whereas prod imports config.yml

so yes theoretically speaking it should work. ha! answering my question on the same question, isnt this quite nice!

Hey Mike!

Ah, so you get auto-completion at all when you try to reference the services? I'm not sure what else to try either :/. Is your PhpStorm project opened just to *one* Symfony project (sometimes people will open a directory that contains many projects, and this can confuse PhpStorm). Are you also on the latest versions of PhpStorm and the plugin? I'm just trying to think of possible reasons for this :).

Cheers!

Thanks for the info.

I tried what you said, but it still doesn't find the doctrine_cache.providers.my_markdown_cache when I start typing markdown. It did however remove all of the markdown.parser.max and markdown.parser.min options. Not really sure what else to try.

Hey Mike!

Yes, please do piggyback - I'd love if we could figure this out for everyone :).

So, you *are* seeing auto-completion on service ids, so that's the first thing to note. I think some people don't get auto-completion at all. And you *are* seeing a lot of valid services in your auto-completion - like markdown.parser.max (that's a real service that *should* auto-complete). The question just is, why aren't you seeing the new doctrine_cache... service?

One way or another, the likely cause is either that (A) your Symfony cache is out of date (fixing that is as simple as running the bin/console debug:container that we ran in the previous chapter) or (B) PhpStorm didn't see the update. From what I know, phpstorm uses a few strategies to know what your services are: it parses your YAML/XML files, but it also looks (I believe) at a dump container metadata file in your var/cache/dev directory. When you first load a page / run bin/console after making the config changes, that new file is dumped. PhpStorm may not be seeing this for some reason. Related to this, in the PhpStorm settings, under the Symfony plugin, there is a section called "Container". Here, you can actually manually point PhpStsorm at this dumped, XML file (it's called var/cache/dev/appDevDebugProjectContainer.xml). I've never had to do this, but perhaps it would make a difference?

This particular example is a little different overall, because we aren't configuring this service directly in one of our YAML files. So, PhpStorm relies on that dumped XML file - again, afaik - whereas normally it would also be parsing our YAML files to find our services.

I hope this helps - let me know if you find anything out!

Hi - If I can just piggyback off of this question. I am also having the same problem. I have the plugin installed and enabled. When I watch your tutorials (which are great by the way!) my autocomplete never really looks the same as yours. For this exact example when I type markdown i see stuff like markdown.parser, markdown.parser.max, markdown.parser.min, etc. even if I try typing in doctrine_cache it does not autocomplete. Any ideas why?

I do have the Symfony plugin for PhpStorm, but I'm guessing you have this? If you do have this, make sure it's enabled for your project: http://knpuniversity.com/sc...

This plugin is the one that gives you this auto-complete.

Hi,
I'm finding out I don't have the autocompletions you have i.e. the doctrine_cache.providers.my_markdown_cache
Do you have any special plugins?