Doctrine DQL
Hey there friends! And thanks for joining me for to a tutorial that's all about the nerdery around running queries in Doctrine. It sounds simple... and it is for a while. But then you start adding joins, grouping, grabbing only specific data instead of full objects, counts... and... well... it gets interesting! This tutorial is about deep-diving into all that good stuff - including running native SQL queries, the Doctrine Query Language, filtering collections, fixing the "N + 1" problem, and a ton more.
Woh, I'm pumped. So let's get rolling!
Project Setup
To INSERT the most query knowledge into your brain I highly recommend coding along with me. You can download the course code from this page. After you unzip it, you'll have a start/ directory with the same code that you see here. There's also a nifty README.md file with all the setup instructions. The final step will be to spin over to your terminal, move into the project, and run:
symfony serve -d
to start a built-in web server at https://127.0.0.1:8000. I'll cheat, click that, and... say "hello" to our latest initiative - Fortune Queries. You see, we have this side business running a multi-national fortune cookie distribution business... and this fancy app helps us track all the fortunes we've bestowed onto our customers.
It's exactly 2 pages: these are the categories, and you can click into one to see its fortunes... including how many have been printed. This is a Symfony 6.2 project, and at this point, it couldn't be simpler. We have a Category entity, a FortuneCookie entity, exactly one controller and no fancy queries.
Side note: this project uses MySQL... but almost everything we're going to talk about will work on Postgres or anything else.
Creating our First Custom Repository Method
Speaking of that one controller, here on the home page, you can see that we're autowiring CategoryRepository and using the easiest way to query for something in Doctrine: findAll().
| // ... lines 1 - 5 | |
| use App\Repository\CategoryRepository; | |
| // ... lines 7 - 10 | |
| class FortuneController extends AbstractController | |
| { | |
| ('/', name: 'app_homepage') | |
| public function index(CategoryRepository $categoryRepository): Response | |
| { | |
| $categories = $categoryRepository->findAll(); | |
| return $this->render('fortune/homepage.html.twig',[ | |
| 'categories' => $categories | |
| ]); | |
| } | |
| // ... lines 22 - 29 | |
| } |
Our first trick will be super simple, but interesting. I want to re-order these categories alphabetically by name. One simple way to do this is by changing findAll() to findBy(). This is normally used to find items WHERE they match a criteria - something like ['name' => 'foo'].
But... you can also just leave this empty and take advantage of the second argument: an order by array. So we could say something like ['name' => 'DESC'].
But... when I need a custom query, I like to create custom repository methods to centralize everything. Head over to the src/Repository/ directory and open up CategoryRepository.php. Inside, we can add whatever methods we want. Let's create a new one called public function findAllOrdered(). This will return an array... and I'll even advertise that this is an array of Category objects.
| // ... lines 1 - 4 | |
| use App\Entity\Category; | |
| // ... lines 6 - 16 | |
| class CategoryRepository extends ServiceEntityRepository | |
| { | |
| // ... lines 19 - 23 | |
| /** | |
| * @return Category[] | |
| */ | |
| public function findAllOrdered(): array | |
| { | |
| } | |
| // ... lines 31 - 73 | |
| } |
Before we fill this in, back here... call it: ->findAllOrdered().
| // ... lines 1 - 10 | |
| class FortuneController extends AbstractController | |
| { | |
| // ... line 13 | |
| public function index(CategoryRepository $categoryRepository): Response | |
| { | |
| $categories = $categoryRepository->findAllOrdered(); | |
| // ... lines 17 - 20 | |
| } | |
| // ... lines 22 - 29 | |
| } |
Delightful!
Hello DQL (Doctrine Query Language)
If you've worked with Doctrine before, you're probably expecting me to use the Query Builder. We will talk about that in a minute. But I want to start even simpler. Doctrine works with a lot of database systems like MySQL, Postgres, MSSQL, and others. Each of these has an SQL language, but they're not all the same. So Doctrine had to invent its own SQL-like language called "DQL", or "Doctrine Query Language". It's fun! It looks a lot like SQL. The biggest difference is probably that we refer to classes and properties instead of tables and columns.
Let's write a DQL query by hand. Say $dql equals SELECT category FROM App\Entity\Category as category. We're aliasing the App\Entity\Category class to the string category in much the same way we might alias a table name to something in SQL. And over here, by just selecting category, we're selecting everything, which means it will return Category objects.
And that's it! To execute this, create a Query object with $query = $this->getEntityManager()->createQuery($dql);. Then run it with return $query->getResult().
| // ... lines 1 - 16 | |
| class CategoryRepository extends ServiceEntityRepository | |
| { | |
| // ... lines 19 - 26 | |
| public function findAllOrdered(): array | |
| { | |
| $dql = 'SELECT category FROM App\Entity\Category as category'; | |
| $query = $this->getEntityManager()->createQuery($dql); | |
| return $query->getResult(); | |
| } | |
| // ... lines 34 - 76 | |
| } |
There's also a $query->execute(), and while it doesn't really matter, I prefer getResult().
When we go over and try that... nothing changes! It is working! We just used DQL directly to make that query!
Adding the DQL ORDER BY
So... what does it look like to add the ORDER BY? You can probably guess how it starts ORDER BY!
The interesting thing is, to order by name, we're not going to refer to the name column in the database. Nope, our Category entity has a $name property, and that's what we're going to refer to. The column is probably also called name... but it could be called unnecessarily_long_column_name and we would still order by the name property.
The point is, because we have a $name property, over here, we can say ORDER BY category.name.
Oh, and in SQL, using the alias is optional - you can say ORDER BY name. But in DQL, it's required, so we must say category.name. Finally, add DESC.
| // ... lines 1 - 26 | |
| public function findAllOrdered(): array | |
| { | |
| // ... line 29 | |
| $query = $this->getEntityManager()->createQuery($dql); | |
| dd($query->getSQL()); | |
| // ... lines 32 - 33 | |
| } | |
| // ... lines 35 - 79 |
If we reload the page now... it's alphabetical!
The DQL -> SQL Transformation
When we write DQL, behind the scenes, Doctrine converts that to SQL and then executes it. It looks to see which database system we're using and translates it into the SQL language for that system. We can see the SQL with dd() (for "dump and die") $query->getSQL().
| // ... lines 1 - 26 | |
| public function findAllOrdered(): array | |
| { | |
| // ... line 29 | |
| $query = $this->getEntityManager()->createQuery($dql); | |
| dd($query->getSQL()); | |
| // ... lines 32 - 33 | |
| } | |
| // ... lines 35 - 79 |
And... there it is! That's the actual SQL query being executed! It has this ugly c0_ alias, but it's what we expect: it grabs every column from that table and returns it. Pretty cool!
By the way, you can also see the query inside our profiler. If we remove that debug and refresh... down here, we can see that we're making seven queries. We'll talk about why there's seven in a bit. But if we click that little icon... boom! There's the first query! You can also see a pretty version of it, as well as a version you can run. If you have any variables inside WHERE clauses, the runnable version will fill those in for you.
Next: We normally don't write DQL by hand. Instead, we build it with the Query Builder. Let's see what that looks like.
25 Comments
If you're on the Symfony 6 track and feel like you're missing a course between "Doctrine, Symfony 6 & the Database" and this one, you are. It's this one: https://symfonycasts.com/screencast/doctrine-relations
Hey Ryan-L,
Yeah, unfortunately that course is not re-written with the latest Symfony 6 yet so that's why it's missing in the Symfony 6 track. Thanks for this reference, might be useful indeed :)
Cheers!
I believe Rufnex is referring to the project's folder and files icons. I was having the same question.
But I found it's the Atom Material Icons plugin, which can be installed via the Marketplace.
yeah, you found a correct package =)
Hey there! forget about my stupid question earlier about the database. OF COURSE should i run a make:migration and then migrate
duh!
sorry sorry sorry - do not publish my post please ;)
Hey Pieter,
No problem! It's still might be useful for some users ;) But I'm happy to hear you were able to figure out the steps yourself. If you ever have any problems following our tutorials - feel free to leave a comment and our support team will help you :)
Cheers!
Hi,
I dived right into this course, downloaded the code, downgraded to php 8.1, did the composer install and think i have everything rolling but then... the database! Where can I find an SQL dump with the tables/data from this course at start?
Hey Pieter,
When you download and unzip the course code, you will find the README file inside. Follow the instructions there to start the project locally :) Specifically for this course, we do recommend to use Docker for the DB:
Read the README for more details. But if you don't want to use Docker - you can run those commands directly in your CLI if you have MySQL installed, just make sure you update the DB credentials in your
.env.local. :)But if you still have any problems starting the project - let us know!
Cheers!
I downloaded the course code and follower the instructions in the README, but my site looks like this screenshot right now. What do I need to do to fix it?
Here is the link to my GitHub repository, if that's useful.
Hey Alex,
Seems you have problems with assets. Did you have
composer installcommand finished without any errors?Also, could you try to run the next commands:
Do you see any errors in the output or did they finish OK? Please, reload your website again, does it help?
Also, do you use
symfony servecommand to spin your website or did you configure a real web server like Apache/Nginx? Could you try to run the website viasymfony serve?Also, you can open Chrome dev tools and in the Network tab watch for the errors regarding the assets, it may contain some hints.
Cheers!
I downloaded course files, but there are no migrations.
@Victor @MolloKhan Guys, can you fix that?
Hey @Jared
Could you tell me what migrations are missing? I don't see any migrations being generated in this tutorial. You only need to create the database and then run
bin/console doctrine:schema:update- check out the README fileLet me know if I'm missing something. Cheers!
Thank you so much, @MolloKhan ! I fixed it.
For now I encountered a new problem: my assets won't show up. I ran
bin/console assets:install, and thebuilddirectory has appeared inpublic, but files likeapp.css, app.jsreturn a404error.Yea, this course uses Webpack instead of Asset Mapper. You need to run
yarn install(ornpm install) and thenyarn encore devCheers!
Doesn't work :( Still got messages in the chrome console:
GET http://localhost/build/app.css net::ERR_ABORTED 404 (Not Found)I run the following commands:
yarn installandyarn encore devA
builddirectory has appeared, files are present in thepublicdirectory, but I'm still getting404errors.I got it. The issue was with the Docker containers: NGINX and PHP-FPM. NGINX uses
fastcgi_passto redirect PHP files to PHP-FPM, but the assets from the build directory are not present in the NGINX container.Thank you @MolloKhan , problem is solved!
Interesting that configs for nginx and php-fpm work like a charm for Symfony 7, but not for Symfony 6.2.
I mean assets, because I had no problems in version 7.
Ha! That's interesting. I suppose they improved Docker integration in Symfony 7
Cheers!
Don't get the project started on Mac with Docker.
"composer install" and "docker compose up -d" finished without errors, but "symfony console doctrine:database:create --if-not-exists" throws:
[critical] Error thrown while running command "doctrine:database:create --if-not-exists". Message: "An exception occurred in the driver: SQLSTATE[HY000] [2002] Connection refused"
I get the exact same error message was this resolved?
Hey Iqpal,
I bet it was, it's a connection problem which means you most probably have invalid credentials configured. Have you followed my advice in the comment below? https://symfonycasts.com/screencast/doctrine-queries/dql#comment-31434
Cheers!
Hey Thilo,
First of all, make sure you install all the dependencies, i.e. Composer install should finish successfully. The error about the DB sounds like DB server is down. Probably try to restart the Docker completely, run the
docker compose upagain and make sure no errors. Let us know if you still have the same DB errorCheers!
Hey, side question ;o) What phpstrom UI do you use? Greetings.
Hey Rufnex,
Are you talking about PhpStorm theme? If so, we use Darcula in our videos :)
Cheers!
"Houston: no signs of life"
Start the conversation!