docker-compose & Exposed Ports
We need to get a database running: MySQL, Postgresql, whatever. If you already have one running, awesome! All you need to do is copy your DATABASE_URL environment variable, open or create a .env.local file, paste, then change it to match whatever your local setup is using. If you decide to do this, feel free to skip ahead to the end of chapter 4 where we configure the server_version.
Docker Just for the Database
For me, I do not have a database running locally on my system... and I'm not going to install one. Instead, I want to use Docker. And, we're going to use Docker in an interesting way. I do have PHP installed locally:
php -v
So I won't use Docker to create a container specifically for PHP. Instead I'm going to use Docker simply to help boot up any services my app needs locally. And right now, I need a database service. Thanks to some magic between Docker and the Symfony binary, this is going to be super easy.
To start, remember when the Doctrine recipe asked us if we wanted Docker configuration? Because we said yes, the recipe gave us docker-compose.yml and docker-compose.override.yml files. When Docker boots, it will read both of these... and they're split into two pieces just in case you want to also use Docker to deploy to production. But we're not going to worry about that: we just want to use Docker to make life easier for local development.
| version: '3' | |
| services: | |
| ###> doctrine/doctrine-bundle ### | |
| database: | |
| image: postgres:${POSTGRES_VERSION:-13}-alpine | |
| environment: | |
| POSTGRES_DB: ${POSTGRES_DB:-app} | |
| # You should definitely change the password in production | |
| POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe} | |
| POSTGRES_USER: ${POSTGRES_USER:-symfony} | |
| volumes: | |
| - db-data:/var/lib/postgresql/data:rw | |
| // ... lines 14 - 22 |
| version: '3' | |
| services: | |
| ###> doctrine/doctrine-bundle ### | |
| database: | |
| ports: | |
| - "5432" | |
| // ... lines 8 - 9 |
These files say that they will boot a single Postgres database container with a user called symfony and password ChangeMe:
Tip
The username changed from symfony to app in the newest recipe version.
It will also expose port 5432 of the container - that's Postgres's normal port - to our host machine on a random port. This means that we're going to be able to talk to the Postgresql Docker container as if it were running on our local machine... as long as we know the random port that Docker chose. We'll see how that works in a minute.
By the way, if you want to use MySQL instead of Postgres, you absolutely can. Feel free to update these files... or delete both of them and run:
php bin/console make:docker:database
to generate a new compose file for MySQL or MariaDB. I'm going to stick with Postgres because it's awesome.
At this point, we're going to start Docker and learn a bit about how to communicate with the database that lives inside. If you're pretty comfortable with Docker, feel free to skip to the next chapter.
Starting the Container
Anyways, let's get our container running. First, make sure you have Docker actually installed on your machine: I won't show that because it varies by operating system. Then, find your terminal and run:
docker-compose up -d
The -d means "run in the background as a daemon". The first time you run this, it'll probably download a bunch of stuff. But eventually, our container should start!
Communicating with the Container
Cool! But now what? How can we talk to the container? Run a command called:
docker-compose ps
This shows info about all the containers currently running... just one for us. The really important thing is that port 5432 in the container is connected to port 50700 on my host machine. This means that if we talk to this port, we will actually be talking to that Postgres database. Oh, and this port is random: it'll be different on your machine... and it'll even change each time we stop and start our container. More on that soon.
But now that we know about port 50700, we can use that to connect to the database. For example, because I'm using Postgres, I could run:
psql --user=symfony --port=50700 --host=127.0.0.1 --password app
That means: connect to Postgres at 127.0.0.1 port 50700 using user symfony and talking to the app database. All of this is configured in the docker-compose.yml file. Copy the ChangeMe password because that last flag tells Postgres to ask for that password. Paste and... we're in!
If you're using MySQL, we can do this same thing with a mysql command.
But, this only works if we have that psql command installed on our local machine. So let's try a different command. Run:
docker-compose ps
again. The container is called database, which comes from our docker-compose.yml file. So we can change the previous command to:
docker-compose exec database psql --username symfony --password app
This time, we're executing the psql command inside the container, so we don't need to install it locally. Type ChangeMe for the password and... we're back in!
The point is: just by running docker-compose up, we have a Postgres database container that we can talk to!
Stopping the Container
Btw, when you're ready to stop the container later, you can run:
docker-compose stop
That basically turns the container off. Or you can run the more common:
docker-compose down
which turns off the containers and removes them. To start back up, it's the same:
docker-compose up -d
But notice that when we run docker-compose ps again, the port on my host machine is a different random port! So, in theory, we could configure the DATABASE_URL variable to point to our Postgres database, including using the correct port. But that random port that keeps changing is going to be annoying!
Fortunately, there's a trick for this! It turns our, our app is already configured, without us doing anything! That's next.
45 Comments
Strongly recommend to drop obsolete
docker-composeand switch todocker compose.Legacy
docker-composeis solo program written in Python and on some OS has to be installed separately.docker composecommand is now part of standard docker binary, is fully compatible with old Python tool and it is surprisingly for quite a long time a standard.https://stackoverflow.com/questions/66514436/difference-between-docker-compose-and-docker-compose
Hey JaroslavTyc,
Thanks for this tip! Yeah,
docker composeis the command you should use these days.Cheers!
For those who use dunglas/symfony-docker, the recipes add few lines inside Dockerfile, so you need to run
docker compose buildone more time to start working with the database, otherwise the exception will occur:An exception occurred in the driver: could not find driverwhen trying to rundoctrine:database:create.Hey droptica_team,
Thank you for sharing this tip with others!
Cheers!
I unfortunately also have a problem connecting. In docker-compose.yml a have the following setup:
Now, when I run the command
psql --user=symfony --port=49153 --host=127.0.0.1 --password appand give the password, I get the error:However, when I use the command
docker-compose exec database psql --user app --password appand give the password, I do get in which makes no sense to me. Could you please demystify?28/08/2024 Update:
Thanks for this tip!
Hello odds. I'm encountering the exact same problem. Did you find a solution ?
by the way,
symfony console doctrine:database:creategives the following error:Hi there!
Sorry for the trouble, but I think I know what the problem is. Since a few weeks ago, the Doctrine recipe comes with Pgsql 14 instead of 13. That's actually fine. However, it's possible that you have an existing volume from running Docker earlier that had a pgsql "data" directory for pgsql 13. And this is causing problem with pgsql 14 operating properly.
This is what happened to me, and it gave me the same error that you're all getting. To see the error, try to start docker without the
-d:This gave me a very clear error:
So, where did this "volume" come from? With the
docker-compose.ymlfile, when you start a database, it creates a volume whose name is generated from the name of the directory you're inside. If you download the course code and move into thestartdirectory, then your directory name isstart. This will cause a volume calledstart_db-datato be created, which stores your database data.You can see all your volumes by running:
So, if you're like me, the problem is caused by using Docker with pgsql 13 previously inside a directory named
start. What's the fix? There are two:1) Delete the old volume - this will only work if that docker container is no longer running:
docker volume rm start_db-data2) Rename the directory you're inside of.
Let me know if that helps!
Cheers!
That's unfortunately not my problem. My version is 14 on both, but it somehow did not make a role:
Now, when I log in with
docker-compose execand try andCREATE ROLE appit says:I`m guessing this has nothing to do with Symfony but more so with either PostgreSQL and/or Docker. If you could point me in the right direction, I would be greatfull.
No clue what's happening, but I downgraded the Postgres_version in docker-compose.yml from 14 to 13 (since that's the version on my production server). Although my local psql is on 14 and the server is now on 13, it seems to work, so far...
Super weird - thanks for the update. If you notice anything else strange, I'd love to know.
Hello @weaverryan
I do not get the same error as your's when I run
docker-compose upHere is what I get :
seems to be logging so we can see the failed connexion attempt
bye the way, here is the content of the
serviceskey in my docker-compose.yml :Hmm, does renaming your directory to something else help? I'm still suspect that there is an existing volume using pgsql 13 that is causing problems. Also, I can't think of why this would make a difference, but oddly your
POSTGRES_PASSWORDis slightly different than the one that comes with the recipe: https://github.com/symfony/recipes/blob/main/doctrine/doctrine-bundle/2.4/manifest.json#L31Cheers!
Hello @weaverryan.
I just started a new symfony project with
symfony newand run thecomposer require doctrinecommande right after, to put your theory to the test.In this new project, everything seems to work perfectly :
I can connect to the database in the container with
psql --user=app --port=56899 --host=127.0.0.1 --password appand the
symfony console doctrine:database:createyells at me something about a database "app" thas already exists... ;)I'm glad to start this new week with Symfony, especially with this problem solved.
Thanks a lot for your help, and for all these useful (and fun) videos.
You rock, have a nice week.
PS : renaming the folder of the mixed_vinyl project works, too
Hey Xesh! Thank you for the update! This tells me that, at least some people had the same issue as me :).
Have a great week!
Hi, I run localhost:8000 for the first time and receive the following error message:
Environment variable not found: "GITHUB_TOKEN".
Tips in case of error:
this command takes 'app' instead of
symfonydocker-compose exec database psql --user **app** --password appalso, change the pwd
!ChangeMe!toChangeMein .env fileHey FriendlyDev,
Thank for this tip! Yeah, you need to use your actual DB credentials you have set up in the .env files
Cheers!
Hello, I am running my symfony local web server with this integration docker. However, even though my docker-compose services are running, I can not get symfony to connect to them.
The debug toolbar says Docker Compose Down under the sf Server menu.
From what I see I don't get any errors regarding this
Any thoughts? If there is some more information I can provide please let me know.
Thanks.
Hey Def,
Hoe do you run your Symfony app? You should not use
bin/consolecalls anymore if you're going to use Docker, you would need to replace all your commands withsymfony consoleinstead - that will handle all the Docker integration for you :)So, e.g. instead of
bin/console doctrine:query:executerunsymfony console doctrine:query:execute, etc.If still does not work - try to "down" all your containers. i.e. run
docker composer downand then up them again withdocker composer up- there might be a misconfiguration etc.Cheers!
Thx Victor for your help
Unfortunatly, it seems like it's a biggest issue with my docker conf because i tried to install a whole new symfony project with a new docker-compose configuration and when i start my local webserver on this new project and up my containers I also have a docker compose down and "Env vars" none, so... yeah
I don't know what to do but thanks
Hey Def,
Hm, not sure then, I don't use Docker daily :) Maybe try to go through Docker docs to reinstall and configure it from scratch, maybe something is missing. Actually, check for updates first, maybe you just need to upgrade to the latest. Btw, laptop restart may help sometimes too.
Cheers!
Just quick note from my journey: if you change the password in .env file and docker-compose.yml before running docker and creating container for the first time, it's all good, but if you changed it later you will have an error password authentication failed for user... if you want to use new password you have to basically delete container and create new one, but you can also return to old password, which you used during creation of db.
Hey Stanislaw,
Thanks for this tip! Yeah, you need to run
docker compose downto drop all the created containers and start them again withdocker compose up.Cheers!
You have to add your user to group "docker". Otherwise you will run into an error.
See: https://stackoverflow.com/questions/64662372/docker-compose-up-error-while-fetching-server-api-version-connection-aborte
sudo gpasswd -a $USER dockerHey,
Thanks for the tip. Can you please share your OS version? Probably it's important because we all use different systems =)
Cheers and happy coding!
Hi. I'm getting invalid compose project
These are the docker apps I'm running.
I'm using the code downloaded from this page, renamed the
startdirectory tomixed_vinylIf I change
db-dataon line 13 of docker-compose.yml to database, the error message is updatedsymfony serve -dserves https://127.0.0.1:8000/ without a problem.Any thoughts? Thanks.
Hey Eric!
Apologies for my very slow reply! Holidays started early this year at my house :p.
Ok, this sounds frustrating :/. So, I CAN repeat your error message, but only if I change
docker-compose.ymlin a specific way. Here is an abbreviated version of mydocker-compose.ymlfile, which came (like in the tutorial) from the DoctrineBundle recipe after installing Doctrine:With this config (well, again, I'm hiding less-important parts, but hopefully you get the idea), things work. How can I repeat your error? By removing the
volumessection. In other words, it seem like, in your config, thevolumessection is missing... or perhaps it's called something different thandb-data. Basically, because we have- db-data:...above, we should have a volume calleddb-data:near the bottom.Let me know if that helps! For reference, here is my full, unchanged,
docker-compose.ymlfile after installing Doctrine (this version works for me):Cheers!
Hi Ryan,
Yes, having
volumes:with a key that matches thevolumes:in services.database.volumes helped. But my problem was I've been usingcolimaas a docker context. I usually run a .ddev/colima with mutagen stack, which works great for Drupal and avoids the subscription service for Docker desktop as well as the Docker desktop app.Once, I changed the docker context back to default (which means docker) and started the docker desktop app (bleh), the docker daemon was started and the symfony app found the database through doctrine right away.
My next questions are out of scope, but do you know how to run the docker daemon without the docker desktop app?
And can Colima be used as a replacement for Docker in Symfony?
Thanks. Enjoy the rest of the holidays.
Hey EricSod!
Ah, ok, this makes more sense then!
Tbh, I'm not familiar with Colima, though it looks super interesting. Unfortunately, because I'm a bit clueless about it (other than reading about it just now for about 10 minutes), I don't know why you had the
volumesproblem when using it. But, as far as Symfony + Docker + Database is concerned thevolumespart isn't important. What's important is that yourdocker-compose.yamlhas a service calleddatabase. So, the whole solution may still be workable, but I don't know enough to say how.I'm a bit clueless here too :). But, from my reading, it sounds like one of the points of colima is to be able to run Docker without needing Docker desktop - but just by running
brew install docker? I have a feeling you already knew that, but let me know.I can't say for certain, but the
symfonybasically looks for adocker-compose.yamlfile, loops over the services inside, then (using the name of each service) it creates some environment variables that point to that service. If you satisfy both of these requirements - regardless of it's all setup, then it should all work. But also, keep in mind that this setup is just an "easy" way to get a database setup in Docker and hooked up with your Symfony app. It should be making your life easier, not harder. If it's not doing that, then don't use it (and that's totally ok!).Cheers!
With the docker desktop app running, set the docker default to colima with
and run
The prompt returns
Symfony profiler reports Server -> Docker Compose -> Down.
Switch the docker default to docker with
run
docker-compose up -dand confirm the Symfony profiler is reporting Docker Compose -> Up.Note: Quit the Docker Desktop app and Symfony profiler reports Server -> Docker Compose -> Down.
Sounds like two problems now. Yay DevOps. :-)
Anyway, the recipe for success is what you're recommending in the tutorial.
Hey EricSod!
Thanks for the detailed reply - and boo when things don't work more easily! :P Happy you're at least not blocked.
Cheers!
Hello, i have a problem when i run this command
psql --user=app --port=52439 --host=127.0.0.1 --password app, the result is :Docker is running on Windows 11, if i execute
docker-compose ps, the result is :What is doing to resolve this mistake ?
Thanks,
Laurent
Hey Laurent-G,
It seems like you're missing that
psqlcommand installed locally, though the error message is a bit weird to me. Do you havepsqlinstalled locally? You should have it installed to be able to connect to the DB from Docker. On my Mac when I executepsql --versionit shows mepsql (PostgreSQL) 14.5 (Homebrew), IIRC I installed the PostgreSQL via Brew package manager locally and it brought me thatpsqlcommand. If you don't have PostgreSQL installed locally - try to install it first. Butbrewis specific for Mac, you should find a way to install PostgreSQL on Windows.Of if you don't want to (or cannot) install it locally - see the next chapter where we execute that
psqlcommand directly from the Docker container.Cheers!
Thanks Victor for your answer.
i don't have installed psql locally. I'm going to see for install psql on my windows system.
i already saw in the next chapter the command to execute psql from Docker container and it's work very good.
Thanks a lot,
Laurent
Hey Laurent-G,
Glad that executing it from the Docker container works for you, thanks for confirming! I bet you just need to install PostgreSQL and it will bring that
psqlcommand into your system automatically. Or you can just ignore it and execute it directly from the Docker container - that's the benefit of having Docker ;)Cheers!
Is there a version of this with a mysql container. Tried using Postgres's but getting data from an existing mysql database to the Postgres container is too hard
figured it out (lost a lot of time because of the stupid ! at the start of the !ChangeMe! password......)
Hey Red,
I'm happy to see you were able to find the solution! And thanks for sharing it with others :)
Cheers!
Hi, running command "docker-compose exec database psql --username symfony --password app" I'm prompted for a password, which I enter (to match the one in the docker compose file which is !ChangeMe! but then... it just hangs. Cursor flashes, but no response at all. I can connect to the database successfully using DBeaver (a GUI database client), it connects and can see the database fine. This is what I see at the terminal:
`$ docker-compose exec database psql --username app --password app
Password: !ChangeMe!
`
Hey Mark,
Hm, could you change that "!ChangeMe!" password in .env files to something without special chars, I guess that the exclamation char (!) may cause issues. And then restart the server and try to connect with the new credentials.
Cheers!
Hi and thanks for this new tutorial!
I've an error when I run
docker-compose exec database psql --username symfony --password app:psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: role "symfony" does not existAm I missing something?
OS: Linux Mint 20.3 Cinnamon 5.2.7
Kernel: 5.4.0-124-generic
Docker version 20.10.17, build 100c70
docker-compose version 1.28.6, build 5db8d86f
Hey Thomas A.!
Oh sheesh - try this instead - changing "symfony" to "app":
Explanation: RIGHT after recording, the recipe was updated - you probably have the new version https://github.com/symfony/recipes/commit/b3395a2477b6d58089f92ef3a0d2e58226825a3a#diff-b52cb1d9a056076027dd6033aa8c573524746ed54fcdeeb5c8eef83431b9da9aL32
Let me know if that helps - we'll need to add a note about it :).
Cheers!
"Houston: no signs of life"
Start the conversation!