Worker Command

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $12.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

Even if I refresh the page, now that our messages aren't being handled immediately... the four most recent photos don't have Ponka in them. That's tragic! Instead, those messages were sent to the doctrine transport and are waiting patiently inside of a messenger_messages table.

So... how can we read these back out and process them? We need something that can fetch each row one-by-one, deserialize each message back into PHP, then pass it to the message bus to be actually handled. That "thing" is a special console command. Run:

php bin/console messenger:consume

You won't see any output... yet... but, unless we messed something up, this is doing exactly what we need: reading each message, deserializing it, and sending it back to the bus for handling.

So... let's go refresh. Woh! It did work! All 4 messages now have Ponka on them! We're saved!

messenger:consume -vv

To make this more interesting, as you can see, it says to run this command with -vv if you want to see what it's doing behind-the-scenes. But... interesting, once the command finished reading and handling all 4 messages... it didn't quit: it's still running. And if we restart it with -vv on the end:

php bin/console messenger:consume -vv

... it does the same. A command that "handles" messages from a queue is called a "worker". And the job of a worker is to watch and wait for new messages to be added to the queue and handle them the instant one is added. It waits and runs... forever! Well, that's not totally true - but more on that later when we talk about deployment.

Let's peek back over in our "queue" - the messenger_messages table:

SELECT * FROM messenger_messages \G

Yep! This holds zero rows because all those messages were processed and removed from the queue. Back at the browser, let's upload... how about... 5 new photos. Woh... that was awesome fast!

Ok, ok, move back to the terminal that's running the worker! We can see it doing its job! It says: "Received message", "Message handled by AddPonkaToImageHandler" then "AddPonkaToImage was handled successfully (acknowledging)". That last part, "acknowledging" means that Messenger notified the Doctrine transport that the message was handled and can be removed from the queue.

Then... it keeps going to the next message... and the next... and the next... until it's done. So if we refresh... Ponka was added to all of these! Let's do it again - upload 5 more photos. And... let's refresh and watch... there's Ponka! We can see them being handled little-by-little. So much wonderful Ponka!

Ok, this would be cooler if our JavaScript automatically refreshed the image when Ponka was added... instead of me needing to refresh the page... but that's a totally different topic, and one that's covered by the Mercure component in Symfony.

And... that's it! This messenger:consume command is something that you'll have running on production all the time. Heck, you might decide to run multiple worker processes. Or, you could even deploy your app to a totally different server - one that's not handling web requests - and run the worker processes there! Then, handling these messages wouldn't use any resources from your web server. We'll talk more about deployment later.

Problem: Database Didn't Update?

Because right now... we have a problem... a kinda weird problem. Refresh the page. Hmm, the original photos all say something like:

Ponka visited 13 minutes ago. Ponka visited 11 minutes ago.

But, since we made things asynchronous, these all say:

Ponka is napping. Check back soon.

Open up the ImagePost entity and find the $ponkaAddedAt property. This is a datetime field, which records when Ponka was added to the photo. The message on the front-end comes from this value.

For the original ones... back when the whole process was synchronous, this field was set successfully. But now... it looks like it isn't. Let's check the database to be sure. Over in MySQL, run:

SELECT * FROM image_post \G

All the way back in the beginning... ponka_added_at was being set. But now they're all null. So... our images are being processed correctly, but, for some reason, this field in the database is not. If we look inside AddPonkaToImageHandler... yea... right here: $imagePost->markPonkaAsAdded(). That sets the property. So... why isn't it saving?

Let's figure out what's going on next and learn a bit more about some "best practices" when it comes to building your message class.

Leave a comment!

  • 2020-05-08 weaverryan

    Hey Ahmedbhs!

    Ah yes, definitely! It's kind of a "different topic", which is why we didn't cover it here :). On a high level, you need your frontend JavaScript to be able to get a "notification" from the server that something changed so that it can update the DOM in some way. There are various ways to do this, but the recommended way in our world for sure is Mercure: https://github.com/symfony/...

    Unfortunately, we don't have a tutorial on it yet - it's highly asked for. But I hope this will help.

    Cheers!

  • 2020-05-05 Ahmedbhs

    Is it possible to not refresh the page every Time, Is there any way to get to change the image on the front automatically ?

  • 2020-04-09 Diego Aguiar

    That's a nice edge case! You may want to report it to PHPStorm website :)

  • 2020-04-09 Daria Motorina

    No, the problem is in running worker from Docker inside PHPStorm terminal :) Updated my first post in discussion.

  • 2020-04-08 Diego Aguiar

    Ohh, so the problem was that it was running out of memory when trying to print errors?

  • 2020-04-08 Daria Motorina

    Hi Diego :)
    I've created DummyMessage - yes, the problem with freezing is the same. Output is freezing even in non-verbose mode when error starts to display in console.
    Updated response: well, I've managed to clone my application from docker to host machine with php 7.4.4. and problem with freezing output is not reproducing. I don't even know - in docker script has 2048 RAM to consume, more than enough, logs are clear...

  • 2020-04-08 Diego Aguiar

    Hey Daria Motorina

    That's weird :)
    Does it only happens when verbose output is enabled?
    What happens if you try to consume a different message? (Maybe a dummy one)

  • 2020-04-07 Daria Motorina

    Hello! I have strange problem with verbose output in messenger:consume command. In video everything works fine at 2:20, but my output freezes at first INFO message no matter what. Example:
    root@7ea27b7cb7c7:/app# php bin/console messenger:consume -vv


    [OK] Consuming messages from transports "async".


    // The worker will automatically exit once it has received a stop signal via the messenger:stop-workers command.

    // Quit the worker with CONTROL-C.

    22:22:28 INFO [messenger] Received message App\Message\AddPonkaToImage ["message" => App\Message\AddPonkaToImage

    I use Docker with php 7.3.6, symfony/messenger 4.3.9.
    There is no error messages in var/log/dev.log (only debug and info), messages are processed successfully.
    Other console commands work fine.
    Even strace command didn't show me any suspicious behavior.

    Do you have any recommendations how to fix or analyze this problem?

    UPD. Thanks to Diego, I analyzed my situation more deeply. The problem is in PhpStorm+Docker. Hope it will help someone in the future :)

  • 2020-02-11 Ahmed Wali

    Thank you weaverryan
    The issue came from timezone i change it and it works

  • 2020-02-10 weaverryan

    Hey Ahmed Wali!

    Hmm, it sounds like a possible timezone issue... though I'm not sure. Your server is responsible for both *setting* the original available_at dates AND for setting which values to look for via the queries. So even if your server and database are on different timezones, I would *expect* that it would make no difference. However, I still think this is probably the issue (and I'm missing some detail). What timezone is your server and database in?

    Cheers!

  • 2020-02-10 Victor Bocharsky

    Hey Ahmed,

    Well done! I'm glad you found the problem yourself! And thanks for letting us know you was able to handle it!

    Cheers!

  • 2020-02-10 Ahmed Wali

    When i upload project to my server ./bin/console messenger:consume -vv
    It needs 3 hours to start handle the messages

    But when i change created_at and available_at to the past 3 hours ago and run cli again it works immediately

    What cause this issue

  • 2020-02-08 Ahmed Wali

    i solved the issue it's in .env.local MESSENGER_TRANSPORT_DSN didn't configured well

  • 2020-02-08 Ahmed Wali

    Hi,
    when it try to run this command ./bin/console messenger:consume -vv
    it's working well in Sy 4
    but
    when i using Sy 5
    it didn't work even though i found messages in database

    i realized log file return different Datetime formate between my Sy 4 and Sy 5:
    Sy 4
    SELECT m.* FROM messenger_messages m WHERE (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) AND (m.queue_name = ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE ["2020-02-01 23:26:53","2020-02-02 00:26:53","default"]
    Sy 5
    SELECT m.* FROM messenger_messages m WHERE (m.delivered_at is null OR m.delivered_at < ?) AND (m.available_at <= ?) AND (m.queue_name = ?) ORDER BY available_at ASC LIMIT 1 FOR UPDATE ["2020-02-08T17:55:23+01:00","2020-02-08T18:55:23+01:00","default"]

    Mysql version: 5.7
    How i can fix it?

  • 2019-12-16 Gilles Hoarau

    Hello Ryan,
    Thanks, I do NOT let you know this was indeed the problem :)

    Learning everyday! Thanks

  • 2019-12-13 weaverryan

    Hey Gilles Hoarau!

    Check that you have this section in your config/packages/dev/monolog.yaml file: https://github.com/symfony/...

    This is what controls outputting logs to stdout when you're using -v. If that is NOT the problem, let me know :).

    Cheers!

  • 2019-12-12 Gilles Hoarau

    Hello Victor,

    Thanks, I do try what you suggest: the following command in dev after clearing the cache. Still, it does not display anything, but messenger is doing its job in the background. Symfony version is @4.3.8.

    php ./bin/console messenger:consume -vv async_priority_high async async_priority_low

    I should try on another environment and another project to spot the error.

  • 2019-11-23 Victor Bocharsky

    Hey Gilles,

    Do you run this command manually? Could you write the exact command you're running? Also, do you run this command in prod or dev environment? Could you try to run the command in the dev env? And please, make sure the cache is cleared. Still no output?

    Cheers!

  • 2019-11-19 Gilles Hoarau

    Hi !
    I do not see any output to the consume command.
    I tried -v, -vv, -vvv with no results. I can see in the database that messages are inserted then deleted, so it works well, but still no output of the worker command.
    I use Symfony 4.3, php 7.2, the monolog bundle 3.5.0 and launch the command on ubuntu.

    Any idea?

  • 2019-07-19 Peter Kosak

    Big thumb up for choosing Vue.js

  • 2019-07-18 Diego Aguiar

    Aha! Excellent, at least it wasn't something bizarre. Cheers!

  • 2019-07-18 Sym

    Was using an outdated version of symfony/messenger. Didn't realise the transport wasnt introduced until 4.3.

  • 2019-07-18 Sym

    Thank you for the quick response. I have symfony/orm-pack installed and the following bundles relating to doctrine:


    ...

    Doctrine\Bundle\DoctrineCacheBundle\DoctrineCacheBundle::class => ['all' => true],
    Doctrine\Bundle\DoctrineBundle\DoctrineBundle::class => ['all' => true],
    Doctrine\Bundle\MigrationsBundle\DoctrineMigrationsBundle::class => ['all' => true],
    Stof\DoctrineExtensionsBundle\StofDoctrineExtensionsBundle::class => ['all' => true],
    ...

    Also, Doctrine is configured in the .env

  • 2019-07-17 Diego Aguiar

    Hey @Sym

    If you have Doctrine installed on your project it should just work out of the box. Double check that you are using Doctrine on your project and if that's the case and the problem persists, please, let us know :)

    Cheers!

  • 2019-07-17 Sym

    Hi,

    I'm receiving "No transport supports the given DSN "doctrine://default"" error after setting up my handler and setting the .env key.

    Any ideas?