Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

Doing Work in the Handler

Video not working?

It looks like your browser may not support the H264 codec. If you're using Linux, try a different browser or try installing the gstreamer0.10-ffmpeg gstreamer0.10-plugins-good packages.

Thanks! This saves us from needing to use Flash or encode videos in multiple formats. And that let's us get back to making more videos :). But as always, please feel free to message us.

Inside our controller, after we save the new file to the filesystem, we're creating a new AddPonkaToImage object and dispatching it to the message bus... or technically the "command" bus... because we're currently using it as a command bus. The end result is that the bus calls the __invoke() method on our handler and passes it that object. Messenger understands the connection between the message object and handler thanks to the argument type-hint and this interface.

Command Bus: Beautifully Disappointing

By the way, you might be thinking:

Wait... the whole point of a "command" bus is to... just "call" this __invoke() method for me? Couldn't I just... ya know... call it myself and skip a layer?

And... yes! It's that simple! It should feel completely underwhelming at first!

But having that "layer", the "bus", in the middle gives us two nice things. First, out code is more decoupled: the code that creates the "command" - our controller in this case - doesn't know or care about our handler. It dispatches the message and moves on. And second, this simple change is going to allow us to execute handlers asynchronously. More on that soon.

Moving code into the Handler

Back to work: all the code to add Ponka to the image is still done inside our controller: this gets an updated version of the image with Ponka inside, another service actually saves the new image onto the filesystem, and this last bit - $imagePost->markAsPonkaAdded() - updates a date field on the entity. It's only a few lines of code... but that's a lot of work!

Copy all of this, remove it, and I'll take my comments out too. Paste all of that into the handler. Ok, no surprise, we have some undefined variables. $ponkaficator, $photoManager and $entityManager are all services.

... lines 1 - 7
class AddPonkaToImageHandler implements MessageHandlerInterface
{
public function __invoke(AddPonkaToImage $addPonkaToImage)
{
$updatedContents = $ponkaficator->ponkafy(
$photoManager->read($imagePost->getFilename())
);
$photoManager->update($imagePost->getFilename(), $updatedContents);
$imagePost->markAsPonkaAdded();
$entityManager->flush();
}
}

In the controller... on top, we were autowiring those services into the controller method. We don't need $ponkaficator anymore.

... lines 1 - 20
class ImagePostController extends AbstractController
{
... lines 23 - 37
public function create(Request $request, ValidatorInterface $validator, PhotoFileManager $photoManager, EntityManagerInterface $entityManager, MessageBusInterface $messageBus)
{
... lines 40 - 63
}
... lines 65 - 95
}

Anyways, how can we get those services in our handler? Here's the really cool thing: the "message" class - AddPonkaToImage is a simple, "model" class. It's kind of like an entity: it doesn't live in the container and we don't autowire it into our classes. If we need an AddPonkaToImage object, we say: new AddPonkaToImage(). If we decide to give that class any constructor arguments - more on that soon - we pass them right here.

But the handler classes are services. And that means we can use, good, old-fashioned dependency injection to get any services we need.

Add public function __construct() with, let's see here, PhotoPonkaficator $ponkaficator, PhotoFileManager $photoManager and... we need the entity manager: EntityManagerInterface $entityManager.

... lines 1 - 5
use App\Photo\PhotoFileManager;
use App\Photo\PhotoPonkaficator;
use Doctrine\ORM\EntityManagerInterface;
... lines 9 - 10
class AddPonkaToImageHandler implements MessageHandlerInterface
{
... lines 13 - 16
public function __construct(PhotoPonkaficator $ponkaficator, PhotoFileManager $photoManager, EntityManagerInterface $entityManager)
{
... lines 19 - 21
}
... lines 23 - 32
}

I'll hit Alt + Enter and select Initialize Fields to create those properties and set them.

... lines 1 - 10
class AddPonkaToImageHandler implements MessageHandlerInterface
{
private $ponkaficator;
private $photoManager;
private $entityManager;
public function __construct(PhotoPonkaficator $ponkaficator, PhotoFileManager $photoManager, EntityManagerInterface $entityManager)
{
$this->ponkaficator = $ponkaficator;
$this->photoManager = $photoManager;
$this->entityManager = $entityManager;
}
... lines 23 - 32
}

Now... let's use them: $this->ponkaficator, $this->photoManager, $this->photoManager again... and $this->entityManager.

... lines 1 - 10
class AddPonkaToImageHandler implements MessageHandlerInterface
{
... lines 13 - 23
public function __invoke(AddPonkaToImage $addPonkaToImage)
{
$updatedContents = $this->ponkaficator->ponkafy(
$this->photoManager->read($imagePost->getFilename())
);
$this->photoManager->update($imagePost->getFilename(), $updatedContents);
... line 30
$this->entityManager->flush();
}
}

Message Class Data

Nice! This leaves us with just one undefined variable: the actual $imagePost that we need to add Ponka to. Let's see... in the controller, we create this ImagePost entity object... which is pretty simple: it holds the filename on the filesystem... and a few other minor pieces of info. This is what we store in the database.

Back in AddPonkaToImageHandler, at a high level, this class needs to know which ImagePost it's supposed to be working on. How can we pass that information from the controller to the handler? By putting it on the message class! Remember, this is our class, and it can hold whatever data we want.

So now that we've discovered that our handler needs the ImagePost object, add a public function __construct() with one argument: ImagePost $imagePost. I'll do my usual Alt+Enter and select "Initialize fields" to create and set that property.

... lines 1 - 4
use App\Entity\ImagePost;
class AddPonkaToImage
{
private $imagePost;
public function __construct(ImagePost $imagePost)
{
$this->imagePost = $imagePost;
}
... lines 15 - 19
}

Down below, we'll need a way to read that property. Add a getter: public function getImagePost() with an ImagePost return type. Inside, return $this->imagePost.

... lines 1 - 6
class AddPonkaToImage
{
... lines 9 - 15
public function getImagePost(): ImagePost
{
return $this->imagePost;
}
}

And really... you can make this class look however you want: we could have made this a public property with no need for a constructor or getter. Or you could replace the constructor with a setImagePost(). This is the way I like to do it... but it doesn't matter: as long as it holds the data you want to pass to the handler... you're good!

Anyways, now we're dangerous! Back in ImagePostController, down here, AddPonkaToImage now needs an argument. Pass it $imagePost.

... lines 1 - 20
class ImagePostController extends AbstractController
{
... lines 23 - 37
public function create(Request $request, ValidatorInterface $validator, PhotoFileManager $photoManager, EntityManagerInterface $entityManager, MessageBusInterface $messageBus)
{
... lines 40 - 59
$message = new AddPonkaToImage($imagePost);
... lines 61 - 63
}
... lines 65 - 95
}

Then, over in the handler, finish this with $imagePost = $addPonkaToImage->getImagePost().

... lines 1 - 10
class AddPonkaToImageHandler implements MessageHandlerInterface
{
... lines 13 - 23
public function __invoke(AddPonkaToImage $addPonkaToImage)
{
$imagePost = $addPonkaToImage->getImagePost();
... lines 27 - 32
}
}

I love it! So that's the power of the message class: it really is like you're writing a message to someone that says:

I want you to do a task and here's all the information that you need to know to do that task.

Then, you hand that off to the message bus, it calls the handler, and the handler has all the info it needs to do that work. It's a simple... but really neat idea.

Let's make sure it all works: move over and refresh just to be safe. Upload a new image and... it still works!

Next: there's already one other job we can move to a command-handler system: deleting an image.

Leave a comment!

20
Login or Register to join the conversation
Default user avatar
Default user avatar Payam Naderi | posted 3 months ago

Hi,
is there any docker setup available for this project?

Reply
Musa Avatar

Hello, I have quite a unique problem that I pray there is a solution for.
Our services require bulk functionality for sending out messages to our customers, this naturally falls into the asynch world as sending out ~20/30k messages in one go is a bit much for both back & frontend.

What makes this problem unique is that we have multiple databases operating from the same codebase for different "users".

-------------------------------------------------------------------------------------------------------------

My problem is as follows:

Which database is used is determined by the URL, e.g https://ourdomain.com/{identifier}/action, this in turn has a listener that sets up the DB connection before any other code is run.

The asynch job message has access to this identifier, and I can use a wrapper for DB connection changing inside the message handler.

However, all the dependencies supplied by the DI container defaults to the predefined database connection (The one used if no identifier is supplied).

So even if I can access the right database with any objects created inside __invoke, any class dependencies provided by the constructor (such as repositories) have the default.

The only options I can think of is:

1. Create the needed objects at runtime inside __invoke.
However, a lot of dependencies are typehinted and injected with interfaces and this would quickly get out of hand.

2. Pass the needed objects to the message envelope.
However, I've tried this before, and this causes issues with doctrine. Since it didn't fetch the object/entity, it assumes that we are doing a create, not an update (https://symfonycasts.com/sc...

---------------------------------------------------------------------------------------------------------------------

Is there any way to tell an asynch job what database to use before it's called? It seems impossible since the constructor is always the first called method and asynch jobs are by nature disconnected.

While waiting for a response I will go with option 2 and see if I can make it work.
Any help is greatly appreciated.

Reply
Musa Avatar

Update
Ended up doing Option 1
Loaded all the dependencies that don't have any DB association via DI container and resolved the rest manually by instantiating new objects and feeding them to the classes that needed them, not pretty but it works.

Would still love a reply in case anyone knows a way around this :)

Reply

Hey David Beck!

So my guess is that you have an event listener on RequestEvent that normally looks at the domain name and does some work to "set up" which database to use. Is that... kind of correct? If so, then (as you know), that listener wouldn't run when your worker is executing a message. But could you accomplish this with a worker listener? I'm thinking about the WorkerMessageReceivedEvent. This is before your message is handled by the worker, but you DO have access to the message itself (and stamps) from this event. Could you read that and reconfigure things before your handler is called?

Btw, you'll probably want to activate the reset_on_message option - https://symfony.com/blog/ne... - so that the container starts "fresh" before each message is processed.

Let me know if this helps!

Cheers!

Reply
Musa Avatar

Hi Ryan, thanks for reaching out on such short notice, really grateful for your activity in this community!

A (event)subscriber is responsible for doing the database switching at runtime for incoming HTTP requests.

And you're completely right, our workers are oblivious to this behaviour.

I'm curious though, does the

WorkerMessageReceivedEvent

live in the same lifecycle as the actual job handler?

I've been thinking about this when reviewing your videos on stamps.

In my mind yesterday, albeit caffeine fuelled and under time pressure, the job handler is just called like any other PHP process on it's own thread (php /path/to/script.php 'Hello, ' 'World!') passing the serialized object(envelope) to the worker in the place of $argv(n).

But if the job life cycle starts with the first middleware in the line, ergo:
Job pushed to queue (HTTP stops here) -> (Internal process starts here) middleware ... n .... -> MessageHandler.
And not:
Job pushed to queue -> middleware ... n ... (HTTP stops here) -> (Internal process starts here) MessageHandler .
Then middleware should work just fine.

I'd love a confirmation on this behaviour, as it's a little hard to draw conclusions based on synchronous examples for messenger.

Reply

Hey David Beck!

Sorry for the slow reply this time :).

> I'm curious though, does the WorkerMessageReceivedEvent live in the same lifecycle as the actual job handler?

I'm not sure exactly what you mean. But I think the answer is "yes". That event is dispatched inside of the "worker process" and it is dispatched right before the handler would actually be called.

> But if the job life cycle starts with the first middleware in the line, ergo:
> Job pushed to queue (HTTP stops here) -> (Internal process starts here) middleware ... n .... -> MessageHandler.
> And not:
> Job pushed to queue -> middleware ... n ... (HTTP stops here) -> (Internal process starts here) MessageHandler .
> Then middleware should work just fine.

It's a mixture of both 😆:

Message dispatched -> middleware -> one middleware pushes to queue (HTTP stops here) -> (Internal process starts) -> Message is loaded from queue -> middleware -> one middleware calls your MessageHandler.

The middleware itself is actually what's responsible for both sending and handling messages. If you added a custom middleware, it should be called BOTH when the message is originally dispatched AND when it is received from the queue and about to be handled. You can check for a ReceivedStamp to see if it was just received.

Cheers!

Reply
Musa Avatar

No worries on the timing at all mate.
I realized that if I had continued on with watching the following videos, the answer was right there.
Thanks for taking the time to explain :)

This worked:


function __construct(ConnectionSwitcher $connectionSwitcher)
{
$this->connectionSwitcher = $connectionSwitcher;
}

public function handle(Envelope $envelope, StackInterface $stack): Envelope
{

if ($envelope->last(ReceivedStamp::class)) {

$message = $envelope->getMessage();

if ($message instanceof AsynchIdentifierMessageInterface) {

$Identifier= $message->getIdentifier();

$this->connectionSwitcher->setIdentifierForRequestLifeCycle($Identifier);
}
}

return $stack->next()->handle($envelope, $stack);
}

However I ran into another issue.

It seems that non-object class variables stay between jobs.
Let me clarify:

1. After the middleware changes my database, the MessageHandlers dependencies are injected and have the correct DB connection.
2. Non-object variables set by the constructor(on a dependency) are persistent across jobs, almost like the object is cached or treated as a cross-job singleton.

Example:

I have a dependency in one of my (Asynch)MessageHandler classes, this dependency handles logging. The logger takes a service and sets the current identifier and prepends it to all messages logged by the instance used.


class Logger {
private string $identifier;
private ExternalLoggingApiService $logger;

function __ construct(FooService $fooService, ExternalLoggingApiService $logger)
{
$this->identifier = $fooService->getIdentifier();
$this->logger = $logger;
}

public function Log(string $message)
{
$this->logger->log("[{$this->identifer}] $message")
}

Job #1 on identifier XXXX. Gets identifier XXXX from fooService and sets it as it's $identifier variable.
Job #2 on identifier YYYY. Ignores fetching identifier YYYY and uses XXXX

This became very apparent when I changed the logger's code to:


class Logger {
private string $identifier = "test";
private ExternalLoggingApiService $logger;
private FooService $fooService;

function __ construct(FooService $fooService, ExternalLoggingApiService $logger)
{
$this->fooService= $fooService;
$this->logger = $logger;
}

public function Log(string $message)
{
$this->identifier = $this->fooService->getIdentifier();
$this->logger->log("[{$this->identifier}] $message")
}

public function getIdentifer()
{
return $this->identifier;
}

I logged (in the MessageHandler) the jobs using the getIdentifier() method to see what was going on:

Job #1, getIdentifier() returns "test" because the function was called before Log(), actual identifier shown in log: XXXX
Job #2, getIdentifier() returns XXXX, actual identifier shown in log: YYYY

If the Logging class dependency was refreshed on each job, the getIdentifier() should always return "test" as the variable has not been reassigned before the actual Log() method is called.

Worth mentioning that calls to the database in Job #2 fetch from the correct database, so these dependencies are working as intended.

So I guess my question is:
Does symfony cache non-object object variables between jobs of the same kind?

I will remove the fetching of identifier from the constructor as it's not good practice anyhow, however,
I'd like to know more about this behaviour in case other dependencies share this behaviour.

1 Reply

Hey David Beck!

Yea, this is an interesting situation. Two things I'll say:

A) Yes, setting the identifier in the constructor is not a *super* great practice. Well, more broadly, making a service "stateful" - meaning it has some data that might change and holds onto some state / "current data" - is kind of an anti-practice. Services should be stateless: you get the same result every time you call methods on them, no matter what. But, that is just a general rule. I DO this type of stuff from time to time - it's useful ;).

B) About the "caching" that you're seeing. For each PHP process, Symfony instantiates a service just one time. During a normal request-response cycle, if you ask for your Logger service, it will be instantiated just once. The SAME thing happens when processing an async message. But in this case, the Logger service will be instantiated just once... even if you handle 5 messages. That's why you're seeing that behavior.

The solution is either to:

1) Not set the state on your service
or
2) Somehow re-set the state after each message or re-initialize it at the start of each message. Depending on your logic, you might be able to do this nicely. OR, iirc, if you make your service implement ResetInterface, then Symfony will call the reset() method each time it "resets" the container (this is not something that happens during a normal request-response process). You can (and probably should, it's just a good idea) force your container to be reset between each message with this new feature: https://symfony.com/blog/ne...

Let me know if this helps!

Cheers!

Reply
Julien R. Avatar
Julien R. Avatar Julien R. | posted 9 months ago

Hello,

I studied this chapter on the text version and it seems you have a typo in the paragraph https://symfonycasts.com/screencast/messenger/handler-work#message-class-data (message class data header) :

This paragraph serves 4 code snippet. The first 2 are currently expanded / collapsed on the exact same lines. But based on what you wrote right above the 2nd, I guess you wanted to expand L15 <--> L19 ? And maybe collapse L9 <---> L14 but as you wish.

Take a look at those snippets fix it if it deserves it, and feel free to delete my comment after !

Cheers !

Reply

Hey Nayte
You're 100% right! Thanks for letting us know. I'm going to fix it right now :)

Cheers!

Reply

If we have a NotificationSendHandler a handler that gonna send a mail to the correct user. The PHP code for sending a mail that is covered by the handler gonna allocated memory In the PHP server not Inside the broker rather rabbitmq. So either we exude asynchonsly or synchronously we consume the same memory, but arsenic is a better use for a server to handle a lot of demands.

Just a question : if we send a notification to 1000 subscribers what's the best way is it to or outside. Do we send one message contain all user to the handler and then make a loop Inside the handler:

public function __invoke(SendNotification $notification)
{
foreach ($message->getUsers() as $user) {
echo 'The mail has been sent with success !\r\n';
$message = (new \Swift_Message('Mail object'))
->setFrom('send@gmail.com')
->setTo($user->getEmail())
->setBody(
$notification->getMessage()
);
$this->mailer->send($message);

or we should have a message per notification and per user !! ?

Reply

Hey Ahmedbhs

> So either we exude asynchonsly or synchronously we consume the same memory, but arsenic is a better use for a server to handle a lot of demands.

If your worker lives in the same server as your application, then yes. When the message gets consume your server may suffer an overload but at least the initial request was already processed, in other words, your user got a faster UX

> if we send a notification to 1000 subscribers what's the best way is it to or outside. Do we send one message contain all user to the handler and then make a loop Inside the handler:

Massive email campaigns are hard to manage, usually is recommended to use a third party service for doing that but if you still want to do it. I would recommend to send like 5 or 10 emails per message

Cheers!

Reply

Hello,

After completing the code in this video, I get this JS error:

vue.esm.js:628 [Vue warn]: Error in v-on handler: "TypeError: Cannot read property 'find' of undefined" found in
---> <imageuploader> at assets/js/components/ImageUploader.vue
<imageapp> at assets/js/components/ImageApp.vue
<root>

I am not sure how to trouble shoot this. Can you help?

this is the JSON that is returned from /api/images:
{"id":7,"originalFilename":"818-banner.jpg","ponkaAddedAt":"2020-02-04T12:16:39-05:00","createdAt":"2020-02-04T12:16:37-05:00","@id":"\/api\/images\/7","url":"\/uploads\/images\/818-banner-5e39a6f5b44a0.jpeg"}

Reply

Hey Skylar,

Hm, let's try to debug it together... Do you have any spots where you call that "find" property? Basically, you can search for "find" text in your JS files. Most probably you just use incorrect object, etc.

Also, please, make sure you followed all the steps in README file of downloaded course code. In particular, make sure you ran "composer install", "yarn install" and "yarn encore dev" before starting working with project. Btw, did you download the course code and started from start/ directory?

Cheers!

Reply

Hi, Thank you for the reply. I traced the error to my system. When I use the Symfony Serve, the XHR response to the /image request is the symfony profiler information. Some how on my system it is not working correctly. When I turn the symfony profiler off, then the ponka app works fine. I have no idea how to fix it. I am using a windows 10 with WSL ubuntu linux. Some where that is causing the issue.

Thanks for your help.

Reply

Hey Skylar Scotlynn Gutman

Try running the web server without SSL. I would also check the Content-Type header of the response because if you are returning JSON, then it should be set to application/json. Let me know if it worked :)

Reply

Thank you Diego.

I solved the problem. First, I updated and upgraded my ubuntu install and upgraded to php 7.4. Then I updated my Symfony CLI file to the latest version. voila. It now works and is serving the correct content for the url requested.

Of course, there is always consequences. My Windows 10 /WSL/Ubuntu install does not support TCP_INFO and now the Symfony CLI app is logging "failed to retrieve TCP_INFO for socket: Protocol not available (92)" I don't even know where to look for Symfony CLI help. But I can just let it run as it is working. yeay!

Reply

Excellent!
You can report issues related to Symfony CLI here https://github.com/symfony/cli

Cheers!

Reply
Gustavo Avatar
Gustavo Avatar Gustavo | posted 2 years ago

Hello,
Is it okay if I have my Handler implement the ServiceSubscriberInterface to get the services (traits, specifically) from my Service folder?

Reply

Hey Gustavo!

Hmm. Typically, if you need a service, you should add a constructor and use dependency injection for each service you need. Service subscribers are a special feature that should generally only be used in some edge-cases when there are performance concerns... and these are rare :). Here's some info on this (in case you haven't seen it): https://symfonycasts.com/sc...

So generally... I would kinda say "no" you shouldn't do this. But if you think you have a special case, let me know.

Cheers!

1 Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial is built with Symfony 4.3, but will work well on Symfony 4.4 or 5.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/annotations": "^1.0", // v1.8.0
        "doctrine/doctrine-bundle": "^1.6.10", // 1.11.2
        "doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
        "doctrine/orm": "^2.5.11", // v2.6.3
        "intervention/image": "^2.4", // 2.4.2
        "league/flysystem-bundle": "^1.0", // 1.1.0
        "phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.1
        "sensio/framework-extra-bundle": "^5.3", // v5.3.1
        "symfony/console": "4.3.*", // v4.3.2
        "symfony/dotenv": "4.3.*", // v4.3.2
        "symfony/flex": "^1.9", // v1.18.7
        "symfony/framework-bundle": "4.3.*", // v4.3.2
        "symfony/messenger": "4.3.*", // v4.3.4
        "symfony/property-access": "4.3.*", // v4.3.2
        "symfony/property-info": "4.3.*", // v4.3.2
        "symfony/serializer": "4.3.*", // v4.3.2
        "symfony/validator": "4.3.*", // v4.3.2
        "symfony/webpack-encore-bundle": "^1.5", // v1.6.2
        "symfony/yaml": "4.3.*" // v4.3.2
    },
    "require-dev": {
        "easycorp/easy-log-handler": "^1.0.7", // v1.0.7
        "symfony/debug-bundle": "4.3.*", // v4.3.2
        "symfony/maker-bundle": "^1.0", // v1.12.0
        "symfony/monolog-bundle": "^3.0", // v3.4.0
        "symfony/stopwatch": "4.3.*", // v4.3.2
        "symfony/twig-bundle": "4.3.*", // v4.3.2
        "symfony/var-dumper": "4.3.*", // v4.3.2
        "symfony/web-profiler-bundle": "4.3.*" // v4.3.2
    }
}