Doing Work in the Handler

Update
Ended up doing Option 1
Loaded all the dependencies that don't have any DB association via DI container and resolved the rest manually by instantiating new objects and feeding them to the classes that needed them, not pretty but it works.

Would still love a reply in case anyone knows a way around this :)

Hey Musa!

So my guess is that you have an event listener on RequestEvent that normally looks at the domain name and does some work to "set up" which database to use. Is that... kind of correct? If so, then (as you know), that listener wouldn't run when your worker is executing a message. But could you accomplish this with a worker listener? I'm thinking about the WorkerMessageReceivedEvent. This is before your message is handled by the worker, but you DO have access to the message itself (and stamps) from this event. Could you read that and reconfigure things before your handler is called?

Btw, you'll probably want to activate the reset_on_message option - https://symfony.com/blog/new-in-symfony-5-4-messenger-improvements#reset-container-services-between-messages - so that the container starts "fresh" before each message is processed.

Let me know if this helps!

Cheers!

Hi Ryan, thanks for reaching out on such short notice, really grateful for your activity in this community!

A (event)subscriber is responsible for doing the database switching at runtime for incoming HTTP requests.

And you're completely right, our workers are oblivious to this behaviour.

I'm curious though, does the <blockquote>WorkerMessageReceivedEvent</blockquote> live in the same lifecycle as the actual job handler?

I've been thinking about this when reviewing your videos on stamps.

In my mind yesterday, albeit caffeine fuelled and under time pressure, the job handler is just called like any other PHP process on it's own thread (php /path/to/script.php 'Hello, ' 'World!') passing the serialized object(envelope) to the worker in the place of $argv(n).

But if the job life cycle starts with the first middleware in the line, ergo:
Job pushed to queue (HTTP stops here) -> (Internal process starts here) middleware ... n .... -> MessageHandler.
And not:
Job pushed to queue -> middleware ... n ... (HTTP stops here) -> (Internal process starts here) MessageHandler .
Then middleware should work just fine.

I'd love a confirmation on this behaviour, as it's a little hard to draw conclusions based on synchronous examples for messenger.

Hey Musa!

Sorry for the slow reply this time :).

> I'm curious though, does the WorkerMessageReceivedEvent live in the same lifecycle as the actual job handler?

I'm not sure exactly what you mean. But I think the answer is "yes". That event is dispatched inside of the "worker process" and it is dispatched right before the handler would actually be called.

> But if the job life cycle starts with the first middleware in the line, ergo:
> Job pushed to queue (HTTP stops here) -> (Internal process starts here) middleware ... n .... -> MessageHandler.
> And not:
> Job pushed to queue -> middleware ... n ... (HTTP stops here) -> (Internal process starts here) MessageHandler .
> Then middleware should work just fine.

It's a mixture of both 😆:

Message dispatched -> middleware -> one middleware pushes to queue (HTTP stops here) -> (Internal process starts) -> Message is loaded from queue -> middleware -> one middleware calls your MessageHandler.

The middleware itself is actually what's responsible for both sending and handling messages. If you added a custom middleware, it should be called BOTH when the message is originally dispatched AND when it is received from the queue and about to be handled. You can check for a ReceivedStamp to see if it was just received.

Cheers!

No worries on the timing at all mate.
I realized that if I had continued on with watching the following videos, the answer was right there.
Thanks for taking the time to explain :)

This worked:

    function __construct(ConnectionSwitcher $connectionSwitcher)
    {
        $this->connectionSwitcher = $connectionSwitcher;
    }

    public function handle(Envelope $envelope, StackInterface $stack): Envelope
    {

        if ($envelope->last(ReceivedStamp::class)) {

            $message = $envelope->getMessage();

            if ($message instanceof AsynchIdentifierMessageInterface) {

                $Identifier= $message->getIdentifier();

                $this->connectionSwitcher->setIdentifierForRequestLifeCycle($Identifier);
            }
        }

        return $stack->next()->handle($envelope, $stack);
    }

<b>However</b> I ran into another issue.

It seems that non-object class variables stay between jobs.
Let me clarify:

1. After the middleware changes my database, the MessageHandlers dependencies are injected and have the correct DB connection.
2. Non-object variables set by the constructor(on a dependency) are persistent across jobs, almost like the object is cached or treated as a cross-job singleton.

<b>Example:</b>

I have a dependency in one of my (Asynch)MessageHandler classes, this dependency handles logging. The logger takes a service and sets the current identifier and prepends it to all messages logged by the instance used.

class Logger {
  private string $identifier;
  private ExternalLoggingApiService $logger;

  function __ construct(FooService $fooService, ExternalLoggingApiService $logger)
  {
    $this->identifier = $fooService->getIdentifier();
    $this->logger = $logger;
  }

  public function Log(string $message)
  {
    $this->logger->log("[{$this->identifer}] $message")
  }

<blockquote>Job #1 on identifier XXXX. Gets identifier XXXX from fooService and sets it as it's $identifier variable.
Job #2 on identifier YYYY. Ignores fetching identifier YYYY and uses XXXX</blockquote>

This became very apparent when I changed the logger's code to:

class Logger {
  private string $identifier = "test";
  private ExternalLoggingApiService $logger;
  private FooService $fooService;

  function __ construct(FooService $fooService, ExternalLoggingApiService $logger)
  {
    $this->fooService= $fooService;
    $this->logger = $logger;
  }

  public function Log(string $message)
  {
    $this->identifier = $this->fooService->getIdentifier();
    $this->logger->log("[{$this->identifier}] $message")
  }

  public function getIdentifer()
  {
    return $this->identifier;
  }

I logged (in the MessageHandler) the jobs using the getIdentifier() method to see what was going on:

<blockquote>Job #1, getIdentifier() returns "test" because the function was called before Log(), actual identifier shown in log: XXXX
Job #2, getIdentifier() returns XXXX, actual identifier shown in log: YYYY</blockquote>

If the Logging class dependency was refreshed on each job, the getIdentifier() should always return "test" as the variable has not been reassigned before the actual Log() method is called.

Worth mentioning that calls to the database in Job #2 fetch from the correct database, so these dependencies are working as intended.

So I guess my question is:
Does symfony cache non-object object variables between jobs of the same kind?

I will remove the fetching of identifier from the constructor as it's not good practice anyhow, however,
I'd like to know more about this behaviour in case other dependencies share this behaviour.

Hey Musa!

Yea, this is an interesting situation. Two things I'll say:

A) Yes, setting the identifier in the constructor is not a *super* great practice. Well, more broadly, making a service "stateful" - meaning it has some data that might change and holds onto some state / "current data" - is kind of an anti-practice. Services should be stateless: you get the same result every time you call methods on them, no matter what. But, that is just a general rule. I DO this type of stuff from time to time - it's useful ;).

B) About the "caching" that you're seeing. For each PHP process, Symfony instantiates a service just one time. During a normal request-response cycle, if you ask for your Logger service, it will be instantiated just once. The SAME thing happens when processing an async message. But in this case, the Logger service will be instantiated just once... even if you handle 5 messages. That's why you're seeing that behavior.

The solution is either to:

1) Not set the state on your service
or
2) Somehow re-set the state after each message or re-initialize it at the start of each message. Depending on your logic, you might be able to do this nicely. OR, iirc, if you make your service implement ResetInterface, then Symfony will call the reset() method each time it "resets" the container (this is not something that happens during a normal request-response process). You can (and probably should, it's just a good idea) force your container to be reset between each message with this new feature: https://symfony.com/blog/ne...

Let me know if this helps!

Cheers!

Hey Julien R.
You're 100% right! Thanks for letting us know. I'm going to fix it right now :)

Cheers!

Hey ahmedbhs

> So either we exude asynchonsly or synchronously we consume the same memory, but arsenic is a better use for a server to handle a lot of demands.

If your worker lives in the same server as your application, then yes. When the message gets consume your server may suffer an overload but at least the initial request was already processed, in other words, your user got a faster UX

> if we send a notification to 1000 subscribers what's the best way is it to or outside. Do we send one message contain all user to the handler and then make a loop Inside the handler:

Massive email campaigns are hard to manage, usually is recommended to use a third party service for doing that but if you still want to do it. I would recommend to send like 5 or 10 emails per message

Cheers!

Hey Skylar,

Hm, let's try to debug it together... Do you have any spots where you call that "find" property? Basically, you can search for "find" text in your JS files. Most probably you just use incorrect object, etc.

Also, please, make sure you followed all the steps in README file of downloaded course code. In particular, make sure you ran "composer install", "yarn install" and "yarn encore dev" before starting working with project. Btw, did you download the course code and started from start/ directory?

Cheers!

Hi, Thank you for the reply. I traced the error to my system. When I use the Symfony Serve, the XHR response to the /image request is the symfony profiler information. Some how on my system it is not working correctly. When I turn the symfony profiler off, then the ponka app works fine. I have no idea how to fix it. I am using a windows 10 with WSL ubuntu linux. Some where that is causing the issue.

Thanks for your help.

Hey Skylar

Try running the web server without SSL. I would also check the Content-Type header of the response because if you are returning JSON, then it should be set to application/json. Let me know if it worked :)

Thank you Diego.

I solved the problem. First, I updated and upgraded my ubuntu install and upgraded to php 7.4. Then I updated my Symfony CLI file to the latest version. voila. It now works and is serving the correct content for the url requested.

Of course, there is always consequences. My Windows 10 /WSL/Ubuntu install does not support TCP_INFO and now the Symfony CLI app is logging "failed to retrieve TCP_INFO for socket: Protocol not available (92)" I don't even know where to look for Symfony CLI help. But I can just let it run as it is working. yeay!

Excellent!
You can report issues related to Symfony CLI here https://github.com/symfony/cli

Cheers!

Hey Gustavo!

Hmm. Typically, if you need a service, you should add a constructor and use dependency injection for each service you need. Service subscribers are a special feature that should generally only be used in some edge-cases when there are performance concerns... and these are rare :). Here's some info on this (in case you haven't seen it): https://symfonycasts.com/sc...

So generally... I would kinda say "no" you shouldn't do this. But if you think you have a special case, let me know.

Cheers!