Chapters
48 Chapters
|
5:05:05
|
Login to bookmark this video
-
Course Code
Subscribe to download the code!Compatible PHP versions: ^7.1.3
Subscribe to download the code!Compatible PHP versions: ^7.1.3
-
This Video
Subscribe to download the video!
Subscribe to download the video!
-
Subtitles
Subscribe to download the subtitles!
Subscribe to download the subtitles!
-
Course Script
Subscribe to download the script!
Subscribe to download the script!
Scroll down to the script below, click on any sentence (including terminal blocks) to jump to that spot in the video!
Subscribe to jump to this part in the video!
Keep on Learning!
If you liked what you've learned so far, dive in! Subscribe to get access to this tutorial plus video, code and script downloads.
This tutorial is built with Symfony 4.3, but will work well on Symfony 4.4 or 5.
What PHP libraries does this tutorial use?
// composer.json
{
"require": {
"php": "^7.1.3",
"ext-ctype": "*",
"ext-iconv": "*",
"composer/package-versions-deprecated": "^1.11", // 1.11.99
"doctrine/annotations": "^1.0", // v1.8.0
"doctrine/doctrine-bundle": "^1.6.10", // 1.11.2
"doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
"doctrine/orm": "^2.5.11", // v2.6.3
"intervention/image": "^2.4", // 2.4.2
"league/flysystem-bundle": "^1.0", // 1.1.0
"phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.1
"sensio/framework-extra-bundle": "^5.3", // v5.3.1
"symfony/console": "4.3.*", // v4.3.2
"symfony/dotenv": "4.3.*", // v4.3.2
"symfony/flex": "^1.9", // v1.21.6
"symfony/framework-bundle": "4.3.*", // v4.3.2
"symfony/messenger": "4.3.*", // v4.3.4
"symfony/property-access": "4.3.*", // v4.3.2
"symfony/property-info": "4.3.*", // v4.3.2
"symfony/serializer": "4.3.*", // v4.3.2
"symfony/validator": "4.3.*", // v4.3.2
"symfony/webpack-encore-bundle": "^1.5", // v1.6.2
"symfony/yaml": "4.3.*" // v4.3.2
},
"require-dev": {
"easycorp/easy-log-handler": "^1.0.7", // v1.0.7
"symfony/debug-bundle": "4.3.*", // v4.3.2
"symfony/maker-bundle": "^1.0", // v1.12.0
"symfony/monolog-bundle": "^3.0", // v3.4.0
"symfony/stopwatch": "4.3.*", // v4.3.2
"symfony/twig-bundle": "4.3.*", // v4.3.2
"symfony/var-dumper": "4.3.*", // v4.3.2
"symfony/web-profiler-bundle": "4.3.*" // v4.3.2
}
}
24 Comments
Would utilizing a custom serializer be the best place to pull in a third party serializer? For instance, I want to utilize the CloudEvent PHP SDK to standardize our messages that we produce in our Symfony apps but also that we'll need to consume from external non Symfony apps. Would I then just utilize the serialize method in the encode() method and the opposite, deserialize in the decode() method? We're utilizing Kafka which really seems to be quite overcomplicated having to utilize messenger->enqueue->messenger-enqueue-transport.
I'm also figuring out how best to integrate the CloudEventImmutable object within our message/message handler class. I think a good approach would be to have a single message class that utilizes the CloudEvent object and then just pass in the application data through parameters to a wrapper class as opposed to creating multiple messages and message handlers. Curious as to best practices for this type of configuration. Loved the series, it was great having the deep dive on a lot of the internals of messenger!
Hey @JE!
I don't have direct experience with CloudEvent, but yea, your thinking seems correct! When a message goes "out" from Messenger, the code goes through
encode(). So you could use the custom serialization from there to, really, serialize into whatever format you want. Then the opposite for when a message is coming in. This is exactly how the core serializer is working (using PHP's nativeserialize()andunserialize()or the other one that uses Symfony's serializer - https://github.com/symfony/symfony/blob/6.4/src/Symfony/Component/Messenger/Transport/Serialization/Serializer.php).Hmm. Again, I don't have experience with the CloudEvent SDK. I can't totally envision your proposal, but the one possible problem I can see is, if you have only a single message and single handler, that one handler (if I'm understanding correctly) will need to "direct traffic" by looking at the application data and probably calling a separate service to do the real work (to avoid having ALL of the handler logic in that one handler). That's actually ok - but if you have this setup, you'll probably want to leverage something like a service subscriber so you can keep each individual separate "faux-handler" service lazy in your real handler. If you did that, it seems cool to me. But I could be totally misunderstanding the situation :p.
Cheers!
Hello,
When I try to serialize the
stamps('stamps' => serialize($allStamps),)the following error occurs:To fix this issue you need to add in the encode function before you get all the stamps the following line:
$envelope = $envelope->withoutStampsOfType(NonSendableStampInterface::class);Hey @Emag-L!
Good tip - the core PhpSerializer does that - https://github.com/symfony/symfony/blob/6.3/src/Symfony/Component/Messenger/Transport/Serialization/PhpSerializer.php#L59
The actual error might be due to Symfony doing something with exceptions that it shouldn't - by complete chance, this issue just came up in a core team chat today :).
Cheers!
Hello !
I am tryin to do the same thing but with SQS, and I have a problem with the stamps:
HTTP 400 returned for "https://sqs.eu-central-1.am...".
Code: InvalidParameterValue
Message: Message (user) attribute 'stamps' value contains invalid binary ch
aracter '#x0'. You must use only the following supported characters: #x9 |
#xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF].
Type: Sender
Detail:
I am using the SQS adapter from messenger with SF 5.1
Does it work differently with AWS SQS ?.
Thank you :)
for anyone that has the same issue,
a general good solution is to configure symfony/messenger to use Symfony\Component\Messenger\Transport\Serialization\Serializer instead of Symfony\Component\Messenger\Transport\Serialization\PhpSerializer.
if any specific needs are in place, e.g. for external message support, you might adapt that code in your project to do what you have to do. for example, i recommend do a data type mapping between "type" header and the class you want to use (this is to prevent the need to provide internal fqcn to external party).
it would be nice to have better support for external messages directly in symfony, for example, instead of using fully-qualified class name, you could do a config map in messenger.yaml. until then, do this way, and you'll sleep well at night.
For me it worked to add and strip slashes.
At decode
At encode
Hey Geoffrey,
Yes, it might be different with different transports. For example, IIRC, Doctrine transport does not allow you to work with binary files because of limitation of MySQL that does not allow storing binary files in a simple text column. That's a known limitation, and if you need store binary files - you would probably need to go with a different transport. Hm, not sure about AWS SQS unfortunately, probably you have the same problem as the Doctrine transport, or it might be something different. Probably the first step would be to figure out what chars exactly are invalid, and if you can change them to valid chars - that would be an easy fix. Otherwise, probably you need to go with a different transport unfortunately. Or sometimes, it might be just a misconfiguration.
Cheers!
Hi Victor,
The problem was about storing the envelopes in the header of a message to SQS. I added it to the body, and now it works.
Thank you !
Hey Geoffrey,
Glad you got it working! And thank you for sharing your solution with us! It might be helpful for others
Cheers!
Hi there
I'm new to Symfony.
I've been asked to create a service to send emails with Mailer using an async transport (I'm using AMQP atm). I'm supposed to use the Symfony serialiser because even though initially we'll consume the messages ourselves, it's possible that in the future we create them for external consumption.
I'm finding lots of problems while trying to achieve this. I'm now creating a new transport to consume messages from the same queue we're putting them up to. I have also created a new External Json serializer. I'm now thinking I'm going to need to extend SendEmailMessage and overwrite the send method so I can use this for the code that actually sends the messages.
So far I'm not sure how I'm going to convert the data into a Mailer object.
Anyway, I just wanted to know if you have any tips for achieving this.
BTW, thanks for your tutorial. You make it easy to understand!
Regards
Hey Janet !
Welcome to Symfony! And thanks for the nice note! I might have a few thoughts that could help :).
1) If you plan on possibly creating messages for external consumption in the future, I wouldn't necessarily, immediately use the Json serializer instead of the normal one. It's totally up to you - but I might just get it working first, and tackle using JSON in the future if you ever have that case :).
2) But, if you DO want to serialize to JSON right now, I think you will not need a custom serializer class or multiple transports. You should be able to just have one transport (this transport would be used for both sending and receiving the messages). Then, for the
serializeroption under that transport, set it tomessenger.transport.symfony_serializer. That is the service id of a built-in serializer for the Messenger component. It's quite smart - it uses the serializer component go serialize to JSON and also stores all the class names of the message (and stamps) as headers on the message. Then, when the message is received, it decodes the JSON back into the original objects. The only thing that you need to worry about is having "serializer-friendly" classes - e.g. classes where you have "getter" and "setter" methods (or public properties) so that the serializer can turn the properties into JSON and then re-set the JSON keys back onto the object later.Let me know if that helps! I could be way off - but it seems like you shouldn't need to do too much work to get this going. In this chapter, we created a custom serializer class because if we are truly receiving messages from an external system, then there is no alternative than for us to write custom code that maps each JSON message into a PHP object by hand. But if the JSON message is actually sent by Messenger, then the built-in
messenger.transport.symfony_serializeris awesome because it adds the extra metadata (as headers) so that the message can be deserialized on "receive" without any custom code.Cheers!
Hi @weaverryan
Thanks for your prompt and helpful reply!
I had actually tried what you're suggesting but I keep getting these two errors when I'm running the consumer:
Could this be due to not having the serializer-friendly class(es) you suggested?
I've created a mailer service class, is that where the getter and setter methods should go?
I'd appreciate any more suggestions you have to help.
Thanks very much :0)
Hey Janet!
Oh boy - it looks like you hit a bug - https://github.com/symfony/symfony/issues/33394
This is why the native, Symfony serializer is a pain for Messenger: the classes need to be constructed just right in order for it to work. In this case, because the RawMessage class doesn't have a getMessage() method on it, the "message" key isn't serialized to JSON and so it's missing during deserialization (I could be wrong about the details, but I'm 95% sure that's what's going on).
There are some workarounds on that issue... but basically, there aren't many easy work arounds. My recommendation might be to create your own
EmailMessageclass that contains all the data you want on your email. Configure this class to be handled async. Then create aEmailMessageHandler. THIS class would use theEmailMessageto create the realEmailobject and deliver it via Mailer. You would then not route Mailer messages to be sync - in other words do NOT do this - https://symfony.com/doc/current/mailer.html#sending-messages-async - you would now want mailer messages to be sent synchronously... which is fine - because you created your own message that is being handled async.Let me know if this helps!
Cheers!
Thanks weaverryan
That's an interesting idea worth trying. I'll let you know how it goes.
:0)
Hi, there is only one part of the Symfony Messenger I do not understand.
At the beginning of the tutorial we have gotten the following error
<br /> Could not decode message using PHP serialization: { <br /> "emoji": 2 <br /> }. <br />Now my question is, why is the failed message not stored in the failed (doctrine queue_name=failed) transport for later handling?
In the messenger.yaml the following is configured
`
framework:
Hey Rainer S.
Looks like this Github issue is related to your situation https://github.com/symfony/...
sadly they haven't fixed the problem yet
Cheers!
Hi,
When I try command php bin/console messenger:consume -vv external_messages, I have this error message :
[Symfony\Component\Debug\Exception\FatalThrowableError]
Argument 1 passed to App\Message\Command\LogEmoji::__construct() must be of the type integer, null given, called in /home/stephane/Hubic/www/
knpuniversity/messenger/src/Messenger/ExternalJsonMessageSerializer.php on line 19
You have any idea of the problem.
Thank
I find the solution : add ? before int and clean my array of emojis.
Hey Stephane,
I'm glad you were able to fix the problem by yourself! And thank you for sharing your solution with others. Though, the fix might depend. If it's ok to have no emoji index - then allowing null is something you would need here. But if every LogEmoji message should contain the index, i.e. should be an integer number - then you should search for the alternative fix. Most probably a LogEmoji message without Emoji index was saved to the DB somehow and now can't be unserialized and processed. So, the fix depends on your needs and your implementation
Cheers!
Hi,
Just a question about the custom serializer. I try to play a little with and if we just implement the decode part, and the handler fail, then, Messenger want to send the message again in the same queue with a RetryStamp and a DelayStamp.
In this example and in the way I've done it, in the client application we only have a Command Message, this one is created from the custom Serializer when we receibed the EventMessage from an other application. But when it fail, we need to re-encode it in an array to re-treat him.
If we do it simply, this create a kind of infinity loop (Delay and Retry Stamp are not added), what is the best way to do ?
`
In Serializer.php line 123:
[Symfony\Component\Messenger\Exception\MessageDecodingFailedException]
Could not decode stamp: The type of the "headers" attribute for class "Symfony\Component\Debug\Exception\FlattenException" must be one of "array" ("NULL" given)..
`
Thanks a lot :)
Hey VERMON!
Wow, awesome question. Honestly, even though I wrote the retry logic... I totally did *not* think about the retry stuff here. You're right that, on a high level, there are a few options here. First, for others, if we receive a message from an external transport while consuming from some transport called "from_external", on failure, Messenger will try to *send* to that same transport.
So, the most natural solution is (1). Basically, make sure your "from_external" transport is configured so that when it *sends*, it will end up in the same queue that it's receiving from (so, you may need to setup some binding keys). Then, as you mentioned, you would need to put the RedeliveryStamp stuff into the message somehow (e.g. headers)... and make sure that you also read it in decode(). That's indeed a tricky flow - I just asked another contributor on Messenger to verify if I'm missing something. I'll update you, assuming I hear from him (everyone is busy!).
Anyways, let me know what you think - and if you get the flow working. I may need to update the tutorial with some notes related to this. Btw, another (not-as-cool) solution is to set your retry limit to 1 and leverage a failure transport. But then you need to manually retry (via the retry commands) after just one failure... which is kinda lame.
Cheers!
Hi @wweaverryan !
Thanks a lot for your reply, finaly I've done the Solution 1 by addind encode and decode of Stamps in the custom serializer. That avoid the unexpected bug of the third solution.
I've do that: https://gist.github.com/mpi... (I can post it here if needed, but a bit long)
In my Serializer I've added 2 functions: encodeStamps and decodeStamps (thath just take care of DelayStamp and RedeliveryStamp, with a lite encoding).
Thanks again :)
Hey VERMON!
Thanks for posting and verifying that this fixed things :). I talked to a co-maintainer on Messenger and what he does is the same basic idea, but with one simple change. To encode the stamps, he uses PHP's serialize() function and then puts it on a header - I think just a single header (e.g. X-Stamps). Then, in decode, he uses unserialize() on that header. And that makes sense: this header would ONLY exist if the message were being redelivered from Messenger. And so, using native PHP serialization for this one field (and JSON for the actual message body) makes a lot of sense.
Cheers!
"Houston: no signs of life"
Start the conversation!