Chapters
-
Course Code
Subscribe to download the code!Compatible PHP versions: ^7.1.3
Subscribe to download the code!Compatible PHP versions: ^7.1.3
-
This Video
Subscribe to download the video!
Subscribe to download the video!
-
Subtitles
Subscribe to download the subtitles!
Subscribe to download the subtitles!
-
Course Script
Subscribe to download the script!
Subscribe to download the script!
Attachments with Async Messenger Emails
Scroll down to the script below, click on any sentence (including terminal blocks) to jump to that spot in the video!
Keep on Learning!
If you liked what you've learned so far, dive in! Subscribe to get access to this tutorial plus video, code and script downloads.
With a Subscription, click any sentence in the script to jump to that part of the video!
Login SubscribeOur registration email is being sent asynchronously via Messenger. And actually, every email our app sends will now be async. Let's double-check that the weekly report emails are still working.
Hit Ctrl+C to stop the worker process and, just to make sure our database if full of fresh data, reload the fixtures:
php bin/console doctrine:fixtures:load
Now run:
php bin/console app:author-weekly-report:send
Problems with Binary Attachments
Ah! Explosion! Incorrect string value? Wow. Okay. What we're seeing is a real-world limitation of the doctrine transport: it can't handle binary data. This may change in Symfony 4.4 - there's a pull request for it - but it may not be merged in time.
Why does our email contain binary data? Remember: the method that creates the author weekly report email also generates a PDF and attaches it. That PDF is binary... so when Messenger tries to put it into a column that doesn't support binary data... boom! Weird explosion.
If this is a problem for you, there are two fixes. First, instead of Doctrine, use another transport - like AMQP. Second, if you need to use doctrine and you do send binary attachments, instead of saying ->attach()
you can say ->attachFromPath()
and pass this a path on the filesystem to the file. By doing this, the path to the file is what is stored in the queue. The only caveat is that the worker needs to have access to the file at that path.
Messenger and Tests
There's one other thing I want to show with messenger. Run the tests!
php bin/phpunit
Awesome! There are a bunch of deprecation notices, but the tests do pass. However, run that Doctrine query again to see the queue:
php bin/console doctrine:query:sql 'SELECT * FROM messenger_messages'
Uh oh... the email - the one from our functional test to the registration page - was added to the queue! Why is that a problem? Well, it's not a huge problem... but if we run the messenger:consume
command...
php bin/console messenger:consume -vv
That would actually send that email! Again, that's not the end of the world... it's just a little odd - the test environment doesn't need to send real emails.
If you've configured your test
environment to use a different database than normal, you're good: your test database queue table will fill up with messages, but you'll never run the messenger:consume
command from that environment anyways.
Overriding the Transport in the test Environment
But there's also a way to solve this directly in Messenger. In .env
, copy MESSENGER_TRANSPORT_DSN
and open up .env.test
. Paste this but replace doctrine
with in-memory
. So: in-memory://
Show Lines
|
// ... lines 1 - 4 |
MESSENGER_TRANSPORT_DSN=in-memory://default |
This transport... is useless! And I love it. When Messenger sends something to an "in-memory" transport, the message... actually goes nowhere - it's just discarded.
Run the tests again:
php bin/phpunit
And... check the database:
php bin/console doctrine:query:sql 'SELECT * FROM messenger_messages'
No messages! Next, lets finish our grand journey through Mailer by integrating our Email styling with Webpack Encore.
20 Comments
Hey Kiuega!
Hmm, I see! Yea, this is "screwy" - I would love to fix this problem properly in WebpackEncoreBundle (which I maintain), but it is tricky!
Here is one thing to try is this: add an event listener to the MessageEvent - https://github.com/symfony/symfony/blob/5.x/src/Symfony/Component/Mailer/Event/MessageEvent.php
This will actually be fired both (right before) the message is queued and also (right before) the message is actually delivered (in the worker). Try doing the entrypoint reset there. If you want to be more specific about it, you could check $event->getEnvelop()
on the event object and see if it has a ReceivedStamp. If it does, then it is being handled by a worker and is about to actually be delivered. If not, then it is not being sent but is just being queued.
Let me know if that works!
Cheers!
Hey Ryan! Thank you for your answer!
I followed your directions, so here's what I get:
class MessageEventListener implements EventSubscriberInterface
{
private $entrypointLookup;
public function __construct(EntrypointLookupInterface $entrypointLookup)
{
$this->entrypointLookup = $entrypointLookup;
}
public static function getSubscribedEvents()
{
return [
MessageEvent::class => 'onMessage',
];
}
public function onMessage(MessageEvent $event)
{
$this->entrypointLookup->reset();
}
}
It works great!
On the other hand to check if the event contains a ReceivedStamp, I have some difficulties.
In fact the line $event->getEnvelope()
seems returns this object:
Symfony\Component\Mailer\DelayedEnvelope and not Symfony\Component\Messenger\Envelope
So, we have this one :
MessageEventListener.php on line 27:
Symfony\Component\Mailer\DelayedEnvelope {#6072 â–¼
-senderSet: false
-recipientsSet: false
-message: Symfony\Bridge\Twig\Mime\TemplatedEmail {#6062 â–¼
-htmlTemplate: "stripe/email/admin/admin_message_to_customer.html.twig"
-textTemplate: null
-context: array:1 [â–¶]
-text: null
-textCharset: null
-html: null
-htmlCharset: null
-attachments: []
-headers: Symfony\Component\Mime\Header\Headers {#6077 â–¶}
-body: null
-message: null
}
-sender: null
-recipients: []
}
I was planning to use this kind of line to see if there was a ReceivedStamp:
if($envelope->last(ReceivedStamp::class)) {
//...
}
But how could I verify it with this object? Does it allow me?
Hey Fabrice!
Ok! One step forward, one more to go :).
About the Envelope, that's my fault: I completely mis-read how the core code works. This is not a Messenger Envelope that we're talking about in this case - it's a "Mailer" envelope. With that in mind, I think what you need to use is simply $event-> isQueued()
. If I'm reading the code correctly (no guarantee) that should be true when the message is about to be queued (and not actually sent) and false when the worker processes it and it's actually about to send :).
Let me know if this helps get that last piece!
Cheers!
Yes it was that ! Good game ! :D
I like attaching a binary file because it avoids having to deal with removing files from the filesystem once the email has been sent.
If I were to pass the file's path on the filesystem using ->attachFromPath(), would there be a clean approach to removing the worker has sent the email? I would like to avoid having to create a cronjob that checks for specific conditions before deleting the files.
Is there a callback from the SendEmailMessage class or a way to run something after the message has been consumed?
Hey apphancer
I'm not sure if there are any callbacks you can hook into, you may want to investigate a little bit but what you can do is to make the *handler* to dispatch another message (the CleanUp message) when it finished its work
Cheers!
Hey !
I'de like to keep using the doctrine transport for emails with attachments and therefore use the attachFromPath()
method. However, I'm using Flysystem to handle files. How can I give a "path" to attachFromPath()
while using Flysystem.
--> Giving the "normal" path (the path I would give Flysystem to retrieve data) does not work and returns an error when trying to run the job in queue.
I think I'm missing something...
Thanks !
Hey Thibault V.!
Hmm. Just so I'm clear: the issue is that you really want to pass a path that should be processed through Flysystem, right? If so, the problem is that, when the Mailer system eventually tries to read your file, it doesn't know to use Flysystem: it just does an fopen
on your file path: https://github.com/symfony/symfony/blob/09645a9103a4eb55c8ff081ea8632179c408dce9/src/Symfony/Component/Mime/Part/DataPart.php#L59
Unfortunately, there's no "simple" way to pass a path that would be resolved later via Flysystem. To do this you could:
A) Register a stream wrapper and pass a path that uses it - e.g. s3://path/to/file
B) Do NOT add an attachment, but instead, create a custom Email class (that extends TemplatedEmail) and set some new/custom property on it that indicates which file should be attached. Then, add a listener to MessageEvent that reads this property and uses it to add the attachment (using a normal attach()). The idea is that this code would run when the email is being sent in the queue. The only trick is that this event (I believe) is also dispatched when the email is originally being queued, so you would need to somehow differentiate between "is this email being queued" and "is this email being sent".
Or, if I've misunderstood somehow and this is working... but it just breaks when running from the queue, let me know :).
Let me know if any of this makes sense ;).
Cheers!
Hey weaverryan,
Thanks for getting back to me ! You understood correctly and i'm in fact trying to attach files from google cloud storage inside some emails.
I'm going to check out the second option for now as it is clearer for me but I would love to know more about the stream wrapper. How would you go about implementing it ? (in my case it would be gs://
;) )
However, since fopen can accept a URL, I'm actually thinking of passing a signed url to the content on cloud storage. I'm finishing another project and haven't had time to test it. What do you think about this ?
Thanks for your feedback :)
Hey Thibault V.!
Ah, signed URL seems like a really interesting idea! The gs://
is a variation of that - you can basically make it so that gs://some/path
uses custom logic in your code to fetch this some/path
and things like fopen
will understand it. It looks like Google Cloud might already support this in their SDK? https://github.com/googleapis/google-cloud-php/blob/master/Storage/src/StreamWrapper.php Or somebody (it's a bit old) added a library to use a stream wrapper with Flysystem https://github.com/twistor/flysystem-stream-wrapper/
Let me know what you figure out!
Cheers!
Hello weaverryan,
I'll look into it and get back to you when I finally get around this feature.
Thanks again !
Once using messenger with mailer, is there a way to send the email immediately (avoiding the queue)?
Hey Kevin B.
It should be sent immediately if you haven't configured messenger routing for Mailer message. But I guess you are looking in Symfony profiler and there is some not exact information.
Cheers!
Hey sadikoff, I was thinking more if you had messenger routing setup but want an important email to be sent immediately, skipping the queue.
I looked into this further and I don't think it's possible.
hm I think it's possible, something like configuring sync transport, route message to both transports sync and async, but configure handler to process messages depending on transport. Here is a clue for you https://symfonycasts.com/sc...
Cheers!
Using the assertions you wrote earlier for Symfony 4.4, the tests fail at $this->assertEmailCount(1);
.
Any info why it considers the email is not sent ?
I would love to see all the tests pass, but does this mean that testing async emails sending is a pain ?
Hey AymDev,
What actual number of count do you have? Is it "0"? Yeah, testing async is a pain :) Because, to actually send the email you need to run worker with "bin/console messenger:consume" command. And anyway, you need to worry about not sending real emails in test mode, because you don't want to hit real users inbox while testing.
Cheers!
Yes I actually have 0 emails sent.
Oh okay that explains it, the message is dispatched but as long as it isn't processed it doesn't count as sent. Seems legit :)
Thank you again for your answers, cheers !
Hey AymDev,
Yes, exactly! Sounds legit for me too :)
Cheers!
"Houston: no signs of life"
Start the conversation!
What PHP libraries does this tutorial use?
// composer.json
{
"require": {
"php": "^7.1.3",
"ext-iconv": "*",
"aws/aws-sdk-php": "^3.87", // 3.110.11
"composer/package-versions-deprecated": "^1.11", // 1.11.99
"doctrine/annotations": "^1.0", // 1.10.1
"doctrine/doctrine-bundle": "^1.6.10", // 1.11.2
"doctrine/doctrine-migrations-bundle": "^1.3|^2.0", // v2.0.0
"doctrine/orm": "^2.5.11", // v2.7.2
"knplabs/knp-markdown-bundle": "^1.7", // 1.7.1
"knplabs/knp-paginator-bundle": "^2.7", // v2.8.0
"knplabs/knp-snappy-bundle": "^1.6", // v1.6.0
"knplabs/knp-time-bundle": "^1.8", // v1.9.1
"league/flysystem-aws-s3-v3": "^1.0", // 1.0.23
"league/flysystem-cached-adapter": "^1.0", // 1.0.9
"league/html-to-markdown": "^4.8", // 4.8.2
"liip/imagine-bundle": "^2.1", // 2.1.0
"nexylan/slack-bundle": "^2.1,<2.2.0", // v2.1.0
"oneup/flysystem-bundle": "^3.0", // 3.1.0
"php-http/guzzle6-adapter": "^1.1", // v1.1.1
"phpdocumentor/reflection-docblock": "^3.0|^4.0", // 4.3.1
"sensio/framework-extra-bundle": "^5.1", // v5.4.1
"stof/doctrine-extensions-bundle": "^1.3", // v1.3.0
"symfony/asset": "^4.0", // v4.3.4
"symfony/console": "^4.0", // v4.3.4
"symfony/flex": "^1.9", // v1.21.6
"symfony/form": "^4.0", // v4.3.4
"symfony/framework-bundle": "^4.0", // v4.3.4
"symfony/mailer": "4.3.*", // v4.3.4
"symfony/messenger": "4.3.*", // v4.3.4
"symfony/property-access": "4.3.*", // v4.3.4
"symfony/property-info": "4.3.*", // v4.3.4
"symfony/security-bundle": "^4.0", // v4.3.4
"symfony/sendgrid-mailer": "4.3.*", // v4.3.4
"symfony/serializer": "4.3.*", // v4.3.4
"symfony/twig-bundle": "^4.0", // v4.3.4
"symfony/validator": "^4.0", // v4.3.4
"symfony/web-server-bundle": "^4.0", // v4.3.4
"symfony/webpack-encore-bundle": "^1.4", // v1.6.2
"symfony/yaml": "^4.0", // v4.3.4
"twig/cssinliner-extra": "^2.12", // v2.12.0
"twig/extensions": "^1.5", // v1.5.4
"twig/extra-bundle": "^2.12|^3.0", // v2.12.1
"twig/inky-extra": "^2.12", // v2.12.0
"twig/twig": "^2.12|^3.0" // v2.13.1
},
"require-dev": {
"doctrine/doctrine-fixtures-bundle": "^3.0", // 3.2.2
"easycorp/easy-log-handler": "^1.0.2", // v1.0.7
"fzaninotto/faker": "^1.7", // v1.8.0
"symfony/browser-kit": "4.3.*", // v4.3.5
"symfony/debug-bundle": "^3.3|^4.0", // v4.3.4
"symfony/dotenv": "^4.0", // v4.3.4
"symfony/maker-bundle": "^1.0", // v1.13.0
"symfony/monolog-bundle": "^3.0", // v3.4.0
"symfony/phpunit-bridge": "^3.3|^4.0", // v4.3.4
"symfony/stopwatch": "4.3.*", // v4.3.4
"symfony/var-dumper": "^3.3|^4.0", // v4.3.4
"symfony/web-profiler-bundle": "4.3.*" // v4.3.4
}
}
Hello ! One problem that can happen is when we have to send multiple async emails. In our "Mailer" service, we made sure to reset the entrypointLookup :
So that works synchronously.
On the other hand, from the moment we switch to asynchronous mode, there may necessarily be problems, since the mail will not be sent just after having reset the entryPointLookup, and therefore the CSS of the mail will not be correctly loaded.
Is there something to counter this?
In our messenger.yaml file, we configure the route as:
<b>'Symfony\Component\Mailer\Messenger\SendEmailMessage': async</b>
One possibility could be to override the event that will receive the SendEmailMessage object, and re-execute the following line just before the actual sending of the email
But I'm not sure how to go about it in this case.
Would this be a good solution?