Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

Attachments with Async Messenger Emails

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $12.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

Our registration email is being sent asynchronously via Messenger. And actually, every email our app sends will now be async. Let's double-check that the weekly report emails are still working.

Hit Ctrl+C to stop the worker process and, just to make sure our database if full of fresh data, reload the fixtures:

php bin/console doctrine:fixtures:load

Now run:

php bin/console app:author-weekly-report:send

Problems with Binary Attachments

Ah! Explosion! Incorrect string value? Wow. Okay. What we're seeing is a real-world limitation of the doctrine transport: it can't handle binary data. This may change in Symfony 4.4 - there's a pull request for it - but it may not be merged in time.

Why does our email contain binary data? Remember: the method that creates the author weekly report email also generates a PDF and attaches it. That PDF is binary... so when Messenger tries to put it into a column that doesn't support binary data... boom! Weird explosion.

If this is a problem for you, there are two fixes. First, instead of Doctrine, use another transport - like AMQP. Second, if you need to use doctrine and you do send binary attachments, instead of saying ->attach() you can say ->attachFromPath() and pass this a path on the filesystem to the file. By doing this, the path to the file is what is stored in the queue. The only caveat is that the worker needs to have access to the file at that path.

Messenger and Tests

There's one other thing I want to show with messenger. Run the tests!

php bin/phpunit

Awesome! There are a bunch of deprecation notices, but the tests do pass. However, run that Doctrine query again to see the queue:

php bin/console doctrine:query:sql 'SELECT * FROM messenger_messages'

Uh oh... the email - the one from our functional test to the registration page - was added to the queue! Why is that a problem? Well, it's not a huge problem... but if we run the messenger:consume command...

php bin/console messenger:consume -vv

That would actually send that email! Again, that's not the end of the world... it's just a little odd - the test environment doesn't need to send real emails.

If you've configured your test environment to use a different database than normal, you're good: your test database queue table will fill up with messages, but you'll never run the messenger:consume command from that environment anyways.

Overriding the Transport in the test Environment

But there's also a way to solve this directly in Messenger. In .env, copy MESSENGER_TRANSPORT_DSN and open up .env.test. Paste this but replace doctrine with in-memory. So: in-memory://

6 lines .env.test
... lines 1 - 4
MESSENGER_TRANSPORT_DSN=in-memory://default

This transport... is useless! And I love it. When Messenger sends something to an "in-memory" transport, the message... actually goes nowhere - it's just discarded.

Run the tests again:

php bin/phpunit

And... check the database:

php bin/console doctrine:query:sql 'SELECT * FROM messenger_messages'

No messages! Next, lets finish our grand journey through Mailer by integrating our Email styling with Webpack Encore.

Leave a comment!

20
Login or Register to join the conversation
Kiuega Avatar

Hello ! One problem that can happen is when we have to send multiple async emails. In our "Mailer" service, we made sure to reset the entrypointLookup :


$this->entrypointLookup->reset();
$email = (new TemplatedEmail())
//....

$this->mailer->send($email);

So that works synchronously.

On the other hand, from the moment we switch to asynchronous mode, there may necessarily be problems, since the mail will not be sent just after having reset the entryPointLookup, and therefore the CSS of the mail will not be correctly loaded.

Is there something to counter this?

In our messenger.yaml file, we configure the route as:

'Symfony\Component\Mailer\Messenger\SendEmailMessage': async

One possibility could be to override the event that will receive the SendEmailMessage object, and re-execute the following line just before the actual sending of the email


$this->entrypointLookup->reset();

But I'm not sure how to go about it in this case.

Would this be a good solution?

Reply

Hey Kiuega!

Hmm, I see! Yea, this is "screwy" - I would love to fix this problem properly in WebpackEncoreBundle (which I maintain), but it is tricky!

Here is one thing to try is this: add an event listener to the MessageEvent - https://github.com/symfony/...

This will actually be fired both (right before) the message is queued and also (right before) the message is actually delivered (in the worker). Try doing the entrypoint reset there. If you want to be more specific about it, you could check $event->getEnvelop() on the event object and see if it has a ReceivedStamp. If it does, then it is being handled by a worker and is about to *actually* be delivered. If not, then it is not being sent but is just being queued.

Let me know if that works!

Cheers!

Reply
Fabrice Avatar

Hey Ryan! Thank you for your answer!

I followed your directions, so here's what I get:


class MessageEventListener implements EventSubscriberInterface
{
private $entrypointLookup;

public function __construct(EntrypointLookupInterface $entrypointLookup)
{
$this->entrypointLookup = $entrypointLookup;
}

public static function getSubscribedEvents()
{
return [
MessageEvent::class => 'onMessage',
];
}

public function onMessage(MessageEvent $event)
{
$this->entrypointLookup->reset();
}
}

It works great!

On the other hand to check if the event contains a ReceivedStamp, I have some difficulties.

In fact the line $event->getEnvelope() seems returns this object:

**Symfony\Component\Mailer\DelayedEnvelope** and not **Symfony\Component\Messenger\Envelope**

So, we have this one :


MessageEventListener.php on line 27:
Symfony\Component\Mailer\DelayedEnvelope {#6072 ▼
-senderSet: false
-recipientsSet: false
-message: Symfony\Bridge\Twig\Mime\TemplatedEmail {#6062 ▼
-htmlTemplate: "stripe/email/admin/admin_message_to_customer.html.twig"
-textTemplate: null
-context: array:1 [▶]
-text: null
-textCharset: null
-html: null
-htmlCharset: null
-attachments: []
-headers: Symfony\Component\Mime\Header\Headers {#6077 ▶}
-body: null
-message: null
}
-sender: null
-recipients: []
}

I was planning to use this kind of line to see if there was a **ReceivedStamp**:


if($envelope->last(ReceivedStamp::class)) {
//...
}

But how could I verify it with this object? Does it allow me?

Reply

Hey Fabrice!

Ok! One step forward, one more to go :).

About the Envelope, that's my fault: I completely mis-read how the core code works. This is not a Messenger Envelope that we're talking about in this case - it's a "Mailer" envelope. With that in mind, I think what you need to use is simply $event-> isQueued(). If I'm reading the code correctly (no guarantee) that should be true when the message is about to be queued (and not actually sent) and false when the worker processes it and it's *actually* about to send :).

Let me know if this helps get that last piece!

Cheers!

Reply
Kiuega Avatar

Yes it was that ! Good game ! :D

1 Reply

I like attaching a binary file because it avoids having to deal with removing files from the filesystem once the email has been sent.
If I were to pass the file's path on the filesystem using ->attachFromPath(), would there be a clean approach to removing the worker has sent the email? I would like to avoid having to create a cronjob that checks for specific conditions before deleting the files.
Is there a callback from the SendEmailMessage class or a way to run something after the message has been consumed?

Reply

Hey apphancer

I'm not sure if there are any callbacks you can hook into, you may want to investigate a little bit but what you can do is to make the *handler* to dispatch another message (the CleanUp message) when it finished its work

Cheers!

Reply
Thibault V. Avatar
Thibault V. Avatar Thibault V. | posted 2 years ago

Hey !

I'de like to keep using the doctrine transport for emails with attachments and therefore use the attachFromPath() method. However, I'm using Flysystem to handle files. How can I give a "path" to attachFromPath() while using Flysystem.
--> Giving the "normal" path (the path I would give Flysystem to retrieve data) does not work and returns an error when trying to run the job in queue.

I think I'm missing something...

Thanks !

Reply

Hey Thibault V.!

Hmm. Just so I'm clear: the issue is that you *really* want to pass a path that should be processed through Flysystem, right? If so, the problem is that, when the Mailer system eventually tries to read your file, it doesn't know to use Flysystem: it just does an fopen on your file path: https://github.com/symfony/...

Unfortunately, there's no "simple" way to pass a path that would be resolved later via Flysystem. To do this you could:

A) Register a stream wrapper and pass a path that uses it - e.g. s3://path/to/file
B) Do NOT add an attachment, but instead, create a custom Email class (that extends TemplatedEmail) and set some new/custom property on it that indicates which file should be attached. Then, add a listener to MessageEvent that reads this property and uses it to add the attachment (using a normal attach()). The idea is that this code would run when the email is being sent in the queue. The only trick is that this event (I believe) is also dispatched when the email is originally being queued, so you would need to somehow differentiate between "is this email being queued" and "is this email being sent".

Or, if I've misunderstood somehow and this *is* working... but it just breaks when running from the queue, let me know :).

Let me know if any of this makes sense ;).

Cheers!

1 Reply
Thibault V. Avatar
Thibault V. Avatar Thibault V. | weaverryan | posted 2 years ago | edited

Hey weaverryan,

Thanks for getting back to me ! You understood correctly and i'm in fact trying to attach files from google cloud storage inside some emails.

I'm going to check out the second option for now as it is clearer for me but I would love to know more about the stream wrapper. How would you go about implementing it ? (in my case it would be gs:// ;) )

However, since fopen can accept a URL, I'm actually thinking of passing a signed url to the content on cloud storage. I'm finishing another project and haven't had time to test it. What do you think about this ?

Thanks for your feedback :)

Reply

Hey Thibault V.!

Ah, signed URL seems like a really interesting idea! The gs:// is a variation of that - you can basically make it so that gs://some/path uses custom logic in your code to fetch this some/path and things like fopen will understand it. It looks like Google Cloud might already support this in their SDK? https://github.com/googleap... Or somebody (it's a bit old) added a library to use a stream wrapper with Flysystem https://github.com/twistor/...

Let me know what you figure out!

Cheers!

1 Reply
Thibault V. Avatar
Thibault V. Avatar Thibault V. | weaverryan | posted 2 years ago | edited

Hello weaverryan,

I'll look into it and get back to you when I finally get around this feature.

Thanks again !

Reply
Kevin B. Avatar
Kevin B. Avatar Kevin B. | posted 2 years ago

Once using messenger with mailer, is there a way to send the email immediately (avoiding the queue)?

Reply

Hey Kevin B.

It should be sent immediately if you haven't configured messenger routing for Mailer message. But I guess you are looking in Symfony profiler and there is some not exact information.

Cheers!

Reply
Kevin B. Avatar
Kevin B. Avatar Kevin B. | sadikoff | posted 2 years ago | edited

Hey sadikoff, I was thinking more if you had messenger routing setup but want an important email to be sent immediately, skipping the queue.

I looked into this further and I don't think it's possible.

Reply

hm I think it's possible, something like configuring sync transport, route message to both transports sync and async, but configure handler to process messages depending on transport. Here is a clue for you https://symfonycasts.com/sc...

Cheers!

Reply
AymDev Avatar

Using the assertions you wrote earlier for Symfony 4.4, the tests fail at $this->assertEmailCount(1);.
Any info why it considers the email is not sent ?

I would love to see all the tests pass, but does this mean that testing async emails sending is a pain ?

Reply

Hey AymDev,

What actual number of count do you have? Is it "0"? Yeah, testing async is a pain :) Because, to actually send the email you need to run worker with "bin/console messenger:consume" command. And anyway, you need to worry about not sending real emails in test mode, because you don't want to hit real users inbox while testing.

Cheers!

Reply
AymDev Avatar

Yes I actually have 0 emails sent.
Oh okay that explains it, the message is dispatched but as long as it isn't processed it doesn't count as sent. Seems legit :)

Thank you again for your answers, cheers !

Reply

Hey AymDev,

Yes, exactly! Sounds legit for me too :)

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial is built on Symfony 4.3, but will work well with Symfony 4.4 or 5.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3",
        "ext-iconv": "*",
        "aws/aws-sdk-php": "^3.87", // 3.110.11
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "knplabs/knp-markdown-bundle": "^1.7", // 1.7.1
        "knplabs/knp-paginator-bundle": "^2.7", // v2.8.0
        "knplabs/knp-snappy-bundle": "^1.6", // v1.6.0
        "knplabs/knp-time-bundle": "^1.8", // v1.9.1
        "league/flysystem-aws-s3-v3": "^1.0", // 1.0.23
        "league/flysystem-cached-adapter": "^1.0", // 1.0.9
        "league/html-to-markdown": "^4.8", // 4.8.2
        "liip/imagine-bundle": "^2.1", // 2.1.0
        "nexylan/slack-bundle": "^2.1,<2.2.0", // v2.1.0
        "oneup/flysystem-bundle": "^3.0", // 3.1.0
        "php-http/guzzle6-adapter": "^1.1", // v1.1.1
        "sensio/framework-extra-bundle": "^5.1", // v5.4.1
        "stof/doctrine-extensions-bundle": "^1.3", // v1.3.0
        "symfony/asset": "^4.0", // v4.3.4
        "symfony/console": "^4.0", // v4.3.4
        "symfony/flex": "^1.9", // v1.17.6
        "symfony/form": "^4.0", // v4.3.4
        "symfony/framework-bundle": "^4.0", // v4.3.4
        "symfony/mailer": "4.3.*", // v4.3.4
        "symfony/messenger": "4.3.*", // v4.3.4
        "symfony/orm-pack": "^1.0", // v1.0.6
        "symfony/security-bundle": "^4.0", // v4.3.4
        "symfony/sendgrid-mailer": "4.3.*", // v4.3.4
        "symfony/serializer-pack": "^1.0", // v1.0.2
        "symfony/twig-bundle": "^4.0", // v4.3.4
        "symfony/twig-pack": "^1.0", // v1.0.0
        "symfony/validator": "^4.0", // v4.3.4
        "symfony/web-server-bundle": "^4.0", // v4.3.4
        "symfony/webpack-encore-bundle": "^1.4", // v1.6.2
        "symfony/yaml": "^4.0", // v4.3.4
        "twig/cssinliner-extra": "^2.12", // v2.12.0
        "twig/extensions": "^1.5", // v1.5.4
        "twig/inky-extra": "^2.12" // v2.12.0
    },
    "require-dev": {
        "doctrine/doctrine-fixtures-bundle": "^3.0", // 3.2.2
        "easycorp/easy-log-handler": "^1.0.2", // v1.0.7
        "fzaninotto/faker": "^1.7", // v1.8.0
        "symfony/browser-kit": "4.3.*", // v4.3.5
        "symfony/debug-bundle": "^3.3|^4.0", // v4.3.4
        "symfony/dotenv": "^4.0", // v4.3.4
        "symfony/maker-bundle": "^1.0", // v1.13.0
        "symfony/monolog-bundle": "^3.0", // v3.4.0
        "symfony/phpunit-bridge": "^3.3|^4.0", // v4.3.4
        "symfony/profiler-pack": "^1.0", // v1.0.4
        "symfony/var-dumper": "^3.3|^4.0" // v4.3.4
    }
}