Private Downloads & Signed URLs

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $10.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

I have one more performance enhancement I want to do. If you click download, it works great! But if these files were bigger, you'd start to notice that the downloads would be kinda slow! Open up ArticleReferenceAdminController and search for download. Remember: we're reading a stream from S3 and sending that directly to the user. That's cool... but it also means that there's a middleman in the process: our server! That slows things down. Couldn't we somehow give the user direct access to the file on S3?

Go back to our bucket, head to its root directory, then click into article_reference. If you click any of these files, each does have a URL. But if you try to go to it, it's not public. That's great because these files are meant to be private... but it sorta ruins our idea of pointing users directly to this URL.

Well, good news! We can have our cake and eat it too... as we say... for some reason in English. Um, we can have the best of both worlds with... signed URLs.

Hello Signed URLs

Signed URLs are not something that we can create with Flysystem - it's specific to S3. So, instead of using our Filesystem object, we'll deal with S3 directly, which turns out to be pretty awesome!

Google for "S3 PHP client signed url" to find their docs about this. Signed URLs let us say:

Hey S3! I want to create a public URL to download this file... but I only want the link to be valid for, like, 20 minutes.

Cool, right! Because the link is temporary, it's ok to let users use it.

We'll do this by interacting with the S3Client object directly... which is super awesome because, a few minutes ago, we registered an S3Client service so we could use it with Flysystem. Half our job is already done!

The other thing we'll need is the bucket name.

Creating the Signed URL

Head back to downloadArticleReference(). Remove the UploaderHelper argument - we won't need that anymore - and add S3Client $s3client. Also add string $s3BucketName.

... lines 1 - 7
use Aws\S3\S3Client;
... lines 9 - 21
class ArticleReferenceAdminController extends BaseController
{
... lines 24 - 127
public function downloadArticleReference(ArticleReference $reference, S3Client $s3Client, string $s3BucketName)
{
... lines 130 - 139
}
... lines 141 - 192
}

That won't autowire, so copy the argument name, open up services.yaml and add a bind for this $s3BucketName:. For the value, copy the environment variable bucket syntax from before and... paste.

... lines 1 - 10
services:
... line 12
_defaults:
... lines 14 - 20
bind:
... lines 22 - 25
$s3BucketName: '%env(AWS_S3_BUCKET_NAME)%'
... lines 27 - 61

Cool! Back in the controller, copy the $disposition line - we're going to put this back in a minute. Then, delete everything after the security check, paste the $disposition line, but comment it out for now.

Ok, let's go steal some code from the docs! We already have the S3Client object, so just grab the rest. Paste that then... let's see... replace my-bucket with the $s3BucketName variable. For Key, that's the file path: $reference->getFilePath(). And, for $request = $s3Client->createPresignedRequest(), you can use whatever lifetime you want. These files are pretty small, so we don't need too much time - but let's make the URLs live for 30 minutes.

... lines 1 - 7
use Aws\S3\S3Client;
... lines 9 - 21
class ArticleReferenceAdminController extends BaseController
{
... lines 24 - 127
public function downloadArticleReference(ArticleReference $reference, S3Client $s3Client, string $s3BucketName)
{
$article = $reference->getArticle();
$this->denyAccessUnlessGranted('MANAGE', $article);
$command = $s3Client->getCommand('GetObject', [
'Bucket' => $s3BucketName,
'Key' => $reference->getFilePath()
]);
$request = $s3Client->createPresignedRequest($command, '+30 minutes');
... lines 138 - 139
}
... lines 141 - 192
}

Now that we have this "request" thing... how can we get the URL? Back on their docs, scroll down... here it is: $request->getUri().

When the user hits our endpoint, what we want to do is redirect them to the URL. Do that with return new RedirectResponse(), (string) - they mentioned that in the docs, it turns the URI into a string - then $request->getUri().

... lines 1 - 12
use Symfony\Component\HttpFoundation\RedirectResponse;
... lines 14 - 21
class ArticleReferenceAdminController extends BaseController
{
... lines 24 - 127
public function downloadArticleReference(ArticleReference $reference, S3Client $s3Client, string $s3BucketName)
{
$article = $reference->getArticle();
$this->denyAccessUnlessGranted('MANAGE', $article);
$command = $s3Client->getCommand('GetObject', [
'Bucket' => $s3BucketName,
'Key' => $reference->getFilePath()
]);
$request = $s3Client->createPresignedRequest($command, '+30 minutes');
return new RedirectResponse((string) $request->getUri());
}
... lines 141 - 192
}

Let's try it! Refresh! And... download! Ha! It works! We're loading this directly from S3. This long URL contains a signature that proves to S3 that the request was pre-authenticated and should last for 30 minutes.

Forcing S3 Response Headers

But we did lose one thing: our Content-Disposition header! This gave us two nice things: it forced the user to download the file instead of loading it "inline", and it controlled the download filename.

Hmm, this is tricky. Now that the user is no longer downloading the file directly from us, we don't really have a way to set custom headers on the response. Well, actually, that's a big ol' lie! There are two ways to do that. First, you can set custom headers on each object in S3. Or you can hint to S3 that you want it to set custom headers on your behalf when the user goes to the signed URL.

How? Add another option to getCommand(): ResponseContentType set to $reference->getMimeType(). That'll hint to S3 that we want it to set a Content-Type header on the download response.

... lines 1 - 21
class ArticleReferenceAdminController extends BaseController
{
... lines 24 - 127
public function downloadArticleReference(ArticleReference $reference, S3Client $s3Client, string $s3BucketName)
{
... lines 130 - 137
$command = $s3Client->getCommand('GetObject', [
'Bucket' => $s3BucketName,
'Key' => $reference->getFilePath(),
'ResponseContentType' => $reference->getMimeType(),
... line 142
]);
... lines 144 - 146
}
... lines 148 - 201

And ResponseContentDisposition. Move the $disposition code up above, then use that value down here.

... lines 1 - 21
class ArticleReferenceAdminController extends BaseController
{
... lines 24 - 127
public function downloadArticleReference(ArticleReference $reference, S3Client $s3Client, string $s3BucketName)
{
... lines 130 - 132
$disposition = HeaderUtils::makeDisposition(
ResponseHeaderBag::DISPOSITION_ATTACHMENT,
$reference->getOriginalFilename()
);
$command = $s3Client->getCommand('GetObject', [
'Bucket' => $s3BucketName,
'Key' => $reference->getFilePath(),
'ResponseContentType' => $reference->getMimeType(),
'ResponseContentDisposition' => $disposition,
]);
... lines 144 - 146
}
... lines 148 - 201

Cool, right? Go download the file one more time. Ha! It downloads and uses the original filename. This is probably the best way to allow users to download private files. Oh, and if you need even faster downloads... cause S3 isn't that fast for large files, you can do the same thing with Cloudfront. Cloudfront is another service that gives users faster access to S3 files, and has a similar process for creating signed URLs.

Ok friends, only one thing left, and it's a fun one! Let's talk about how our file upload endpoint might look different if we were building a pure API.

Leave a comment!

  • 2019-07-25 Ionut Stoican

    Hello,

    I managed to get it working. After I set up my distribution, I just replaced my s3 bucket host parameter from my symfony parameters.yml with my Cloudfront domain name and it worked.

    Next step is to make signed URLs, I will dig a little before trying.

    Thanks a lot for your feedback.

  • 2019-07-25 weaverryan

    Hey Ionut Stoican!

    Fair question :). CloudFront can be used for two separate things... and how you integrate it is a bit different for those 2 different things.

    1) You can upload files to S3, but then you want CloudFront to actually serve these to your users. In this case, you set up a cloud front distribution that "pulls" from your S3 bucket. Getting the permissions/config correct *can* be a pain, from experience :/. But once you have it set up, you will have a new cloudfront URL that you can use instead of your S3 domain. Then, you can just replace your S3 URL in your code that we created in this tutorial with the new CloudFront URL. Basically, on the Symfony side, you're just rendering a different hostname (the CloudFront host name) to the assets.

    2) You can make CloudFront serve your static JavaScript/CSS assets. This is similar, except that you tell CloudFront to "pull" from your "origin". That's a fancy way of telling cloudfront that when someone asks for /css/main.css on CloudFront, to actually fetch it from YourDomain.com/css/main.css. Once you have this setup... you (once again) just need to make sure that your static assets all point to the cloud front domain in your application. How you do this depends a bit on if you're using Webpack Encore, but probably includes setting the "assets base_urls" setting to your new cloud front distribution domain. https://symfony.com/doc/cur...

    So, it is kind of a tricky topic... but actually has nothing to do with uploading... it's ALL about CloudFront configuration. You can also create signed URLs from CloudFront. You would configure a distribution like we did in part (1) above then get signed URLs from it. The config & authentication is kind of a pain - CloudFront is never friendly :/.

    If you have any specific questions, you're welcome to ask - we've implemented this a few times in the past.

    Cheers!

  • 2019-07-20 Ionut Stoican

    Hello,

    How would we go about integrating cloudfront into the symfony application?
    There aren't any good tutorials on the web. This was the main thing I was looking for in this course.

  • 2019-05-16 Ricardo Gonçalves

    Thanks weaverryan .

    In the past, I can't remember a case where the exact filename was a requirement, translating ñ to n or ç to c is actually normal. In any case, if needed, I'll set up the headers manually and well, let's hope the end user has the correct browser.

    Cheers!

  • 2019-05-16 weaverryan

    Ah, indeed! I learned something! Apparently the original HTTP spec says that you're only allowed to use ASCII characters in your string, which is why Symfony enforces that. But apparently many browsers *do* allow non-ASCII characters... but maybe not all of them :/ - https://stackoverflow.com/q...

    So, the answer is that if you DO need non-ASCII characters, you'll need to build the Content-Disposition header manually, but then it also may not work in all browsers :/. If you want to remove the characters, https://github.com/doctrine... might do it, but the character encoding stuff is not my string suit.

    Cheers!

  • 2019-05-16 Ricardo Gonçalves

    Everything works fine up until I use makeDisposition. I get an error like "the filename fallback must only contain ASCII characters.".
    Storing in the db is not an issue, everything worked correctly until this moment, and now that I think about it, it was working fine before using S3 (cannot be 100% because I stumbled upon this by accident, but I could swear the downloader worked correctly before).

  • 2019-05-16 weaverryan

    Hey Ricardo Gonçalves!

    Hmm... interesting question! I wonder at what point you start seeing issues? Storing the ç in the database should be fine (as long as you have utf8 encoding on your database). And that character should also be totally fine being used in a URL. At what point are you seeing the character having issues?

    Cheers!

  • 2019-05-16 Ricardo Gonçalves

    Hello Ryan,

    Great tutorial as always. I have just one question here, what if the filename has some non ASCII characters?? For example, a file with my lastname "Gonçalves.pdf". How would you suggest we could serve the file with its original filename?

    I used mb_convert_encoding but the result is not great.

    Thanks