Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

The Secrets Vault

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $10.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

How do does deployment work with environment variables? Because, in a real app, we're going to have a bunch of sensitive environment variables - like database username & password, API keys and more.

Deployment 101

I don't want to get too far into the topic of deployment right now, but here's the general idea. Step 1: get your code onto your production machine and run composer install to populate the vendor/ directory. Step 2, somehow create a .env.local file with all your production values. And step 3, run:

php bin/console cache:clear

to clear the production cache. The Symfony documentation has more details, but... it's basically that simple.

The trickiest part is step 2: creating the .env.local file. Somehow, your deployment system needs to have access to the sensitive environment variable values so it can populate this file. But, since we're not committing those to our repository... where should we store them?

Hello Secrets Vaults

To solve this problem, a concept called a "Secrets Vault" has been invented. And there are actually several cloud-based vaults where you can store your secrets in their system, then securely read them while you're deploying. Those are excellent options.

Symfony also comes with its own secrets vault, which is super cool because it allows us to commit our sensitive values - called secrets - into Git!

Dumping the Sentry Connection Details

To help us see this, in QuestionController::show(), add a third argument: HubInterface $sentryHub. Below, dump($sentryHub):

Tip

In newer versions of SentryBundle, you may need to dump($sentryHub->getClient()) to see the data that we see here.

... lines 1 - 6
use Sentry\State\HubInterface;
... lines 8 - 12
class QuestionController extends AbstractController
{
... lines 15 - 42
public function show($slug, MarkdownHelper $markdownHelper, HubInterface $sentryHub)
{
dump($sentryHub);
... lines 46 - 64
}
}

The main purpose of SentryBundle's services is not for us to interact with them directly like this. But this will be a handy way to quickly see our SENTRY_DSN value. By the way, I found this interface, of course, by using debug:autowiring.

Check it out: back on your browser, I'll close the sentry.io tab and refresh. Down on the web debug toolbar, it dumps a Hub object. If you expand the stack property... and expand again, again, and again, there it is! By digging, we can see that our production SENTRY_DSN value is being used.

Creating the Vault

Here's the goal: we're going to move the SENTRY_DSN environment value into our vault. A vault is basically a collection of encrypted values. And in Symfony, you'll have two vaults: one for the dev environment - which will contain non-sensitive default values - and a separate one for the prod environment with the real values.

So, each "secret" will need to be set in both vaults. Let's start by putting SENTRY_DSN into the dev vault. How do we do that? Find your terminal and run a shiny new command:

php bin/console secrets:set SENTRY_DSN

Because we're in the dev environment, this will populate the dev vault. And, a good value in dev is actually an empty string. If SENTRY_DSN is empty, Sentry is disabled.

So, just hit "Enter". Ah!

Warning: No value provided, aborting.

This is a bug in Symfony where it doesn't allow an empty secret. It's already been fixed and an empty value is allowed in Symfony 5.0.8. So hopefully, you won't get this.

If you do, we can work around it: re-run the command with a - on the end, which tells Symfony to read from STDIN.

php bin/console secrets:set SENTRY_DSN -

Then, as crazy as it sounds, hit Control+D - as in "dog". That was just a fancy way to set SENTRY_DSN to an empty string.

The config/secrets/ Public And Private Key Files

Because this was the first secret that we set into the dev vault, it automatically initialized the vault. It says:

Sodium keys have been generated at config/secrets/dev/

Let's go check that out! Open up the new config/secrets/ directory. Excellent: this has a dev/ sub-directory because we just created the dev vault.

The dev.encrypt.public.php file returns the key that's used to add or update secrets:

... lines 1 - 2
return "\xE2\xDBrR\x9Fz\xC7\xED\x2B\x23\x0F\xB0\x14\x00\xD2d\xC0N\x3AU\xD3M\xC5\xA8\x012\x80\xA2\xA2\xEBFq";

It's used to encrypt secrets. The dev.decrypt.private.php file does the opposite: its value is used to decrypt the secrets so that our app can read them:

... lines 1 - 2
return "\x86\xB75m\xEFM\x11\x04\x15\xFB\x03\xC8\xF5\xA2b9\xF0eU\xFF\xEA\xD5\x0F\x06\xAC\x05\x89\xC8\x08\x7F\x8A\x9E\xE2\xDBrR\x9Fz\xC7\xED\x2B\x23\x0F\xB0\x14\x00\xD2d\xC0N\x3AU\xD3M\xC5\xA8\x012\x80\xA2\xA2\xEBFq";

The decrypt key is usually a sensitive value that we would not commit to the repository. However, we usually do commit the decrypt key for the dev vault for two reasons. First, the values in the dev vault are hopefully not very sensitive. And second, we do want other developers on our team to be able to decrypt the dev secrets locally. Otherwise... their code won't work.

This directory will also contain one file per secret. We will commit this because it's encrypted.

Creating the prod Vault

Let's repeat the same process to put SENTRY_DSN into the prod vault. Run the command again but also pass --env=prod:

php bin/console secrets:set SENTRY_DSN --env=prod

For the value, open .env.local, copy the long DSN string, then paste here. You won't see the value because the command is hiding it for security purposes.

And... boom! This generated the prod vault and encrypted the secret. Check out config/secrets/prod. It has the same files, but the output had one extra, angry looking note:

DO NOT COMMIT THE DECRYPTION KEY FOR THE PROD ENVIRONMENT

It's talking about prod.decrypt.private.php. This file does need to be here in order for our app to decrypt & read the prod secrets. But we are not going to commit it. This is the one sensitive value that your deploy script will need to know about.

Tip

Instead of creating the prod.decrypt.private.php file when deploying, you can also set the key on a SYMFONY_DECRYPTION_SECRET environment variable. See Production Secrets for more info.

And notice how this is a different color in my editor? That's because... in our .gitignore file, we are already ignoring the prod.decrypt.private.php file:

18 lines .gitignore
... line 1
###> symfony/framework-bundle ###
... lines 3 - 5
/config/secrets/prod/prod.decrypt.private.php
... lines 7 - 9
###
... lines 11 - 18

Committing the Secrets Vaults

Cool! Let's commit them! At your terminal... run:

git add config/secrets

And then:

git status

Yes! This added the encrypted secret values themselves, both the encrypt and decrypt keys for the dev environment, but only the encrypt key for prod. Other developers will be able to add new keys for prod, but not read them. Isn't encryption cool?

Now that our vaults are set up, let's use these secret values in our app! Doing that will be easier than you think. Let's tackle it next.

Leave a comment!

28
Login or Register to join the conversation
Andrey Avatar

I'm still trying to wrap my head around this concept of secrets vault. So in production, there is one file which is not committed but needs to be deployed, but it allows also to read all the sensitive values. Isn't it exactly what .env.local does? Can someone clarify the conceptual difference here?

Reply

Hey Andrey V.

> So in production, there is one file which is not committed but needs to be deployed, but it allows also to read all the sensitive values.
Yes, that's the prod.decrypt.private.php file. You should not commit it

> Isn't it exactly what .env.local does?
In a general level, the difference between two files is that the .env file contains all your sensitive data but it's not encrypted, anyone who has read access to it can steal your production credentials.
By using secrets you're increasing your security by having 2 files (public and private key), if someone has access to only one of them, then you're still safe but of course, if someone steals your private key and has access to your repository, you're in trouble

Cheers!

Reply
Andrey Avatar
public and private key


Symfony has encrypt key and decrypt key; decrypt key must be present locally in order for the whole app to work. This means that whoever else works with me on the project, will still have access to all the secrets.

Reply

Yes, and that's why you can create a dev set of keys, so, you can share those with your team and hide somewhere else the decrypt file for your production public key

Reply
Kevin Avatar

The decrypt key is usually a sensitive value that we would not commit to the repository. However, we usually do commit the decrypt key for the dev vault for two reasons. First, the values in the dev vault are hopefully not very sensitive. And second, we do want other developers on our team to be able to decrypt the dev secrets locally. Otherwise... their code won't work.

A little unclear about this. What kind of information is stored in the dev vault if sensitive data should not be there? In the real world what are some examples of things developers on a team need to decrypt locally?

Reply

Hey Kevin!

That's a fair question :). Suppose you're using some 3rd party service - e.g. Stripe for payments. To use Stripe, you have some secret key - let's say it's called STRIPE_SECRET. On your prod vault, you would store the real, sensitive production value. But locally, while developing, you STILL need to set STRIPE_SECRET to *something* so that you can use the site and test payment. In the case of Stripe, they have a "sandbox" system with its own "secret". And so, in the dev vault, you would set the STRIPE_SECRET to the "sandbox" value. This isn't really sensitive data. Well, you probably would not want it to be shared publicly, of course :). But within your team, you really want everyone to have easy access to this. Hence, you put it in your dev vault and commit the dev decrypt key to the vault so that anyone can read it.

Let me know if that helps :). This idea of committing the dev decrypt key to the repo is not something that everyone agrees on.

Cheers!

1 Reply
Kevin Avatar

Great example. Thank you!

Reply

Hmm I'm just not entirely sold yet on why to use Symfony secrets due to the what the documentation states about deploying to production. Because you need to either upload the private key and keep it there, or use the private key to decrypt them to the local vault, which creates the ".env.prod.local" file, with decrypted secrets (and then delete the private key afterwards).

You could just add a (or multiple) secret(s) with the contents of the dotenv file to your CI/CD with the help of like say Azure or Github secrets, and then write the content of those secrets to the ".env.prod.local" file, and end up with the same result. Or am I missing something crucial here?

Reply

Hey Senet

Yea, it's usually easier to set up the application secrets via your host provider. But, this is a built-in way to manage secrets in case your host does not give you any convenient way to handle secrets

Cheers!

Reply
Jakub Avatar

Im using 5.0.11 Symfony and this problem still exists. What's more ^D doesnt work for me. Nothing happens. Any ideas what am I doing wrong?

Reply
Jakub Avatar

Ok, got this. I were missing that
composer require paragonie/sodium_compat

Reply

Hey HudyWeas,

Great, I'm happy you were able to solve this yourself, well done!

Cheers!

Reply

This concept I already saw on Docker. The single security improvement that I can see, is that you can commit many prod variables and put effort only in keep the decrypt key far away of public access. But, in case of an invasion on the server, the vault won't ensure any protection. Let me know whether have a second benefit.

Reply

Hey @Daniel

> in case of an invasion on the server
If that ever happen I'd say you have other bigger security issues to worry about. I'm not sure if there is a way to protect your prod environment variables in that case.

> Let me know whether have a second benefit.
It's an easier way to work with env variables. You don't depend now on your hosting provider (they usually offer you a way to inject env vars). Perhaps there are other benefits but that's the only one I can think right now

Cheers!

Reply
Jörg daniel F. Avatar
Jörg daniel F. Avatar Jörg daniel F. | posted 1 year ago

Hello,
when I try "HubInterface $sentryHub" and dumb that. so I get a "Sentry\State\HubAdapter {#5831}" an not like in the video "Sentry\State\Hub"
what did I wrong?

Reply

Hey Jörg daniel F.!

That's probably ok. In fact, it's kind of cool :). This is because you're using a newer version of the bundle that I am. And for some reason (https://github.com/getsentr... they changed this service from using Hub to HudAdapter (both implement HubInterface). The reason I think this is cool is the fact that autowiring forces us to type-hint the interface (HubInterface). Then, the fact that the underlying implementation changed doesn't really matter to us: all we care about is that we're passed a HubInterface, which both Hub and HubAdapter implement :).

Anyways, the short answer is that you're not doing anything wrong!

Cheers!

1 Reply
Nick-F Avatar

Ryan, you're not getting the issue. When "HubAdapter" is dumped, it is completely empty and does not have any of the information that "Hub" dumped out, as shown in this video.

1 Reply

Hey Nick F.!

Ah! I didn't realize it was empty in newer version - just that it had a different class name like stated in the original comment :). I'll check into adding a note to the video.

Thanks!

Reply
Nick-F Avatar

You can't just run dump($sentryHub); anymore because the HubAdapter won't have any data attached to it
You have to run
dump($sentryHub->getClient()->getOptions());
which will give you the data that he's looking at in the video

Reply

Hey Name->Nick,

Thank you for this tip!

Cheers!

Reply

The question if we don't git commit the private key in production, how the application gonna decrypt my env variables ?

Reply

Hey ahmedbhs

That's a good question and the answer relies on your hosting server, or deployment mechanism. It may provide you a way to define environment variables where you can manually set up with your prod's key. Another way is to SSH onto your server and install the key but as I said it depends on how you deploy your application

Cheers!

Reply
Tobias I. Avatar
Tobias I. Avatar Tobias I. | posted 2 years ago

Hey! there

While following this video, I thought an obvious environment variable to encrypt in the way shown would be APP_SECRET. But after setting the value for my dev secrets and commenting out the variable in .env, I get Environment variable not found: "APP_SECRET". when refreshing any page. Why is that?

Reply

Hey Tobias I.!

That is *indeed* weird! I can't repeat it on this project: I removed the APP_SECRET env var, went to a page that uses this, and DID see the "Environment variable not found" error. Then I added it to the dev vault and... boom! Everything worked. So, it could be something specific to your app, but I can't imagine what it would be. Are you using the code from this project? Doing something else? What does your stack trace look like? That may help indicate who is trying to reference this.

Cheers!

Reply
Tobias I. Avatar

Hey weaverryan

Sorry for the late reply. I just tested this again on a fresh install of symfony and it works as expected. In my original project i made a silly typo, so of course it was yelling at me.

One thing I noticed (not necessarily with the secret vault but with doctrine) is, if you want to store the production database user and password in the vault, you have to remove the DATABASE_URL key from your .env file as it overrides any other config. This behavior is described in the docs, but it threw me off for a seconds.

1 Reply

Hey Tobias I.!

Ah, happy you figured it out! Nice!

> if you want to store the production database user and password in the vault, you have to remove the DATABASE_URL key from your .env file as it overrides any other config

Yep, this is "by design", but can be tricky because you don't get an error when this happens. The way I think about secrets vs env vars is: either something is 100% a secret (and not an env var at all) or 100% an env var (and not a secret at all) - if you go "half way", things won't work. Of course, you can use this "env var overrides secret" to your advantage to, for example, override a secret in the test environment. But first, yes, you need to understand not to cross these ideas :).

Cheers!

Reply
Default user avatar
Default user avatar wuwu5431 | posted 2 years ago

Is this "Secret Vault" section optional or essential ?

Reply

I'd say it's optional if you already have a way to store your secrets. If not, then it's valuable to learn about Symfony Secrets so you can store all your sensitive data in a nice and convenient way

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial also works great for Symfony 6!

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.3.0 || ^8.0.0",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "knplabs/knp-markdown-bundle": "^1.8", // 1.9.0
        "sensio/framework-extra-bundle": "^6.0", // v6.2.1
        "sentry/sentry-symfony": "^4.0", // 4.0.3
        "symfony/asset": "5.0.*", // v5.0.11
        "symfony/console": "5.0.*", // v5.0.11
        "symfony/debug-bundle": "5.0.*", // v5.0.11
        "symfony/dotenv": "5.0.*", // v5.0.11
        "symfony/flex": "^1.3.1", // v1.17.5
        "symfony/framework-bundle": "5.0.*", // v5.0.11
        "symfony/monolog-bundle": "^3.0", // v3.6.0
        "symfony/profiler-pack": "*", // v1.0.5
        "symfony/routing": "5.1.*", // v5.1.11
        "symfony/twig-pack": "^1.0", // v1.0.1
        "symfony/var-dumper": "5.0.*", // v5.0.11
        "symfony/webpack-encore-bundle": "^1.7", // v1.8.0
        "symfony/yaml": "5.0.*" // v5.0.11
    },
    "require-dev": {
        "symfony/maker-bundle": "^1.15", // v1.23.0
        "symfony/profiler-pack": "^1.0" // v1.0.5
    }
}