Chapters

24 Chapters | 2:45:42 |

01

Setup: Server Provisioning

8:38
02

Ansistrano Role Installation

5:50
03

Anatomy of an Ansistrano Deploy

4:40
04

Deploy with git

3:46
05

Virtual Host Setup

5:54
06

Deploying Keys & Private Repos

8:40
07

Ansistrano Stages & Shared Files

3:11
08

Deploy Hooks & parameters.yml

5:25
09

parameters.yml: Handling Secret Config

6:48
10

Composer & Cache Permissions

7:46
11

Building Webpack Encore Assets

8:56
12

Database Setup

6:21
13

Safe Migrations

7:10
14

Faster Deploy with Shared Files

7:01
15

Logs, Sessions & File Permissions

5:32
16

Cache Permission Secrets

6:34
17

Optimizing with Cache

3:14
18

Priming cache.app

7:00
19

Cleanup & GitHub OAuth Token

7:30
20

Deploying to Multiple Servers

7:11
21

Load Balancer & Reverse Proxy Setup

9:36
22

CircleCI: Auto-Deploy my Code!

10:29
23

Optimizing Performance!

11:14
24

When things go wrong: Rollback

7:11

> Dev Tools > Animated Deployment with Ansistrano

Buy Access to Course

22.

CircleCI: Auto-Deploy my Code!

Autoplay

Keep on Learning!

If you liked what you've learned so far, dive in! Subscribe to get
access to this tutorial plus video, code and script downloads.

Start your All-Access Pass

Buy just this tutorial for $9.00

Previous Chapter

Next Chapter

With a Subscription, click any sentence in the script to jump to that part of the video!

This is not a tutorial about testing... but we couldn't resist! Our project actually does have a small test suite. Find your local terminal. To run them, execute:

./vendor/bin/simple-phpunit

This is a wrapper around PHPUnit. It will install some dependencies the first time you try it and then... go tests go! They pass! Despite our best efforts, we haven't broken anything.

So here is my lofty goal: I want to configure our project with continuous integration on CircleCI and have CircleCI deploy for us, if the tests pass. Woh.

CircleCI Setup

In your browser, go to https://circleci.com and login. I'll make sure I'm under my own personal organization. Then go to projects and add a new project: our's is called ansistrano-deploy.

To use CircleCI, we will need a config.yml file. Don't worry about that yet! Live dangerously and just click "Start Building": this will activate a GitHub webhook so that each code push will automatically create a new CircleCI build. The power!

Actually, this starts our first build! But since we don't have that config.yml file yet, it's not useful.

Creating .circleci/config.yml

Head back to your editor. If you downloaded the "start" code for the course, you should have a tutorial/ directory with a circleci-config.yml file inside. To make CircleCI use this, create a new .circleci directory and paste it there: but call it just config.yml:

            Copy Code

                        87 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
              build_and_test:
        
                working_directory: ~/mootube
        
                docker:
        
                  - image: php:7.1
        
                  - image: mysql:5.7
        
                    environment:
        
                      MYSQL_ALLOW_EMPTY_PASSWORD: yes
        
                steps:
        
                  # Installation
        
                  - run:
        
                      name: Install System Packages
        
                      command: apt-get update && apt-get -y install git unzip zlib1g-dev
        
                  - run:
        
                      name: Install PHP Extensions
        
                      command: docker-php-ext-install pdo pdo_mysql zip
        
                  - run:
        
                      name: Install Composer
        
                      command: |
        
                        php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
        
                        php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
        
                        php composer-setup.php && \
        
                        php -r "unlink('composer-setup.php');" && \
        
                        chmod +x ./composer.phar && \
        
                        mv ./composer.phar /usr/local/bin/composer
        
                  # Dependencies
        
                  - checkout
        
                  - restore_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
        
                  - run: composer install --prefer-dist --no-interaction
        
                  # Force pulling Simple PHPUnit dependencies to be able to cache them as well
        
                  - run: ./vendor/bin/simple-phpunit --version
        
                  - save_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
        
                      paths:
        
                        - '/root/.composer/cache'
        
                        - './vendor'
        
                  # Database
        
                  - run: ./bin/console doctrine:database:create --env=test
        
                  - run: ./bin/console doctrine:schema:create --env=test
        
                  - run: ./bin/console hautelook_alice:doctrine:fixtures:load --no-interaction
        
                  # To use server:start we need to install pcntl extension
        
                  - run:
        
                      name: Run web server in background
        
                      command: ./bin/console server:run
        
                      background: true
        
                  # Testing
        
                  - run: ./bin/console lint:yaml app/config
        
                  - run: ./bin/console lint:twig app/Resources
        
                  - run: ./vendor/bin/simple-phpunit
        
              deploy:
        
                working_directory: ~/mootube
        
                docker:
        
                  - image: ansible/ansible:ubuntu1604
        
                steps:
        
                  # Installation
        
                  - run:
        
                      name: Install System Packages
        
                      command: pip install --upgrade pip && pip install ansible
        
                  # Dependencies
        
                  - checkout
        
                  - restore_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
        
                  - run: ansible-galaxy install -r ansible/requirements.yml
        
                  - save_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
        
                      paths:
        
                        - '/root/.ansible/roles'
        
                  # @TODO Deploy to AWS here...
        
            workflows:
        
              version: 2
        
              build_test_and_deploy:
        
                jobs:
        
                  - build_and_test
        
                  - deploy:
        
                      requires:
        
                        - build_and_test

We will talk about this file in a minute... but heck! Let's get crazy and just try it first! Back on your local terminal, add that directory and commit:

git add .circleci/
git commit -m "Adding CircleCI config"

Push wrecklessly to master! This should create a new build... there it is! It's build #7... because - to be totally honest - I was doing a bit of practicing before recording. I usually try to hide that... but I'm busted this time...

Anyways, click into the build. Ah, we're on some "Workflow" screen, and you can see two different builds: build_and_test and deploy.

Builds and Workflows in config.yml

Go back to config.yml. Under jobs, we have one called build_and_test:

            Copy Code

                            Show All Lines

                        87 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
              build_and_test:
        
                Show Lines

                                                    // ... lines 4 - 87

It sets up our environment, installs composer, configures the database and... eventually, runs the tests!

            Copy Code

                            Show All Lines

                        87 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
              build_and_test:
        
                working_directory: ~/mootube
        
                docker:
        
                  - image: php:7.1
        
                  - image: mysql:5.7
        
                    environment:
        
                      MYSQL_ALLOW_EMPTY_PASSWORD: yes
        
                steps:
        
                  # Installation
        
                  - run:
        
                      name: Install System Packages
        
                      command: apt-get update && apt-get -y install git unzip zlib1g-dev
        
                  - run:
        
                      name: Install PHP Extensions
        
                      command: docker-php-ext-install pdo pdo_mysql zip
        
                  - run:
        
                      name: Install Composer
        
                      command: |
        
                        php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && \
        
                        php -r "if (hash_file('SHA384', 'composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" && \
        
                        php composer-setup.php && \
        
                        php -r "unlink('composer-setup.php');" && \
        
                        chmod +x ./composer.phar && \
        
                        mv ./composer.phar /usr/local/bin/composer
        
                  # Dependencies
        
                  - checkout
        
                  - restore_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
        
                  - run: composer install --prefer-dist --no-interaction
        
                  # Force pulling Simple PHPUnit dependencies to be able to cache them as well
        
                  - run: ./vendor/bin/simple-phpunit --version
        
                  - save_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./composer.lock" }}-v1
        
                      paths:
        
                        - '/root/.composer/cache'
        
                        - './vendor'
        
                  # Database
        
                  - run: ./bin/console doctrine:database:create --env=test
        
                  - run: ./bin/console doctrine:schema:create --env=test
        
                  - run: ./bin/console hautelook_alice:doctrine:fixtures:load --no-interaction
        
                  # To use server:start we need to install pcntl extension
        
                  - run:
        
                      name: Run web server in background
        
                      command: ./bin/console server:run
        
                      background: true
        
                  # Testing
        
                  - run: ./bin/console lint:yaml app/config
        
                  - run: ./bin/console lint:twig app/Resources
        
                  - run: ./vendor/bin/simple-phpunit
        
                Show Lines

                                                    // ... lines 56 - 87

But we also have a second job: deploy:

            Copy Code

                            Show All Lines

                        87 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 87

The whole point of this job is to install Ansible and get ready to run our Ansistrano deploy. We're not actually doing this yet... but the environment should be ready:

            Copy Code

                            Show All Lines

                        87 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                working_directory: ~/mootube
        
                docker:
        
                  - image: ansible/ansible:ubuntu1604
        
                steps:
        
                  # Installation
        
                  - run:
        
                      name: Install System Packages
        
                      command: pip install --upgrade pip && pip install ansible
        
                  # Dependencies
        
                  - checkout
        
                  - restore_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
        
                  - run: ansible-galaxy install -r ansible/requirements.yml
        
                  - save_cache:
        
                      key: mootube-{{ .Branch }}-{{ checksum "./ansible/requirements.yml" }}-v1
        
                      paths:
        
                        - '/root/.ansible/roles'
        
                  # @TODO Deploy to AWS here...
        
                Show Lines

                                                    // ... lines 78 - 87

The real magic is down below under workflows:

            Copy Code

                            Show All Lines

                        87 lines
            |
            .circleci/config.yml
        
                Show Lines

                                                    // ... lines 1 - 78
                            
            workflows:
        
              version: 2
        
              build_test_and_deploy:
        
                jobs:
        
                  - build_and_test
        
                  - deploy:
        
                      requires:
        
                        - build_and_test

The one workflow lists both builds. But, thanks to the requires config, the deploy job will only run if build_and_test is successful. That's super cool.

Back on CircleCI, that job did finish successfully, and deploy automatically started. This should setup our Ansible-friendly environment... but it will not actually deploy yet.

CircleCI Environment Vars and the Vault Pass

It's time to fix that! In config.yml, under deploy, run the normal deploy command: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --ask-vault-pass:

            Copy Code

                            Show All Lines

                        88 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 76
                            
                  # Deploy
        
                  - run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --ask-vault-pass
        
                Show Lines

                                                    // ... lines 79 - 88

And in theory... that's all we need! But... do you see the problem? Yep: that --ask-vault-pass option is not going to play well with CircleCI.

We need a different solution. Another option you can pass to Ansible is --vault-password-file that points to a file that holds the password. That's better... but how can we put the password in a file... without committing that file to our repository?

The answer! Science! Well yes, but more specifically, environment variables!

Back in CircleCI, configure the project. Find "Environment Variables" and add a new one called ANSIBLE_VAULT_PASS set to beefpass. Back in config.yml, before deploying, we can echo that variable into a file: how about ./ansible/.vault-pass.txt:

            Copy Code

                            Show All Lines

                        90 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 76
                            
                  # Deploy
        
                  - run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
        
                Show Lines

                                                    // ... lines 79 - 90

Use that on the next line: --vault-password-file= and then the path:

            Copy Code

                            Show All Lines

                        90 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 76
                            
                  # Deploy
        
                  - run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
        
                  - run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --vault-password-file=./ansible/.vault-pass.txt
        
                Show Lines

                                                    // ... lines 80 - 90

To be extra safe, delete it on the next line:

            Copy Code

                            Show All Lines

                        90 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 76
                            
                  # Deploy
        
                  - run: echo $ANSIBLE_VAULT_PASS > ./ansible/.vault-pass.txt
        
                  - run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini --vault-password-file=./ansible/.vault-pass.txt
        
                  - run: rm ./ansible/.vault-pass.txt
        
                Show Lines

                                                    // ... lines 81 - 90

And... I'll fix my ugly YAML.

Setting Ansible Variables

Ok, problem solved! Time to deploy, right!? Well... remember how we added that prompt at the beginning of each deploy? Yep, that's going to break things too! No worries: Ansible gives us a way to set variables from the command line. When we do that, the prompt will not appear. How? Add a -e option with: git_branch=master:

            Copy Code

                            Show All Lines

                        90 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                Show Lines

                                                    // ... lines 58 - 78
                            
                  - run: ansible-playbook ansible/deploy.yml -i ansible/hosts.ini -e "git_branch=master" --vault-password-file=./ansible/.vault-pass.txt
        
                Show Lines

                                                    // ... lines 80 - 90

Disabling Host Key Checking

Ready to deploy... now!? Um... not so fast. Scroll up a little. Under the docker image, we need to add one environment variable: ANSIBLE_HOST_KEY_CHECKING set to no:

            Copy Code

                            Show All Lines

                        92 lines
            |
            .circleci/config.yml
        
            version: 2
        
            jobs:
        
                Show Lines

                                                    // ... lines 3 - 56
                            
              deploy:
        
                working_directory: ~/mootube
        
                docker:
        
                  - image: ansible/ansible:ubuntu1604
        
                    environment:
        
                      ANSIBLE_HOST_KEY_CHECKING: no
        
                Show Lines

                                                    // ... lines 63 - 92

Whenever you SSH to a machine for the first time, SSH prompts you to verify the fingerprint of that server. This disables that. If you have a highly sensitive environment, you may need to look into actually storing the fingerprints to your servers instead of just disabling this check.

Finally... I think we're ready! Go back to your local terminal, commit the changes, and push!

Adding ssh Keys

Go check it out. Ah, here is the new build: the build_and_test job starts off immediately. Let's fast-forward. But watch, when it finishes.... yes! Visually, you can see it activate the second job: deploy.

Inside this job, it sets up the environment first. When it starts running our tasks... woh! It fails! Ah:

Failed to connect to host via ssh... no such identity... permission denied

Of course! CircleCI is trying to SSH onto our servers, but it does not have access. This works on our local machine because, when we deploy to the aws hosts, the group_vars/aws.yml file is loaded. This tells Ansible to look for the SSH key at ~/.ssh/KnpU-Tutorial.pem:

            Copy Code

                            Show All Lines

                        6 lines
            |
            ansible/group_vars/aws.yml
        
            ---
        
                Show Lines

                                                    // ... line 2
                            
            ansible_ssh_private_key_file: ~/.ssh/KnpU-Tutorial.pem
        
                Show Lines

                                                    // ... lines 4 - 6

That path does not exist in CircleCI.

So... Hmmm... We could leverage environment variables to create this file... but great news! CircleCI gives us an easier way. Open up the key file and copy all of its contents. Then, in CircleCI, configure the project and look for "SSH Permissions". Add a new one: paste the key, but leave the host name empty. This will tell CircleCI to use this key for all hosts.

We are ready! In CircleCI, I'll just click rebuild. It skips straight to the deploy job and starts setting up the environment. Then... yes! It's running our playbook! OMG, go tell your co-workers! The machines are deploying the site to the other machines! It takes a minute or two... but it finishes! CircleCI just deployed our site automatically.

There's no visible difference, but we are setup!

Next, let's talk about some performance optimizations that we need to make to our deploy.

2 Comments

Sort By

Permana Jayanta 5 years ago edited

Somehow I always get error on checkout step.

Directory (/home/circleci/MY_PROJECT_FOLDER) you are trying to checkout to is not empty and not a git repository

Probably someone had this kind of error before?

| Reply |

weaverryan SFCASTS Permana Jayanta 5 years ago edited

Yo Permana Jayanta !

Hmm, that's very interesting! I haven't gotten this before. Before this "step" that fails on CircleCI, I would add a ls -lR /home/circleci/MY_PROJECT_FOLDER - let's see what's inside that directory and find out why it's not empty!

Cheers!