Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine
This tutorial has a new version, check it out!

CircleCI Artifacts

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $10.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

The hardest thing about CI is when tests fail on CircleCI, but pass locally. When you have a lot of functional tests, this will happen more often than you think. Usually, it's a timing issue: you click to open a JavaScript modal and then click a link in that modal. This works locally... but for some reason that modal loads a little bit slower in CircleCI. Your test fails because you try to click the link too quickly. We talk about this in detail in our Behat tutorial.

The "Rebuild with SSH" option is great for this. But an even better tool is artifacts. Very simply, artifacts are a way for you to save data - like logs or browser screenshots when a test fails - and make it accessible from the web interface. Imagine seeing 4 test failures and seeing 4 screenshots right on the UI of what the browser looked like the moment those tests failed! We actually have a blog post about getting this setup. That post uses CircleCI version 1.0 - but once we talk about artifacts, you should be able to translate to version 2 without a problem.

Artifacts in Action

Let's see artifacts in action. Go back to the config file. At the end of the phpunit command add --log-junit ~/phpunit/junit.xml.

... lines 1 - 5
jobs:
build:
... lines 8 - 18
steps:
... lines 20 - 38
- run: ./vendor/bin/phpunit --log-junit ~/phpunit/junit.xml
... lines 40 - 46

This flag tells PHPUnit to output some logs in a standard "JUnit" format. This is basically a detailed diagnostic of what happened during the tests.

Now, add two more steps: store_test_results with a path option set to ~/phpunit. And another one called store_artifacts with that same option.

... lines 1 - 5
jobs:
build:
... lines 8 - 18
steps:
... lines 20 - 40
- store_test_results:
path: ~/phpunit
... line 43
- store_artifacts:
path: ~/phpunit

Let's commit this first, trigger the build, and then talk. Commit wildly... then push!

Understanding CircleCI Steps

A CircleCI build consists of these "steps", and each step uses a built-in step "type". The most common and useful type is run. But we're also using checkout near the top and now store_test_results and store_artifacts. These are all explained on that config reference page.

store_artifacts is the simpler of the two: whatever files we store as an artifact will become available for us to see & download via the web UI - or API. That means that we'll be able to see the junit.xml file... or any logs or screenshots we choose to store.

The store_test_results is also really cool. Thanks to this, CircleCI will parse the junit.xml file and learn things about your tests: like how long they took to run and their favorite color is... err... maybe not that one.

Anyways, let's go find our build! It passed! Wow, and it only took 48 seconds!

First, look under "Artifacts". Yea! Here is our junit.xml file. It's not super attractive, but you get the point. On KnpU, this is full of screenshots for any tests that failed.

Now, click on "Test Summary". Cool! 29 tests and 0 failures. And it even knows which tests are the slowest.

Go back and look at the previous build - build #3. The "Test Summary" part was empty! This data was filled in thanks to the store_test_results step. It's not mission-critical, but it's free functionality!

Other CI Uses

By the way, you can also use continuous integration for other things beyond tests, like enforcing code standards. By installing the php-cs-fixer, you could easily make your build fail if someone pushes code that doesn't follow the standards.

And... we're done! Thanks for traveling along with me, dodging dinosaurs, and putting them back into their Enclosures. It's a thankless job, but somebody has to do it.

With testing as a part of your toolkit, your life will be so much better. Tests allow us to create features faster and code more aggressively. They give us the confidence that we're not going to break something important on the site. And you guys know me: I'm a pragmatist. I don't test for some philosophical reason about code quality. I test because it allows me to ship a high-quality app with confidence.

And besides, using continuous integration is super fun! If that's not reason enough to write tests, I don't know what is.

Ok guys, seeya next time!

Leave a comment!

13
Login or Register to join the conversation

Once again, you have created a wonderful tutorial! I learned a lot and like the humor. Thank you so much!

3 Reply
Marcel D. Avatar
Marcel D. Avatar Marcel D. | posted 3 years ago | edited

This course is so wonderful!

I learned a lot with it. I knew a little bit about tests, but it helped me consolidating my knowledge and gain more confidence on testing. I learned a lot of new terms and concepts. This course is so complete! It is far from a superficial TDD course that only teaches how to do TDD. This course covered good practices, test doubles, CLI options, functional tests and even continuous integration.

Congrats for the well done course there, Leanna, Andrew and team. ;)

1 Reply
Default user avatar
Default user avatar Andrew M. | Marcel D. | posted 3 years ago | edited

Thank you for the kind words Marcel D. very glad it helped you to learn :)

1 Reply

❤️You've made our day Marcel D.! Thanks for the kind words - I've shared them with the team!

1 Reply

How to setup artifacts?

Reply

Hey Akshit,

You can look at this example where we configure storing screenshots on failed Behat tests as artifacts: https://symfonycasts.com/bl.... But storing artifacts is as easy as a one line copy/pasted code :)


# .circleci/config.yml
version: 2
jobs:
build:
# ...
steps:
- store_artifacts:
path: var/behat_screenshots/

In case you're talking about CircleCI v2 config

I hope this helps!

Cheers!

Reply
Arne K. Avatar
Arne K. Avatar Arne K. | posted 3 years ago

Thanks for a great and incredibly useful course! I've always known I should do testing, but often it was put off for more "urgent" coding. This course took a very practical approach to teaching it, and I have already employed some techniques on two different projects. Keep up the good work!

Reply

Thanks Arne K.!

If you want to go further you can check out our PHPSpec course or Behat course, those tools are so fun and powerful to use.

Cheers!

Reply

The only thing missing in this tutorial is functional testing secure areas of the app.

Reply

Hey Skylar

Thanks for your feedback. If you are in the need, there is an example about how to do it in the documentation: http://symfony.com/doc/curr...

Or if you preffer, you can take advange of LiipFunctionalTestBundle, it comes with free functionality for login in users and many other useful things

Cheers!

Reply

Hey Skylar!

I'll add one more thing in addition to what Diego said, which is actually less helpful, but might clear things even more :). If you want to functionally test a secured area, you don't need to do anything special. The only difference is that the first thing you will do at the beginning of each test is go to /login, fill in the username/password and press submit. You can easily write this code in a way that is re-usable across your tests. The only downside is that this slow your tests down further. But, on the bright side, it's a *true* test: you're using your site JUST like you would in real life. LiipFunctionalTestBundle has some "tricks" to get around this, if you're interested :).

But if you have more specific questions, let us know!

Cheers!

Reply

Wonderful tutorial, Thank you so much :) , I am really appreciate this class of tutorial

Reply

Hey Jiménez,

Thank YOU for being our user ;) And thanks for your kind words, they help us to make more awesome tutorials :)

Cheers!

1 Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

While the fundamentals of PHPUnit haven't changed, this tutorial *is* built on an older version of Symfony and PHPUnit.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.0, <7.4",
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/doctrine-bundle": "^1.6", // 1.10.3
        "doctrine/orm": "^2.5", // v2.7.2
        "incenteev/composer-parameter-handler": "^2.0", // v2.1.2
        "sensio/distribution-bundle": "^5.0.19", // v5.0.21
        "sensio/framework-extra-bundle": "^3.0.2", // v3.0.28
        "symfony/monolog-bundle": "^3.1.0", // v3.1.2
        "symfony/polyfill-apcu": "^1.0", // v1.6.0
        "symfony/swiftmailer-bundle": "^2.3.10", // v2.6.7
        "symfony/symfony": "3.3.*", // v3.3.13
        "twig/twig": "^1.0||^2.0" // v2.4.4
    },
    "require-dev": {
        "doctrine/data-fixtures": "^1.3", // 1.3.3
        "doctrine/doctrine-fixtures-bundle": "^2.3", // v2.4.1
        "liip/functional-test-bundle": "^1.8", // 1.8.0
        "phpunit/phpunit": "^6.3", // 6.5.2
        "sensio/generator-bundle": "^3.0", // v3.1.6
        "symfony/phpunit-bridge": "^3.0" // v3.4.30
    }
}