We use cookies on this site to enhance your user experience. By clicking "OK, I Agree" or using our site, you consent to the use of cookies unless you have disabled them. View our cookie policy to learn more.
OK, I AGREE
TRACK
Symfony 3 >
COURSE
Symfony RESTful API: Basics++ (Course 3)
Buy Access to Course
This tutorial has a new version, check it out!

Pagination Design and Test

Video not working?

It looks like your browser may not support the H264 codec. If you're using Linux, try a different browser or try installing the gstreamer0.10-ffmpeg gstreamer0.10-plugins-good packages.

Thanks! This saves us from needing to use Flash or encode videos in multiple formats. And that let's us get back to making more videos :). But as always, please feel free to message us.

Next Chapter
Edit on Github

Tip

In this course we're using Symfony 2, but starting in episode 4, we use Symfony 3. If you'd like to see the finished code for this tutorial in Symfony 3, download the code from episode 4 and check out the start directory!

Hey, Guys! Welcome to Episode 3 of our REST in Symfony Series. In this episode, we're going to cover some really important details we haven't talked about yet, like pagination, filtering, and taking the serializer and doing really cool and custom things with it.

If you're following along with me, use the same project we've been building. If you're just joining, where have you been? Ah, it's fine: download the code from this page and move into the start/ directory. Start up the built-in PHP web server to get things running:

./app/console server:run

Designing how Pagination should Work

Let's talk about pagination first, because the /api/programmers endpoint doesn't have it. Eventually, once someone talks about our cool app on Reddit, we're going to have a lot of programmers here: too many to return all at once. First, think about pagination on the web. How does it work? Usually, it's done with query parameters: something like ?page=1, ?page=2, and so on. Sometimes, it's done in the URL - like /products/1 and /products/2. For API's, query parameters is better.

Second, on the web, we don't make the user guess those URLs: we give them links, like "next" and "previous", and maybe even "first" and "last".

So why would building an API be any different? Let's use query parameters and include links to help the API client get around.

Adding a Test

Like always, we're gonna start with a test because it's the easiest way to try things out and it helps us think about the API's design. In ProgrammerControllerTest, find the testProgrammersCollection() method and copy this to make a new test for pagination:

196 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 88
}
... lines 90 - 194
}

To make this interesting, we need more programmers - like 25. Add a for loop to do this: for i=0; i<25; i++. In each loop, create a programmer with the super creative name of Programmer plus the $i value. This means that we'll have programmers zero through 24. The avatarNumber is required, but we don't care about its value:

196 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
for ($i = 0; $i < 25; $i++) {
$this->createProgrammer(array(
'nickname' => 'Programmer'.$i,
'avatarNumber' => 3,
));
}
// page 1
... lines 82 - 88
}
... lines 90 - 194
}

Keep the same URL and the 200 status code assertion. Below, start basic with a sanity check for page 1: assert that the programmer with index 5 is equal to Programmer5:

196 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 80
// page 1
$response = $this->client->get('/api/programmers');
$this->assertEquals(200, $response->getStatusCode());
$this->asserter()->assertResponsePropertyEquals(
$response,
'programmers[5].nickname',
'Programmer5'
);
}
... lines 90 - 194
}

I'll use multiple lines to keep things clear. Index 5 is actually the 6th programmer, but since we start with Programmer0, this should definitely be Programmer5.

Adding count and total

It might also be useful to tell the API client how many results are on this page and how many results there are in total. I want to show 10 results per page in the API so add a line that looks for a new property called count that's set to 10. Let's also have another property called total. That'll be the total number of results. In this case, that should be 25:

200 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 89
$this->asserter()->assertResponsePropertyEquals($response, 'count', 10);
$this->asserter()->assertResponsePropertyEquals($response, 'total', 25);
... line 92
}
... lines 94 - 198
}

Finally, the API response needs to have those links! And by "links", I mean that I want to add a new field - maybe called "next" - whose value will be the URL to get the next page of results. Use the asserter again and change this to assertResponsePropertyExists(). Let's assert that there is an _links.next key, which means the JSON will have an _links key and a next key under that:

200 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 91
$this->asserter()->assertResponsePropertyExists($response, '_links.next');
}
... lines 94 - 198
}

By moving things under _links, it makes it a little more obvious that next isn't a property of a programmer, but something different: a link.

Oh, and you probably saw my mistake above: change the line above to total, not count.

And here's where things get really cool. In our test, we need to make a request to page 2 and make sure we see the next 10 programmers. Instead of hardcoding the URL, we can read the next link and use that for the next request. It's like the API version of clicking links!

Use $this->asserter() and then a method called readResponseProperty() to read the _links.next property. Now, add $response = $this->client->get($nextUrl) to go to the next page:

211 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 93
// page 2
$nextLink = $this->asserter()->readResponseProperty($response, '_links.next');
$response = $this->client->get($nextLink);
... lines 97 - 103
}
... lines 105 - 209
}

Ok, let's test page 2! Copy some of the asserts that we just wrote. This time, the programmer with index 5 should be Programmer15 because we're looking at results 11 through 20. Next, the count should still be 10, and the total still 25 - but let's save a little code and remove that line:

211 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 96
$this->assertEquals(200, $response->getStatusCode());
$this->asserter()->assertResponsePropertyEquals(
$response,
'programmers[5].nickname',
'Programmer15'
);
$this->asserter()->assertResponsePropertyEquals($response, 'count', 10);
}
... lines 105 - 209
}

The next link is nice. But we can do even more by also having a first link, a last link and a prev link unless we're on page 1. Copy the code from earlier that clicked the next link. Ooh, and let me fixing my formatting!

This time, use the _links.last key and update the variable to be $lastUrl. When we make a request to the final page, programmers[4] will be the last programmer because we started with index 0. The name should be Programmer24. And on this last page, count should be just 5:

223 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 104
$lastLink = $this->asserter()->readResponseProperty($response, '_links.last');
$response = $this->client->get($lastLink);
$this->assertEquals(200, $response->getStatusCode());
$this->asserter()->assertResponsePropertyEquals(
$response,
'programmers[4].nickname',
'Programmer24'
);
... line 114
$this->asserter()->assertResponsePropertyEquals($response, 'count', 5);
}
... lines 117 - 221
}

I'm also going to use the asserter with assertResponsePropertyDoesNotExist() to make sure that there is no programmer here with index 5. Specifically, check for no programmers[5].nickname path:

223 lines src/AppBundle/Tests/Controller/Api/ProgrammerControllerTest.php
... lines 1 - 5
class ProgrammerControllerTest extends ApiTestCase
{
... lines 8 - 71
public function testGETProgrammersCollectionPaginated()
{
... lines 74 - 113
$this->asserter()->assertResponsePropertyDoesNotExist($response, 'programmers[5].name');
$this->asserter()->assertResponsePropertyEquals($response, 'count', 5);
}
... lines 117 - 221
}

There's a small bug in my asserter code: if I just check for programmers[5], it thinks it exists but is set to null. That's why I'm checking for the nickname key.

That's it! Our pagination system is now really well-defined. Next, we'll bring this all to life.

Leave a comment!

  • 2017-07-10 krewetka

    "you ultimately load a lot of fixtures that you might not need on that test"?

    You can simply load just one fixture file with only these 25 programmers in this test...then if you load just one file then it is easy to check quickly what is in this file ;)

  • 2017-03-13 Владимир Кривошапов

    Thanks a lot for your quick answer, you answered on my question!

  • 2017-03-13 weaverryan

    Yo Владимир Кривошапов!

    Cool question :). There is no one, correct answer about this actually! The most important thing is this: at the start of each test, your database should be in a "known" state. And there are two philosophies for doing this:

    A) Start each test with an empty database (or, at least, empty the tables that are relevant to the test) and then manually add whatever data you need right inside the test. This is the approach you see in this tutorial.

    B) Start each test by loading some known fixtures, e.g. like Alice.

    Approach (B) is a bit simpler, but it has 2 disadvantages:

    1) It's a bit slower - you ultimately load a lot of fixtures that you might not need on that test. However, this can be overcome with some trick. For example, if you configured your test environment to use an SQLite database, then you could load all your fixtures once, then commit that final .db file. Then, at the beginning of each test, copy that .db file to the correct location so that your code uses. With this trick, you get a fresh database before each test... and all you needed to do was copy a single file.

    2) Loading fixtures is a bit less clear. This is the bigger reason why I don't typically do this in tests. For example, if you have 25 programmers in your fixtures, then you could simply load your fixtures before the test, and everything would work great. But when you read the test... it's not so clear. I wonder, "why are there 25 users? Where are they coming from". When your data-loading code isn't right in your test, you lose some clarity.

    I DO love fixtures - it's just a great way to have dummy data so that playing with your site is easy. But, I don't usually do it in tests... but that is not a 100% rule :).

    Cheers!

  • 2017-03-13 Владимир Кривошапов

    Hi Ryan! Thanks for the great tutorial!

    You describe (in other tutorial) approach with loading mock data (using Alice and doctrine:fixtures:load)
    But in this tutorial when you write tests you load mock data inside each method, like:


    public function testGETProgrammersCollectionPaginated()
        {
            for ($i = 0; $i < 25; $i++) {
                $this->createProgrammer(array(
                    'nickname' => 'Programmer'.$i,
                    'avatarNumber' => 3,
                ));
            }
        }

    Why you avoid to use Alice and doctrine:fixtures:load approach in this tutorial, I think it's much more clear?
    And second can we use Alice for test RESTfull APIs and is it much proper way then load mock data inside every test method?

Edit on Github
Next Chapter