Flag of Ukraine
SymfonyCasts stands united with the people of Ukraine

PropertyFilter: Sparse Fieldsets

Keep on Learning!

If you liked what you've learned so far, dive in!
Subscribe to get access to this tutorial plus
video, code and script downloads.

Start your All-Access Pass
Buy just this tutorial for $12.00

With a Subscription, click any sentence in the script to jump to that part of the video!

Login Subscribe

In just a few minutes, we've given our API clients the ability to filter by published cheese listings and search by title and description. They may also need the ability to filter by price. That sounds like a job for... RangeFilter! Add another @ApiFilter() with RangeFilter::class. Let's immediately go up and add the use statement for that - the one for the ORM. Then, properties={"price"}.

... lines 1 - 7
use ApiPlatform\Core\Bridge\Doctrine\Orm\Filter\RangeFilter;
... lines 9 - 14
/**
* @ApiResource(
... lines 17 - 27
* @ApiFilter(RangeFilter::class, properties={"price"})
... line 29
*/
class CheeseListing
... lines 32 - 150

This filter is a bit nuts. Flip over, refresh the docs, and look at the GET collection operation. Woh! We now have a bunch of filter boxes, for price between, greater than, less than, greater than or equal, etc. Let's look for everything greater than 20 and... Execute. This adds ?price[gt]=20 to the URL. Oh, except, that's a search for everything greater than 20 cents! Try 1000 instead.

This returns just one item and, once again, it advertises the new filters down inside hydra:search.

Filters are super fun. Tons of filters come built-in, but you can totally add your own. From a high-level, a filter is basically a way for you to modify the Doctrine query that's made when fetching a collection.

Adding a Short Description

There's one more filter I want to talk about... and it's a bit special: instead of returning less results, it's all about returning less fields. Let's pretend that most descriptions are super long and contain HTML. On the front-end, we want to be able to fetch a collection of cheese listings, but we're only going to display a very short version of the description. To make that super easy, let's add a new field that returns this. Search for getDescription() and add a new method below called public function getShortDescription(). This will return a nullable string, in case description isn't set yet. Let's immediately add this to a group - cheese_listing:read so that it shows up in the API.

... lines 1 - 30
class CheeseListing
{
... lines 33 - 90
/**
* @Groups("cheese_listing:read")
*/
public function getShortDescription(): ?string
{
... lines 96 - 100
}
... lines 102 - 160
}

Inside, if the description is already less than 40 characters, just return it. Otherwise, return a substr of the description - get the first 40 characters, then a little ... at the end. Oh, and, in a real project, to make this better - you should probably use strip_tags() on description before doing any of this so that we don't split any HTML tags.

... lines 1 - 93
public function getShortDescription(): string
{
if (strlen($this->description) < 40) {
return $this->description;
}
return substr($this->description, 0, 40).'...';
}
... lines 102 - 162

Refresh the docs... then open the GET item operation. Let's look for cheese listing id 1. And... there it is! The description was just barely longer than 40 characters. I'll copy the URL, put it into a new tab, and add .jsonld on the end to see this better.

At this point, adding the new field was nothing special. But... if some parts of my frontend only need the shortDescription... it's a bit wasteful for the API to also send the description field... especially if that field is really, really big! Is it possible for an API client to tell our API to not return certain fields?

Hello PropertyFilter

At the top of our class, add another filter with PropertyFilter::class. Move up, type use PropertyFilter and hit tab to auto-complete. This time, there's only one of these classes.

... lines 1 - 9
use ApiPlatform\Core\Serializer\Filter\PropertyFilter;
... lines 11 - 15
/**
* @ApiResource(
... lines 18 - 29
* @ApiFilter(PropertyFilter::class)
... line 31
*/
class CheeseListing
... lines 34 - 164

This filter does have some options, but it works perfectly well without doing anything else.

Go refresh our docs. Hmm, this doesn't make any difference here... this isn't a feature of our API that can be expressed in the OpenAPI spec doc.

But, this resource in our API does have a new super-power. In the other tab, choose the exact properties you want with ?properties[]=title&properties[]=shortDescription. Hit it! Beautiful! We still get the standard JSON-LD fields, but then we only get back those two fields. This idea is sometimes called a "sparse fieldset", and it's a great way to allow your API client to ask for exactly what they want, while still organizing everything around concrete API resources.

Oh, and the user can't try to select new fields that aren't a part of our original data - you can't try to get isPublished - it just doesn't work, though you can enable this.

Next: let's talk about pagination. Yea, APIs totally need pagination! If we have 10,000 cheese listings in the database, we can't return all of them at once.

Leave a comment!

13
Login or Register to join the conversation

2:40 We can do it more fancy way, using symfony/string component. https://symfony.com/doc/cur...

1 Reply

Ohh the String component, you're right, it would be super fancy to do it that way :)
Thanks for sharing it. Cheers!

Reply
AO Avatar

Is it possible to specify properties[] on a POST request to control the data that is returned after success? A developer tried by POSTing the required data in the body and using the ?properties[] syntax, but instead of returning the fields he wanted, it seemed to filter out any data he had sent (that wasn't included in the properties array), making the POST payload invalid. Is that expected?

Reply
weaverryan Avatar weaverryan | SFCASTS | AO | posted 2 years ago | edited

Hey AO!

> A developer tried by POSTing the required data in the body and using the ?properties[] syntax, but instead of returning the fields he wanted, it seemed to filter out any data he had sent (that wasn't included in the properties array), making the POST payload invalid.

THAT is really interesting. I'm not sure if it's expected, but I do see (now) that this would happen. The PropertyFilter works by modifying the (de)normalization *context* - https://github.com/api-plat... - and it is called both at the start of the request during read/deserialization AND later during serialization. Basically, you only want it to happen during deserialization, which totally makes sense to me :p. Unfortunately, that doesn't look possible currently - the filter isn't set up for this. You could request this as an option on the PropertyFilter (that would make sense to me), but it doesn't work that way currently.

One option might be to sub-class PropertyFilter, use that in your annotation, and override apply(). I've never done this before, but I can't think of why it wouldn't work. Then, in your overridden apply, check the $normalization argument and ONLY call the parent apply() method if $normalization is true.

I'm not 100% sure this will work, but it's worth a shot ;).

Cheers!

Reply
AO Avatar

Thanks for the reply, super useful! I ended up adding a POST normalization group that only sends ID back for now but I might take a look at that solution out of work hours :D

Reply

Awesome! Thanks for sharing your solution - it seems simple enough at least :).

Cheers!

Reply
Frederick E. Avatar
Frederick E. Avatar Frederick E. | posted 3 years ago

Hi, Please how can I implement a filter that will filter my returned resources based on the available collections. For example the below returned data shows a collection property named 'exam'. I want the filter to return only the one with exam IRI property of "/api/exams/4". Thank you.

{
"@context": "/api/contexts/groups",
"@id": "/api/groups/fetch",
"@type": "hydra:Collection",
"hydra:member": [
{
"@id": "/api/groups/1",
"@type": "groups",
"groupName": "Narcotic Officer",
"exam": [
"/api/exams/4"
]
},
{
"@id": "/api/groups/2",
"@type": "groups",
"groupName": "Narcotic Assistant",
"exam": []
},
{
"@id": "/api/groups/3",
"@type": "groups",
"groupName": "Narcotic Officer2",
"exam": []
}
],
"hydra:totalItems": 3,
"hydra:search": {
"@type": "hydra:IriTemplate",
"hydra:template": "/api/groups/fetch{?}",
"hydra:variableRepresentation": "BasicRepresentation",
"hydra:mapping": []
}
}
Reply

Hey Frederick E.!

Excellent question! Indeed, we didn't talk or think about how it might look to filter for a specific item (e.g. Group) based on the value/existence of some collection property. I just played with this, and fortunately, I think it's quite easy. I would:

1) On your Group entity, add a SearchFilter:


/**
* @ApiFilter(SearchFilter::class, properties={
* "exam": "exact"
* })
*/
class Group {

2) That's it! You should now see an option in Swagger to filter by this. You'll enter the IRI into the box. So, to only return "Groups" that have the exam /api/exams/4, it would look like https://localhost:9022/api/users?exam=%2Fapi%2Fgroups%2F4 - where the %2f things are URL-encoded slashes.

Let me know if this helps!

Cheers!

Reply
Frederick E. Avatar
Frederick E. Avatar Frederick E. | weaverryan | posted 3 years ago | edited

Awesome!!! Thank you weaverryan that helped....

Reply
Ajie62 Avatar

Hi, when creating the getShortDescription() method, you indicate it "will return a nullable string, in case description isn't set yet", but in the code you didn't add the "?" for the return type of the method, right before "string". I wanted to update it myself but when I arrived in the GitHub repository, I just saw "[[[ code('5bacd98cdf') ]]]" and didn't know where to find this code. :/ It's not a very big issue, but it's worth fixing I think :p

Reply

Hey Ajie62!

You're right! Very nice catch! The video is right - the code is wrong. Thanks for trying to fix it... unfortunately (as you discovered!) this is one part of our process that is not contributor-friendly (I wish it were, but it's not). However, we're going to update that code block and add the nullable ? :)

Cheers!

Reply
Ajie62 Avatar

I'm glad I can help at least a little.... :)

Reply

FYI, code block was fixed in https://github.com/SymfonyC...

Thank you for reporting it!

Cheers!

Reply
Cat in space

"Houston: no signs of life"
Start the conversation!

This tutorial works great for Symfony 5 and API Platform 2.5/2.6.

What PHP libraries does this tutorial use?

// composer.json
{
    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "api-platform/core": "^2.1", // v2.4.3
        "composer/package-versions-deprecated": "^1.11", // 1.11.99
        "doctrine/annotations": "^1.0", // 1.10.2
        "doctrine/doctrine-bundle": "^1.6", // 1.11.2
        "doctrine/doctrine-migrations-bundle": "^2.0", // v2.0.0
        "doctrine/orm": "^2.4.5", // v2.7.2
        "nelmio/cors-bundle": "^1.5", // 1.5.5
        "nesbot/carbon": "^2.17", // 2.19.2
        "phpdocumentor/reflection-docblock": "^3.0 || ^4.0", // 4.3.1
        "symfony/asset": "4.2.*|4.3.*|4.4.*", // v4.3.11
        "symfony/console": "4.2.*", // v4.2.12
        "symfony/dotenv": "4.2.*", // v4.2.12
        "symfony/expression-language": "4.2.*|4.3.*|4.4.*", // v4.3.11
        "symfony/flex": "^1.1", // v1.17.6
        "symfony/framework-bundle": "4.2.*", // v4.2.12
        "symfony/security-bundle": "4.2.*|4.3.*", // v4.3.3
        "symfony/twig-bundle": "4.2.*|4.3.*", // v4.2.12
        "symfony/validator": "4.2.*|4.3.*", // v4.3.11
        "symfony/yaml": "4.2.*" // v4.2.12
    },
    "require-dev": {
        "symfony/maker-bundle": "^1.11", // v1.11.6
        "symfony/stopwatch": "4.2.*|4.3.*", // v4.2.9
        "symfony/web-profiler-bundle": "4.2.*|4.3.*" // v4.2.9
    }
}