Serialization Groups: Choosing Fields
Keep on Learning!
If you liked what you've learned so far, dive in! Subscribe to get access to this tutorial plus video, code and script downloads.
With a Subscription, click any sentence in the script to jump to that part of the video!
Login SubscribeRight now, whether or not a field in our class is readable or writable in the API is entirely determined by whether or not that property is readable or writable in our class (basically, whether or not it has a getter or setter method). But what if you need a getter or setter... but don't want that field exposed in the API? For that, we have two options.
A DTO Class?
Option número uno: create a DTO class for the API resource. This is something we'll save for another day... in a future tutorial. But in a nutshell, it's where you create a dedicated class for your DragonTreasure API... and then move the ApiResource attribute onto that. The key thing is that you'll design the new class to look exactly like your API... because modeling your API will be its only job. It takes a little more work to set things up, but the advantage is that you then have a dedicated class for your API. Done!
Hello Serialization Groups
The second solution, and the one we're going to use, is serialization groups. Check it out. Over on the ApiResource attribute, add a new option called normalizationContext. If you recall, "normalization" is the process of going from an object to an array, like when you're making a GET request to read a treasure. The normalizationContext is basically options that are passed to the serializer during that process. And the one option that's most important is groups. Set that to one group called treasure:read:
| // ... lines 1 - 16 | |
| ( | |
| // ... lines 18 - 26 | |
| normalizationContext: [ | |
| 'groups' => ['treasure:read'], | |
| ] | |
| ) | |
| class DragonTreasure | |
| { | |
| // ... lines 33 - 140 | |
| } |
We'll talk about what this does in a minute. But you can see the pattern I'm using for the group: the name of the class (it could be dragon_treasure if we wanted) then :read... because normalization means that we're reading this class. You can name these groups however you want: this is my standard.
So... what does that do? Let's find out! Refresh the documentation... and, to make life easier, go to the URL: /api/dragon_treasures.jsonld. Whoops! It's just treasures.jsonld now. There we go. And... absolutely nothing is returned! Ok, we have the hydra fields, but this hydra:member contains the array of treasures. It is returning one treasure... but other than @id and @type... there are no actual fields!
How Serialization Groups Work
Here's the deal. As soon as we add a normalizationContext with a group, when our object is normalized, the serializer will only include properties that have this group on it. And since we haven't added any groups to our properties, it returns nothing.
How do we add groups? With another attribute! Above the $name property, say #[Groups], hit "tab" to add its use statement and then treasure:read. Repeat this above the $description field... because we want that to be readable... and then the $value field... and finally $coolFactor:
| // ... lines 1 - 14 | |
| use Symfony\Component\Serializer\Annotation\Groups; | |
| // ... lines 16 - 31 | |
| class DragonTreasure | |
| { | |
| // ... lines 34 - 39 | |
| (['treasure:read']) | |
| private ?string $name = null; | |
| // ... lines 42 - 43 | |
| (['treasure:read']) | |
| private ?string $description = null; | |
| // ... lines 46 - 50 | |
| (['treasure:read']) | |
| private ?int $value = null; | |
| // ... lines 53 - 54 | |
| (['treasure:read']) | |
| private ?int $coolFactor = null; | |
| // ... lines 57 - 145 | |
| } |
Good start. Move over and refresh the endpoint. Now... got it! We see name, description, value, and coolFactor.
DenormalizationContext: Controlling Writable Groups
We now have control over which fields are readable... and we can do the same thing to choose which fields should be writeable in the API. That's called "de-normalization", and I bet you can guess what we're going to do. Copy normalizationContext, paste, change it to denormalizationContext... and use treasure:write:
| // ... lines 1 - 17 | |
| ( | |
| // ... lines 19 - 30 | |
| denormalizationContext: [ | |
| 'groups' => ['treasure:write'], | |
| ] | |
| ) | |
| class DragonTreasure | |
| { | |
| // ... lines 37 - 148 | |
| } |
Now head down to the $name property and add treasure:write. I'm going to skip $description (remember that we actually deleted our setDescription() method earlier on purpose)... but add this to $value... and $coolFactor:
| // ... lines 1 - 34 | |
| class DragonTreasure | |
| { | |
| // ... lines 37 - 42 | |
| (['treasure:read', 'treasure:write']) | |
| private ?string $name = null; | |
| // ... lines 45 - 53 | |
| (['treasure:read', 'treasure:write']) | |
| private ?int $value = null; | |
| // ... lines 56 - 57 | |
| (['treasure:read', 'treasure:write']) | |
| private ?int $coolFactor = null; | |
| // ... lines 60 - 148 | |
| } |
Oh, it's mad at me! As soon as we pass multiple groups, we need to make this an array. Add some [] around those three properties. Much happier.
To check if this is A-OK, refresh the documentation... open up the PUT endpoint, and... sweet! We see name, value, and coolFactor, which are currently the only fields that are writable in our API.
Adding Groups To Methods
We are missing a few things, though. Earlier, we made a getPlunderedAtAgo() method...
| // ... lines 1 - 34 | |
| class DragonTreasure | |
| { | |
| // ... lines 37 - 132 | |
| public function getPlunderedAtAgo(): string | |
| { | |
| return Carbon::instance($this->plunderedAt)->diffForHumans(); | |
| } | |
| // ... lines 137 - 148 | |
| } |
and we want this to be included when we read our resource. Right now, if we we check the endpoint, it's not there.
To fix this, we can also add groups above methods. Say #[Groups(['treasure:read'])]:
| // ... lines 1 - 34 | |
| class DragonTreasure | |
| { | |
| // ... lines 37 - 132 | |
| (['treasure:read']) | |
| public function getPlunderedAtAgo(): string | |
| { | |
| return Carbon::instance($this->plunderedAt)->diffForHumans(); | |
| } | |
| // ... lines 138 - 149 | |
| } |
And when we go check... voilà, it pops up.
Let's also find the setTextDescription() method... and do the same thing: #[Groups([treasure:write])]:
| // ... lines 1 - 34 | |
| class DragonTreasure | |
| { | |
| // ... lines 37 - 93 | |
| (['treasure:write']) | |
| public function setTextDescription(string $description): self | |
| { | |
| // ... lines 97 - 99 | |
| } | |
| // ... lines 101 - 150 | |
| } |
Awesome! If we head back to the documentation, the field is not currently there... but when we refresh... and check out the PUT endpoint again... textDescription is back!
Re-Adding Methods
Hey, now we can re-add any of the getter or setter methods we removed earlier! Like, maybe I do need a setDescription() method in my code for something. Copy setName() to be lazy, paste and change "name" to "description" in a few places.
Got it! And even though we have that setter back, when we look at the PUT endpoint, description doesn't show up. We have complete control over our fields thanks to the denormalization groups. Do the same thing for setPlunderedAt()... because sometimes it's handy - in data fixtures especially - to be able to set this manually.
And... done!
Adding Field Defaults
So we know that fetching a resource works. Now let's see if we can create a new resource. Click on the POST endpoint, hit "Try it out", and... let's fill in some info about our new treasure, which is, of course, a Giant jar of pickles. This is very valuable and has a coolFactor of 10. I'll also add a description... though this jar of pickles speaks for itself.
When we try this... oh, dear... we get a 500 error:
An exception occurred while executing a query: Not null violation,
nullvalue in columnisPublished.
We slimmed our API down to only the fields that we want writeable... but there's still one property that must be set in the database. Scroll up and find isPublished. Yup, it currently defaults to null. Change that to = false... and now the property will never be null.
If we try it... the Giant jar of pickles is pickled into the database! It works!
Next: let's explore a few more cool serialization tricks to give us even more control.
24 Comments
if it helps anybody, the moment you modify the metadata and add attributes, etc, you need to clear the cache unless you run the profiler debug toolbar which does that for you. I was stuck wondering why it didn't worked and that was the reason, then after installing the debugger everything worked fine just by reloading.
Hey Reborto,
Thank you for this tip! Yeah, it's always a good idea to clear the cache first in any weird case, i.e. when things should work but somehow work not like you expect.
Cheers!
Hi Ryan,
In this chapter you are talking about the DTO solution, are you going to do a course to explain how it works in Api Platform v3?
I've always used them in v2, to separate the Entities from the Api. But in v3 I encounter a problem I don't know how to solve.
The @id field in DTO looks like "@id": "/.well-known/genid/01e546d3f38c0b5d3b8a", and I don't find anyway to get the IRI instead. Do you know if it's possible?
Thank you in advance
Hey @Auro!
Yes! But not until episode 3 so we can give them proper attention.
Hmm. This is new to me. It's generating something called a "skolem", which I know almost nothing about. And so, I'm giving advice... without really understanding ;). Looking at the code, which is deep and complex, you could try:
A) Setting
force_resource_class: trueas undernormalizationContextof yourApiResource.B) There is a new, undocumented feature in API Platform 3.1 that allows you to, sort of, "tie" your DTO to your entity a bit closer. Looking at the code, this seems related - but there are a lot of layers to it, and I'm honestly not sure what it does or doesn't do. But I believe, above your
ApiResourcein your DTO, you would add<br />stateOptions: new DoctrineORMOptions(entityClass: MyClass::class)<br />Part of the tricky thing here is that DTO can mean at least 2 different things: a class that actually has the
ApiResourceattribute on it and IS the resource, or you have that attribute on an entity, then use theinputoroutputconfig.Cheers!
Hi @weaverryan,
Thank you for your answer.
I've tried the two options, and this is my feedback:
A. This solution works perfectly if you use id as default identifier. Sadly is not my case, I use uuid, and I think there is a bug when using uuid.
in the IriConverter class, method generateSymfonyRoute, it calls the following:
$identifiers = $this->identifiersExtractor->getIdentifiersFromItem($resource, $identifiersExtractorOperation, $context);and in the IdentifiersExtractor class the first line of the getIdentifiersFromItem method
if (!$this->isResourceClass($this->getObjectClass($item))) { return ['id' => $this->propertyAccessor->getValue($item, 'id')]; }If I use id instead of uuid o replace this line by uuid in both case it works and generate correctly the @id.
Instead of hardcoding the id, i think that it should dynamically use the identifier property
B. Digging into the code, there is a stateOptions but it looks to only works with elasticsearchOptions for the moment.
I have an ultimate question about DTOs, what would be the equivalent of the DataTransformerInitializerInterface?
I'm trying to initialize/hydrate the DTO when working with PUT/PATCH methods, but i don't find any way to have the object initialized before the validation step.
Hi @Auro!
Thanks for the update - sorry it's proving so tricky!
I don't know this code well. But I agree that it seems odd that it is looking specifically for an
idpropertyIn short, I don't know yet... because I haven't dug into this - that will be soon for episode 3. They do talk about this briefly in the upgrade doc - https://api-platform.com/docs/core/upgrade-guide/#datatransformers-and-dto-support - you're supposed to use state providers to load the DTO data - https://api-platform.com/docs/core/dto/#implementing-a-write-operation-with-an-input-different-from-the-resource - however, I'm not sure this works yet with relation to validation.
Sorry I can't be more helpful! If you find anything out, I'd love to know :)
Thank you for your help @weaverryan ,
For the first point i've open an issue
https://github.com/api-platform/api-platform/issues/2411
For the second one, I will wait for the episode 3. I'm already using state providers and it works fine for POST methods. The problem is how to bypass validate and hydrate the DTO before the state provider code is executed.
Just to give some feeback, i've found a solution that works, but not really sure if it's a hack or the correct way to go.
https://github.com/api-platform/core/issues/5451
Awesome - thank you for sharing this. I'll have to dig more deeply into it when I look at DTO's. I'm totally unfamiliar with having an operation - like
Get()right on the class vs inside ofApiResource🤔Just in case one of you happen to know the answer.
how could i add the serialization groups using laravel. t
Hey @jefry ,
I personally don't know much about how to do it in Laravel. Probably try to ask ChatGPT it may give you some hints.
Cheers!
hi,i have question: i have Article Entity, and have 2 or more ManyToOne relations,like Category Entity。i want display category name in GetCollection. so i create normalizationContext groups,yes it is working.so i created 100000 article just for test, when i use GetCollection,get collection is slow.1 and more second returnd. any optimization solution?
Hey Yangzhi,
Oh my, that's a lot of articles :) I haven't had such a use case yet, so can't tell you for sure how to overcome it, but I may give you some hints and directions to look into.
First of all, make sure you have SQL indexes on those IDs, double check
idfields, and also checkcategory_idthat is used for building the relation. Query profiling with tools likeEXPLAINmay help a lot to understand if it's a query problem or it's due to a big data you're trying to return.Also, you may want to use the Doctrine cache in the doctrine.yaml config. Or you can even use HTTP cache in this case to speed up further hits of this endpoint.
You may also consider using "Eager Loading" for relations.
The built-in pagination may also help in this case, as it helps to restrict the number of returned items. This is probably the best solution for such a case btw.
Also, double check if you have configured your Serialization Groups good, probably you can simplify the data what is returned in relation to minimum discarding extra data you don't need in that endpoint.
I hope this helps!
Cheers!
Hi guys, what the difference between normalization or denormalization contexts placed in ApiResource scope and explicitly for operations get/post/patch/...?
Hi @triemli
it will configure serializer behavior for serialize and deserialize operations, you will use groups to select what values you will need to pass from Entity to JSON and vice versa.
Cheers!
Hi Ryan,
I watched the API Platform 2 course. I plan to watch this one but for now I did not find the time to do it. I still searched if there was the same course "Automatic Serialization Groups" that was in API Platform 2 course in API Platform 3 course. I could not find it, and it seems that the old version does not work anymore. Do you have any idea about how to do something similar ?
Thank you for your excellent work !
Hey @Aurelien-A!
It's funny you ask about that - it was maybe the ONE thing I didn't include in the API Platform 3 tutorial and at least 2 people have asked about it :P. Here is the other conversation - https://symfonycasts.com/screencast/api-platform/install#comment-29468 - it looks like the feature should be implemented the same way... it's just that some class names changed.
Let me know if you try it and hit any hiccups.
Cheers!
Hello Ryan, you were right. With a little reworking of the code in your original course, I believe I've managed to reproduce the same operation. I'm leaving the code in question available here, perhaps it could be useful to someone.
Thanks for your help and congratulations for your work on Symfony :)
And it was very useful to me. Thank you very much. Might be good to add that to the course or even to the Api Platform docs. I know a lot of people who implemented the version from the Api Platform 2 Course on production, because that way of auto generation groups is very helpful. Might be a time-saver for everyone upgrading. Thank you
Ah, thank you for posing that! ❤️❤️❤️ Nice work!
Just to signal a little fault :
DenormlizationContextshould be DenormalizationContext in the title "DenormlizationContext: Controlling Writable Groups" ! (i see another but cannot find it after reading the course twice!) But very good job, many thanks !Ah, thank you Pierre! Fixed up now :) https://github.com/SymfonyCasts/api-platform3/commit/d542951c3fb0f33f31eecf9a642edf5da1a02386
Hi support Team,
maybe you can share few examples with DTO ?
Thanks in advance
Hey @Mepcuk!
That's coming! Not until episode 3, but definitely it's coming. It was better to wait until ep3 than try to smash it in earlier - it deserves some space :).
Cheers!
"Houston: no signs of life"
Start the conversation!