Dtos, Mapping & Max Depth of Relations
Keep on Learning!
If you liked what you've learned so far, dive in! Subscribe to get access to this tutorial plus video, code and script downloads.
With a Subscription, click any sentence in the script to jump to that part of the video!
Login SubscribeHead to /api/users.jsonld to see... a circular reference coming from the serializer. Yikes! Let's think: API Platform serializes whatever we return from the state provider. So head there.... and find where the collection is created. Dump the DTOs. These are what's being serialized, so the problem must be here.
| // ... lines 1 - 14 | |
| class EntityToDtoStateProvider implements ProviderInterface | |
| { | |
| // ... lines 17 - 25 | |
| public function provide(Operation $operation, array $uriVariables = [], array $context = []): object|array|null | |
| { | |
| // ... line 28 | |
| if ($operation instanceof CollectionOperationInterface) { | |
| // ... lines 30 - 36 | |
| dd($dtos); | |
| // ... lines 38 - 44 | |
| } | |
| // ... lines 46 - 53 | |
| } | |
| // ... lines 55 - 59 | |
| } |
Refresh and... no surprise: we see 5 UserApi objects. Ah, but here's the problem: the dragonTreasures field holds an array of DragonTreasure entity objects... and each has an owner that points to a User entity... and that points back to a collection of DragonTreasure entities... which causes the serializer to serializer forever and ever. But that's not even the real problem! I know, I'm full of good news. The real problem is that the UserApi object should really relate to a DragonTreasureApi, not a DragonTreasure entity.
Over in UserApi, this will now be an array of DragonTreasureApi. Once we start going the DTO route, for maximum smoothness, we should relate DTOs to other DTOs... instead of mixing them with entities.
| // ... lines 1 - 42 | |
| class UserApi | |
| { | |
| // ... lines 45 - 61 | |
| /** | |
| * @var array<int, DragonTreasureApi> | |
| */ | |
| (writable: false) | |
| public array $dragonTreasures = []; | |
| // ... lines 67 - 69 | |
| } |
To populate the DTO objects, go to the mapper: UserEntityToApiMapper. Down here, for dragonTreasures, we can't do this anymore because that will give us DragonTreasure entity objects. What we basically want to do is convert from DragonTreasure to DragonTreasureApi. And so, once again, it's micro mapper to the rescue!
Micro-Mapping DragonTreasure -> DragonTreasureApi
Add public function __construct() with private MicroMapperInterface $microMapper. Down here, add some fancy code: $dto->dragonTreasures = set to array_map(), with a function that has a DragonTreasure argument. We'll finish that in a second... but first pass the array that it will loop over: $entity->getPublishedDragonTreasures()->toArray().
So: we get an array of the published DragonTreasure objects and PHP loops over them and calls our function for each one - passing the DragonTreasure. Whatever we return will become an item inside a new array that's set onto dragonTreasures. And what we want to return is a DragonTreasureApi object. Do that with $this->microMapper->map($dragonTreasure, DragonTreasureApi::class).
| // ... lines 1 - 4 | |
| use App\ApiResource\DragonTreasureApi; | |
| // ... line 6 | |
| use App\Entity\DragonTreasure; | |
| // ... lines 8 - 10 | |
| use Symfonycasts\MicroMapper\MicroMapperInterface; | |
| // ... lines 12 - 13 | |
| class UserEntityToApiMapper implements MapperInterface | |
| { | |
| public function __construct( | |
| private MicroMapperInterface $microMapper, | |
| ) | |
| { | |
| } | |
| // ... lines 21 - 41 | |
| $dto->dragonTreasures = array_map(function(DragonTreasure $dragonTreasure) { | |
| return $this->microMapper->map($dragonTreasure, DragonTreasureApi::class); | |
| }, $entity->getPublishedDragonTreasures()->getValues()); | |
| // ... lines 45 - 47 | |
| } | |
| } |
Circular Relationships
Cool! When we refresh to try it... we're greeted with a different circular reference problem. Fun! This one comes from MicroMapper... and it's a problem that will happen whenever you have relationships that refer to each other.
Think about it: we ask Micro Mapper to convert a DragonTreasure entity to DragonTreasureApi. Simple. To do that, it uses our mapper. And guess what? In our mapper, we ask it to convert the owner - a User entity - to an instance of UserApi. To do that, micro mapper goes back to UserEntityToApiMapper and... the process repeats. We're in a loop: to convert a User entity, we need to convert a DragonTreasure entity... which means we need to convert its owner... which is that same User entity.
Setting Mapping Depth
The fix lives in your mapper, when calling the map() function. Pass a third argument, which is a "context"... kind of an array of options. You can pass whatever you want, but Micro Mapper itself only has 1 option that it cares about. Set MicroMapperInterface::MAX_DEPTH to 1.
| // ... lines 1 - 4 | |
| use App\ApiResource\DragonTreasureApi; | |
| // ... line 6 | |
| use App\Entity\DragonTreasure; | |
| // ... lines 8 - 10 | |
| use Symfonycasts\MicroMapper\MicroMapperInterface; | |
| // ... lines 12 - 13 | |
| class UserEntityToApiMapper implements MapperInterface | |
| { | |
| public function __construct( | |
| private MicroMapperInterface $microMapper, | |
| ) | |
| { | |
| } | |
| // ... lines 21 - 41 | |
| $dto->dragonTreasures = array_map(function(DragonTreasure $dragonTreasure) { | |
| return $this->microMapper->map($dragonTreasure, DragonTreasureApi::class, [ | |
| MicroMapperInterface::MAX_DEPTH => 1, | |
| ]); | |
| }, $entity->getPublishedDragonTreasures()->getValues()); | |
| // ... lines 47 - 49 | |
| } | |
| } |
Let's see what that does. When we refresh... look at the dump, which comes from the state provider. It maps the User entities to UserApi objects... and we see 5. We can also see that the dragonTreasures property is populated with DragonTreasureApi objects. So it did do the mapping from DragonTreasure to DragonTreasureApi. But when it went to map the owner of that DragonTreasure to a UserApi, it's there... but it's empty. It's a shallow mapping.
When we pass MAX_DEPTH => 1, we're saying:
Yo! I want you to fully map this
DragonTreasureentity toDragonTreasureApi. That is depth 1. But if the micro mapper is called again to map any deeper, skip that.
Well, not exactly skip. When the mapper is called the 2nd time to map the User entity to UserApi, it calls the load() method on that mapper... but not populate(). So we end up with a UserApi object with an id... but nothing else. That fixes our circular loop. And, we don't really care that the owner property is an empty object... because our JSON never renders that deeply!
Watch. Remove the dd() so we can see the results. And... perfect! The result is exactly what we expect! For DragonTreasures, we're only showing the IRI.
So, as a rule, when calling micro mapper from inside a mapper class, you'll probably want to set MAX_DEPTH to 1. Heck, we could set MAX_DEPTH to 0! Though the only reason to do that would be a slight performance improvement.
This time, when we map $dragonTreasure to DragonTreasureApi, try MAX_DEPTH => 0.
| // ... lines 1 - 13 | |
| class UserEntityToApiMapper implements MapperInterface | |
| { | |
| // ... lines 16 - 32 | |
| public function populate(object $from, object $to, array $context): object | |
| { | |
| // ... lines 35 - 41 | |
| $dto->dragonTreasures = array_map(function(DragonTreasure $dragonTreasure) { | |
| return $this->microMapper->map($dragonTreasure, DragonTreasureApi::class, [ | |
| MicroMapperInterface::MAX_DEPTH => 0, | |
| ]); | |
| }, $entity->getPublishedDragonTreasures()->getValues()); | |
| // ... lines 47 - 49 | |
| } | |
| } |
This will cause the depth to be hit immediately. When it goes to map the DragonTreasure entity to DragonTreasureApi, it will use the mapper, but only call the load() method. The populate() method will never be called. Put the dd() back. What we end up with is a shallow object for DragonTreasureApi.
This might seem weird, but it's technically okay... because this dragonTreasures array is going to be rendered as IRI strings... and the only thing API Platform needs to build that IRI is... the id! Check it out! Remove the dump and reload the page. It looks exactly the same. We just saved ourselves a tiny bit of work.
So, to be on the safe side - in case you embed the object - use MAX_DEPTH => 1. But if you know that you're using IRIs, you can set MAX_DEPTH to 0.
Over here, let's do the same thing: MicroMapperInterface::MAX_DEPTH set to 0 because we know that we're only showing the IRI here as well.
| // ... lines 1 - 13 | |
| class DragonTreasureEntityToApiMapper implements MapperInterface | |
| { | |
| // ... lines 16 - 33 | |
| public function populate(object $from, object $to, array $context): object | |
| { | |
| // ... lines 36 - 41 | |
| $dto->owner = $this->microMapper->map($entity->getOwner(), UserApi::class, [ | |
| MicroMapperInterface::MAX_DEPTH => 0, | |
| ]); | |
| // ... lines 45 - 52 | |
| } | |
| } |
Forcing a JSON Array
One other thing you may have noticed is that dragonTreasures suddenly looks like an object - with its squiggly braces instead of square brackets. Well, in PHP it is an array - array_map returns an array with the 0 key set to something and the 2 key to set to something. But because of the missing 1 key, when it's serialized to JSON it looks like an associative array, or an "object" in JSON.
If we change the toArray() to getValues() and refresh the page... perfect! We're back to a regular array of items.
Next: We can read from our new DragonTreasureApi resource, but we can't write to it yet. Let's create a DragonTreasureApiToEntityMapper and re-add things like security and validation.
10 Comments
Hi,
Using
$entities = $this->collectionProvider->provide(...)causes an N+1 query problem: Doctrine lazily loads associations per entity, leading to many queries and slow performance.So, Icheck if the repository implements EagerLoadingRepositoryInterface and, if so, fetch paginated entities with relations eager-loaded. Map them to DTOs and return via TraversablePaginator. Otherwise, fall back to the default provider.
Repository
Is this a good solution, or can it be optimized further?
Thnks
Hey Anas,
I think you're on the right track! I have never do this myself but it sounds logical to me, and if it helped you with that N+1 problem - great job!
But if your goal is generic performance improvement, I would suggest you to take a look into API Platform’s built-in
EagerLoadingExtensionand only disable lazy joins where really needed. So yes, your solution works and is valid, but the optimized API Platform way is usually to let itsEagerLoadingExtensionhandle this, and only override with a customQueryCollectionExtensionif you need fine-grained control.I hope that helps!
Cheers!
Hi,
Apologies, I'm relatively new to Symfony/API Platform, so my terminology maybe slightly off. But I have been following this tutorial and creating my own API, and I am having an issue whereby when using
DTOs, aManyToManyrelationship is being returned as the fully hydrated model, not theIRIsas I would of expected.Broadly, this is an example what I have put in place (I have removed the mapper from the provider etc to make this as stripped back as possible, so what is being done is minimal):
The test provider just wraps the Doctrine Provider and maps the returned entity to the
BookDTO.The
ownerwill come back as expected, but theauthorswill come back fully hydrated, and not be converted to IRIs. I'm not sure why it works fine for the owner but not authors:Note that if I tag the related Doctrine Entities with
APIResourceit all works perfectly. Also note that theOneToManyon the model seems to be working as expected as well.I did originally have a
UserDTObut have stripped it out to make the example simpler (and to try and get this working with the Doctrine entity first).Any help would be greatly appreciated, I'm sure I'm doing something basic wrong.
Hey @jorganson!
This is almost certainly too late to help, but here we go anyway!
I'm not sure about this, but generally speaking: if you have a relation AND it doesn't contain any groups that your serializing the paren with AND the related class is an
#[ApiResource]you should get an IRI.Sorry for such a slow reply!
Being in this chapter of the course, my already tired head tells me that if I want to add custom data that comes from business logic operations, I would have to do it here in the created Mapper, injecting the service that performs the logic I need and then assign it in populate() as it is done for example with
$dto->isMine = $this->security->getUser() && $this->security->getUser() === $entity->getOwner();<br />Am I right ? or better somewhere else ?
Hey @Rodrypaladin!
I'd say that you're right. Technically, the place for this stuff a would be the data provider. But we've purposely made our provider generic & reusable: pushing all of the specific details into the mapper. So this makes the mapper, in a sense, the data provider code that's specific to just one ApiResource and thus the place I'd add the logic.
tl;dr yes, I agree with you!
Cheers!
Hello, how can we ensure that in the following example we at least have the entire nested object?
I can't seem to get more than an array of URIs, even when increasing MAX_DEPTH to 2 or more.
I would like to get the top level information for each dragon, including the road collection.
Unfortunately, I figured out that the annotation /** @var */ does not seems to work.
You might try with this, it's more verbose, but it definitely solves all my issues with serialization :
(readableLink allows to serialize relation as an entity, writeableLink allows to create new entities through relations in a Post, you might not need this one here)
Hope this helps :)
@Jeremy it works if you add the variable
$dragonTreasuresname to the@varannotation. Then you don't need the#[ApiProperty]directive.Hey @Billy!
The
MAX_DEPTHwill make sure that the embedded PHP objects have their data. But the determination of whether those embedded PHP objects will become an IRI or embedded data in the JSON is done by the serialization groups. What's annoying is that, with DTO's, you don't need serialization groups: your DTO only holds the fields that will be in your API. But... if you have an embedded object that is also an#[ApiResource]and you want it to be returned as embedded data in JSON, then you need to use serialization groups to opt into this: https://symfonycasts.com/screencast/api-platform/embedded#embedding-vs-iri-via-normalization-groupsBtw, another option that you might choose is to embed an object in your DTO that is... just a random object, and not an
#[ApiResource]. In this case, the data is embedded always - https://symfonycasts.com/screencast/api-platform-extending/embedded-objectCheers!
"Houston: no signs of life"
Start the conversation!