Using API Platform to build ticketing system (Antonio Peric-Mazar)

Hello? can you hear me? Yeah. Cool. Hi. Morning. How are you enjoying the conference? Yeah. Cool. Uh, first I want to thank you, the entire Symfony team for inviting me to speak here. Uh, I did many of conferences. This one was on my bucket list for awhile, so I'm really, really happy to be here. Also, thanks to the sponsors for making this happen. Uh, before I start, I want to get to know more about you. So first, who is using Symfony 5 in production? There is a guy, okay. Who is using API platform. Okay. Like 30% of the people. Cool. Uh, so my name is Antonio Perić-Mažar. I'm from Croatia from split. I'm the CEO of Locastic. I'm also co founder of the Litto, it is not software development agency. It is different industry. And we also, two years ago co found a Tinel Meetup, which is in our industry.

And every month we bring one foreign speaker to our home city to do of course free meetup, with beers, pizza and hanging around. Uh, we are very proud about that. Just few words about company. We are doing bunch of Symfony backend based projects. We are also doing a lot of user experience projects working with the banks, telecom operators. We are not like big company only 20 people, but we are like really do, I would say good things. Uh, what is our, what is the context of my token API platform experience?

Experience

So one of the biggest projects that we did, which is based on Symfony and API platform is ticketing platform for GFNY organization. That is the franchise business that is running in 26 countries and supporting our users from 82 countries in eight different languages including Hebrew and Indonesia. We are having that in production for a year and a half serving approximately 60,000 tickets per year.

The complexity of this project is not like high traffic or like high availability. It's more like in domain that the main is really, really complex on the front end it's React with Redux on the backend is fully Symfony with API platform. With API platform. We also did some social networks chat based, and some matching similar to the Tinder. I assume that some of you know what's the Tinder and also with a few like enterprise CRM, ERP applications so that that is something that I want to share some parts of our experience in this talk. When I planned the talk I was like aiming to talk only about this ticketing system but actually when I start writing on a paper I noticed there is like bunch of things that are repeating so I pulled a few of them and I will try to show you as best as I can what what we are doing and how we are doing the things.

What is the API platform because only like 40% people is using the API platform. I will just do quick introduction to the API platform, so if we want to trust Fabien Potencier air, that's the, that's the most powerful API platform in any language in the world. It is based on Symfony. It is dedicated to API dream projects. It contains PHP library to create fully PHP features, APIs, supporting industry standards, providing some Javascript libraries shipped with Docker and Kubernetes integration and it's now Symfony official API stack. It's very, it's very powerful tool containing many feature like simple creating crud, data validation, pagination, filtering, sorting, hypermedia, GraphQL support, whatever. Basically whatever you need to build modern APIs in matter of minutes. So just for example, I want to show you how simple it is to create crud in literally for the seconds.

So basically the first thing that we need to do is create some model, some entity, regular entity in a Symfony. So it has a name, it has ID, nothing, nothing special. The second thing is that we need to map that to the our ORM system and we need, if we want to have some validation and the only thing that we need to expose this as a resource, as the API resources to do one line of configuration. I'm using yaml. You can use XML you can use annotations, you can use whatever you want, but this is the only thing that you need to do. And you are getting fully operational crud for API, fully operational API resource with beautiful documentation. You can play it, you can test it here it, it works perfectly. So what we are getting, we are getting two types of operation that's collection operations and that's item operations to our mandatory GET on a collection and GET on the item.

Others can be removed or modified or whatever. Now this is the as as we don't have controllers, we don't have anything here. We just have configuration for the API resources. We have bunch of extension points that we can use to build our business logic. So the best thing about API platform is that it's not focusing you to think about framework itself, it's you, it's you are focusing even more than with Symfony 4 on building your value, business value for your users. Okay, so at the top of the end is the kernel events. That is regular Symfony events. But suggestion is to use other extension points because kernel events works only with the rest APIs and other extension points will work even if you are using GraphQL as your, as your APIs, uh, it is using action domain responder pattern which is like better alternative for HTTP comparing to MVC how it works, so user GET like sends something to the URI which is requesting some action. Action asks domain to do some logic and then responder or gives respond to the end user.

If you divide that in the API platform, we have separation like this on the left side in the action part we have operational resources and actions that we did with configuration. Then we have data providers, data persister and other things which contains some kind of hard domain logic and then we have serialization response which is actually responder with the things that I show you as the extension point. You can basically do the changes in any of these free parts. We have also serialization group divides in two contexts. One is normalized context. Second one is deserialized context. Why this is very powerful. It's because in this way with very simple configuration, we can have different read write models without even writing the custom code for that so we can normalize a context. You can specify which fields are you requiring. For example, for creating or updating the resource and with deserialize you can say what you want to expose to the user.

Okay, so how, how the regular project looks like with the, with the API platform and a Symfony. The things that I will talk today, it's more like about big Symfony projects which are using the API platform as one part of the application. The one part that is actually communicating with any part, with any client's application. It can be your watch, it can be your fridge, it can be your single page application, it can be your computer, it can be your car, whatever you want. We are exposing API to the API platform. We are getting some amazing thing to implement our logic. So today you will see that and our projects, we are quite, I know that we are at the conferences talking about a lot about decoupling from the framework, but I think that some points we are missing to talk how to leverage the framework.

Like for example, in our case, the fully decoupling from the Symfony is not something that you are doing because in the future we will never move from the Symfony even more maybe from the PHP but not from the Symfony. So leveraging the framework, it's a huge advantage for us, especially if we have nice structure of the projects and how we can do that. So first advice that I have for you is for configuration. Use the yaml if you have a longterm project. Did this configuration can be quite extensive and I know that annotations are really easy. Nice to start. But imagine this having inner annotations, this is what I selected. It's the configuration for only one resource. This can be even improved this file, because we can separate this in few files, which will be yaml files of course. But imagine this having in one huge annotation with bunch of other things.

It can be really, really messy. I personally use annotation for a demo because they will save me a bunch of time, but if we are doing like longterm projects, we prefer yaml. You can use whatever you want. This is just my advice.

User Management & Security

Okay. First thing that every application has is users. So let's see. Few tips and tricks, how you can do user management or security. If you have listened to the security talk yesterday, probably some of the things will be familiar to you, but there are few touchpoints that I want you to have intention to that. So basically if you are using API platform, you don't want to use FOSUserBundle. I assume that you are all aware of FOSUserBundle but FOSUserBundle. It's not built for REST. It's built for some quick user management and it brings a lot of over head with itself.

As your project grows, you will have more and more problem overriding the things that are built in FOSUserBundle. For example, I'm working at the moment at one legacy project and I spent the days removing some parts of the FOSUserBundle, finally I'm clean so I can, I can change it. What you can use, it's actually Doctrine User Provider which is included in Symfony 4 so it's simple and easy to integrate. Then you need to, only thing that you need to do is bin/console maker:user. You will get fully working user with everything setup in security and the only thing in rest API that you need to implement some action. For example, registration action. So this is how it looks. It needs to implement UserInterface and I set up already some some groups, some from writes some from read, so normalization and the denormalization context.

Second thing is that I need to map that to my ORM and database and then I need configure the resources, as you can see here, this is a little bigger configuration because I set up the normalization context and denormalization context. Okay, so the next thing is what I want to do, I want to hash password. When I'm saving, when I'm storing user to the database, I want to implement registration. The only thing that I need to do is create UserDataPersister. In the UserDataPersister I have the method which is the second method which is actually the first after, after the construct which returns do I support this method for this entity? If it is true it will do persist or remove. Depends what we need to do and in the persist method what I'm doing, I'm hashing the plain password, removing the plain password, saving User to the database.

I can send the email here, I can do whatever logic I want so like I am putting some custom logic in a place in the API platform. What we are using mostly for for for security for a login is JSON web tokens can, which are like standards that are lightweight and simply identification system stateless and they are storing, token to the browser, local storage and then can be authenticated. This depends on the, from the project from the project. For some projects you will need or too or something else. But this works for, for for example, for this ticketing system, this works perfectly. It works very simple. You send request to the server with your user name and password. Server signs the token returns this to you and which every new request you are sending that to the server. What is very important thing that I want you to pay notice here is that insight.

When we are decrypting this encoded string, we are actually getting the data about user. We can get any data that we want to start by. For example, we will store email. Why this is okay for our, just before I tell you why this is very important for of course for this, for this authentication, we have two bundles which are LexikJWTAuthenticationBundle and JWTRefreshTokenBundle. JWTRefreshTokenBundle is to use after your token is expired to get a new one. So basically after the login you will get a token and refresh token. Token will sign, we'll have some short time to live. Uh, also one very nice thing too that a lot of developers miss from the documentation is that we have user checker and security component and this can save a lot of time for you. Basically the Symony is using some default UserChecker, but you can implement your own just implementing this interface and then the only thing that you need is put one line of the setup in security.yml.

In this way you can do some additional check. It can use any fields that you want for a user, like is it expire, is it blocked, is it deleted, did I ban it or whatever you want. It gives you very, very good flexibility and two points. Preauthorization and PostAuthorization. Resource and operational level on on API platform are quite powerful to set up. So basically what this configuration says, it says that if you want to access to the resource book, you need to be at least the role user. You need to logged in. If you want to create new book, you need to be role admin. If you want to update the book, you need to be owner of that book and you need to be admin. Okay. And what's even powerful is the using voters. You all know what the voters are. Okay, cool.

So the same way that you are using voters in any Symfony project with this slide, two lines of the configuration, you can use them in a API platform projects.

JWT Tip

When I told you that to pay attention on a JWT token, it was about this for example, you're usually your user is stored in your database. You have DoctrineUserProvider, but sometimes in some projects it can happen that your user is stored to some third party API. And then communication with that third party API is not like all this possible or it's like too, too slow. For example, three seconds. You don't want users to wait every time for three second. So what's the nice thing to do? It's actually to create database last user. Since we have email inside JWT token, we can, we don't need to authorize user to load user every time from the database.

We can just authorize them from the email because we can trust to the JWT token and this is very, very nice thing to have to have in mind how to do this. It's again just the line of configuration. We need to set up the default JWT, lexik_jwt provider and we need to say our firewall for the API. Can you use that provider for the user all done. The only thing that you need to have in mind is that if you want to get user information, you need to create that manually. You cannot do token get user. Okay.

Multi Language API

Creating multi language API. This is something if you are doing like big projects usually today you will need to create a multi language APIs. API platform is not supporting that out of the box so when we got this requirement we need to find some solution about it.

What we actually did as we are doing a lot of it to Sylius and are also doing amazing work same as, API platform's people. We took the idea about Sylius translation, how, how they are working there and implement that with the API platform. We created open source bundle, publish it to the GitHub and also there is blog post explaining how you can use that. It's, it's actually quite easy what you need after you install the bundle you need to extend your entity that you want to be translatable with the AbstractTranslatable and add first method which is createTranslation that method. It's loading the translation entity that we will implement later. And the second thing is that we need to implement a translation relation to to our uh, to our PostTranslation entity. Of course we need to create some virtual fields which will be used to get single language query and then we need to implement translation entity which will contain all the fields that we want to translate.

After we did all of this, we need to set up like few lines of configuration. We need to set up the translation groups for normalizing context and we need to set up translation groups for some filters or whatever we want to use. And how we are using this. It's quite easy when we are creating a new resource we are sending all the languages that we want to create so for example in this example we are creating English and Dutch, sorry Germany and we are sending all the files. We see that we have local, local code inside. If we want to get some language we can query only one language. Then we will use this virtual properties and we will get like only title and the content for the English language. If we want to get all the languages we need to say to send dynamically groups, translation and we will get all the languages listed.

Second thing that you need to figure out how to solve when you have multi language API is static translations. There are two solutions. One solution is to use LexikTranslationBundle which stores translation to the database but you can export it to the files. We wanted to have solution that writes directly to the file so that we don't need to have some cron jobs or manually dropping the translations to the, to the files. So there is a guide also on our blog how it's, it's too much code but it's really easy concept. So if you want to learn how to write your own transplantation, you can go there and check it.

Manipulating Context

Uh, next thing that I want to talk is manipulating contexts. This example that I have on a slide is purely from the API documentation, but this is very important thing that we have in API Platform.

Why it is important. It is important because for one of the basic examples is that we don't have, we don't need to have two separate APIs for admin and for the regular users. So we can have the same access points, but based on their roles we can have different responses. How this work, again we have some API resource here you can see annotation configuration and then we are adding serializer groups. So you can see at the active that we have book:output and we have admin:input and on the name we have book:output and book:input. So what we are doing, we are creating a service, which is our BookContextBuilder. We are decorating API platform, serializer context_builder and we are building our custom logic inside and this is very simple concept. Again, we are just checking what we are doing like is are we talking about book resource, do are we doing denormalization or normalization is user logged and do they have admin role? If they have just check for the context groups and add group admin:input period, that's it. You will get a different response for user and for admin user.

Symfony Messenger Component

Oh, Symfony Messenger component please. Who's using Symfony Messenger component. Okay. Not much people but like a lot 20%. Symfony Messenger component is a very simple component to start using it. It allows us to have like communication between queues, between different application, easy to implement, easy to use, making asynchronous communication easy. It just works in very simple way. You have sender, you have handler, sender sends a message, handler takes that message, do some logic. We have some bus transportation between that. We can have some middleware, but you can read that in documentation. Why I'm talking about this because with Symfony messenger and API platform, we can have Command Query Responsibility Segregation or CQRS pattern. It's very easy how it works. Basically you just need to set up configuration. Again, you will say a message that we are using messenger, that output is false for example, and response on this action will be 202, like status. Okay.

Then the handler will do other things. We'll do heavy, heavy things. This is the simple thing. I want to talk about more, more advanced thing. Also you can use that in the image in ImageMediaDataPersister for example, data persisters skier, like if you have image that you want to resize, you can dispatch a message than some Lambda or cloud, whatever can do the work, return the image or whatever. Uh, also you can use it in events, but as I said, like try to use other extension points. This is the reason why messenger is very important for me. If you are talking about, this is the project that I'm actually working on. It's fully hosted on a Google cloud platform. It is event sourced, distributed base. I don't know. Architecture. Okay, so what we have in the blue is the thing that is actually hosting our Symfony application in the yellow and orange are cloud functions from the, from the, from the Google cloud platform and Firebase and other things.

We have a bunch of the pub and sub services communicating between and the API platform is this small red.here. Literally API platform exposing the rest API to the, to the client's app and everything in the behind. As I stated in the beginning of my talk is Symfony, so basically Symfony is always like more complex part than just API platform. So if you're talking about Symfony and the messenger, if you read documentation it's always sending the message to some application and then that same application is consuming the message. That works perfectly. But usually that's not how the things work. Usually you are at least communicating between two different Symfony applications only or even in, in better case you are communicating with some third party applications, nodeJS, GO applications, whatever and Symfony works perfect in first case, it works not so good in second case it doesn't work at all out of the box.

In the third case actually you need to do, can I say out of the box? It doesn't work just with configuration that you set up in your Symfony application. Why? This is how Symfony dispatch message it looks like. We have body which actually contains our message. Then we have a bunch of headers which are describing which handler will be used and some which command bus, blah blah blah like a bunch of configuration that is needed in our Symfony application. If you dispatch this message app message CommunicationMessage to some other Symfony application, there is huge, there is huge chance that you are not using the same namespace so this won't work if you are dispatching the message from the node JS application for example, which was my case, you are not having properties on the headers. You don't have the body which is string actually contains escape, escape JSON and properties and the headers are proper JSON. You actually have this, we will have some JSON like no description, nothing. Then me and colleague, developer and a team, we start working and he was dispatching the message. I was getting errors, errors. I start researching and then the simple solution was can you add the headers and other things that we need and at the first was like, huh, no, but that's the only message that we are consuming at the moment from the other side like, okay, let's do that. That's the quickest way, but that's not like sustainable way. We want to do that better. So how you can do that is actually you can write your custom ExternalJsonMessageSerializer what that message JSON serializer do. It's actually receives your message, decodes from the string to array and then based on some parameter, if you have it in a message, it can checks what is happening, what it is, what it is receiving, and then can create the message that your SymfonyMessengerHandler needs so that it's possible to do.

Of course you need to have some parameter which you can, which you can catch on this point. We were lucky enough so we'd have this key based on the key I know that's testing or communication or caching or whatever and I have variables which I know that they are body of my message. Again, you need to create your message as I said you, we need you. We'll return that. That will work. Also you need to set up some configuration because in in a default configurations Symfony Messenger is using the default Symfony Messenger, deserializer second solution that is also possible is that using the HappyMessageSerializer why I didn't put the code for a message serializer because it has the dependency you need to have body property in a JSON and with this custom serializer we can even outweigh that. Also there is a blog post Symfony Messenger on AWS Lambda that Tobias wrote. I think so, yeah. Also have in mind that messenger component is similar to ORM components. It will work in most of the cases, but if you have some like really, really specific cases, you will need to build something by yourself at some point or at least like really hard customizations.

Handling Emails

Handling emails with new Symfony Mailer component that is easier than ever. Like we have a bunch of emails that we are sending bunch of notifications in, in, in all, in all applications that are mentioned at the beginning. So how to do is like read the Symfony Mailer component. You have really easy integration with Symfony Messenger. So that means you can send them asynchronously. You have also support for the load balancer. So if you have like three or four different transports for the emails, it will spread them like with the round Robin method, uh, it's also high level availability.

If a fail over, if something fails, it will go to second then the third transportation, for the email, it supports out of the box. It supports Amazon, SES, Gmail, MailChimp, Mandrill, Mailgun, PostMark and SendGrid. And that's for me personally, that is the best thing that happened with this component because usually setting up the emails was pain in, really painful. So yeah, that is something if you are not using, check that it's much better than just Swift mailer that we had like a few years ago or even a few months ago.

Creating Reports (exports)

Uh, creating reports and exports. When you are doing ticketing systems, that is like something that you do on a daily basis. What we had as a problem was importing data from the few different sources, storing that in a database, doing some transformation and exporting in a total different format to our users, so how we did the importing first we set up the crawlers with the Symfony Messenger component.

We crawl all the data that we need. We could not do transportation on a fly because there was too much relationship between all the data that we are getting, so we need to store that to the database. Then do transformation. Then again store that to the database to get it in a format that we want to use. Now that when we have the format that we can store in data base that we can read that we can do some operation with that we needed to export that to CSV files. It's the same if you are doing XML it's the same if you are doing XML files, what you need to do is actually create custom operation for export. You need to change the format to the CSV. For example, in this case you need to say what is DTO Data Transformer Object for your output in this case is OrderExport class, which group we will use for in this case OrderExport and that in this case is a POST method.

I got the question why it is POST not GET? The POST because when we are creating something we always use the past and in this situation we are creating some document that does not exist. So that's our logic. Again, you can use GET if you want. Next thing is that you need to create this DTO object. It's simple model. Sam as you have model for for reading and writing to your database by entity. You have this DTO here which contains bunch of properties, get and set methods so you can go with the public that that depends on you. A second thing is that you need to implement DataTransformer, DataTransformer again is a simple class that do only one thing. Transform data from one format to some other format. Not, not a format, wrong word. Like literally from one object to another object and doing some like changes on, on the way how we store the data.

So what we have here, we have export on the left side, which is created in, ah at the beginning of the transfer method. And we are setting the values and transforming if it is needed for exporting to the uh, to the, to the end user. What is very important here if you didn't, uh, if you didn't notice this pagination is enabled false. Why it is that because in the report we want to have all the data that we have in a database. By default we will get 30 of of that. And that's it. That's how you do exports. How you do reports in any format that you want. You just need to create objects that will contain data and export it to the, to the users.

Real-time Applications

Real time applications with API platform. Who, who is using or who heard about Mercure. Okay, cool. So in like, this is like what we can do now easily. And what are the real time application? What the real time applications are. So like if you do update on the web, it will be updated on all mobile devices that are connected together. In the past there was a way to build real time applications, but usually that was like some, let's call it weird solutions. It will be ready with NodeJS combination that they will be subscribed to PHP will publish to the Redis, Node will read it, and then to the web sockets will send it to all the clients that are connected. There was Pusher solution like hosted solution, ReactPHP did something. I think they did actually a good job there. But to be honest, the PHP is not built for real time applications.

PHP is built for request response. We all know that. So what actually changed here is that we have this Mercure HUB which is created by Kevin. Kevin did a lot of amazing work with the Symfony, with the Mercure. If you listen yesterday. So basically what we are doing after we create something, we passed that resource to the Mercure hub. And then Mercure hub sends server-side events to all clients that are connected to Mercure hub. It cannot go in opposite directions. So it's not a duplex communication as web sockets, but in 99% cases this will be solution that will work for your project. So if you are doing chat application, if you are doing push notifications, that will work. Now this is just a short description. It's written in Go, it's based on, it's automatic to HTTP to HTTPS, it has CORS support, Cloud Native, it's open source and bunch of other things.

This is very interesting. It works very easy out of the box with API platform. So basically you only need to say it's Mercure true on any changes that you do with your resource. It will be automatically sent to the Mercure hub and it will be, if you're, if some users are connected, they will get the data. This part of the JavaScript is the freelance of the code that you need to subscribe to the mercure and get the data. Uh, once the event happens.

Testing

If you are doing a big applications like this, you need to write a test. So what what we learned during during this applications is that for example, we don't believe in 100% coverage of the tests because that usually will means that we will lose a lot of time. But we believe in smart testing like we want to test the critical cases, we want to test any bug that happened.

We want to have like really good structure tests. Also for legacy code we have policy like if we got some bug or something we will first cover that with a test and we will resolve that. And also like if you are not very experienced, maybe TDD will be hard at the beginning for you. So just start writing the tests. But I want to talk a little bit about API testing tools. First tool is that you have it out of the box with with API platform is HTTP client in API platform which which manipulates the Symfony HTTPKernel directly. So it give boost of performance and you will have much faster tests. If we are comparing that with the test over the network. Also it's good to consider another tool which is APITestCase. They are quite similar, I would say from the, from the Sylius project and in API test case you get a lot of helper methods that can save you a bunch of time like testing is is JSON equals something?

Is JSON contains something is this, is this matching the schema that we defined for the resource for example book, book object. A second very useful thing is PHP Matcher because if you are creating dummy data you don't want to create data like Antonio surname Peric, SymfonyCon. You want to just check is this string type is this integer type is this date is this array contains something that is possible with the PHP Matcher and this will save bunch of time for you. There is plenty of methods inside that so you can basically check any type of expression that you have inside your response. If you are setting up the new project and you don't have a data Faker, which is, which works with AliceBundle under the hood, it's very, very useful tool to create dummy data which looks real data. Kevin used that yesterday for, for his demo at the end of the talk.

So the data were created with the AliceBundle. Also you can use Postman test. Postman tests are very nice but they will be slower than API test case both from the Sylius and from the API platform because the Postman can later be used as ah interactive specification of your API for your front end developers. The tests are written in NodeJS but they are very, very simple to write and then you can integrate them with the Newman, Newman in your console or in your CI and just run it before deploying to the server to check is everything okay with uh, with uh, with your, with your tests, with are API, sorry. Also some tools for checking test quality like Infection, PHPStan, Continuous Integration, other things. I think you are listening a lot about these tools. So I will just mention them and yeah, use them.

They are good tools. Okay.

Last slide before I'm done is actually the picture that Fabien Potencier tweeted a few days ago about the book he's writing and about architecture of the modern modern application built with a Symfony. So the applications that I, that I mentioned today that I described some parts of how they work are actually built in this way. We have a bunch of models which are integrated to work together. Some models are communicating directly to the API inside the code. Some are communicating to the message component, depends on the things, what are they doing and the API, the API is like I'll say API is only one part which is exposing that to the user but also it give us like really nice extension points where we can put a bunch of our logic. Usually of course in this kind of applications you have a lot of console command applications which are doing a lot of background job as a workers or as cron jobs or or whatever. Uh, one last thing, like conclusion API platform and a Symfony especially Symfony 4 are really awesome tools and I think we all should be happy that we have that good framework in PHP and that we are using that as a part of our everyday work. Thank you.