Building really fast applications (Tobias Nyholm)

Hey, my name is Tobias and I'm going to talk to you about building really fast applications. This talk is not titled building quick applications. It's not title building, fairly quick applications. It's called building fast applications. The kind of applications you see in the movies, and I'm doing this talk because I think we should care about performance I'm also doing the talk because if you know about performance, it gives you better understanding what your application's actually doing. And it's fun. Performance is fun, eh. Short about me. I work at a company called Happyr. I do plenty of Symfony stuff. I'm also running PHP Meetup in Stockholm. And I do Open source. I started my open source career in um, in a conference. It was a SymfonyCon in Paris. PSR6 just came out. So me and a few other friends were like, yeah, let's, let's build implementations for PSR6 and PSR6 is caching, correct. So we build PHP-cache we built like, 20 different implementations of memcache, Redis, the Couchbase database or whatever. And I do plenty of other things in open source too. The most important, the most fun thing here is the NSA, NSA totally violates class privacy. So it's easier to access private properties, methods using NSA. Um, let's get started. Um, frameworks are slow. This is something I see on the internet time to time. Last time I saw it was today in a tweet. And you all heard this, right? Yes, I heard this. Yes. Thank you. Who here uses the framework? Show of hands.

Fair...

You who rose your hand? What is the framework?

What is a Framework

I've been traveling, traveling Europe, mostly asking these questions to different people both in public and private. And for every person I ask, I get a different answers. Some here in the front whispered like, Oh, a collection of components. And Nicholas Grekas said on stage yesterday that if using the dependency injection container, that's when using Symfony, I dunno, but before we, before we're gonna, before we get to claim that something slow, we need to know what it is. So back in the days, like 10 years ago, I tried to use the Zend, I download the Zend, started to read installation instructions. And there was plenty of files. It was folders after folders, and file after file. I don't even know where to write hello world to make something happen. So I quickly gave up like don't make get wrong. Symfony wasn't too much better.

Like if you did Symfony 10 years ago you did your SVN checkout and you got plenty of files and nowadays that's something different. Like Symfony 5 barely has any files when you start with it, you get like four folders and five files. So in in, in Symfony 5 or in Symfony 4 it's you who are responsible for what code you adding. You are responsible what dependencies you are downloading and if you download slow code, your application will be slower. And so I want to say framework is just our code or the code we responsible of. So let's dig into, is our code really slow?

Let's talk about Performance

Let's start talking about performance. There are three important rules in performance. These rules are always true no matter what you're doing in computer science. Rule number one is buy a better server. Hardware is way, way cheaper than software.

And so this is what you gotta do always like unless you're doing something crazy, this is what your going to do. If work, by crazy, I mean if you have more than 10 servers in production, you should buy another one. Rule number two, use Varnish. Most people do use Varnish. It's super awesome, super great and this is not that kind of talk like using Varnish is basically all the talk is, Oh let Varnish handle 95% of traffic and everything's super fast, super smooth. There's plenty of resources on this online. There's plenty of talks, plenty of books, plenty of blog articles. You should read them and learn about them. It's, it's Varnish is great. However, it's not what this talk is going to be about. If you see a talk that's named how we handle 1 million requests a day or how we take care of 10,000 requests a second, it's definitely a Varnish talk.

Rule number two is like caching, like, but we're not going to talk more about Varnish. I want to talk about this. I want to talk about rule number three, run less code. This is the only way how you actually speed up the execution of your code. You all recognize this, nobody knows this, but you all recognize this. This is machine code. It's basically when you write in PHP, when write echo hello world, it gets compiled to opcache and opcodes and then it got compiled to machine code. Each of these lines is an instruction for your CPU. Each of these line is like one tick in CPU, so if you have a CPU that is one gigahertz CPU, it means that it can run 1 million of these lines every second and every ever these, every code do, every line of PHP we write it generates multiple of ticks and the more ticks you're having, the slower your application will be. So basically more code, more ticks. It's going to be slower. I researched this a little bit because I'm trying to be a good, a good scientist. So I looked at previous Symfony versions, I looked at how many lines of code that are in them, how many lines of code were executed when running Hello World. How many function calls there was, how much memory and how much time it took to execute.

So the point I'm trying to make here is the less code you have, the less code you're running, the less memory it'll take and the quicker it will go. And this is not super, it's not super accurate. Like you see that Symfony 3 was adding some more code than Symfony 2. And also it's not really fair to Symfony 1, because I run this in development mode, but you get the idea right. Whenever asking questions like I wanted to, yeah, thank you. Sure. Okay. But this is only for the hello world application and our application of doing hello world. Like if we were doing hello world, we should do something like this. This is by far quicker than everything every framework ever. So what we need to figure out is what is, what is our application doing? We're not doing hello world, we're not doing something super generic.

We have a business problem and we need a specific solution to this business problem. So first thing we need to do when creating application, we've got to figure out what is the application doing. And this is very hard to make a generic answer to. The best generic answer is basically Symfony skeleton. Like Symfony skeleton is a great base to build out towards. But if you, if you do something specific like, like my URL shortener, I don't, I don't need something super generic, I just need something that's specific for me. So this is an application that's actually in production and it didn't take more than an hour to to write, test and deploy. And I think now you see what it's doing, just takes the URL and rewrites it and sends to somewhere else. And I think we all can agree that if I was using a framework for this, that would be overkill.

Do you need this?

Right? And you could also argue that since I'm using, if I'm using PHP, that would be overkill. I should use something like Nginx rewrite rules or something, but I don't know Nginx very well, that's why I'm writing this in PHP. So whenever you start a new application, you need to figure out where on this scale your application going to be, is it going to be somewhere over here with my redirect application or is it going to be somewhere over here and a big fat application using forms and stuff. Like you need to think about this when starting a new project. So what I like to do whenever I start a new project and when I download new dependency, I like play a game and it's called do you need this? So basically do you need security, if you're on a private network? Do you need forms, if you're running a micro service? Do you need a router like Symfony 4 is the fastest router ever written in PHP, but if you only have five routes, do you really need it? An IF statement is good enough, do you need a kernel if you have a small code base? Do you need dependency injection dependency injection is great, but do we really need it if you only have 10 classes and also do we need classes?

Do you need HTTP? Like if you're running locally, do you need HTTP and if you do need HTTP, do we need HTTP Foundation or are you happy with superglobals? Like you should know that getting a user IP from superglobals is pretty much impossible and HTTP Foundation helps you with that. However, your application may not need the user IP using the Messenger component is great. I love the Messenger component and it's a great way to read from queues and write to queues, but do really need it. Do you need that fancy code that abstraction, maybe they AMQP extension is good enough. In writing the database. Do you need doctrine? Do you need doctrine ORM or happy with doctrine PDO. What I'm trying to force myself and now telling you to force yourself like you should. It's a balance.

I do need, I do use a library with good well-tested code and nice API and they'll take care of some time. It's an all day everything. Or do you write the code yourself having something that's super specific for your use case, but that means that you have to take care of edge cases yourself and you may have to have rough APIs to work with. So it's basically do I want to use a library and maybe develop really quickly or do I want to go with no library with a specific use case, maybe developing more slowly like the one I'm a quick application is slow development or like it's a balance. And I want you to think about this before it required a new dependency.

Building a small application

So when when building something small like small and less complex than Symfony skeleton I like to start from nothing. I like to start in writing my index.php file myself and the result of this is nothing really I can publish. There's nothing really I can like, Oh this is my new framework, which is super nice because I'm trying to solve a specific business case. And that doesn't really make sense to make it public. I, however, I did write a small library, I call it superslim, which basically is a small framework for educational purposes. So what I'm going to tell you now, the next five or 10 minutes is it's super slim. If you pay attention, don't, don't, you don't need to Google it. Superslim one word.

So what I do, I like to use the middleware pattern. It's the same thing as, it's the same thing as PSR 15 so I start with my index.php file. I create a request from the globals and I create the response and somewhere down there I also echo out the body. The result of this file is as expected nothing, but I want to add more. I want to add more content in my response. The first thing I do when adding an anonymous function with it takes a request and response and then a callable to the next anonymous function. And this specific one is just checks that it's a get request. And if I want to be more cooler and add another anonymous function, and that adds something to the response. So when I have my array of middlewares or an array of anonymous function, I give them all to a runner and then I say, Hey, runner, make sure run yourself and give me back the response. And if I run this now I'll get something at least something a little bit more cool.

So you may think, Oh, this runner's is where every, every, every magic thing happens. But it's really isn't. This is, this is the full runner. It just takes the middleware, run them one by one, like run middleware A, B, and C, and then that's it. So you may think like, Oh, this anonymous functions, they are, they're toxic or they looked bad, or they give me theu give me a really wrong feeling. So if, so we just slap on an interface.

So I wrote, I've written many applications using this pattern. And this is a simple application. I think it's a, I'm a geocoder basically. So the idea here is that all the code in this application, I've written myself, I've know this written this Runner, I written this PHP file, I've written all this middlewares sure. In the cache middleware I decided to use Symfony cache. So, but I made the decision to include that cache library. It's all the responsibilities on me. Um, and it might be tricky to understand, okay, sure I like this index.php file, but where does the magic happen? So in my geocoder the last middleware is a router and the router looks like this. So it's a switch case statement. And if I see a request coming to from IP I just call the ipController and the ipAction and that controller looks like any controller that's, a normal Symfony controller.

So consider this application again, the caching I added a cache layer, I was super happy with it. But when developing it came became an issue because I added cache and then I did a change in my controller and I couldn't really see much change. So it would make sense to have different configuration in development and production. So I decided to add a custom kernel and a custom kernel also allow me to have a yaml configuration and a dependency injection. So I created a class I wrote to myself, I call it Kernel and a constructor with a environment parameter and a Boolean if I'm the debugging or not and in the handle function that's the same code as I had in my index.php file. I add my middlewares and I give it to a runner. Then I run the runner, there's a boot function also.

The boot function basically looks at if they have a CachedContainer, if I have a CachedContainer, use it. If I don't have a CacheContainer, create a new ContainerBuilder, add some parameters to it and then look at the services.yml file and the services_environment.yml file and look at, ah, for configuration and then a compile it and a dump it to the file system for, for optimizations for in future. And then I use the compiled container. So with this small class, I can write configuration like this and this supports auto wiring, this supports out in a figure or not resilient. It's auto wiring without configure, you need to do a trick, you need to add code for this entry boot function. So this is basically saying like if you see a service which implements the EventSubscriberInterface, add this kernel.event_subscriber tag to it.

Or if you see something that looks like a command, add this command to it. And then the Drager system compiler passes to make sure every service is up and running properly. So if you want to have more auto configure functions, you need to add more and more boilerplate like this. And in my case, my, the code stayed like this. I didn't have to add anything else. But if you want to continue working on this project, you may add so much boilerplates. At some point you're like, Oh, it's too much. I can't take the, I don't know what all these lines is doing. And if so, feel free to use the FrameworkBundle because this is actually what a framework bundle does for you. It brings some nice configurations for your components and it registers services to make sure auto configure is working properly.

Improve Performance on Legacy Applications

I also recognize that you don't, all of us don't have the luxury of creating new applications. Most of us are sitting on old legacy applications and we just want to make it faster. So, um, let's try to improve the performance on legacy applications. Uh, you all know the basics. This is the basics. They, and let's make them more generic. And there's a reason that these are in order, like the goes from cheap to very expensive. And there is, again, there's no point of making number three unless, unless it's, you think it's fun or unless you're doing something crazy, like you have more than 20, 30 production servers. So I'm gonna sip some water for effect and then I'm going to start.

So I'm going to try something like this. I'm going to give you some tips now and I got to have a tip counter up there so you can keep talking. The tips. I will share the slides later. So don't, I mean, so you can go over them and I won't give tips on architecture. Like if you have the wrong architecture, that's a very specific problem. So I can really get tips on this, but I going to try, I'll try to give some tips. So are we ready now? I'm a little bit nervous because this, I'm going to try to have a high speed with these tips, so if you knows something I'm going to quickly move on and I'm not sure how well I'm going to do. So I'm a little bit nervous. Anyhow, the first tip is basically you should look at the bottom of this list to buy a better server, the more powerful services at the bottom.

Symfony Cache

Um, and, but we're going to began to talk about the caching a little bit, caching in Symfony has been way, way improved in the last versions, like Symfony implements, three caching interfaces. They implement the great PSR 6, they implement the okay PSR 16 the simple cache and the implement, the excellent Symfony cache and there's adapters to memory, adapters read, Redis memcache, APCU and file and plenty more. And what you should do is you should have one or two of these chains in the configuring application. So cache can either be small and very fast or slow and very big like memory, small and fast and file is slow and big. Also in this scenario, memcache is also slow and big. So create a chain of caches like this. So you do the fast first. If you are missing, then you go to the slower one and make sure when you configure, Redis, memcache don't use TLS because that could increase the handshake could take up to two millisecond, 200 milliseconds.

Doctrine Cache

You should also make sure to configure the cache with doctrine because this is a lifesaver. Like this is probably if you use Flex it's probably already configured. Please be sure you have checked this because when someone told me this, it's saved like 25% of all requests on my application. So doctrine has three caches that the memcache metadata cache which is basically configuration and have query cache which translates from DQL to SQL and it has a result cache which is basically the result of the query.

Result Cache

Ah here is an example of how using result cache, so say you building a CMS like nobody really cares if your page is one hour old from data is not so you can, you can easily, you can easily store it this for an hour. This will obviously be a way better fit for complex queries.

I was going to say that this will, running this code will actually create DQL and invoke doctrine. If you wrapped his function around a Symfony cache, you will not use doctrine and not using doctrine is less code than using doctrine so please, please try and experiment with this.

Cache HTTP Calls

This is not about super important. Often get, often gets forgotten. You should cache HTTP calls because if we fight and optimize for milliseconds and then you do an HTTP call, that takes a second like what's the point of fighting for the milliseconds and showing you this. This is a simple Twitter client that is fetching all the latest tweets and say that you have two visitors every seconds and it takes a half an hour for this. The average time, this is math saying if we measured for 10 minutes, the average time will be an half second. Every request to make an HTTP called the Twitter and it takes a half a second.

So this is long and boring. I know, but you get the idea, right, because this is building up to something. So if you're using cache, you could, I mean I get everything using Symfony Cache I say cache, please give me the latest tweets. That's my cache key. If it's a miss, then call this, call this callable. And on the callable here, I say, just cache this for one minute. It's not a lot. It's just one minute. And with this only one minute caching, the average time waiting for Twitter is now 4 milliseconds and I've been asking around plenty and they confirm. So please cache your, please cache HTTP calls, even even just a little bit and you don't have to be fancy. It's a cache service. I mean you can use local cache and properties. Say you have a simple class calculating the area of a circle and if I call this, every time I call this, I give this a radius, every time I call it I'll start calculating pie and this is obviously time-consuming and we don't have to do this every call.

We can use a local cache to store result and also yes, I'm blown away. This is how we calculate PI. This is actually an Excel, I think that's funny. Building, ah building a web application, it's all about parsing a request and delivering response. Everything that's not part of delivering response you should postpone to later. Ryan, you showed us in the previous talk like you use async for pretty much everything you can use async for. Something that's super slow is code generation like Symfony dependency injection container helps us with this. No yaml or XML is read in production. Symfony dependency injection does this for us and read everything and generates the PHP code with our, with our configuration. So again, if you're not using the dependency injection container, please use it because it's improves performance massively. So when you define all your services, you still need to instantiate them and instead, oops instantiating services are still slow.

Lazy Services

So that's why you should use Lazy services. So if you have a fat service with a lot of the dependencies to instantiate this, we need to first instantiate all the dependencies. And then instantiate this class, and this can take a lot of time, and especially if all the services are not used in every, every time your using the service though, if all the dependencies use every time using service. So say and lazy services, you can say this service should be lazy, which means don't instantiate it until we using it. So this is when you writing a RequestListener, for instance, most of your request listeners looks like this. So we check if this is the master request, then you continue. And if this starts with API and then continue, but most of the time it doesn't start with API. So you just cancel, you do nothing. And we don't want to spend a lot of time instantiating this expensive event listener if not going to do any job and the work. So that's why lazy services is super, super helpful to speed up performance. However, when defining a service that's lazy, it is global, maybe you won't only want lazy services on the request listeners, and this is where service subscribers comes into play.

Service Subscribers

I don't know if you know about service subscribers, but that they're, Oh, I barely knows service subscribers if they're fairly fairly new, like in Symfony 3.4, Anyhow, how do you, do you know how to use service subscribers? Nobody raised their hand. Okay, so basically what you do, you're going to say this event request listener, I will be interested in these three services and these three services will not be, will not be instantiated until you say, Hey, locator, give me the service. So this is a great way to have a lot of heavy lifting request listeners, but without affecting performance, they're only heavy in when you actually using them. Makes sense. Lovely. So this is cool and all. Let's move on to doctrine. Look at this five lines of code. It's a form. For yourself, think about potential issues with this code. I'm going to show you a less obvious one. You see, I have a daytime immutable for one year. This will generate a new query to the database every second. If I'm just a little bit smart, I can say that I fixed the time. This will help my data database to cache the query. There is more issues with the data layer. For instance, I forgot to use indexes. This is the simple web profiler. And if I click the explain query, I will see that I have about 300,000 users and this will take a lot of time. So if I just add my index to this query, a table, I can see that it takes, I get what goes from 200 milliseconds down to less than one to get this query. But we're not done. I assume some of you spotted more issues with this code?

Nope.

Um, if there is a hundred user created in last year, this will generate 100 queries and, Oh, sorry. Sorry. This will generate one query but 100 objects because this will, the hydration is the expensive thing in doctrines lifecycle. So I fetch my I fetch all my rows and then I create a user object for all of these rows. And if I'm only using two fields here, like I should only query for what I need. So if I change this code a little bit to make sure to query only the ID and the email, I won't create 100 objects. And this will reduce memory massively.

1 + n problem

Massively, like in your application won't crash when you seen this form. Another doctrine related thing is the 1 + n problem. Basically this query over here, I'm fetching the web shop for the this country and then I'm looping over all the products in the web shop to display the name. This will generate one query for the web shop. And then if I have a hundred products it will generate 100 additional queries because we are using lazy loading. So a better idea would be to actually query for the products immediately. This will only generate one query.

Data Transfer

Another doctrine related thing is a data transfer. Say that the product contains a lot of, a lot of megabytes of data. You have images there, you have a lot of product descriptions and every time you fetch your products from the database you take, you transfer data from the database into memory and maybe not really using that memory, maybe not even using the product description. So there's three possible solutions for this. You can use partial objects, which means that you're only getting few fields. However does issue with that is if I give you a product, you don't know if the partial product or the full product, I can also split this product entity into two tables. Like you have a product and then I have a one to one mapping to metadata and the metadata has all this, all the heavy lifting things.

So when I have a product that's quick and easy, no memory, trans, no memory, not a lot of data is transferred from database to memory or I can use another feature which is new Doctrine 2.4. It's the NEW keyword which basically says I can keep my product just like this, like with all the heavy data, but I can create a model looking like this but only the two properties I, this is my simple product and when I query for this I can say give me a simple product and this is how we instantiated. This will instantiate just, uh this instantiate a simple product, not the full, fat one.

PHP.ini

Other things you should consider is your PHP.ini file. Like make sure you don't do the default from your vendor. Let's make sure we configure sensible, default yourself. This basically makes sure to enable opcache and it sets a sensible defaults for the real path cache. And bunch of other things. There's plenty, Oh, make sure you know what you're doing and make sure it's someone else tells you this is correct thing for your application.

Front-end

The same thing also goes through with front-end stuff like you may have a super quick backend but the front end is slow and heavy and same rules here like run less code. For instance, here's something on top of my head like make sure you have small HTML pages. Like make sure we have a limited number of cookies, only include the JS and CSS script that you actually are using and you can warm up the DNS cache. You can make sure that your scripts are executed asynchronously. You should always use Gzip and HTTP2 and there's one thing with icons, like if you have icons, make sure you use the font swap. So either. You start loading new page and it's white and then you're waiting for fonts to load and when fonts loaded then you show your content or you show the content at once and then you swap off the fonts when they are getting there. I wrote a blog post how you do do this with Google fonts etc and about four weeks later Google updated so they do it automatically for you. So that means two things. That is, font swap is a good idea, and Google reads my blog.

You should also make sure to optimize and lazy load images, you should use CDNs and you should make sure you have good cache control headers like this will cache it for one year, but it would also tell the browse, this file is immutable. It will never ever change. So even so you still, if you have assets, make sure they are immutable because that will be a performance and webpack encore is excellent for helping you with this like front and assets could be a mouthful, but using webpack encore, it helps you with code splitting in the optimize the images, etc, etc. And you're lucky because there are plenty of tools helping you to audit your frontend like Chrome Lighthouse is one of those tools and it's all right. It's perfect.

I show you these rules. There's actually a fourth one...

Have you ever logged into PayPal? Logging you in securely? Sure you are, sure you are. They have a slow system somehow and they made it a feature. I mean that's super smart but I mean all those other things that might be like you've seen before. Sorry for super being super dark. It's basically placeholders. Facebook does this, Slack does this and this even react plugins that helps you to display something. So it looks like your content loaded a little bit quicker than it actually did.

Profile your Application

So if you apply any of these tips and tricks, you should make sure to profile your application. Like you should make a baseline measurement, do a PR with a change and then profile if it actually was a positive change or not. And I cannot stress this enough, like a few weeks ago I created something was an annotation warmer which makes you annotation, go to Symfony cache. I was pretty happy about it and I told Tiwan and Tiguan like, Hmm, I think this will be slower because good reasons but I couldn't really tell him anything because I forgot the profile, the before and after difference. So make sure to always remember to profile your applications before and after you do changes. And profiling also a good way to figure out where your architecture is strong and is also a good way to figure out where your application's wrong. And there are a few profilers out there. The most popular popular one is Tideways and Blackfire. And here's an example of Blackfire. Could you tell me why this particular application is slow?

Too much code? I'm happy to say that I can give you a small hint. It's basically class, class loading like composer is great doing class loading, especially since PHP 7 but it's not excellent. So I'm going to show you some slides. Like if you were at SymfonyCon three years ago there were a person called Andrew Carter and he showed a lot of great things. But these slides, these slides are the best two slides I've ever seen. So let's play a game. Let's play a game of the HTTP pipeline being a restaurant. So the client is a customer, the server is the restaurant and the request is an order and the response is obviously the food and the application who created the response, the application, the one who creates the response is the chef. And the one who facilitates all this is a waiter or waitress. So if the HTTP pipeline is the restaurant, the customer will enter the restaurant. The waitress would take the order, waitress creates a chef, chefs makes food, waitress gives foods customer, and then the waitress brutally murders the chef. Why? Why would we do this? Like if you were a management consultant and came through this restaurant, what is the one tip you would give them? So, and it takes a lot of time to create a new chef.

So in PHP more technical, we do something like this, a browser connects to Nginx and Nginx talks of FastCGi to PHPfpm, PHPfpm has a pool of processes already warm and running. This is good because creating a process is very expensive. And when the request comes in, we create an application, the application delivers the response and then we kill the application. We're killing the chef. One request more comes in and we kill it. And you probably have a lot of traffic. So you probably doing this a lot of time. But what if it could not kill a chef? What if we could keep the application loaded when they fancy animation is done I'll click. Excellent. So what if it did something like this instead? So FastCGI talks directly with a PHP process and this will keep the application loaded over multiple requests and we don't kill the chef.

Sure. Since PHP is not built for this kind of Witchery, we have to kill it one time to time to just make sure we don't have memory issues. But this is like this is, this will speed up your application significantly. And if you, if you want to do this, there's a few libraries I like FastCGI Daemon. And if you basically put your application inside the anonymous function, and you can do this with my fancy, fancy self built library or a full Symfony framework too. So this is a way for us not to kill a chef, but you may ask, Hey Tobias, next week, where PHP 7.4 what about preloading? So preloading is an excellent way to doing pretty much exact same thing. So with preloading, you can make sure to have like 90 95% of your chef still there. And that's way better than nothing, right?

PHP 7.4 Preloading

So how to use preloading, it is that Symfony simple. You basically add this line to your php.ini config and Symfony will automatically generate the classes or your chef. The classes that you use the most since this is not a PHP 7.4 is unstable and PHP 7.4 is not stable and there is some issues with this and the latest master. This Symfony feature not is completely stable either. There will be plenty of improvements if you tag your class. If you, you can tag your service as container hot path. It will end up in this file. Also, it will be ways for bundles. Make sure you can add things. Add classes to this file too. So I'm going to end my talk now I'm going to say frameworks are not slow. It is you as a developer who has the responsibility to, you have responsibility. Whatever you add to application, you have responsibility for every code line of every dependency you add to your application. And if you add a lot of things you will have a lot of code and yes that will be slow. But if you follow all the tips and tricks I've showed you, you can easily go under 10 milliseconds. Like in fact, you can go under less than five milliseconds. And this is rarely needed, right? We want to do this because it's fun, but it's super possible. And I've done a talk a few times before and all of a sudden at the end of the talk. People like, Oh, Tobias, have you tested this? Really? Oh, Tobias, can you use this in production? Yes, you can. Thank you for your attention.