Crawling the Web with the New Symfony Components (Adiel Cristo)

Video not working?

It looks like your browser may not support the H264 codec. If you're using Linux, try a different browser or try installing the gstreamer0.10-ffmpeg gstreamer0.10-plugins-good packages.

Thanks! This saves us from needing to use Flash or encode videos in multiple formats. And that let's us get back to making more videos :). But as always, please feel free to message us.

Tip

SymfonyCon 2019 Amsterdam presentation by Adiel Cristo.

When developing an application, a common feature we need to implement is gathering data from other sources. These data are available in various forms, usually unstructured, and behind some JS application, making them harder to reach.

To make things worse, as the application evolves, we need to get more data, from even more sources. But don't worry, things can be easier!

In this talk we'll use the Symfony's HttpClient, Messenger and Panther to build a crawler, first as a simple console application, then evolving to a distributed one.

Transcript & caption for the talk will be added soon.

Leave a comment!