Dailymotion, powered by symfony

Dailymotion

Dailymotion, one of the world's largest independent video entertainment website, is now powered by symfony. Dailymotion is among the top 60 websites worldwide. In December 2008, more than 41,9 million unique users visited the website and they viewed more than 922 million online videos (source: comScore, December 2008). Dailymotion is also the number 2 French site in the world.

Founded in 2005, the Dailymotion website has always been powered by PHP and managed by a team of in-house developers. Over the years, a custom framework has been developed to power the platform.

But mid-2008, they decided to move away from their home-made framework and migrate to an Open-Source framework.

Dailymotion by the numbers

What to know all the nitty-gritty details of the Dailymotion architecture? Here are some interesting numbers:

• 50 PHP machines serving about 5000 PHP pages per second
• 17 DB machines
• 30 memcache machines for a total of 210 Gb of memory storage
• 60 Gb/s of bandwidth
• 1.2 petabytes of storage

And that does not account for the machines that encode the videos, host the search engine, ... All in all, they have more than 430 machines!

Why migrate to an Open-Source framework?

Even if Dailymotion has more than 10 developers dedicated to the website development, they wanted to be more focused on their business, instead of being forced to maintain a framework.

Over time, they started to feel the pain of maintaining their own framework: the maintenance of the framework itself started to take way too much time: adding new features, documenting them, training new developers, keep everything coherent, etc...

By choosing an Open-Source framework with a large community, they free themselves from everything other than their business.

Olivier Poitrey, one of the co-founder and CTO of Dailymotion, sums up the main advantages of moving to a framework:

• Don't reinvent the wheel
• Maintain less code
• Better compliance to coding conventions
• Implement new processes like unit/functional testing easily

Why symfony?

Thanks to the creativity of their marketing department, the Dailymotion developers add new features on a nearly everyday basis. So, rewriting the website from scratch was out of question. They just could not have afforded to stop all new developments, waiting for the new code base to be finished.

They needed a way to migrate step by step from the old code base to the new one, and keep adding new features at the same time. And symfony was the ideal candidate. Why?

First, because the symfony framework is one of the most well-known PHP frameworks, and one used by many high-traffic websites.

Then, the framework is well decoupled, meaning that each component can be used by itself, without the whole framework.

Eventually, the other big selling point of symfony was its flexibility. It is very easy to override the default behaviors or add new ones if needed. And that's exactly what they needed.

The migration process

The main challenge of the migration process was to avoid breaking too many things by introducing small doses of symfony.

The first change was to make symfony handle the request, dispatch it to the old code, get back the response content, and send back the response to the browser. Wrapping the existing code was made possible by using the symfony 2.0 sfRequestHandler class. We decided to use symfony 2.0 for its raw speed, even if it is not public yet.

By the way, that is the classic process of the symfony framework evolution. Sensio Labs uses symfony for its customers and a lot of new features are added based on their needs and feedback. For the Dailymotion migration, Sensio Labs helped Dailymotion define the migration strategy and the target architecture.

The next step was the migration of the mod_rewrite rules to the symfony routing. Thanks to the great flexibility of the Symfony routing, Dailymotion has been able to extend the default behavior to fit its unique needs: object parameters, labeled variables, segment grouping, arbitrary ordered optional segments, centralized formats, ...

Along the way, the Dailymotion team contributed back code to the symfony community. Some code has already been committed and is already available as part of the symfony 1.2 release. Some other enhancements are proposed as tickets.

One of the other big changes was the introduction of massive unit and functional tests. To migrate with confidence, more than 5000 unit and functional tests have been written, using lime and the symfony test browser.

During phase 1, the following components of symfony has been used:

What's next?

Even if the Dailymotion website has only been powered by symfony for less than a week, the next migration steps are already planned:

• Introduction of symfony controllers
• Usage of symfony's facilities like sfI18n, sfUser, sfCookieSessionStorage, ...
• Usage of symfony's 2.0 service container
• Usage of symfony's model system (Doctrine)

Are you still using YouTube for your videos? I think it's time we all switch to Dailymotion ;)