"In February 2005, Booking.com started their initial set of A/B testing experiments for which they had more than 1000 experiments in parallel and 150,000 total experiments. However, they observed a less than 25% success rate. Manuel stated that the goal wasn't to be right; it was to learn fast. These experiments ultimately built their Data-Driven DNA."
"Booking.com's original tech stack was built on Perl libraries and MySQL that offered asynchronous replication and commercial support. They had only one master database in 2005 that has grown into approximately 6800 database instances in 2020. Their MySQL setup is also unique because they don't have specialized hardware, stored procedures, Universal Disk Formats (UDFs), database views, and a cache layer."
"Their 'secret sauce,' as Manuel characterized it, consisted of smaller databases (with a 2TB limit) that fit in Non-Volatile Memory Express (NVMe) solid state drives. They observed point queries that were less than 350 microseconds. This model was successful until their data grew too large. To remedy this, Booking.com added Apache Hadoop for distributed storage and processing at scale."
Booking.com's 20-year technology evolution demonstrates a progression from simple Perl and MySQL infrastructure to complex distributed systems supporting AI. Beginning in 2005 with over 1,000 parallel A/B experiments despite a 25% success rate, the company prioritized learning velocity over immediate accuracy, establishing a data-driven culture. Their infrastructure evolved through three layers: Data Management, Machine Learning Engineering, and Domain Intelligence. The original MySQL architecture, optimized with smaller 2TB databases on NVMe drives achieving sub-350 microsecond query times, eventually required scaling through Apache Hadoop. By 2011, two on-premise Hadoop clusters with 60,000 cores and 200 petabytes of storage enabled processing at scale, supporting the company's transition toward AI integration.
#data-driven-architecture #distributed-systems #ab-testing #database-scaling #ai-infrastructure-evolution
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]