I've been a fan of Figma for a while, and one thing that has always impressed me is their ability to scale their services to meet the needs of their growing user base. However, as I recently learned, even a company with Figma's resources and expertise can run into challenges when it comes to scaling their database infrastructure.
Figma has been a heavy user of PostgreSQL for some time, but as they've grown, they've encountered a number of issues with the database's default scaling capabilities. PostgreSQL is an all-in-one box, with the query layer, storage layer, and implementation details all squeezed into one package. While this can work well for small to medium-sized applications, it can become a major bottleneck as an application scales to millions of users and billions of rows.
Figma's database team learned this the hard way, as they embarked on a 9-month journey to horizontally shard their PostgreSQL stack. This process involved splitting groups of related tables, like Figma files or organizations, into their own vertical partitions. This allowed Figma to make incremental scaling gains and maintain enough runway to stay ahead of their growth.
However, vertical partitioning was just the first step. Figma knew that they needed to go even further to unlock nearly infinite scalability. This is where horizontal sharding came in.
Horizontal sharding involves splitting a single table or group of tables across multiple physical database instances. This allows a table to support any number of shards at the physical layer, and enables the database to scale out further by simply running a physical shard split. This capability is crucial for removing major scaling challenges and achieving full horizontal sharding, which Figma knew would be a multi-year effort.
The process of horizontally sharding a table or group of tables is complex, and involves breaking up the data and spreading it across multiple physical databases. This can result in the loss of reliability and consistency properties that are taken for granted in traditional SQL databases. However, with careful planning and execution, it is possible to achieve full horizontal sharding and unlock nearly infinite scalability.
Figma's experience with scaling their PostgreSQL stack is a great example of the challenges and complexities that can arise as an application grows. While it's certainly possible to build an application without worrying about these issues, it's important to be aware of them and have a plan in place for addressing them as they arise.
In the case of Figma, they were able to successfully horizontally shard their PostgreSQL stack and achieve nearly infinite scalability. This is a testament to their expertise and dedication to providing the best possible service to their users.
In conclusion, scaling a database infrastructure is a complex and challenging task, but it is essential for ensuring that an application can meet the needs of its users as it grows. Figma's experience with scaling their PostgreSQL stack is a great example of the challenges and complexities that can arise, and the importance of having a plan in place for addressing them. Whether you're building a small application or a large, complex system, it's important to be aware of the scaling challenges that you may face and to have a strategy for addressing them.