Sharding for unlimited scale

With vertical sharding, each individual table resides on a single server, but collectively the tables are spread out across many servers.

Perhaps you have a database with several hundred tables. Most of them are small, but two are large - 1TB each. Instead of housing everything on one server, we can use vertical sharding. The two large tables will get placed on their own dedicated servers, and the rest can remain on a shared server. This will still appear as a single, unified database from the application's perspective, since it connects to the database through a proxy layer.

Horizontal sharding is another way to spread tables out across servers. With horizontal sharding, we take the rows of an individual (large) table and spread them out across many servers. The proxy layer maintains metadata to keep track of which rows of a table live on which server, thus allowing it to effectively fulfill queries on this data.

Perhaps we have a database with many small tables, and one huge table that has 1 trillion rows and is 2TB. The large table could get spread across four shards, each managing 500GB of the data. The proxy layer will manage routing queries to the appropriate shard.

The sharding strategy is the technique used to distribute the data. A common sharding strategy is to use ID hashing as the shard key. With this technique, we choose one of the ID columns from the table. Each time we receive a new row, Vitess generates a hash of this ID. Each shard server is responsible for storing the rows for a range of hashes, and new row gets sent to the appropriate server.

A good choice of shard key can lead to excellent performance, but a poor choice can be detrimental to your database. PlanetScale enterprise support provides guidance on how to shard your database effectively.