System Design
Horizontal vs Vertical Scaling
Vertical scaling involves adding resources to a node while horizontal scaling involves adding more nodes. While it is easier to simply add more memory to a server, it is costly and limited. Something as large as Facebook or Google could not possibly be served from a single machine. As a application grows, it will eventually need to spread out horizontally. Horizontal scaling also means that if any one node fails, your application will still be live.
Load Balancer
Load balancers will distribute incoming traffic to different servers so that no one machine is handling everything. Every machine must be a clone of each other so each can handle the same tasks. The load balancer can work as simply as just choosing a machine at random.
Database Denormalization and NoSQL
A normalized SQL database will have one value per row and column and use join queries to access associated collections in other tables. This can become expensive, so it may eventually become worthwhile to start allowing multiple values in a field. This will likely mean that you're duplicating data, which can also be expensive when you need to update that data, since you'll need to do multiple writes.
NoSQL databases will generally make it easy to stored nested data. This means that, given the right situation, you can retrieve all the data you need with a single query.
Database Partitioning (Sharding)
Just as splitting up your servers becomes necessary, so too does splitting up your database. You can have all of you servers access a single database, but eventually that will have performance issues. When you partition your database, you will also need ways of figuring out where data is. The following are some popular approaches. Often, multiple approaches will be used on a large app.
Vertical Partitioning
Vertical partitioning usually means you partition by a feature of the application. All the tables for related to subscriptions on YouTube might be on a single machine, for example. However, even that can grow too large and be forced to split up even more (especially for something as large as YouTube). That will likely mean using a different strategy.
Key-Based (or Hash-Based) Partitioning
This is a process similar to building a hash table. You take some value from the data to be stored (likely the id), hash it, and then mod that value against the number of servers you have. Unfortunately, if you need to add servers, you will also have to reallocated all of the data.
Directory-Based Partitioning
This involves creating a lookup table (a separate machine), which tracks which server has the data you're looking for. This will make it possible to easily add more servers, but has its own limitations. The lookup table itself cannot scale infinitely, and it can be a single point of failure, causing the entire application to go down if it fails.
Caching
In-memory cache is very vast, but you're going to be limited to how much data it can hold. On a large app, it's important to choose carefully what gets cached, how it gets cached, and for how long it should be cached. A popular website like Hacker News doesn't want to run its complex query algorithm for the top 30 submissions for every request. That algorithm will generally be run once per minute or so and cached. Then, everyone who goes to the home page will see that response from the cache.
Asynchronous Processing & Queues
Any process that takes a long time and can block a user experience should be done asynchronously. You don't want a user to have to wait around for something that doesn't need to be done right away. This may involve using a separate thread with a priority queue to line up tasks that can be done later. It can also involve pre-processing, as is the case in the Hacker News example above.