Modern high-traffic websites need to handle a tremendous number of simultaneous requests. To address this, organizations have adopted horizontal and vertical scaling, which involves adding more servers or increasing the power of existing ones. However, these methods alone don’t guarantee a service will remain continuously available.
When a single server is responsible for all incoming requests, it can easily become overwhelmed, leading to slow performance or a complete system crash. To overcome this, we use horizontal scaling, which involves adding more servers to handle the load. This is where load balancing comes into play to efficiently distribute incoming network traffic across the group of backend servers, often referred to as a server pool.
Networking Components
To manage and optimize network traffic, several key components are used:
- Load Balancer (LB): A load balancer distributes incoming traffic to multiple servers. It continuously monitors the status of its backend servers and will stop sending requests to any server that is unresponsive, overloaded, or has a high error rate. When a server is unresponsive, is incapable of handling new requests, or exhibits an increased error rate, the LB halts traffic to that server and routes it to another available server. LBs can operate at different levels of the OSI model, such as Layer 4 (Transport) and Layer 7 (Application), using various algorithms to make traffic-routing decisions.
- Reverse Proxy: A reverse proxy acts as a gateway between the internet and your web servers. It intercepts incoming client requests and routes them to the appropriate backend server. Reverse proxies are used to improve security by hiding the details of your backend servers from clients. They also offload tasks like SSL termination, content caching, and compression to improve performance and scalability.
- Forward Proxy: Unlike a reverse proxy, a forward proxy sits between client devices and the internet. It mediates outbound traffic, forwarding a client’s request to a web resource on their behalf. Forward proxies are often used in corporate networks to enforce web access policies, filter content, and enhance privacy by masking client IP addresses.
- API Gateway: An API gateway is a specialized type of reverse proxy that provides a single, unified entry point for client applications to access multiple backend services. They are commonly used in microservice architectures to simplify client access and manage the API lifecycle, handling tasks such as authentication, rate limiting, and security policy enforcement.
AWS offers Elastic Load Balancers (ELBs) in different types as application, network, and gateway LBs, which we will cover in more detail in Chapter 9.