Load Balancer Explained: Nginx Setup on VPS

When your website grows and a single server can no longer handle all incoming traffic — or when that single server goes down and takes your entire site with it — a load balancer is the solution. It sits in front of your backend servers and distributes incoming requests across all of them automatically, keeping your service online even when one server fails.

This guide explains what a load balancer is, which algorithm to choose, and how to configure Nginx as a load balancer on Ubuntu VPS from scratch.

What Is a Load Balancer?

A load balancer is a device or software that acts as the single entry point for all incoming traffic, then forwards each request to one of several backend servers based on a chosen algorithm. Users always connect to the load balancer — never directly to the backend servers.

Load Balancing Algorithms

The algorithm determines how the load balancer decides which backend server receives each request. Choose based on your workload:

1. Round Robin (Default)

Requests cycle through servers in order: A → B → C → A → B → C → ... Best when all servers have similar specs and each request requires similar resources.

2. Least Connections

Sends each new request to the server with the fewest active connections at that moment. Best for long-running requests (API calls, file uploads) where some requests take much longer than others.

3. IP Hash

Uses the client's IP address to determine which server receives the request. The same IP always goes to the same server (session persistence). Best for applications that store sessions in server memory rather than shared Redis or a database.

4. Weighted Round Robin

Round Robin with weights — higher-spec servers receive more requests. For example, Server A (weight=3) handles 3 out of every 4 requests; Server B (weight=1) handles 1 out of 4.

Setting Up Nginx Load Balancer on Ubuntu VPS

Nginx supports load balancing natively — no extra software needed. This example uses 3 VPS instances:

Step 1 — Install Nginx on Load Balancer

SSH into VPS-LB and install Nginx:

sudo apt update && sudo apt install nginx -y
sudo systemctl enable nginx
sudo systemctl start nginx

Step 2 — Create the Upstream Block

Create a new site configuration for the load balancer:

sudo nano /etc/nginx/sites-available/load-balancer

Paste the following (Round Robin is the default):

upstream backend_pool {
    # Round Robin (default) — uncomment to change algorithm
    # least_conn;     ← Least Connections
    # ip_hash;        ← IP Hash (Session Persistence)

    server 10.0.0.2:80;
    server 10.0.0.3:80;
}

server {
    listen 80;
    server_name yourdomain.com www.yourdomain.com;

    location / {
        proxy_pass http://backend_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Step 3 — Enable the Configuration

# Create symlink to enable
sudo ln -s /etc/nginx/sites-available/load-balancer /etc/nginx/sites-enabled/

# Test configuration syntax
sudo nginx -t

# Reload Nginx (no downtime)
sudo systemctl reload nginx

Step 4 — Use Weighted Round Robin

If VPS-B1 has a higher spec than VPS-B2, assign it a higher weight so it handles more traffic:

upstream backend_pool {
    server 10.0.0.2:80 weight=3;  # handles 3 of every 4 requests
    server 10.0.0.3:80 weight=1;  # handles 1 of every 4 requests
}

Step 5 — Configure Health Checks

Nginx can detect when a backend is unresponsive and temporarily stop sending traffic to it:

upstream backend_pool {
    server 10.0.0.2:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.3:80 max_fails=3 fail_timeout=30s;
}

Add proxy_next_upstream for more granular passive health checking:

location / {
    proxy_pass http://backend_pool;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    # If backend returns an error, try the next server immediately
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_connect_timeout 5s;
    proxy_read_timeout 60s;
}

Step 6 — Add SSL to the Load Balancer

Users connect only to the load balancer, so install SSL there. Backend servers communicate over HTTP on the private network (SSL termination at the load balancer):

# Install Certbot for Let's Encrypt
sudo apt install certbot python3-certbot-nginx -y
sudo certbot --nginx -d yourdomain.com -d www.yourdomain.com

Certbot automatically updates your Nginx config to handle HTTPS and sets up auto-renewal.

Step 7 — Test Traffic Distribution

Place a small test script on each backend to confirm the load balancer is routing correctly:

# On VPS-B1
echo '<?php echo "Response from: " . gethostname(); ?>' > /var/www/html/hostname.php

# On VPS-B2
echo '<?php echo "Response from: " . gethostname(); ?>' > /var/www/html/hostname.php

Then hit the endpoint multiple times from your local machine:

for i in {1..6}; do curl -s http://yourdomain.com/hostname.php; echo; done

You should see responses alternating between vps-b1 and vps-b2.

Test Failover — Take One Backend Offline

# Stop Nginx on VPS-B1 to simulate a server failure
sudo systemctl stop nginx

# Your site should still respond from VPS-B2
curl http://yourdomain.com/

Add a New Backend Without Downtime

To add a new backend server, update the config and reload Nginx — no restart needed:

# Add VPS-B3 to the pool
upstream backend_pool {
    server 10.0.0.2:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.3:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.4:80 max_fails=3 fail_timeout=30s;  # ← new server
}

# Reload with zero downtime
sudo nginx -t && sudo systemctl reload nginx

Monitor Logs

# Watch access log in real time
sudo tail -f /var/log/nginx/access.log

# Watch error log
sudo tail -f /var/log/nginx/error.log

Enable Nginx Status Module for Real-Time Monitoring

Nginx ships with a built-in stub_status module that provides live connection statistics. Add the following location block to your load balancer server config to enable it securely:

server {
    listen 80;
    server_name yourdomain.com;

    location /nginx_status {
        stub_status on;
        allow 127.0.0.1;       # allow localhost only
        allow 10.0.0.0/24;     # allow private network
        deny all;              # block everyone else
    }

    location / {
        proxy_pass http://backend_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

After reloading Nginx, poll the status endpoint:

curl http://localhost/nginx_status

A typical response looks like this:

Active connections: 45
server accepts handled requests
 1024 1024 3891
Reading: 0 Writing: 3 Waiting: 42

Solving Session Problems with Redis Shared Session Storage

The most common issue when introducing a load balancer to an existing web application is session inconsistency. A user logs in, gets routed to Backend 1, then the next request goes to Backend 2 — which has no knowledge of that session — and the user is abruptly logged out. The correct fix is Redis as a shared session store that all backend servers read from and write to.

Install Redis (on a dedicated VPS or on the load balancer)

sudo apt install redis-server -y
sudo systemctl enable redis-server

# Allow backends to connect — edit /etc/redis/redis.conf
# Change: bind 127.0.0.1 → bind 127.0.0.1 10.0.0.1
sudo systemctl restart redis-server

Configure PHP to Use Redis Sessions (on every backend)

# Edit /etc/php/8.x/fpm/php.ini on each backend
session.save_handler = redis
session.save_path = "tcp://10.0.0.1:6379?auth=yourpassword"

Load Balancer vs Reverse Proxy — What Is the Difference?

Nginx can act as both a reverse proxy and a load balancer. The two terms are related but distinct:

In practice, Nginx's load balancer is a reverse proxy with an upstream pool. The distinction is about how many backends you're proxying to and whether a distribution algorithm is involved.

Recommended Architecture for Thailand and Singapore VPS

AsiaGB offers VPS in both Thailand and Singapore, making it straightforward to build a multi-region load balancing setup that serves users across Southeast Asia with low latency: