Setting Up HAProxy Load Balancer on VPS Step by Step

As your application grows and traffic increases, a single server often cannot handle the load. The solution is to set up multiple backend servers and use a load balancer to distribute incoming traffic among them. HAProxy is a powerful, open-source load balancer and reverse proxy that can handle millions of concurrent connections. It automatically detects unhealthy servers, distributes traffic intelligently, and terminates SSL connections, making it ideal for building scalable, high-performance infrastructure.

Why Use HAProxy Over Nginx?

While both Nginx and HAProxy can perform load balancing, they have different design philosophies. Nginx is a general-purpose web server with load balancing capabilities, whereas HAProxy is purpose-built specifically for traffic management and load balancing. Here's why HAProxy stands out:

• Superior Connection Handling — HAProxy can manage over 1 million concurrent connections on standard hardware using the epoll mechanism
• Active Health Checks — HAProxy actively probes backend servers and automatically removes unhealthy ones from the rotation
• Session Persistence (Sticky Sessions) — Bind client sessions to specific backend servers for applications with in-memory session storage
• SSL Termination — HAProxy handles HTTPS connections from clients and can communicate with backends over HTTP, reducing CPU overhead
• Detailed Logging — Comprehensive request logging with timing information for deep performance analysis
• Advanced Routing — Route traffic based on domain, path, headers, or any combination for complex multi-application setups

Prerequisites

• Ubuntu 20.04+ or Debian-based VPS with systemd
• At least 2-3 backend servers running web services (HTTP/HTTPS)
• Domain name pointing to the HAProxy server's IP via DNS A Record
• Ports 80 and 443 open in firewall with no other services using them
• Basic terminal and text editor knowledge (nano or vim)
• sudo access to install packages and modify system files

Step 1 — Installing HAProxy

# Update package list
sudo apt update

# Install HAProxy and Certbot for SSL management
sudo apt install -y haproxy certbot

# Verify installation
haproxy -v

You should see HAProxy version 2.x or higher. If your distribution provides an older version, add the HAProxy PPA repository for the latest stable release.

Step 2 — Creating the Main Configuration File

HAProxy's main configuration file is located at /etc/haproxy/haproxy.cfg. Back up the original and create a new configuration:

sudo cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.backup
sudo nano /etc/haproxy/haproxy.cfg

Replace the contents with this comprehensive HAProxy configuration:

global
  log stdout local0
  log stdout local1 notice
  chroot /var/lib/haproxy
  stats socket /run/haproxy/admin.sock mode 660 level admin
  stats timeout 30s
  user haproxy
  group haproxy
  daemon
  maxconn 262144              # Maximum concurrent connections

  # SSL security configuration (A+ rating)
  ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
  ssl-default-bind-ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256

defaults
  log     global
  mode    http
  option  httplog
  option  denyall-on-abort
  timeout connect 5000
  timeout client  50000
  timeout server  50000
  retries 3

# Frontend — receives traffic from clients
frontend myweb
  bind *:80
  bind *:443 ssl crt /etc/letsencrypt/live/domain.com/fullchain.pem key /etc/letsencrypt/live/domain.com/privkey.pem

  # Redirect HTTP to HTTPS
  http-request redirect scheme https code 301 if !{ ssl_fc }

  # Set header indicating HTTPS origin
  http-request set-header X-Forwarded-Proto https if { ssl_fc }

  # Detailed HTTP logging
  option httplog

  # Capture Host header for debugging
  capture request header Host len 64

  # Route all traffic to backend pool
  default_backend webservers

# Backend — processes traffic on actual servers
backend webservers
  balance roundrobin                        # Distribution algorithm

  # Health checking
  option httpchk GET /health HTTP/1.1
  http-check expect status 200
  default-server inter 5s fall 2 rise 2

  # Define backend servers
  server backend1 192.168.1.10:8080 check   # IP:Port of backend 1
  server backend2 192.168.1.11:8080 check   # IP:Port of backend 2
  server backend3 192.168.1.12:8080 check   # IP:Port of backend 3

  # Session persistence via cookie
  cookie SERVERID insert indirect nocache
  server-template srv 1-10 0.0.0.0:80 disabled cookie srv

# Statistics interface
listen stats
  bind *:8404
  stats enable
  stats uri /stats
  stats refresh 30s
  stats show-legends

Required modifications:

• Replace domain.com with your actual domain name
• Replace IP addresses 192.168.1.10, 192.168.1.11, 192.168.1.12 with your backend servers' real IPs
• Change port 8080 if your backend services listen on different ports
• Adjust /health endpoint based on your application's health check path

Step 3 — Obtaining SSL Certificate from Let's Encrypt

Before starting HAProxy, obtain an SSL certificate using Certbot:

# Stop HAProxy if running
sudo systemctl stop haproxy

# Request SSL certificate (replace domain.com and email)
sudo certbot certonly --standalone -d domain.com -d www.domain.com --email [email protected]

# Respond to prompts as required

# Verify certificate files
ls -la /etc/letsencrypt/live/domain.com/

You should see two important files: fullchain.pem (certificate chain) and privkey.pem (private key).

Step 4 — Starting HAProxy Service

# Test configuration syntax
sudo haproxy -c -f /etc/haproxy/haproxy.cfg

# If no errors, enable and start the service
sudo systemctl enable haproxy
sudo systemctl start haproxy

# Check service status
sudo systemctl status haproxy

# Monitor logs in real-time
sudo tail -f /var/log/haproxy.log

Step 5 — Configuring Sticky Sessions

If your application stores session data in server memory, enable sticky sessions to route clients to the same backend:

# Edit HAProxy configuration
sudo nano /etc/haproxy/haproxy.cfg

# In the backend pool, add cookie configuration:
backend webservers
  balance roundrobin
  cookie SERVERID insert indirect nocache

  server backend1 192.168.1.10:8080 check cookie backend1
  server backend2 192.168.1.11:8080 check cookie backend2
  server backend3 192.168.1.12:8080 check cookie backend3

# Reload HAProxy to apply changes
sudo systemctl reload haproxy

HAProxy will inject a cookie named SERVERID into responses. Subsequent requests from the same client will be routed to the same backend based on this cookie.

Step 6 — Setting Up Automatic SSL Renewal

SSL certificates expire every 90 days. Set up automatic renewal with a post-renewal hook:

# Create renewal hook directory
sudo mkdir -p /etc/letsencrypt/renewal-hooks/post

# Create the renewal hook script
sudo nano /etc/letsencrypt/renewal-hooks/post/haproxy.sh

Add this content:

#!/bin/bash
# HAProxy SSL Certificate Renewal Hook
# This runs after certificate renewal is successful

# Reload HAProxy with new certificate
systemctl reload haproxy

# Optional: Send email notification
echo "HAProxy SSL certificate renewed for $(date)" | mail -s "SSL Renewed" [email protected]

Save and make it executable:

# Set execute permissions
sudo chmod +x /etc/letsencrypt/renewal-hooks/post/haproxy.sh

# Test renewal dry-run (doesn't actually renew)
sudo certbot renew --dry-run

# Automatic renewal happens via cron daily

Load Balancing Algorithms

HAProxy offers several algorithms for distributing traffic. Modify the balance setting in your backend pool:

1. Roundrobin (Default)

balance roundrobin
# Sends requests sequentially to each backend: srv1, srv2, srv3, srv1, srv2, ...

2. Least Connections

balance leastconn
# Routes to the backend with the fewest active connections
# Ideal for long-lived connections like WebSockets or streaming

3. Source IP Hash

balance source
# Hashes client IP to consistently route to the same backend
# No cookie needed, but session data is lost if backend goes down

4. URI Hash

balance uri
# Routes based on the request URI, useful for content-based distribution

Advanced Health Check Configuration

# Sophisticated health checking with custom timing
backend webservers
  # Check every 3 seconds, 2-second timeout
  # 3 failures mark server down, 2 successes mark it up
  option httpchk GET /health HTTP/1.1\r\nHost:\ domain.com
  default-server inter 3s fall 3 rise 2 timeout 2s

  server backend1 192.168.1.10:8080 check slowstart 60s
  server backend2 192.168.1.11:8080 check slowstart 60s
  server backend3 192.168.1.12:8080 check slowstart 60s

The slowstart option gradually increases traffic to newly recovered servers, preventing thundering herd scenarios.

Troubleshooting Common Issues

HAProxy Won't Start

# Check for configuration syntax errors
sudo haproxy -c -f /etc/haproxy/haproxy.cfg

# Common issues:
# - Incorrect domain name (doesn't match certificate files)
# - Wrong SSL file paths
# - Ports 80/443 already in use by other services

Backend Servers Not Responding to Health Checks

# Test connectivity to backend manually
curl -v http://192.168.1.10:8080/health

# Check if firewall blocks port 8080
sudo ufw allow 8080

# Or add security group rule in cloud provider (AWS/GCP)

SSL Certificate Not Working

# Verify certificate file permissions
sudo ls -la /etc/letsencrypt/live/domain.com/

# Grant haproxy user permission to read certificate
sudo chown -R haproxy:haproxy /etc/letsencrypt/live/domain.com/

# Reload HAProxy
sudo systemctl reload haproxy

Monitoring and Logging

Real-time Statistics

# Query HAProxy statistics via socket
echo "show stats" | sudo socat stdio /run/haproxy/admin.sock | head -30

Forward Logs to Syslog for Analysis

# Install rsyslog if not present
sudo apt install -y rsyslog

# Create HAProxy-specific syslog config
sudo nano /etc/rsyslog.d/99-haproxy.conf

Add this line:

:programname, isequal, "haproxy" /var/log/haproxy/haproxy.log
& ~

Restart rsyslog:

sudo systemctl restart rsyslog

HAProxy vs Other Load Balancing Solutions