Nginx Configuration Developers Need to Know: Load Balancing, Health Checks, and Rate Limiting Explained

In high-concurrency, distributed systems, Nginx is not only a reverse proxy but also the first line of defense for service stability. Recently, while working with a client on high availability tests, I discovered that some functions could be perfectly handled using Nginx configurations.

📌 This article applies to Nginx Open Source (non-Plus), and all configurations have been verified in production environments.

1. Choosing a Load Balancing Strategy

Nginx’s upstream module supports various load balancing algorithms. Choosing a reasonable one can significantly improve system stability and resource utilization.

upstream backend {
    least_conn;  # Recommended strategy
    server 10.0.0.1:8000 max_fails=3 fail_timeout=30s;
    server 10.0.0.2:8000 max_fails=3 fail_timeout=30s;
}

Comparison of Common Strategies

💡 Suggestion: Unless there is a strong session binding requirement, prioritize using least_conn.

2. Health Check and Failure Recovery Mechanism

Nginx Open Source only supports passive health checks.

Key Parameter Explanation

Failure Recovery Process

1. Backend is marked as down due to errors;
2. During fail_timeout, Nginx will not forward requests to it;
3. After timeout, Nginx automatically attempts to send a request to probe;
4. If successful, it recovers; if failed, counting restarts.

✅ This is “Passive Recovery” — relying on real traffic to detect if the backend has recovered.

3. What Situations Trigger Retry?

By default, only connection errors (like connection refused) or timeouts are considered failures. HTTP 5xx responses (like 500/502/503) are NOT considered failures by default!

If you want to retry on 5xx as well, you need explicit configuration:

location / {
    proxy_pass http://backend;
    proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
    proxy_next_upstream_tries 3;
    proxy_next_upstream_timeout 10s;
}

🔔 Note: Only enable 5xx retry for idempotent requests (GET/HEAD/DELETE).

4. Timeout Configuration Details

Timeouts are the main cause of 502/504 errors. Three key parameters:

• proxy_connect_timeout: Timeout for establishing TCP connection
• proxy_send_timeout: Timeout for sending request body intervals
• proxy_read_timeout: Timeout for reading response body intervals

Timeout Parameter Suggestions

🎯 Golden Rule: Timeout = Backend P99 × 1.5 ~ 2.0

5. Rate Limiting Configuration: Anti-Scraping & Anti-DoS

1. Concurrency Connection Limit (limit_conn)

http {
    limit_conn_zone $binary_remote_addr zone=conn_limit:10m;
    limit_conn_status 429;

    server {
        limit_conn conn_limit 40;

        location / {
            proxy_pass http://backend;
        }
    }
}

• limit_conn_zone: Defines a shared memory zone to track current concurrent connections for each key.
• $binary_remote_addr: Uses the client IP in binary form as the key (saves memory, equivalent to $remote_addr but more efficient).
• zone=conn_limit:10m:
  • Creates a shared memory zone named conn_limit;
  • Size is 10MB, capable of supporting connection state records for about 160,000 concurrent IPs (each record is about 32–64 bytes).
• Enable connection limiting in that server (or can be in location).
• When a client’s concurrent connections exceed the limit, Nginx returns HTTP 429 (Too Many Requests) status code.
• By default, Nginx returns 503. Explicitly specifying 429 here makes the semantics clearer (indicating “too many requests” or “too many connections”).
• conn_limit: References the limit_conn_zone name defined earlier.
• 40: Indicates each client IP allows at most 40 simultaneous connections to this server (or location).
• limit_conn_status 429: Returns 429 Too Many Requests.

2. Request Rate Limit (limit_req)

http {
    limit_req_zone $binary_remote_addr zone=req_rate:10m rate=3r/s;
    limit_req_status 429;

    server {
        limit_req zone=req_rate burst=5 nodelay;

        location /api/ {
            proxy_pass http://backend;
        }
    }
}

• limit_req_zone: Defines a shared memory zone to store rate limiting states (e.g., request counts per IP).
• $binary_remote_addr: Uses the client’s binary format IP address as the rate limiting key.
• zone=req_rate:10m:
  • Creates a shared memory zone named req_rate;
  • Size is 10MB, storing states for about 160,000 IP addresses.
• rate=3r/s: Limits the request rate for each IP to at most 3 requests per second (i.e., on average 3 requests allowed per second).
• burst=5: Allows a burst of 5 requests.
• nodelay: Indicates burst requests are processed immediately.

Comparison: limit_req vs limit_conn

6. Summary and Best Practices

• Use least_conn as the default load balancing strategy.
• Implement automatic failure isolation and recovery via max_fails + fail_timeout.
• Explicitly configure proxy_next_upstream to retry 5xx errors (idempotent interfaces only).
• Timeout settings must be based on actual backend performance.
• Dual Rate Limiting: limit_conn + limit_req to defend against attacks from different dimensions.
• Return 429 uniformly for clear semantics.

🌟 An excellent system is not one without failures, but one that handles failures gracefully.

References

• https://cloud.tencent.com.cn/developer/article/2567587?policyId=1004
• https://cloud.tencent.com.cn/developer/article/2567587?policyId=1003
• Nginx Complete Guide