Reverse‑Proxy Architecture (nginx → Apache)

This page documents the complete reverse‑proxy setup for the AlmaLinux frontend server. It replaces the former IIS reverse‑proxy and Windows‑based certificate renewal system. The configuration is now fully Linux‑native, stable, and successor‑friendly.

Overview

Frontend: AlmaLinux + nginx
Backend: Apache (Drupal + MediaWiki)
Certificates: Let’s Encrypt (certbot)
Automatic renewal: systemd timer
Windows/IIS: fully retired (no bindings, no renewals, no proxies)

The nginx server terminates HTTPS and forwards traffic to the internal Apache instance.

Directory Layout

/etc/nginx/nginx.conf – global nginx configuration
/etc/nginx/conf.d/ – per‑site reverse‑proxy configs
/var/www/letsencrypt/ – ACME challenge directory
/etc/letsencrypt/live/<domain>/ – certificate files

ACME Challenge Directory

Certbot uses a shared directory for HTTP‑01 validation:

/var/www/letsencrypt/

nginx exposes this path for both domains.

Site: research.costasano.club

HTTP (port 80)

Redirects all traffic to HTTPS, except ACME challenges.

server {
    listen 80;
    listen [::]:80;
    server_name research.costasano.club;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    return 301 https://$host$request_uri;
}

HTTPS (port 443)

Terminates TLS and proxies to Apache.

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;

    server_name research.costasano.club;

    ssl_certificate     /etc/letsencrypt/live/research.costasano.club/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/research.costasano.club/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    location / {
        proxy_pass http://192.168.33.231;

        proxy_set_header Host research.costasano.club;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;

        proxy_buffering on;
        proxy_buffers 16 16k;
        proxy_buffer_size 16k;

        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }
}

Site: mwiki.costasano.club

HTTP (port 80)

server {
    listen 80;
    listen [::]:80;
    server_name mwiki.costasano.club;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    return 301 https://$host$request_uri;
}

HTTPS (port 443)

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;

    server_name mwiki.costasano.club;

    ssl_certificate     /etc/letsencrypt/live/mwiki.costasano.club/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mwiki.costasano.club/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    location / {
        proxy_pass http://192.168.33.231;

        proxy_set_header Host mwiki.costasano.club;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;

        proxy_buffering on;
        proxy_buffers 16 16k;
        proxy_buffer_size 16k;

        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }
}

Certificate Renewal

Certbot is installed natively on AlmaLinux.

Renewal Timer

systemctl status certbot-renew.timer

The timer runs daily and triggers renewal when certificates approach expiry.

Manual Dry‑Run Test

sudo certbot renew --dry-run

Windows/IIS Retirement

The former IIS reverse‑proxy and win‑acme renewal system were removed:

IIS sites deleted
win‑acme renewal entries cancelled
scheduled tasks removed
old certificates removed from certlm.msc
no bindings remain on ports 80/443

Verification Checklist

nginx configs validated: nginx -t
nginx reloaded: systemctl reload nginx
certbot timer active
ACME challenge reachable
backend Apache reachable
Windows silent on ports 80/443

Architecture Diagram (ASCII)

The following diagram shows the full request flow from the internet to the backend applications.

                 +----------------------+
                 |      Internet        |
                 +----------+-----------+
                            |
                            v
                 +----------------------+
                 |   AlmaLinux Server   |
                 |      (frontend)      |
                 +----------+-----------+
                            |
                HTTPS :443 / HTTP :80
                            |
                            v
                    +---------------+
                    |    nginx      |
                    |  reverse proxy|
                    +-------+-------+
                            |
        +-------------------+-------------------+
        |                                       |
        v                                       v
+---------------------+               +---------------------+
| research.costasano  |               | mwiki.costasano     |
| .club (vhost)       |               | .club (vhost)       |
+----------+----------+               +----------+----------+
           |                                     |
           | proxy_pass http://192.168.33.231    |
           +-------------------------+-----------+
                                     |
                                     v
                         +------------------------+
                         |   Apache (backend)     |
                         |  192.168.33.231        |
                         +-----------+------------+
                                     |
                 +-------------------+-------------------+
                 |                                       |
                 v                                       v
        +-------------------+                  +-------------------+
        |   Drupal site     |                  |  MediaWiki site   |
        | research.*        |                  | mwiki.*           |
        +-------------------+                  +-------------------+


ACME / Let’s Encrypt HTTP-01 validation:

    Let’s Encrypt  --->  nginx  --->  /var/www/letsencrypt/
                         (both vhosts expose
                          /.well-known/acme-challenge/)

Certificates:

    /etc/letsencrypt/live/research.costasano.club/
    /etc/letsencrypt/live/mwiki.costasano.club/

Renewal:

    systemd timer: certbot-renew.timer
    service:       certbot-renew.service

Rationale for Replacing IIS with nginx

This section explains why the former Windows/IIS reverse‑proxy was retired in favour of a native Linux‑based nginx setup. The decision was driven by stability, clarity, maintainability, and long‑term operational safety.

1. Architectural Simplicity

The previous setup required:

Windows Server
IIS with URL Rewrite + ARR
win-acme for certificate renewal
Manual bindings for each hostname
Coordination between Windows and the Linux backend

This created a multi‑platform dependency chain. The new setup collapses the entire frontend into a single, predictable layer:

AlmaLinux
nginx
certbot (systemd‑managed)

Fewer moving parts → fewer failure points.

2. Elimination of Cross‑Platform Coupling

The old architecture required Windows to proxy traffic to Linux. This meant:

Two operating systems had to stay in sync
Two certificate systems had to be maintained
Two sets of logs had to be consulted during troubleshooting

By moving the reverse‑proxy to the same machine that hosts the backend, the system becomes self‑contained and easier to reason about.

3. Cleaner TLS and Certificate Management

The Windows setup used win‑acme, scheduled tasks, and IIS bindings. This caused:

Duplicate certificate stores
Renewal tasks that were easy to forget
Certificates living on a machine that no longer served the content

The new setup uses:

certbot
systemd timers
a shared ACME challenge directory
certificates stored exactly where they are used

This is simpler, safer, and fully automated.

4. nginx Is Purpose‑Built for Reverse‑Proxying

nginx excels at:

high‑performance proxying
buffering
header management
HTTP/2
TLS termination

IIS can do these things, but only through multiple modules (ARR, URL Rewrite) and with more complexity. nginx provides them natively and predictably.

5. Reduced Operational Overhead

The Windows/IIS layer required:

OS patching
IIS updates
win‑acme maintenance
Monitoring of scheduled tasks
Manual cleanup of old bindings and certificates

The nginx layer requires:

a single systemd timer
a single config directory
one place to check logs

This reduces cognitive load for both you and any future maintainer.

6. Successor‑Friendly Documentation and Maintenance

The new architecture is:

linear
transparent
fully documented
reproducible
platform‑consistent

A successor only needs to understand:

nginx vhost files
certbot
Apache backend routing

No Windows knowledge is required.

7. Removal of Legacy Dependencies

The IIS reverse‑proxy was originally introduced as a workaround during earlier infrastructure phases. Now that the backend is stable on AlmaLinux, the Windows layer no longer serves a purpose and only adds risk.

Removing it:

reduces attack surface
removes unused services
avoids silent port conflicts
simplifies backups and disaster recovery

Summary

The migration from IIS to nginx was not just a technical change but an architectural improvement. It aligns the system with:

clarity
maintainability
long‑term stability
reduced operational burden
successor‑friendly design

The result is a cleaner, more predictable, and more robust infrastructure.

Before vs After: Reverse‑Proxy Architecture

The table below summarizes the differences between the former IIS‑based setup and the current nginx‑based architecture.

Aspect	Before: Windows + IIS	After: AlmaLinux + nginx
Reverse‑proxy layer	IIS with ARR + URL Rewrite (added modules, multi‑step configuration)	nginx (native reverse‑proxy, single config file per site)
Operating system	Windows Server (separate from backend)	AlmaLinux (same platform as backend)
Certificate management	win‑acme (scheduled task, IIS bindings) Certificates stored in Windows certificate store	certbot (systemd timer) Certificates stored directly under /etc/letsencrypt/
ACME challenge handling	IIS rewrite rules + filesystem mapping More fragile and harder to debug	Direct nginx location block Shared ACME directory for all vhosts
Number of moving parts	High: Windows + IIS + ARR + Rewrite + win‑acme	Low: nginx + certbot
Logging	Split across Windows Event Viewer, IIS logs, win‑acme logs	Unified: nginx logs + certbot logs
Maintenance burden	Requires Windows patching, IIS updates, scheduled task monitoring	Minimal: systemd‑managed services, simple config reloads
Failure modes	Certificate renewal failures often silent Bindings could break after updates Proxy rules spread across GUI modules	Transparent and predictable Config validated with nginx -t Renewal handled by systemd
Security surface	Larger (Windows + IIS + ARR + Rewrite)	Smaller (nginx only)
Successor‑friendliness	Requires Windows + IIS knowledge Proxy logic hidden behind GUI layers	Fully text‑based, documented, reproducible Easy for any Linux‑familiar maintainer
Architectural clarity	Cross‑platform coupling (Windows → Linux)	Single‑platform, self‑contained frontend

Status

The reverse‑proxy setup is stable, maintainable, and fully Linux‑native.

Troubleshooting Appendix

This appendix lists common issues that may occur in the nginx → Apache reverse‑proxy setup, along with clear diagnostic steps and expected outcomes. It is designed to help future maintainers quickly identify where a failure originates.

1. Site Not Loading (General)

If a site does not load at all:

Check nginx status:

systemctl status nginx

Validate configuration:

nginx -t

Reload nginx:

systemctl reload nginx

If nginx fails to start or reload, the error message will point to the exact file and line.

2. HTTPS Works but HTTP Does Not Redirect

Symptoms:

Visiting http://domain shows an error
Visiting https://domain works normally

Check the port 80 server block for the domain:

It must include the ACME location block
It must end with: return 301 https://$host$request_uri;

Restart nginx after correcting:

systemctl reload nginx

3. Certificate Renewal Fails

Symptoms:

Certbot emails warnings
Browser shows certificate expiring soon
Dry‑run renewal fails

Diagnostics:

Test ACME challenge manually:

curl http://domain/.well-known/acme-challenge/test

Expected:

404 Not Found (normal)
NOT a 403 or 500

Check permissions:

ls -ld /var/www/letsencrypt

Run a dry‑run:

sudo certbot renew --dry-run

If renewal fails, the error message will indicate whether nginx routing or filesystem access is the cause.

4. Backend (Apache) Not Responding

Symptoms:

nginx returns 502 Bad Gateway
nginx returns 504 Gateway Timeout

Diagnostics:

Check Apache:

systemctl status httpd

Test backend directly:

curl http://192.168.33.231

If Apache is down, restart it:

systemctl restart httpd

If Apache is up but unreachable, check firewall rules.

5. Wrong Site Appears (Host Header Issue)

Symptoms:

Visiting research.* shows the MediaWiki site
Visiting mwiki.* shows the Drupal site

Cause:

Missing or incorrect proxy_set_header Host ... in nginx

Fix: Ensure each vhost sets the correct Host header:

proxy_set_header Host research.costasano.club;

Reload nginx afterwards.

6. ACME Challenge Returns 404

Symptoms:

Certbot renewal fails
Browser shows 404 for ACME path

Check that both vhosts contain:

location /.well-known/acme-challenge/ {
    root /var/www/letsencrypt;
}

Ensure the directory exists:

ls /var/www/letsencrypt

7. Windows Still Interfering (Legacy Setup)

Symptoms:

Port conflicts
Unexpected certificates
Old bindings still active

Diagnostics on Windows:

Check ports:

netstat -ano | findstr :80
netstat -ano | findstr :443

Check IIS:

Ensure no sites remain in IIS Manager.

Check scheduled tasks:

Remove any leftover win‑acme tasks.

If Windows is still listening on 80/443, disable the Web Server (IIS) role entirely.

8. Browser Shows Old Certificate

Symptoms:

Browser still shows the old expiry date
Renewal succeeded on the server

Possible causes:

Browser caching
Intermediate certificate caching
CDN or proxy caching (if applicable)

Fix:

Hard refresh (Ctrl+F5)
Test with curl:

curl -v https://domain

The certificate shown by curl is authoritative.

9. nginx Reload Fails After Editing Config

Symptoms:

systemctl reload nginx fails
nginx refuses to start

Run:

nginx -t

This will show:

the file
the line
the exact syntax error

Fix the issue and reload nginx.

10. Slow Responses or Timeouts

Symptoms:

Pages load slowly
MediaWiki or Drupal feels sluggish

Check proxy timeouts:

proxy_read_timeout 60s;
proxy_send_timeout 60s;

Check backend load:

top
systemctl status httpd

If Apache is overloaded, investigate modules, PHP-FPM, or database performance.

Summary

This appendix provides a structured approach to diagnosing issues in the nginx → Apache reverse‑proxy setup. Most problems fall into one of three categories:

nginx configuration
certificate renewal
backend availability

Following the steps above should isolate the cause quickly and safely.

Notes for AlmaLinux 10.1 Security Changes

AlmaLinux 10.1 introduces several security‑related changes that affect nginx, certbot, and backend communication. The following points must be checked whenever the system is updated or rebuilt.

1. SELinux: Stricter Defaults

AlmaLinux 10.1 enforces tighter SELinux rules, especially for:

nginx reading files outside /usr/share/nginx/html
ACME challenge directories
proxy connections to backend servers

Ensure:

semanage fcontext -a -t httpd_sys_content_t "/var/www/letsencrypt(/.*)?"
restorecon -Rv /var/www/letsencrypt

If nginx cannot reach Apache:

setsebool -P httpd_can_network_connect 1

2. FirewallD: More Restrictive Defaults

After updates, FirewallD may re‑apply stricter profiles.

Verify:

firewall-cmd --list-services

Ensure:

http
https

are enabled permanently.

3. OpenSSL 3.x Enforcement

AlmaLinux 10.1 uses OpenSSL 3.x with stricter cipher policies.

Ensure nginx uses:

TLSv1.2
TLSv1.3

and avoid deprecated ciphers.

4. Systemd Hardening of Services

Some systemd units (nginx, certbot) may receive tightened sandboxing.

If certbot renewal fails after an update:

journalctl -u certbot-renew.service

Check for:

permission denials
sandbox restrictions
missing ACME directory access

5. nginx: HTTP/2 and TLS Requirements

AlmaLinux 10.1 enforces modern TLS for HTTP/2.

Ensure:

http2 on;
ssl_protocols TLSv1.2 TLSv1.3;

HTTP/2 will silently disable itself if insecure ciphers are used.

6. Apache: Private IP Binding Verification

After updates, Apache may rebind or change Listen directives.

Verify:

grep -R "Listen" /etc/httpd/conf*

Expected:

Apache listens on 192.168.33.231:80
NOT on 0.0.0.0:80 (to avoid conflicts)

7. Certbot: Renewal Timer Integrity

System updates may reset or disable timers.

Verify:

systemctl status certbot-renew.timer
systemctl list-timers | grep certbot

Expected:

Timer active
Next run scheduled within 24 hours

8. Kernel-Level Networking Changes

AlmaLinux 10.1 includes updated nftables and TCP stack behaviour.

If proxying becomes slow or unstable:

sysctl net.ipv4.ip_local_port_range
sysctl net.core.somaxconn

Ensure defaults are not overly restrictive.

Summary

AlmaLinux 10.1 is more secure by default, but this can affect nginx, certbot, and Apache interactions. After any major update, verify:

SELinux permissions
FirewallD rules
nginx TLS configuration
Apache binding
certbot timers

This ensures the reverse‑proxy remains stable and predictable.