Deployment

Production deployment best practices for R-VPN servers.

Architecture

Single Server

For basic deployment:

Internet ──► Server (Port 443) ──► NAT ──► Internet
            - TLS Termination
            - VPN Processing
            - Decoy Website

High Availability

For production with redundancy:

              ┌─────────────┐
Internet ────►│   Load      │
              │  Balancer   │
              └──────┬──────┘
                     │
         ┌───────────┴───────────┐
         ▼                       ▼
   ┌──────────┐            ┌──────────┐
   │ Server 1 │            │ Server 2 │
   └──────────┘            └──────────┘
         │                       │
         └───────────┬───────────┘
                     ▼
              ┌──────────┐
              │ Database │
              │ (Redis)  │
              └──────────┘

Production Checklist

Security

• TLS certificates from trusted CA (Let's Encrypt recommended)
• Firewall configured (only 443/tcp and optionally 80/tcp open)
• Fail2ban installed and configured
• Regular security updates (apt update && apt upgrade)
• SSH key-based authentication (password auth disabled)
• Root login disabled (use sudo)
• Server identity key permissions set to 600
• Private key files backed up securely
• Rate limiting configured appropriately
• Decoy website deployed for stealth

Performance

• Adequate bandwidth for expected users (1 Mbps per concurrent user minimum)
• Sufficient RAM (2GB+ recommended, 4GB+ for high load)
• SSD storage for logs and temporary data
• Network latency under 100ms to target users
• File descriptor limits increased (65536+)
• TCP buffers optimized for high-throughput
• CPU with AES-NI support for TLS acceleration

Reliability

• Monitoring with journalctl configured
• Log rotation configured
• Automated backups scheduled
• Health checks configured
• Restart on failure (systemd)
• TLS auto-renewal configured
• Recovery scripts tested
• Disaster recovery plan documented

DNS Configuration

Main Domain

vpn.example.com.    A       1.2.3.4
vpn.example.com.    AAAA    2001:db8::1    # Optional IPv6

Client Configuration

Clients connect to: - Server: wss://vpn.example.com/connect - Prekey bundle: https://vpn.example.com/prekey-bundle.json

DNS TTL Recommendations

For production deployments: - A/AAAA records: 300-600 seconds (5-10 minutes) for quick failover - During normal operation: 3600 seconds (1 hour) for better caching

TLS with Let's Encrypt Auto-Renewal

Initial Certificate Setup

# Install certbot
sudo apt update
sudo apt install certbot

# Get certificate (standalone mode - stops any service on port 80)
sudo certbot certonly --standalone -d vpn.example.com

# Copy certificates to R-VPN directory
sudo cp /etc/letsencrypt/live/vpn.example.com/fullchain.pem /etc/rvpn/cert.pem
sudo cp /etc/letsencrypt/live/vpn.example.com/privkey.pem /etc/rvpn/key.pem

# Set secure permissions
sudo chmod 600 /etc/rvpn/key.pem
sudo chmod 644 /etc/rvpn/cert.pem

Auto-Renewal with Hook

Create the renewal hook script:

# Create deployment hook
sudo tee /etc/letsencrypt/renewal-hooks/deploy/rvpn-server << 'EOF'
#!/bin/bash
# R-VPN Server Certificate Renewal Hook

set -e

DOMAIN="vpn.example.com"
RVPN_CERT_DIR="/etc/rvpn"
LE_DIR="/etc/letsencrypt/live/${DOMAIN}"

# Copy new certificates
cp "${LE_DIR}/fullchain.pem" "${RVPN_CERT_DIR}/cert.pem"
cp "${LE_DIR}/privkey.pem" "${RVPN_CERT_DIR}/key.pem"

# Set permissions
chmod 600 "${RVPN_CERT_DIR}/key.pem"
chmod 644 "${RVPN_CERT_DIR}/cert.pem"

# Reload server (graceful certificate reload)
systemctl reload rvpn-server 2>/dev/null || systemctl restart rvpn-server

echo "$(date): R-VPN certificates renewed and server reloaded" >> /var/log/rvpn/cert-renewal.log
EOF

sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/rvpn-server

Test Auto-Renewal

# Test the renewal process (dry run)
sudo certbot renew --dry-run

# Force renewal for testing
sudo certbot renew --force-renewal

# Check renewal timer status
sudo systemctl status certbot.timer
sudo certbot certificates

Certificate Paths

[server]
tls_cert_file = "/etc/letsencrypt/live/vpn.example.com/fullchain.pem"
tls_key_file = "/etc/letsencrypt/live/vpn.example.com/privkey.pem"

Or using copied files:

[server]
tls_cert_file = "/etc/rvpn/cert.pem"
tls_key_file = "/etc/rvpn/key.pem"

Monitoring with journalctl

Basic Log Viewing

# View logs in real-time
sudo journalctl -u rvpn-server -f

# View last 100 lines
sudo journalctl -u rvpn-server -n 100

# View logs since last boot
sudo journalctl -u rvpn-server --since today

# View logs from specific time
sudo journalctl -u rvpn-server --since "2024-01-01 10:00:00" --until "2024-01-01 12:00:00"

Filtering by Log Level

# Errors only
sudo journalctl -u rvpn-server -p err

# Warnings and above
sudo journalctl -u rvpn-server -p warning

# All logs with priority levels
sudo journalctl -u rvpn-server -o verbose | grep PRIORITY

Persistent Logging Configuration

Ensure persistent logging is enabled:

# Check if persistent logging is enabled
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal

# Restart journald
sudo systemctl restart systemd-journald

# Configure journal retention
sudo tee /etc/systemd/journald.conf.d/rvpn.conf << 'EOF'
[Journal]
SystemMaxUse=500M
SystemMaxFileSize=50M
MaxFileSec=7day
EOF

sudo systemctl restart systemd-journald

Log Export and Analysis

# Export logs to file
sudo journalctl -u rvpn-server --since "24 hours ago" > /tmp/rvpn-logs.txt

# Export as JSON for analysis
sudo journalctl -u rvpn-server -o json --since today > /tmp/rvpn-logs.json

# Count errors by hour
sudo journalctl -u rvpn-server --since today -p err -o short | awk '{print $1, $2, $3}' | uniq -c

Log Rotation

Systemd Journal Rotation

Journald handles rotation automatically. Configure retention:

# Create custom journal configuration
sudo tee /etc/systemd/journald.conf.d/99-rvpn.conf << 'EOF'
[Journal]
# Maximum disk space for journal
SystemMaxUse=1G
# Maximum size per journal file
SystemMaxFileSize=100M
# Keep logs for 30 days
MaxRetentionSec=30day
EOF

sudo systemctl restart systemd-journald

Application Log Rotation (if file logging enabled)

Create /etc/logrotate.d/rvpn:

/var/log/rvpn/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root adm
    sharedscripts
    postrotate
        systemctl reload rvpn-server 2>/dev/null || true
    endscript
}

Testing Log Rotation

# Test logrotate configuration
sudo logrotate -d /etc/logrotate.d/rvpn

# Force rotation
sudo logrotate -f /etc/logrotate.d/rvpn

# Check rotated logs
ls -la /var/log/rvpn/

Backup and Recovery Scripts

Backup Script

Create /usr/local/bin/rvpn-backup.sh:

#!/bin/bash
# R-VPN Server Backup Script

set -e

BACKUP_DIR="/backup/rvpn"
DATE=$(date +%Y%m%d_%H%M%S)
RETENTION_DAYS=30

# Create backup directory
mkdir -p "${BACKUP_DIR}"

# Backup keys and configuration
tar -czf "${BACKUP_DIR}/rvpn-backup-${DATE}.tar.gz" \
    -C /etc/rvpn \
    --exclude='*.log' \
    . 2>/dev/null || true

# Backup prekey bundles
tar -czf "${BACKUP_DIR}/rvpn-data-${DATE}.tar.gz" \
    -C /var/lib/rvpn \
    . 2>/dev/null || true

# Backup systemd service if customized
if [ -f /etc/systemd/system/rvpn-server.service ]; then
    cp /etc/systemd/system/rvpn-server.service "${BACKUP_DIR}/rvpn-server.service-${DATE}"
fi

# Create backup manifest
cat > "${BACKUP_DIR}/manifest-${DATE}.txt" << EOF
R-VPN Backup Manifest
=====================
Date: $(date)
Hostname: $(hostname)
Version: $(rvpn-server --version 2>/dev/null || echo "unknown")

Backup Contents:
- Configuration files (/etc/rvpn)
- Data files (/var/lib/rvpn)
- Systemd service (if customized)

To restore:
1. Stop rvpn-server: systemctl stop rvpn-server
2. Extract config: tar -xzf rvpn-backup-${DATE}.tar.gz -C /etc/rvpn
3. Extract data: tar -xzf rvpn-data-${DATE}.tar.gz -C /var/lib/rvpn
4. Restore service: cp rvpn-server.service-${DATE} /etc/systemd/system/rvpn-server.service
5. Reload systemd: systemctl daemon-reload
6. Start server: systemctl start rvpn-server
EOF

# Clean up old backups
find "${BACKUP_DIR}" -name "*.tar.gz" -mtime +${RETENTION_DAYS} -delete
find "${BACKUP_DIR}" -name "*.service-*" -mtime +${RETENTION_DAYS} -delete
find "${BACKUP_DIR}" -name "manifest-*.txt" -mtime +${RETENTION_DAYS} -delete

echo "Backup completed: ${BACKUP_DIR}/rvpn-backup-${DATE}.tar.gz"

Make executable and schedule:

sudo chmod +x /usr/local/bin/rvpn-backup.sh

# Add to crontab (daily at 3 AM)
echo "0 3 * * * root /usr/local/bin/rvpn-backup.sh >> /var/log/rvpn/backup.log 2>&1" | sudo tee /etc/cron.d/rvpn-backup

Recovery Script

Create /usr/local/bin/rvpn-restore.sh:

#!/bin/bash
# R-VPN Server Recovery Script

set -e

if [ $# -lt 1 ]; then
    echo "Usage: $0 <backup-date> [backup-dir]"
    echo "Example: $0 20240115_030000"
    exit 1
fi

BACKUP_DATE="$1"
BACKUP_DIR="${2:-/backup/rvpn}"
CONFIG_BACKUP="${BACKUP_DIR}/rvpn-backup-${BACKUP_DATE}.tar.gz"
DATA_BACKUP="${BACKUP_DIR}/rvpn-data-${BACKUP_DATE}.tar.gz"

if [ ! -f "${CONFIG_BACKUP}" ]; then
    echo "Error: Config backup not found: ${CONFIG_BACKUP}"
    exit 1
fi

echo "=== R-VPN Server Recovery ==="
echo "Backup date: ${BACKUP_DATE}"
echo ""

# Confirm
read -p "This will overwrite current configuration. Continue? (yes/no): " confirm
if [ "$confirm" != "yes" ]; then
    echo "Aborted."
    exit 0
fi

# Stop server
echo "Stopping rvpn-server..."
systemctl stop rvpn-server

# Backup current config (just in case)
echo "Creating safety backup of current config..."
tar -czf "${BACKUP_DIR}/rvpn-pre-restore-$(date +%Y%m%d_%H%M%S).tar.gz" -C /etc/rvpn . 2>/dev/null || true

# Restore configuration
echo "Restoring configuration..."
mkdir -p /etc/rvpn
tar -xzf "${CONFIG_BACKUP}" -C /etc/rvpn

# Restore data
echo "Restoring data..."
if [ -f "${DATA_BACKUP}" ]; then
    mkdir -p /var/lib/rvpn
    tar -xzf "${DATA_BACKUP}" -C /var/lib/rvpn
fi

# Restore systemd service if exists
SERVICE_BACKUP="${BACKUP_DIR}/rvpn-server.service-${BACKUP_DATE}"
if [ -f "${SERVICE_BACKUP}" ]; then
    echo "Restoring systemd service..."
    cp "${SERVICE_BACKUP}" /etc/systemd/system/rvpn-server.service
    systemctl daemon-reload
fi

# Set permissions
echo "Setting permissions..."
chmod 600 /etc/rvpn/*.key 2>/dev/null || true
chmod 644 /etc/rvpn/*.pem 2>/dev/null || true
chmod 755 /var/lib/rvpn

# Start server
echo "Starting rvpn-server..."
systemctl start rvpn-server

# Check status
sleep 2
if systemctl is-active --quiet rvpn-server; then
    echo ""
    echo "=== Recovery completed successfully ==="
    systemctl status rvpn-server --no-pager
else
    echo ""
    echo "=== WARNING: Server failed to start ==="
    echo "Check logs: journalctl -u rvpn-server -n 50"
    exit 1
fi

Make executable:

sudo chmod +x /usr/local/bin/rvpn-restore.sh

Remote Backup (Optional)

# Add to backup script for remote sync
# rsync -avz --delete /backup/rvpn/ backup-server:/backups/rvpn-$(hostname)/

Performance Tuning

File Descriptors

Increase system-wide file descriptor limits:

# Edit limits.conf
sudo tee -a /etc/security/limits.conf << 'EOF'
# R-VPN Server
* soft nofile 65536
* hard nofile 65536
rvpn soft nofile 65536
rvpn hard nofile 65536
EOF

# Edit systemd service limits
sudo mkdir -p /etc/systemd/system/rvpn-server.service.d/
sudo tee /etc/systemd/system/rvpn-server.service.d/limits.conf << 'EOF'
[Service]
LimitNOFILE=65536
LimitNOFILESoft=65536
EOF

sudo systemctl daemon-reload
sudo systemctl restart rvpn-server

# Verify
ulimit -n
cat /proc/$(pgrep rvpn-server)/limits | grep "Max open files"

TCP Settings

Optimize TCP for high-throughput VPN:

# Create sysctl configuration
sudo tee /etc/sysctl.d/99-rvpn.conf << 'EOF'
# R-VPN Performance Tuning

# Increase TCP buffer sizes
net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216

# TCP memory tuning
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Enable TCP window scaling
net.ipv4.tcp_window_scaling = 1

# Increase connection tracking
net.netfilter.nf_conntrack_max = 524288
net.netfilter.nf_conntrack_tcp_timeout_established = 600

# Reduce TCP keepalive time
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 3

# Enable TCP fast open
net.ipv4.tcp_fastopen = 3

# Increase local port range
net.ipv4.ip_local_port_range = 1024 65535

# Enable BBR congestion control (if available)
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
EOF

# Apply settings
sudo sysctl --system

# Verify BBR is available and enabled
sysctl net.ipv4.tcp_available_congestion_control
sysctl net.ipv4.tcp_congestion_control

Kernel Parameters for VPN

# Add to /etc/sysctl.d/99-rvpn.conf

# Enable IP forwarding
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1

# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

# Enable SYN cookies
net.ipv4.tcp_syncookies = 1

# Reduce SYN backlog
net.ipv4.tcp_max_syn_backlog = 65536

# Increase netdev budget for high packet rates
net.core.netdev_budget = 50000
net.core.netdev_budget_usecs = 5000

# Increase max connections in SYN queue
net.ipv4.tcp_max_syn_backlog = 65536
net.core.somaxconn = 65535
EOF

Network Interface Tuning

# Tune network interface (replace eth0 with your interface)
sudo tee /etc/networkd-dispatcher/routable.d/50-rvpn-tune << 'EOF'
#!/bin/bash
# Network tuning for R-VPN

IFACE="eth0"

# Increase ring buffer sizes
ethtool -G ${IFACE} rx 4096 tx 4096 2>/dev/null || true

# Enable offload features
ethtool -K ${IFACE} gro on 2>/dev/null || true
ethtool -K ${IFACE} gso on 2>/dev/null || true
EOF

sudo chmod +x /etc/networkd-dispatcher/routable.d/50-rvpn-tune

Monitoring Performance

# Check current connections
ss -s

# Monitor bandwidth usage
iftop -i eth0

# Check CPU usage by process
top -p $(pgrep rvpn-server)

# Monitor network throughput
sar -n DEV 1

# Check for packet drops
ethtool -S eth0 | grep -i drop

Docker Deployment

Dockerfile

# Build stage
FROM rust:1.75-slim-bookworm as builder

WORKDIR /app
COPY . .

# Install dependencies
RUN apt-get update && apt-get install -y \
    pkg-config \
    libssl-dev \
    && rm -rf /var/lib/apt/lists/*

# Build release binary
RUN cargo build --release -p rvpn-server

# Runtime stage
FROM debian:bookworm-slim

# Install runtime dependencies
RUN apt-get update && apt-get install -y \
    ca-certificates \
    iptables \
    iproute2 \
    && rm -rf /var/lib/apt/lists/*

# Create non-root user
RUN useradd -r -s /sbin/nologin -M rvpn

# Copy binary
COPY --from=builder /app/target/release/rvpn-server /usr/local/bin/

# Create directories
RUN mkdir -p /etc/rvpn /var/lib/rvpn /var/log/rvpn && \
    chown -R rvpn:rvpn /etc/rvpn /var/lib/rvpn /var/log/rvpn

# Expose ports
EXPOSE 443/tcp
EXPOSE 80/tcp

# Switch to non-root user
USER rvpn

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD curl -f -k https://localhost:443/ || exit 1

ENTRYPOINT ["rvpn-server"]
CMD ["--config", "/etc/rvpn/server.toml"]

Docker Compose

Create docker-compose.yml:

version: '3.8'

services:
  rvpn-server:
    build: .
    image: rvpn-server:latest
    container_name: rvpn-server
    restart: unless-stopped

    # Network mode for VPN functionality
    network_mode: host

    # Or use port mapping (limited functionality)
    # ports:
    #   - "443:443/tcp"
    #   - "80:80/tcp"

    volumes:
      # Configuration
      - ./config:/etc/rvpn:ro
      # Data persistence
      - rvpn-data:/var/lib/rvpn
      # Logs
      - rvpn-logs:/var/log/rvpn
      # TLS certificates
      - /etc/letsencrypt:/etc/letsencrypt:ro

    # Required capabilities for VPN
    cap_add:
      - NET_ADMIN
      - NET_RAW

    # Sysctl for networking
    sysctls:
      - net.ipv4.ip_forward=1
      - net.ipv6.conf.all.forwarding=1

    environment:
      - RUST_LOG=info

    healthcheck:
      test: ["CMD", "curl", "-f", "-k", "https://localhost:443/"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 10s

volumes:
  rvpn-data:
  rvpn-logs:

Building and Running

# Build image
docker build -t rvpn-server:latest .

# Run with docker
docker run -d \
  --name rvpn-server \
  --restart unless-stopped \
  --network host \
  --cap-add NET_ADMIN \
  --cap-add NET_RAW \
  -v /etc/rvpn:/etc/rvpn:ro \
  -v /var/lib/rvpn:/var/lib/rvpn \
  -v /etc/letsencrypt:/etc/letsencrypt:ro \
  rvpn-server:latest

# Or use docker-compose
docker-compose up -d

# View logs
docker logs -f rvpn-server

# Stop
docker-compose down

Docker Swarm Deployment

version: '3.8'

services:
  rvpn-server:
    image: rvpn-server:latest
    deploy:
      replicas: 1
      restart_policy:
        condition: any
        delay: 5s
        max_attempts: 3
      resources:
        limits:
          cpus: '2.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 256M
    networks:
      - rvpn-network
    volumes:
      - /etc/rvpn:/etc/rvpn:ro
      - rvpn-data:/var/lib/rvpn
    configs:
      - source: rvpn_config
        target: /etc/rvpn/server.toml

networks:
  rvpn-network:
    driver: overlay
    attachable: true

volumes:
  rvpn-data:

configs:
  rvpn_config:
    external: true

Troubleshooting

High Connection Count

Check active connections:

# Summary of connections
ss -s

# Connections per IP (top talkers)
ss -tn | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head -20

# Connection states
ss -tan | awk '{print $1}' | sort | uniq -c

# Check for SYN flood
ss -tan state syn-recv | wc -l

# Monitor connection rate
watch -n 1 'ss -tan | wc -l'

Mitigation:

# Add connection rate limiting
sudo iptables -A INPUT -p tcp --dport 443 -m connlimit --connlimit-above 100 -j DROP

# Enable SYN cookies
sudo sysctl -w net.ipv4.tcp_syncookies=1

Memory Issues

Check memory usage:

# Process memory
ps aux | grep rvpn-server

# Detailed memory info
cat /proc/$(pgrep rvpn-server)/status | grep -i vm

# System memory
free -h

# Memory over time
vmstat 1 10

# OOM killer log
dmesg | grep -i "out of memory"

# Check for memory leaks
pmap $(pgrep rvpn-server)

Resolution:

# Restart service
sudo systemctl restart rvpn-server

# Add swap if needed
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

Network Issues

Check firewall:

# List all rules
sudo iptables -L -n -v

# Check specific port
sudo ss -tlnp | grep 443

# Test port from outside
nc -zv vpn.example.com 443

# Check routing
ip route

# Check interface stats
ip -s link show eth0

# Trace path
mtr vpn.example.com

Common fixes:

# Reset iptables (careful!)
sudo iptables -F
sudo iptables -X
sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

# Re-add basic rules
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -j DROP

TLS Certificate Issues

# Verify certificate
openssl x509 -in /etc/rvpn/cert.pem -text -noout

# Check certificate dates
openssl x509 -in /etc/rvpn/cert.pem -noout -dates

# Test TLS connection
openssl s_client -connect vpn.example.com:443 -tls1_3 -servername vpn.example.com

# Check certificate chain
openssl s_client -connect vpn.example.com:443 -showcerts </dev/null

# Verify Let's Encrypt renewal
sudo certbot certificates

Service Won't Start

# Check service status
sudo systemctl status rvpn-server

# Check for errors
sudo journalctl -u rvpn-server -n 50

# Test config syntax
cat /etc/rvpn/server.toml

# Check file permissions
ls -la /etc/rvpn/
ls -la /var/lib/rvpn/

# Test binary directly
sudo rvpn-server --config /etc/rvpn/server.toml -vv

# Check for port conflicts
sudo ss -tlnp | grep 443

Performance Issues

# CPU profiling
perf top -p $(pgrep rvpn-server)

# Network latency
ping vpn.example.com

# Bandwidth test
iperf3 -c vpn.example.com

# Check for packet loss
netstat -i

# DNS resolution time
dig vpn.example.com

Next Steps

• Setup - Initial setup
• Configuration - Configuration reference