Prometheus + Grafana VPS Monitoring: Building a Server Monitoring System from Scratch

black flat screen tv showing game

Running quant strategies and proxy nodes without monitoring is extremely risky. Out-of-memory crashes at 3 AM, disks filling up, strategies crashing—without alerts, you might not notice for hours.

This guide builds a complete monitoring stack from scratch: Node Exporter collects metrics on each monitored machine, Prometheus aggregates and stores them, Grafana visualizes the data, and Telegram delivers alerts.

Architecture

Monitored VPS (N machines)    Central Monitoring VPS (1 machine)
┌──────────────┐              ┌─────────────────────┐
│ Node Exporter│ ──9100──→    │ Prometheus          │
│ Node Exporter│ ──9100──→    │ (collect & store)   │
│ Node Exporter│ ──9100──→    │ Grafana             │
└──────────────┘              │ (visualize)         │
                              │ Alertmanager        │
                              │ (alerts)            │
                              └─────────────────────┘

The approach: dedicate a small VPS (1 core, 1 GB RAM is plenty) as the monitoring hub; install Node Exporter on all other machines to expose metrics.

Step 1: Install Node Exporter on Monitored Machines

Do this on every VPS you want to monitor.

# Download Node Exporter (verify the latest version)
cd /tmp
wget https://github.com/prometheus/node_exporter/releases/download/v1.8.2/node_exporter-1.8.2.linux-amd64.tar.gz
tar xzf node_exporter-1.8.2.linux-amd64.tar.gz
mv node_exporter-1.8.2.linux-amd64/node_exporter /usr/local/bin/

Create a systemd service:

cat > /etc/systemd/system/node_exporter.service << 'EOF'
[Unit]
Description=Node Exporter
After=network.target

[Service]
Type=simple
User=nobody
ExecStart=/usr/local/bin/node_exporter --web.listen-address=:9100
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable node_exporter --now

Verify it’s running correctly:

curl http://localhost:9100/metrics | head -20
# If you see lots of metrics output, it's working

Firewall setup:

Node Exporter’s port 9100 should only be accessible to your Prometheus server, never the public internet:

# Allow only your monitoring center IP to access 9100
ufw allow from monitoring-hub-IP to any port 9100
# Don't run `ufw allow 9100` ← this opens it to everyone

Step 2: Install Prometheus on the Monitoring Hub

This server actively pulls metrics from all machines.

# Create a dedicated user
useradd --no-create-home --shell /bin/false prometheus

# Download Prometheus
cd /tmp
wget https://github.com/prometheus/prometheus/releases/download/v2.51.2/prometheus-2.51.2.linux-amd64.tar.gz
tar xzf prometheus-2.51.2.linux-amd64.tar.gz

# Install
mkdir -p /etc/prometheus /var/lib/prometheus
cp prometheus-2.51.2.linux-amd64/prometheus /usr/local/bin/
cp prometheus-2.51.2.linux-amd64/promtool /usr/local/bin/
cp -r prometheus-2.51.2.linux-amd64/consoles /etc/prometheus/
cp -r prometheus-2.51.2.linux-amd64/console_libraries /etc/prometheus/

chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus

Create the config file /etc/prometheus/prometheus.yml:

global:
  scrape_interval: 30s      # Collect metrics every 30 seconds
  evaluation_interval: 30s

alerting:
  alertmanagers:
    - static_configs:
        - targets: ['localhost:9093']

rule_files:
  - /etc/prometheus/rules/*.yml

scrape_configs:
  - job_name: 'node'
    static_configs:
      - targets:
          - 'aws-hk-IP:9100'     # Hong Kong
          - 'aws-us-IP:9100'     # US
          - 'aws-jp-IP:9100'     # Japan
          - 'dmit-us-IP:9100'    # DMIT US
        labels:
          env: 'production'

Create a systemd service:

cat > /etc/systemd/system/prometheus.service << 'EOF'
[Unit]
Description=Prometheus
After=network.target

[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/var/lib/prometheus \
  --storage.tsdb.retention.time=30d \
  --web.listen-address=:9090
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable prometheus --now

Visit http://monitoring-hub-IP:9090 to verify; check Status → Targets to see all monitored machines’ status.

Step 3: Install Grafana

# Add Grafana APT repository
apt-get install -y apt-transport-https software-properties-common
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | \
  tee /etc/apt/sources.list.d/grafana.list

apt-get update
apt-get install -y grafana

systemctl enable grafana-server --now

Grafana runs on port 3000 by default; visit http://monitoring-hub-IP:3000:

Default user: admin
Default password: admin (you’ll be prompted to change it after login)

Add Prometheus as a Data Source

Left menu → Connections → Data sources
Click Add data source → select Prometheus
Enter http://localhost:9090 for the URL
Click Save & test; if it says “Data source is working,” you’re good

Import a Pre-Built Dashboard

No need to build dashboards from scratch—import a mature one from the community:

Left menu → Dashboards → Import
Enter Dashboard ID 1860 (Node Exporter Full, the most popular)
Select your Prometheus data source
Click Import

This dashboard includes CPU usage, memory, disk I/O, network throughput, system load, and filesystem space—very comprehensive.

Step 4: Set Up Telegram Alerts

Dashboards are nice, but you need instant notifications when things break.

Create a Telegram Bot

Find @BotFather in Telegram
Send /newbot and give your bot a name
Save the returned Bot Token (format: 1234567890:ABCD...)
Send the bot a message, then visit https://api.telegram.org/botYOUR_TOKEN/getUpdates to get your chat_id

Install Alertmanager

cd /tmp
wget https://github.com/prometheus/alertmanager/releases/download/v0.27.0/alertmanager-0.27.0.linux-amd64.tar.gz
tar xzf alertmanager-0.27.0.linux-amd64.tar.gz
cp alertmanager-0.27.0.linux-amd64/alertmanager /usr/local/bin/
mkdir -p /etc/alertmanager

Create /etc/alertmanager/alertmanager.yml:

global:
  resolve_timeout: 5m

route:
  receiver: telegram
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h

receivers:
  - name: telegram
    telegram_configs:
      - bot_token: 'YOUR_BOT_TOKEN'
        chat_id: YOUR_CHAT_ID
        message: |
          🚨 Alert: {{ .GroupLabels.alertname }}
          Host: {{ .GroupLabels.instance }}
          Details: {{ range .Alerts }}{{ .Annotations.description }}{{ end }}
        send_resolved: true

cat > /etc/systemd/system/alertmanager.service << 'EOF'
[Unit]
Description=Alertmanager
After=network.target

[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/bin/alertmanager \
  --config.file=/etc/alertmanager/alertmanager.yml \
  --storage.path=/var/lib/alertmanager
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable alertmanager --now

Configure Alert Rules

Create /etc/prometheus/rules/node_alerts.yml:

groups:
  - name: node_alerts
    rules:
      # CPU over 85% for 5 minutes
      - alert: HighCPUUsage
        expr: 100 - (avg by(instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 85
        for: 5m
        annotations:
          description: "{{ $labels.instance }} CPU {{ $value | printf \"%.1f\" }}% utilized"

      # Available memory below 10%
      - alert: LowMemory
        expr: (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100 < 10
        for: 2m
        annotations:
          description: "{{ $labels.instance }} only {{ $value | printf \"%.1f\" }}% memory available"

      # Disk usage over 85%
      - alert: DiskSpaceLow
        expr: (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100 < 15
        for: 5m
        annotations:
          description: "{{ $labels.instance }} only {{ $value | printf \"%.1f\" }}% disk space left"

      # Machine offline (no metrics for 2+ minutes)
      - alert: InstanceDown
        expr: up == 0
        for: 2m
        annotations:
          description: "{{ $labels.instance }} offline for 2+ minutes"

Reload the configuration:

systemctl reload prometheus
# Or:
curl -X POST http://localhost:9090/-/reload

Step 5: Secure Grafana with Nginx Reverse Proxy

Don’t expose Grafana’s port 3000 directly to the internet—put Nginx in front:

apt install nginx -y

cat > /etc/nginx/sites-available/grafana << 'EOF'
server {
    listen 80;
    server_name grafana.yourdomain.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name grafana.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/grafana.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/grafana.yourdomain.com/privkey.pem;

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
EOF

ln -s /etc/nginx/sites-available/grafana /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx

Use acme.sh to auto-provision SSL certificates:

curl https://get.acme.sh | sh
~/.acme.sh/acme.sh --issue -d grafana.yourdomain.com --nginx
~/.acme.sh/acme.sh --install-cert -d grafana.yourdomain.com \
  --cert-file /etc/letsencrypt/live/grafana.yourdomain.com/fullchain.pem \
  --key-file /etc/letsencrypt/live/grafana.yourdomain.com/privkey.pem \
  --reloadcmd "systemctl reload nginx"

What You Get

Once it’s set up, you’ll have:

Real-time CPU / memory / disk curves for each machine
Network in/out throughput, so you can spot which nodes are saturated
System load trends—you’ll instantly see when your quant strategies hammer the CPU
Machine goes down or metrics spike? Telegram alerts you instantly

Further reading:

📷 Hero image · generated by Flux Schnell AI · Panshi brand style

Architecture#

Step 1: Install Node Exporter on Monitored Machines#

Step 2: Install Prometheus on the Monitoring Hub#

Step 3: Install Grafana#

Add Prometheus as a Data Source#

Import a Pre-Built Dashboard#

Step 4: Set Up Telegram Alerts#

Create a Telegram Bot#

Install Alertmanager#

Configure Alert Rules#

Step 5: Secure Grafana with Nginx Reverse Proxy#

What You Get#