A Quant Strategy Needs More Than Just Code: Building Your Complete Monitoring & Alerting Stack
Running a quant strategy requires far more than the strategy itself—you need a full suite of tools to execute it reliably, monitor its state, and get alerted when things go wrong. This post shares my complete tooling setup, from strategy execution through alerts.
Overall Architecture
Strategy Layer (Python)
↓ Trading Orders
Binance API
↓ Execution Results
State Files / CF KV
↓ Metrics
Prometheus + node_exporter
↓ Storage
Grafana Visualization
↓ Alert Rules
Alertmanager → Telegram
Strategy Execution Layer
Tech Stack
- Language: Python 3.11
- Binance SDK:
python-binance - Scheduled Tasks: APScheduler or system cron
- Dependency Management:
requirements.txt+ venv
Directory Structure
quant/
├── strategy/
│ ├── main.py # Core strategy logic (signal generation + order placement)
│ ├── signals.py # Indicator calculations (technical indicator library)
│ └── risk.py # Position sizing, stop-loss logic
├── exchange/
│ └── binance_client.py # Binance API wrapper
├── monitor/
│ └── exporter.py # Expose state metrics in Prometheus format
├── config/
│ └── settings.py # Configuration (read from environment variables)
└── run.py # Entry point
Key Code: Exposing State Metrics
While your strategy runs, write key metrics in Prometheus text file format for node_exporter to scrape:
# monitor/exporter.py
import os
import time
METRICS_FILE = "/var/lib/node_exporter/textfile_collector/quant.prom"
def write_metrics(data: dict):
"""
data example:
{
"pnl_total": 12.5, # cumulative return %
"pnl_today": 0.8, # today's return %
"position": 1, # 1=long, -1=short, 0=flat
"last_signal_ts": 1776200000, # last signal timestamp
"last_run_ts": 1776235000, # last strategy run time
}
"""
lines = [
'# HELP quant_pnl_total Cumulative return (percent)',
'# TYPE quant_pnl_total gauge',
f'quant_pnl_total {data["pnl_total"]}',
'',
'# HELP quant_pnl_today Daily return (percent)',
'# TYPE quant_pnl_today gauge',
f'quant_pnl_today {data["pnl_today"]}',
'',
'# HELP quant_position Current position direction (1=long,-1=short,0=flat)',
'# TYPE quant_position gauge',
f'quant_position {data["position"]}',
'',
'# HELP quant_last_signal_timestamp Last signal time',
'# TYPE quant_last_signal_timestamp gauge',
f'quant_last_signal_timestamp {data["last_signal_ts"]}',
'',
'# HELP quant_last_run_timestamp Last strategy run time',
'# TYPE quant_last_run_timestamp gauge',
f'quant_last_run_timestamp {int(time.time())}',
'',
]
tmp = METRICS_FILE + ".tmp"
with open(tmp, "w") as f:
f.write("\n".join(lines))
os.rename(tmp, METRICS_FILE)
Server Configuration
Choosing Your Server
A quant strategy has three core server requirements:
- Stability: No random downtime—it needs to run 24/7
- Low latency: Close to Binance servers (Singapore or Tokyo for faster order execution)
- Isolation: Don’t share resources with other high-load tasks
I run on AWS Singapore (aws-sg). Binance’s primary servers are in Singapore, so order latency is 1-5ms.
systemd Daemon
Don’t run your strategy with nohup—use systemd:
# /etc/systemd/system/quant-strategy.service
[Unit]
Description=Quant Strategy
After=network.target
[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/quant
ExecStart=/home/ubuntu/quant/venv/bin/python run.py
Restart=always
RestartSec=10
EnvironmentFile=/home/ubuntu/quant/.env
[Install]
WantedBy=multi-user.target
systemctl enable quant-strategy
systemctl start quant-strategy
systemctl status quant-strategy
If your strategy crashes, systemd automatically restarts it with a 10-second delay.
Environment Variable Management
Keep API keys in a .env file, never commit to Git:
# .env (600 permissions only)
BINANCE_API_KEY=xxxx
BINANCE_API_SECRET=xxxx
TELEGRAM_BOT_TOKEN=xxxx
TELEGRAM_CHAT_ID=xxxx
# settings.py
from dotenv import load_dotenv
import os
load_dotenv()
BINANCE_API_KEY = os.environ["BINANCE_API_KEY"]
BINANCE_API_SECRET = os.environ["BINANCE_API_SECRET"]
Prometheus Monitoring Configuration
Add your strategy server’s node_exporter to prometheus.yml:
scrape_configs:
- job_name: node-remote-linux
static_configs:
- targets:
- your-strategy-server-IP:9100
node_exporter automatically collects metrics from any *.prom files in /var/lib/node_exporter/textfile_collector/—no extra configuration needed.
Grafana Dashboard
Your core strategy monitoring panel should include:
Profit Section (top)
- Cumulative return time series chart
- Daily PnL stat card
- Current position direction (long/short/flat)
Risk Section (middle)
- Maximum drawdown
- Daily return bar chart (last 30 days)
- Sharpe ratio (annualized)
Health Section (bottom)
- Time since last strategy run
- Time since last signal
- Heartbeat status (green = healthy, red = problem)
Key Panel: Strategy Heartbeat
# Grafana PromQL
(time() - quant_last_run_timestamp) / 60
This query shows how many minutes have passed since the strategy last ran. Color thresholds:
- < 65 minutes: green (healthy for hourly strategy)
- 65-120 minutes: yellow (minor delay)
120 minutes: red (strategy likely crashed)
Alert Configuration
Alertmanager + Telegram
Define alert rules in alerts.yml:
groups:
- name: quant-alerts
rules:
# Strategy heartbeat timeout
- alert: QuantHeartbeatTimeout
expr: (time() - quant_last_run_timestamp) > 7200
for: 5m
labels:
severity: critical
annotations:
summary: "Quant strategy heartbeat timeout"
description: "Strategy hasn't run for over 2 hours—check server status"
# Significant daily loss
- alert: QuantLargeLoss
expr: quant_pnl_today < -3
for: 1m
labels:
severity: warning
annotations:
summary: "Daily loss exceeds 3%"
description: "Daily return: {{ $value }}%—monitor closely"
Telegram alert template (configure in alertmanager.yml):
receivers:
- name: telegram
telegram_configs:
- bot_token: "your-BOT-TOKEN"
chat_id: your-CHAT-ID
message: |
🚨 {{ .GroupLabels.alertname }}
{{ range .Alerts }}
{{ .Annotations.summary }}
{{ .Annotations.description }}
{{ end }}
Direct Telegram Alerts from Strategy Code
Beyond Alertmanager, push alerts directly from your strategy code:
import requests
def send_telegram(message: str):
bot_token = os.environ["TELEGRAM_BOT_TOKEN"]
chat_id = os.environ["TELEGRAM_CHAT_ID"]
url = f"https://api.telegram.org/bot{bot_token}/sendMessage"
requests.post(url, json={"chat_id": chat_id, "text": message})
# Push alert after placing order
send_telegram(f"📊 Signal: {direction}\nPrice: {price}\nSize: {size}")
Summary: Tool Selection Rationale
| Tool | Alternatives | Why Choose This |
|---|---|---|
| Python | Go, JavaScript | Richest quant ecosystem (pandas, ta-lib, etc.) |
| Binance python-binance | Custom REST wrapper | Official SDK—saves reinventing the wheel |
| systemd | supervisor, PM2 | Built-in, stable, centralized log management |
| Prometheus + node_exporter | InfluxDB + Telegraf | Reuses existing monitoring stack, no new components |
| Alertmanager | Custom alert scripts | Full-featured rules, silencing, grouping |
| Telegram | Email, DingTalk | Immediate mobile push, simple API |
→ Read Polaris Strategy Deep Dive → Read Binance API From Zero to First Order → Read Prometheus + Grafana VPS Monitoring Setup
📷 Cover · Generated with Flux Schnell AI · panshi.io brand style