API Documentation
WebSocket, SSE, and TCP endpoints for real-time certificate streaming
What is Certstream?
Real-time certificate transparency log aggregation
Overview
Certstream aggregates certificates from Certificate Transparency logs and streams them in real-time. This is a Rust rewrite that works as a drop-in replacement for certstream-server (Elixir) and certstream-server-go.
Why Rust?
- ~12MB memory idle, ~19MB under load
- Sub-millisecond latency (tested 0.33ms min)
- Handles 50,000+ concurrent connections
- ~6% CPU with 500+ clients
- Single binary, no dependencies
Features
Core capabilities and v1.0.2 additions
Core Features
- WebSocket, SSE, and raw TCP streaming
- Server-side filtering (regex support for domain, issuer, log source)
- Pre-serialized messages (serialize once, broadcast to all)
- Works with existing certstream clients
- Prometheus metrics
- TLS support
- 60+ CT logs monitored
v1.0.2 New Features
- State Persistence: Resume from last position after restart
- Rate Limiting: Configurable requests per second with burst
- Connection Limiting: Max total and per-IP connections
- Token Authentication: Bearer token based auth
- Hot Reload: Config changes without restart
- CT Log Health: Automatic retry, circuit breaker, health checks
Endpoints
Connect using WebSocket, Server-Sent Events, or raw TCP
| Endpoint | Protocol | Description |
|---|---|---|
wss://ws.certstream.dev/ |
WebSocket | Lite stream (no DER/chain data) |
wss://ws.certstream.dev/full-stream |
WebSocket | Full stream with DER and certificate chain |
wss://ws.certstream.dev/domains-only |
WebSocket | Domain names only (minimal data) |
https://ws.certstream.dev/sse |
SSE | Server-Sent Events lite stream |
https://ws.certstream.dev/sse?stream=full |
SSE | Server-Sent Events full stream |
https://ws.certstream.dev/sse?stream=domains |
SSE | Server-Sent Events domains only |
ws.certstream.dev:8081 |
TCP | Raw TCP (send f/d/nothing for full/domains/lite) |
https://ws.certstream.dev/health |
HTTP | Health check endpoint |
https://ws.certstream.dev/metrics |
HTTP | Prometheus metrics |
https://ws.certstream.dev/example.json |
HTTP | Sample message format |
Message Format
JSON structure for certificate update messages
{
"message_type": "certificate_update",
"data": {
"update_type": "X509LogEntry",
"leaf_cert": {
"subject": {
"CN": "example.com",
"O": "Example Inc"
},
"issuer": {
"CN": "R3",
"O": "Let's Encrypt"
},
"serial_number": "04:8A:3F:...",
"not_before": 1703721600,
"not_after": 1735257600,
"fingerprint": "SHA256:AB:CD:EF:...",
"all_domains": [
"example.com",
"www.example.com",
"mail.example.com"
],
"as_der": "BASE64..." // full-stream only
},
"chain": [...], // full-stream only
"cert_index": 1234567890,
"seen": 1703808000.123,
"source": {
"url": "ct.googleapis.com/logs/argon2024",
"name": "Google Argon 2024"
}
}
}
Stream Types
Choose the right stream for your use case
Lite Stream (default)
Contains certificate metadata, domains, issuer info, and timestamps. No DER-encoded certificate data or chain. Best for most monitoring use cases.
Full Stream
Includes everything in lite stream plus base64-encoded DER certificate and full chain. Use when you need to verify or store complete certificates.
Domains Only
Minimal payload with just domain names, timestamp, and source. Lowest bandwidth option for domain-focused monitoring like phishing detection.
Quick Start
Connect and start receiving certificates in seconds
const ws = new WebSocket('wss://ws.certstream.dev/'); ws.onmessage = (event) => { const data = JSON.parse(event.data); if (data.message_type === 'certificate_update') { console.log(data.data.leaf_cert.all_domains); } };
# Lite stream curl -N https://ws.certstream.dev/sse # Full stream curl -N "https://ws.certstream.dev/sse?stream=full" # Domains only curl -N "https://ws.certstream.dev/sse?stream=domains"
Client Examples
Connect using your preferred language
# pip install certstream import certstream def callback(message, context): if message['message_type'] == 'certificate_update': domains = message['data']['leaf_cert']['all_domains'] print(domains) certstream.listen_for_events(callback, url='ws://localhost:8080/')
import "github.com/CaliDog/certstream-go" stream, _ := certstream.CertStreamEventStream(false) for event := range stream { fmt.Println(event.Data.LeafCert.AllDomains) }
const WebSocket = require('ws'); const ws = new WebSocket('ws://localhost:8080/'); ws.on('message', (data) => { const msg = JSON.parse(data); if (msg.message_type === 'certificate_update') { console.log(msg.data.leaf_cert.all_domains); } });
# WebSocket with token wscat -c ws://localhost:8080/ -H "Authorization: Bearer your-token" # SSE with token curl -H "Authorization: Bearer your-token" http://localhost:8080/sse
Server-Side Filtering
Filter certificates on the server to save bandwidth
| Parameter | Description | Example |
|---|---|---|
domain |
Domain regex pattern | .*\.google\.com |
issuer |
Issuer pattern | Let.*Encrypt |
source |
CT log name | Cloudflare |
exclude_domain |
Exclude domains | .*test.* |
exclude_issuer |
Exclude issuers | Self-Signed |
mode |
Match mode | any or all |
# .com domains from DigiCert websocat "ws://localhost:8080/?domain=.*\.com&issuer=DigiCert" # Google domains via SSE curl -N "http://localhost:8080/sse?domain=.*\.google\.com" # Exclude test domains websocat "ws://localhost:8080/?exclude_domain=.*test.*"
Self-Hosting
Run your own instance with Docker or from source
# Quick start docker run -d -p 8080:8080 -p 8081:8081 reloading01/certstream-server-rust:latest # With custom config docker run -d \ -p 8080:8080 \ -p 8081:8081 \ -v ./config.yaml:/app/config.yaml \ -v ./state:/app/state \ reloading01/certstream-server-rust:latest # Docker Compose version: '3.8' services: certstream: image: reloading01/certstream-server-rust:latest ports: - "8080:8080" - "8081:8081" volumes: - ./config.yaml:/app/config.yaml - ./state:/app/state restart: unless-stopped
# Docker build docker build -t certstream-server-rust . docker run -d -p 8080:8080 certstream-server-rust # Cargo build cargo build --release ./target/release/certstream-server-rust # Docker Compose docker compose up -d
Environment Variables
Configure the server using environment variables
General Settings
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_HOST |
0.0.0.0 | Bind address |
CERTSTREAM_PORT |
8080 | HTTP/WebSocket port |
CERTSTREAM_LOG_LEVEL |
info | debug, info, warn, error |
CERTSTREAM_BUFFER_SIZE |
1000 | Broadcast buffer |
Protocols
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_WS_ENABLED |
true | Enable WebSocket |
CERTSTREAM_SSE_ENABLED |
false | Enable SSE |
CERTSTREAM_TCP_ENABLED |
false | Enable TCP |
CERTSTREAM_TCP_PORT |
8081 | TCP port |
Rate Limiting
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_RATE_LIMIT_ENABLED |
false | Enable rate limiting |
CERTSTREAM_RATE_LIMIT_PER_SECOND |
10 | Requests per second |
CERTSTREAM_RATE_LIMIT_BURST_SIZE |
20 | Burst allowance |
Connection Limiting
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_CONNECTION_LIMIT_ENABLED |
false | Enable connection limits |
CERTSTREAM_CONNECTION_LIMIT_MAX_CONNECTIONS |
10000 | Max total connections |
CERTSTREAM_CONNECTION_LIMIT_PER_IP_LIMIT |
100 | Max per IP |
Authentication
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_AUTH_ENABLED |
false | Enable token auth |
CERTSTREAM_AUTH_TOKENS |
- | Comma-separated tokens |
CERTSTREAM_AUTH_HEADER_NAME |
Authorization | Auth header |
CT Log Settings
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_CT_LOG_STATE_FILE |
- | State file path |
CERTSTREAM_CT_LOG_RETRY_MAX_ATTEMPTS |
3 | Max retry attempts |
CERTSTREAM_CT_LOG_REQUEST_TIMEOUT_SECS |
30 | Request timeout |
CERTSTREAM_CT_LOG_BATCH_SIZE |
256 | Entries per batch |
Hot Reload
| Variable | Default | Description |
|---|---|---|
CERTSTREAM_HOT_RELOAD_ENABLED |
false | Enable hot reload |
CERTSTREAM_HOT_RELOAD_WATCH_PATH |
- | Config file to watch |
Configuration File
Full YAML configuration example
host: "0.0.0.0" port: 8080 log_level: "info" buffer_size: 1000 ct_logs_url: "https://www.gstatic.com/ct/log_list/v3/all_logs_list.json" protocols: websocket: true sse: true tcp: true tcp_port: 8081 ct_log: state_file: "/data/state.json" batch_size: 256 poll_interval_ms: 500 retry_max_attempts: 3 request_timeout_secs: 30 rate_limit: enabled: true per_second: 10 burst_size: 20 connection_limit: enabled: true max_connections: 10000 per_ip_limit: 100 auth: enabled: false tokens: - "secret-token-1" - "secret-token-2" header_name: "Authorization" hot_reload: enabled: true custom_logs: - name: "My Custom CT Log" url: "https://ct.example.com/log"
Config search order: CERTSTREAM_CONFIG env var → ./config.yaml → ./config.yml → /etc/certstream/config.yaml
Performance
Load tested with 1,000 concurrent WebSocket clients
| Metric | Rust | Go |
|---|---|---|
| Memory (idle) | ~12 MB | ~100 MB |
| Memory (avg under load) | 22 MB | 254 MB |
| CPU (avg under load) | ~15% | ~34% |
| Latency (avg) | 3.4ms | 31ms |
| Latency (min) | 0.16ms | 1.7ms |
| Throughput | 677K msg | 267K msg |
Result: 12x less memory, 9x faster latency, 2.5x higher throughput.
CT Logs Monitored
60+ Certificate Transparency logs including
Argon, Xenon, Solera, Submariner
Cloudflare
Nimbus
DigiCert
Wyvern, Sphinx
Sectigo
Elephant, Tiger, Dodo
Let's Encrypt
Sapling, Clicky
Others
TrustAsia, Nordu, and more
Use Cases
Common applications for certificate monitoring
Phishing Detection
Monitor for certificates that mimic legitimate domains to detect phishing campaigns early.
Brand Monitoring
Track certificates issued for your brand name or typosquatting variations.
Certificate Research
Analyze certificate issuance patterns, CA behavior, and cryptographic practices.
Compliance Tracking
Monitor certificate lifecycle and ensure compliance with security policies.
Related Projects
Other certstream implementations and clients
certstream-server
Original Elixir implementation by CaliDog
certstream-server-go
Go implementation
certstream-python
Python client library