GitHub

What is Certstream?

Real-time certificate transparency log aggregation

Overview

Certstream aggregates certificates from Certificate Transparency logs and streams them in real-time. This is a Rust rewrite that works as a drop-in replacement for certstream-server (Elixir) and certstream-server-go.

Why Rust?

  • ~12MB memory idle, ~19MB under load
  • Sub-millisecond latency (tested 0.33ms min)
  • Handles 50,000+ concurrent connections
  • ~6% CPU with 500+ clients
  • Single binary, no dependencies

Features

Core capabilities and v1.0.2 additions

Core Features

  • WebSocket, SSE, and raw TCP streaming
  • Server-side filtering (regex support for domain, issuer, log source)
  • Pre-serialized messages (serialize once, broadcast to all)
  • Works with existing certstream clients
  • Prometheus metrics
  • TLS support
  • 60+ CT logs monitored

v1.0.2 New Features

  • State Persistence: Resume from last position after restart
  • Rate Limiting: Configurable requests per second with burst
  • Connection Limiting: Max total and per-IP connections
  • Token Authentication: Bearer token based auth
  • Hot Reload: Config changes without restart
  • CT Log Health: Automatic retry, circuit breaker, health checks

Endpoints

Connect using WebSocket, Server-Sent Events, or raw TCP

Endpoint Protocol Description
wss://ws.certstream.dev/ WebSocket Lite stream (no DER/chain data)
wss://ws.certstream.dev/full-stream WebSocket Full stream with DER and certificate chain
wss://ws.certstream.dev/domains-only WebSocket Domain names only (minimal data)
https://ws.certstream.dev/sse SSE Server-Sent Events lite stream
https://ws.certstream.dev/sse?stream=full SSE Server-Sent Events full stream
https://ws.certstream.dev/sse?stream=domains SSE Server-Sent Events domains only
ws.certstream.dev:8081 TCP Raw TCP (send f/d/nothing for full/domains/lite)
https://ws.certstream.dev/health HTTP Health check endpoint
https://ws.certstream.dev/metrics HTTP Prometheus metrics
https://ws.certstream.dev/example.json HTTP Sample message format

Message Format

JSON structure for certificate update messages

JSON certificate_update
{
  "message_type": "certificate_update",
  "data": {
    "update_type": "X509LogEntry",
    "leaf_cert": {
      "subject": {
        "CN": "example.com",
        "O": "Example Inc"
      },
      "issuer": {
        "CN": "R3",
        "O": "Let's Encrypt"
      },
      "serial_number": "04:8A:3F:...",
      "not_before": 1703721600,
      "not_after": 1735257600,
      "fingerprint": "SHA256:AB:CD:EF:...",
      "all_domains": [
        "example.com",
        "www.example.com",
        "mail.example.com"
      ],
      "as_der": "BASE64..."  // full-stream only
    },
    "chain": [...],  // full-stream only
    "cert_index": 1234567890,
    "seen": 1703808000.123,
    "source": {
      "url": "ct.googleapis.com/logs/argon2024",
      "name": "Google Argon 2024"
    }
  }
}

Stream Types

Choose the right stream for your use case

Lite Stream (default)

Contains certificate metadata, domains, issuer info, and timestamps. No DER-encoded certificate data or chain. Best for most monitoring use cases.

Full Stream

Includes everything in lite stream plus base64-encoded DER certificate and full chain. Use when you need to verify or store complete certificates.

Domains Only

Minimal payload with just domain names, timestamp, and source. Lowest bandwidth option for domain-focused monitoring like phishing detection.

Quick Start

Connect and start receiving certificates in seconds

JavaScript Browser
const ws = new WebSocket('wss://ws.certstream.dev/');

ws.onmessage = (event) => {
    const data = JSON.parse(event.data);
    if (data.message_type === 'certificate_update') {
        console.log(data.data.leaf_cert.all_domains);
    }
};
cURL SSE
# Lite stream
curl -N https://ws.certstream.dev/sse

# Full stream
curl -N "https://ws.certstream.dev/sse?stream=full"

# Domains only
curl -N "https://ws.certstream.dev/sse?stream=domains"

Client Examples

Connect using your preferred language

Python certstream
# pip install certstream
import certstream

def callback(message, context):
    if message['message_type'] == 'certificate_update':
        domains = message['data']['leaf_cert']['all_domains']
        print(domains)

certstream.listen_for_events(callback, url='ws://localhost:8080/')
Go certstream-go
import "github.com/CaliDog/certstream-go"

stream, _ := certstream.CertStreamEventStream(false)
for event := range stream {
    fmt.Println(event.Data.LeafCert.AllDomains)
}
JavaScript Node.js
const WebSocket = require('ws');
const ws = new WebSocket('ws://localhost:8080/');

ws.on('message', (data) => {
    const msg = JSON.parse(data);
    if (msg.message_type === 'certificate_update') {
        console.log(msg.data.leaf_cert.all_domains);
    }
});
Bash With Auth
# WebSocket with token
wscat -c ws://localhost:8080/ -H "Authorization: Bearer your-token"

# SSE with token
curl -H "Authorization: Bearer your-token" http://localhost:8080/sse

Server-Side Filtering

Filter certificates on the server to save bandwidth

Parameter Description Example
domain Domain regex pattern .*\.google\.com
issuer Issuer pattern Let.*Encrypt
source CT log name Cloudflare
exclude_domain Exclude domains .*test.*
exclude_issuer Exclude issuers Self-Signed
mode Match mode any or all
Examples Filtering
# .com domains from DigiCert
websocat "ws://localhost:8080/?domain=.*\.com&issuer=DigiCert"

# Google domains via SSE
curl -N "http://localhost:8080/sse?domain=.*\.google\.com"

# Exclude test domains
websocat "ws://localhost:8080/?exclude_domain=.*test.*"

Self-Hosting

Run your own instance with Docker or from source

Docker Recommended
# Quick start
docker run -d -p 8080:8080 -p 8081:8081 reloading01/certstream-server-rust:latest

# With custom config
docker run -d \
  -p 8080:8080 \
  -p 8081:8081 \
  -v ./config.yaml:/app/config.yaml \
  -v ./state:/app/state \
  reloading01/certstream-server-rust:latest

# Docker Compose
version: '3.8'
services:
  certstream:
    image: reloading01/certstream-server-rust:latest
    ports:
      - "8080:8080"
      - "8081:8081"
    volumes:
      - ./config.yaml:/app/config.yaml
      - ./state:/app/state
    restart: unless-stopped
Build From Source
# Docker build
docker build -t certstream-server-rust .
docker run -d -p 8080:8080 certstream-server-rust

# Cargo build
cargo build --release
./target/release/certstream-server-rust

# Docker Compose
docker compose up -d

Environment Variables

Configure the server using environment variables

General Settings

Variable Default Description
CERTSTREAM_HOST 0.0.0.0 Bind address
CERTSTREAM_PORT 8080 HTTP/WebSocket port
CERTSTREAM_LOG_LEVEL info debug, info, warn, error
CERTSTREAM_BUFFER_SIZE 1000 Broadcast buffer

Protocols

Variable Default Description
CERTSTREAM_WS_ENABLED true Enable WebSocket
CERTSTREAM_SSE_ENABLED false Enable SSE
CERTSTREAM_TCP_ENABLED false Enable TCP
CERTSTREAM_TCP_PORT 8081 TCP port

Rate Limiting

Variable Default Description
CERTSTREAM_RATE_LIMIT_ENABLED false Enable rate limiting
CERTSTREAM_RATE_LIMIT_PER_SECOND 10 Requests per second
CERTSTREAM_RATE_LIMIT_BURST_SIZE 20 Burst allowance

Connection Limiting

Variable Default Description
CERTSTREAM_CONNECTION_LIMIT_ENABLED false Enable connection limits
CERTSTREAM_CONNECTION_LIMIT_MAX_CONNECTIONS 10000 Max total connections
CERTSTREAM_CONNECTION_LIMIT_PER_IP_LIMIT 100 Max per IP

Authentication

Variable Default Description
CERTSTREAM_AUTH_ENABLED false Enable token auth
CERTSTREAM_AUTH_TOKENS - Comma-separated tokens
CERTSTREAM_AUTH_HEADER_NAME Authorization Auth header

CT Log Settings

Variable Default Description
CERTSTREAM_CT_LOG_STATE_FILE - State file path
CERTSTREAM_CT_LOG_RETRY_MAX_ATTEMPTS 3 Max retry attempts
CERTSTREAM_CT_LOG_REQUEST_TIMEOUT_SECS 30 Request timeout
CERTSTREAM_CT_LOG_BATCH_SIZE 256 Entries per batch

Hot Reload

Variable Default Description
CERTSTREAM_HOT_RELOAD_ENABLED false Enable hot reload
CERTSTREAM_HOT_RELOAD_WATCH_PATH - Config file to watch

Configuration File

Full YAML configuration example

YAML config.yaml
host: "0.0.0.0"
port: 8080
log_level: "info"
buffer_size: 1000
ct_logs_url: "https://www.gstatic.com/ct/log_list/v3/all_logs_list.json"

protocols:
  websocket: true
  sse: true
  tcp: true
  tcp_port: 8081

ct_log:
  state_file: "/data/state.json"
  batch_size: 256
  poll_interval_ms: 500
  retry_max_attempts: 3
  request_timeout_secs: 30

rate_limit:
  enabled: true
  per_second: 10
  burst_size: 20

connection_limit:
  enabled: true
  max_connections: 10000
  per_ip_limit: 100

auth:
  enabled: false
  tokens:
    - "secret-token-1"
    - "secret-token-2"
  header_name: "Authorization"

hot_reload:
  enabled: true

custom_logs:
  - name: "My Custom CT Log"
    url: "https://ct.example.com/log"

Config search order: CERTSTREAM_CONFIG env var → ./config.yaml./config.yml/etc/certstream/config.yaml

Performance

Load tested with 1,000 concurrent WebSocket clients

Metric Rust Go
Memory (idle) ~12 MB ~100 MB
Memory (avg under load) 22 MB 254 MB
CPU (avg under load) ~15% ~34%
Latency (avg) 3.4ms 31ms
Latency (min) 0.16ms 1.7ms
Throughput 677K msg 267K msg

Result: 12x less memory, 9x faster latency, 2.5x higher throughput.

CT Logs Monitored

60+ Certificate Transparency logs including

Google

Argon, Xenon, Solera, Submariner

Cloudflare

Nimbus

DigiCert

Wyvern, Sphinx

Sectigo

Elephant, Tiger, Dodo

Let's Encrypt

Sapling, Clicky

Others

TrustAsia, Nordu, and more

Use Cases

Common applications for certificate monitoring

Phishing Detection

Monitor for certificates that mimic legitimate domains to detect phishing campaigns early.

Brand Monitoring

Track certificates issued for your brand name or typosquatting variations.

Certificate Research

Analyze certificate issuance patterns, CA behavior, and cryptographic practices.

Compliance Tracking

Monitor certificate lifecycle and ensure compliance with security policies.

Related Projects

Other certstream implementations and clients

certstream-server

Original Elixir implementation by CaliDog

certstream-server-go

Go implementation

certstream-python

Python client library