Back to blog

Building stamusctl · Part 2

Docker plumbing and PCAP replay

Circuit breakers for Docker, binary protocol parsing for log streaming, and how readpcap spins up a temporary Suricata container to turn a PCAP file into indexed security data.

In the previous part I covered what stamusctl does and how the template system works. This part is about the Docker integration and how readpcap turns a packet capture file into searchable security data.

Wrapping Docker Compose

stamusctl wraps docker compose through the Go Docker client. compose up, compose down, compose ps, compose logs, compose restart all proxy through.

The wrapping isn’t just pass-through. Docker operations fail in transient ways: connection reset, TLS handshake timeout, daemon not ready after a restart. A naive wrapper would surface a cryptic Docker error and leave the user confused. stamusctl retries with a circuit breaker:

type CircuitBreaker struct {
    state        string  // "closed", "open", "half-open"
    failureCount int
    threshold    int
    timeout      time.Duration
}

Three states: closed (normal), open (too many failures, back off), half-open (try one request to test recovery). Exponential backoff doubles on each retry. The breaker opens after 3 consecutive transient failures and resets after a configurable timeout.

Transient error detection checks for specific failure modes: connection refused, reset, EOF, broken pipe, TLS timeout, context deadline exceeded. Anything else is treated as a real error and surfaced immediately. This distinction matters because retrying a “permission denied” error is pointless, but retrying a “connection reset” after the Docker daemon restarted is exactly right.

Without this, stamusctl would hammer a Docker daemon that’s restarting and either hang or produce a wall of identical error messages. With it, the tool backs off, waits, and retries cleanly.

readpcap: from PCAP file to indexed events

This is my favorite feature in the whole tool. You run stamusctl compose readpcap capture.pcap and a few minutes later your Scout hunting interface is full of data from that capture. Here’s what happens under the hood.

stamusctl creates a temporary Docker container running Suricata in offline mode:

config := container.Config{
    Image:      suricataImage,
    Entrypoint: []string{"/docker-entrypoint.sh"},
    Cmd: []string{
        "-vvv", "-k", "none",
        "-r", "/replay/" + pcapName,
        "--runmode", "autofp",
        "-l", "/var/log/suricata",
    },
}

hostConfig := container.HostConfig{
    AutoRemove: true,
    Mounts: []mount.Mount{
        {Type: mount.TypeBind, Source: configPath + "/suricata/etc",
         Target: "/etc/suricata"},
        {Type: mount.TypeBind, Source: pcapPath,
         Target: "/replay/" + pcapName, ReadOnly: true},
    },
    CapAdd: []string{"net_admin", "sys_nice"},
}

The container mounts the existing Suricata config from the running stack (so your rules apply) and the PCAP file as read-only. Suricata processes the capture in autofp mode (multi-threaded), writes EVE JSON events to the log directory. Fluentd, which is already running as part of the CE stack, picks up those events and sends them to OpenSearch. The container auto-removes after it finishes.

Suricata needs net_admin and sys_nice capabilities even for offline PCAP replay. It’s a quirk of how Suricata initializes its capture engine. Without those capabilities the container just exits with an unhelpful error.

The Suricata image isn’t hardcoded. stamusctl resolves it from the running Compose config. It tries docker compose config first (the most reliable method), falls back to parsing the local YAML, then an environment variable, then a default image. This way readpcap always uses the same Suricata version as the running stack, so rules and EVE output format stay compatible.

Docker’s multiplexed log protocol

Log streaming from the readpcap container was the part that took longest to get right. Docker’s container.Logs API returns a multiplexed stream, not plain text. Each frame has an 8-byte binary header:

  • Byte 0: stream type (1 = stdout, 2 = stderr)
  • Bytes 1-3: padding (zero)
  • Bytes 4-7: frame size as big-endian uint32
hdr := make([]byte, 8)
for {
    out.Read(hdr)
    count := binary.BigEndian.Uint32(hdr[4:])
    dat := make([]byte, count)
    out.Read(dat)
    switch hdr[0] {
    case 1:
        fmt.Fprint(os.Stdout, string(dat))
    default:
        fmt.Fprint(os.Stderr, string(dat))
    }
}

If you just pipe the raw stream to stdout (which is what most Docker SDK examples do with io.Copy), you get binary garbage interspersed with your log output. You have to parse the frame headers and route each chunk to the right output. This is documented in Docker’s API reference but not in most Go SDK tutorials.

The readpcap command streams Suricata’s verbose output (-vvv) in real-time so you can watch it process the capture. When it finishes and the container exits (auto-removed), the events are already in OpenSearch and visible in Scout.

The whole experience from “I have a PCAP file” to “I can see every DNS query, HTTP request, and TLS handshake in a hunting interface” is a few minutes. For someone evaluating whether Clear NDR is right for their network, this is the fastest possible demo.