cryload 6.0.0

#
cryload - HTTP load testing for CI/CD

Cross-platform, single-binary HTTP load testing CLI. A modern alternative to `ab` / `wrk` / `hey`, written in Crystal.

##
Quick start

    curl -sSfL https://raw.githubusercontent.com/sdogruyol/cryload/master/scripts/install.sh | sh
    cryload https://example.com -n 1000 -c 50

1000 requests, 50 concurrent connections, JSON/CSV/plain output. That's it.

##
Why cryload?

Existing tools work fine on your laptop. cryload is built for the one place that matters most: CI/CD pipelines.

| Problem | Solution | |---------|----------| | "Did my deploy break performance?" | Set latency thresholds that fail your build | | "Where do I put the results?" | JSON and CSV output, ready to parse | | "Different OS in CI vs local?" | Single binary for Linux, macOS, Windows | | "Which tool works in all three?" | cryload does |

If you need a graph on your laptop, use wrk. If you need to fail a pipeline when p99 goes over 200ms, use cryload.

##
Features

- ⚡ Multi-core load generation (`--workers`), concurrent connections (`-c`)
- ⏱️ Duration or request count mode
- 📊 Latency percentiles: p50, p75, p90, p95, p99, p999 + histogram
- 🎯 Honest latency: coordinated-omission-corrected percentiles and rate attainment
- 🔍 Phase breakdown: DNS, connect, TLS, TTFB — see where the time actually went
- 🗂️ Per-URL and per-status breakdowns, not just one global number
- 🎯 CI threshold matrix: `--max-p50` … `--max-p999`, `--max-avg`, `--max-latency`, `--min-rps`, `--max-fail-rate`, per-endpoint `--url-threshold`
- 🚦 Exit code taxonomy: slow service, unreachable target and bad config are different codes
- 📦 JSON / CSV / quiet output for pipelines, with a versioned schema
- 🔒 Rate limiting, warmup, keep-alive control, request deadlines, TLS skip
- 🌐 Multi-URL, redirects, proxy, cookies, custom success codes
- 🖥️ Cross-platform: Linux, macOS, Windows - single binary

##
Performance

cryload is fast. Written in Crystal, compiled to native code, and multi-core by default since 6.0.0.

All figures below are the median of 5 runs, with the observed spread, because a single run of a load generator on a shared machine is noise, not a number.

| Test | Throughput (median of 5) | Spread | p99 | |------|--------------------------|--------|-----| | `--workers 8 -c 128 -d 3` | 329,440 req/sec | 296k – 342k | 1.95 ms | | `--workers 16 -c 128 -d 3` | 280,079 req/sec | 278k – 301k | 2.34 ms | | `--workers 4 -c 128 -d 3` | 210,668 req/sec | 190k – 240k | 2.38 ms | | `--workers 1 -c 128 -d 3` (v5-equivalent single-thread path) | 109,161 req/sec | 108k – 114k | 2.94 ms | | `--workers 1 -c 64 -d 3` | 138,106 req/sec | 110k – 143k | 1.32 ms |

| | | |---|---| | Peak RSS (`--workers 8 -c 128`, 2-byte bodies) | 22.9 MB | | Peak RSS (`--workers 1 -c 64`, 2-byte bodies) | 15.7 MB | | Peak RSS (`--workers 8 -c 32`, 100 KB bodies) | 12.9 MB | | Binary size | ~1.4 MB stripped, single file, no runtime dependencies |

Large responses cost less memory than many small ones, because bodies are streamed through one fixed buffer per worker rather than materialised as a String.

Multi-core buys 3.0x on this hardware. Note that `--workers 16` measures slower than `--workers 8`: once the schedulers outnumber the useful work they fight each other and the target for cores. That is why the default is `min(cpu cores, connections)` rather than "all of them", and why `--workers` is a dial worth turning rather than a number to max out.

Methodology. The target is a local Crystal `HTTP::Server` returning a 2-byte body, running as 7 processes sharing the port via `SO_REUSEPORT` so that the target is not the bottleneck. Hardware is an AMD Ryzen AI 9 465 (20 threads), over loopback, release build, 3-second runs, median of 5.

These are loopback numbers on one machine, not a claim about your network or your application. They exist to show the load generator's own ceiling — the number you need before you can trust that a measurement is about your server. Reproduce them with the commands in the table:

    cryload http://localhost:3000 --workers 8 -c 128 -d 3
    cryload http://localhost:3000 --workers 1 -c 128 -d 3

If throughput rises when you raise `--workers`, your earlier numbers were measuring the load generator, not the target. And look at that spread column before you set a `--min-rps` gate from a single local run.

No JVM, no Node. Just a single binary that starts instantly and uses almost no memory.

##
Honest latency

Most load testers measure latency from the moment a request is sent. That sounds reasonable until the target stalls. While it is stalled, no requests get sent — so no slow measurements get recorded — and the tool reports the handful of fast responses from before and after the freeze. A target that stops responding makes the numbers look better. This is called coordinated omission.

It is not a rounding error. Here is cryload's own regression test: freeze the target for 1 second during a `-q 500 -d 5` run.

| | v5 | v6 | |---|---|---| | Service p99 (measured from the send) | 0.37 ms | 0.56 ms | | p99 the gate actually reads | 0.37 ms | 951.96 ms (`corrected_latency_ms.p99`) | | Scheduled requests | 2500 | 2500 | | Requests actually issued | 2010 | 2500 | | Dropped without a word | 490 | 0 (`rate.skipped_requests`) | | Peak send delay reported | — | 981 ms (`rate.schedule_drift_ms`) | | `--max-p99 200` verdict | PASS, exit 0 | FAIL, exit 1 |

Read the second row twice. Same run, same server, same one-second outage: v5 reported a p99 of 0.37 ms and passed the gate, while quietly discarding 490 of the 2500 requests it was supposed to send. v6 reports 951.96 ms and fails the build. Note that v6's service p99 is still sub-millisecond and that is correct — the requests really were served fast once they got out. The outage lives in the waiting, which is exactly what a user would have felt.

v6 fixes this in three places:

- `corrected_latency_ms` charges each request for the time it spent waiting for its scheduled send slot, plus its service time. A request that was supposed to go out at t=1.0s but went out at t=2.0s is a 1-second-plus request, because that is what a real user would have experienced.
- `rate.skipped_requests` and `rate.attainment_percent` report the schedule slots that were never issued, so a run can no longer claim a rate it did not achieve.
- Threshold gates evaluate corrected latency by default, so `--max-p99 250` fails on that frozen target instead of passing. Add `--fail-on-rate-miss` to also fail when the attained rate falls more than 1% short.

Corrected and uncorrected latency are identical when `--rate` is not set: without a request schedule there is no send delay to correct for. Pass `--latency-correction off` for v5 gate semantics.

##
Installation

###
Option 1: Install script (recommended)

Linux / macOS:

    curl -sSfL https://raw.githubusercontent.com/sdogruyol/cryload/master/scripts/install.sh | sh -s

Windows (PowerShell):

    iwr -useb https://raw.githubusercontent.com/sdogruyol/cryload/master/scripts/install.ps1 | iex

###
Option 2: Prebuilt binary

Download from Releases:

    chmod +x cryload-linux
    ./cryload-linux --help

###
Option 3: Docker

    docker run --rm ghcr.io/sdogruyol/cryload https://example.com -n 1000 -c 50

Multi-arch (amd64/arm64) image, ~9 MB. Use `--network host` on Linux to reach services on the host's localhost. Tags follow the release version without the `v` prefix: `6.0.0`, `6.0`, `6`, `latest`.

###
Option 4: GitHub Action

    - uses: sdogruyol/cryload@v6
      with:
        url: http://127.0.0.1:3000/api
        requests: 500
        max_p99: "250"

See docs/github-action.md for all inputs and outputs.

###
Option 5: Build from source

Requires Crystal >= 1.21.0.

    git clone https://github.com/sdogruyol/cryload.git && cd cryload
    shards build --release

##
Usage

    cryload <url> [options]

| Option | Description | |--------|-------------| | `-n`, `--numbers` | Number of requests | | `-d`, `--duration` | Test duration (`30`, `30s`, `2m`, `1h30m`, `500ms`) | | `-c`, `--connections` | Concurrent connections (default: 10) | | `--workers` | OS threads running the load (default: `min(cpus, connections)`; `--workers 1` reproduces v5 behaviour) | | `-m`, `--method` | HTTP method (default: GET) | | `-b`, `--body` | Request body | | `--body-file` | Read body from file | | `--body-stdin` | Read body from stdin | | `-H`, `--header` | Repeatable header (`-H "Key: Value"`) | | `-a`, `--basic-auth` | Basic auth (`user:password`) | | `--user-agent` | User-Agent override | | `--host-header` | Explicit Host header | | `--timeout` | Connect/read/write timeout, with units | | `--request-timeout` | Total deadline for one request, including body download | | `-q`, `--rate` | Rate limit in req/sec, fractional allowed (`-q 0.5`) | | `--latency-correction` | `on` (default) or `off` — gate on corrected latency | | `-L`, `--follow-redirects` | Follow redirects | | `--disable-keepalive` | New connection per request; setup cost counts as latency | | `--output-format` | `text`, `json`, `csv`, `quiet` | | `--json` | Shorthand for `--output-format json` | | `--success-status` | Custom success codes/ranges | | `--insecure` | Skip TLS verification | | `--warmup` | Untimed warmup before the benchmark, with units | | `--proxy` | HTTP(S) proxy | | `--cookie` | Repeatable cookie (`name=value`) | | `--urls-file` | Load target URLs from file | | `--random-path` | Append random path per request | | `--progress` / `--no-progress` | Live stderr progress (default: on only when stderr is a TTY) | | `-V`, `--version` | Print the installed release |

Threshold flags are in CI/CD below.

###
Common examples

    # 10K requests, 100 concurrent
    cryload http://localhost:3000 -n 10000 -c 100

    # 30 seconds, 50 connections
    cryload http://localhost:3000 -d 30s -c 50

    # POST with JSON body
    cryload http://localhost:3000/api -n 500 -m POST \
      -H "Content-Type: application/json" \
      -b '{"name":"cry"}' --timeout 5

See docs/examples.md for more.

##
CI/CD

cryload is built for pipelines. Use `--json` or `--output-format csv` for structured output, `--output-format quiet` for exit-code-only checks.

###
Threshold matrix

Any breached threshold exits `1`.

| Flag | Effect | |------|--------| | `--max-p50 MS` | Exit 1 if p50 exceeds MS | | `--max-p75 MS` | Exit 1 if p75 exceeds MS | | `--max-p90 MS` | Exit 1 if p90 exceeds MS | | `--max-p95 MS` | Exit 1 if p95 exceeds MS | | `--max-p99 MS` | Exit 1 if p99 exceeds MS | | `--max-p999 MS` | Exit 1 if p999 exceeds MS | | `--max-avg MS` | Exit 1 if average latency exceeds MS | | `--max-latency MS` | Exit 1 if the maximum observed latency exceeds MS | | `--min-rps N` | Exit 1 if throughput is below N req/s | | `--max-fail-rate PCT` | Exit 1 if the failure rate exceeds PCT | | `--fail-on-error` | Exit 1 on any HTTP or transport error | | `--fail-on-transport-error` | Exit 1 on any transport error | | `--fail-on-rate-miss` | Exit 1 if the attained rate is more than 1% below `--rate` |

Latency gates evaluate corrected latency by default; `--latency-correction off` restores v5 semantics. See Honest latency.

###
Per-endpoint gates

`--url-threshold "PATTERN METRIC VALUE"` is repeatable and scopes a gate to the target URLs whose text contains `PATTERN`:

    cryload --urls-file urls.txt -d 30s -c 50 \
      --url-threshold "/api/users max-p99 120" \
      --url-threshold "/api/search max-p99 400" \
      --url-threshold "/api/orders max-fail-rate 0.5" \
      --min-rps 2000

`METRIC` is one of `max-p50 max-p75 max-p90 max-p95 max-p99 max-p999 max-avg max-latency max-fail-rate min-rps`. A pattern that matches no target URL is a config error (exit `2`), so a renamed endpoint fails the build instead of silently un-gating itself.

###
Exit codes

| Code | Meaning | |------|---------| | `0` | Run finished, no threshold breached | | `1` | A threshold was breached | | `2` | Usage/config error (bad flag, bad URL, missing file, unresolvable `--url-threshold` pattern, fd limit too low) | | `3` | Target unreachable: zero HTTP responses and at least one transport error | | `130` | Interrupted (SIGINT) | | `143` | Terminated (SIGTERM) |

A breached threshold takes precedence over a signal code. The same information is in the JSON report as `verdict.exit_code` and `verdict.reason`.

###
GitHub Actions example

Use the official action — it installs cryload, runs the benchmark, and exposes the JSON results as step outputs:

    - name: Latency SLA
      id: bench
      uses: sdogruyol/cryload@v6
      with:
        url: http://localhost:3000/api
        requests: 500
        max_p99: "250"

    - name: Show p99
      if: always()
      run: echo "p99 was ${{ steps.bench.outputs.p99 }} ms"

See docs/github-action.md for all inputs and outputs. Or install the binary directly:

    - name: Install cryload
      run: curl -sSfL https://raw.githubusercontent.com/sdogruyol/cryload/master/scripts/install.sh | sh -s

    - name: Latency SLA
      run: |
        cryload http://localhost:3000/api -d 30s -q 500 \
          --max-p99 250 --fail-on-rate-miss --json > result.json
        jq -e '.schema_version == 1' result.json
        jq -r '.thresholds.breached[] | "FAIL \(.name) \(.metric)=\(.actual) limit=\(.limit)"' result.json

`--no-progress` is no longer needed: progress is only enabled when stderr is a TTY, so CI logs stay clean by default.

##
How cryload compares

| Feature | cryload | ab | hey | oha | wrk | |---------|:-------:|:--:|:---:|:---:|:---:| | CI/CD output (JSON/CSV/quiet) | ✅ | - | JSON | JSON | - | | CI threshold exit codes | ✅ | - | - | - | - | | Per-endpoint threshold gates | ✅ | - | - | - | - | | Exit code taxonomy | ✅ | - | - | - | - | | Coordinated-omission correction | ✅ | - | - | - | - | | Latency phase breakdown | ✅ | - | partial | partial | - | | Rate limiting (`--rate`) | ✅ | - | partial | ✅ | - | | Multi-core load generation | ✅ | - | ✅ | ✅ | ✅ | | HTTP/2 | - | - | ✅ | ✅ | - | | Scriptable load (Lua) | - | - | - | - | ✅ | | Cross-platform binary | ✅ | Linux | ✅ | ✅ | Linux |

Full matrix: docs/comparison.md.

##
Built with Crystal

cryload is written in Crystal. Ruby-like syntax, compiled speed, single-binary deployment.

##
FAQ

Why not just use ab / hey / wrk?

Those tools are great for local benchmarks, and wrk still wins on Lua-scripted scenarios. cryload is built for CI/CD: coordinated-omission-corrected latency, a threshold matrix you can scope per endpoint, an exit code taxonomy, JSON/CSV output with a versioned schema, and cross-platform binaries. Since 6.0.0 it also saturates every core by default, so multi-core is no longer a reason to reach for another tool. If you want to fail a pipeline when p99 goes over 200ms — and be sure that p99 is honest — use cryload.

Can I use cryload for DDoS?

No. cryload is designed for testing your own servers and CI pipelines. Do not use it against targets you don't own.

Does cryload support HTTP/2?

Not yet. HTTP/1.1 only for now. HTTP/2 is on the roadmap.

What is coordinated omission and why should I care?

Most load testers start the latency clock when a request is sent. When the target stalls, nothing gets sent, so nothing slow gets measured — and the report shows only the fast responses from either side of the stall. A target that stops responding therefore looks faster. In cryload's own regression test, freezing the target for one second during a `-q 500 -d 5` run made v5 report p99 = 0.37 ms while it silently dropped 490 of 2500 scheduled requests. v6 charges each request for the time it waited for its scheduled slot (`corrected_latency_ms`), reports the slots it never issued (`rate.skipped_requests`), and gates on corrected latency by default. See Honest latency.

Why did my exit code change from 1 to 2/3?

v5 returned `1` for every failure. v6 splits them so a pipeline can tell them apart: `1` is a breached threshold (a real regression to triage), `2` is a usage or config error (a broken command line or workflow file), and `3` is an unreachable target (nothing answered — usually a broken test harness, not a slow service). Signals now exit `130`/`143` instead of `0`, so a killed run no longer looks like a pass. If your script did `if cryload ...; then`, it still behaves the same; if it matched on `== 1`, widen it to `!= 0` or switch on the code — docs/examples.md has a template.

Is there a Docker image?

Yes: `docker pull ghcr.io/sdogruyol/cryload`. The single binary is still the lightest option, but the image is handy for containerized pipelines (GitLab CI, Kubernetes jobs, etc.).

Why is it written in Crystal?

Crystal compiles to a single native binary with no runtime. It starts instantly, uses minimal memory, and delivers C-like performance with Ruby-like syntax.

##
Sponsors

If cryload helps your CI pipeline, consider sponsoring. Every dollar helps me keep building open source tools full time.

##
License

MIT