Protocol Canary

Protocol Canary is a command-line tool that checks whether a Stellar application or piece of infrastructure remains compatible with a target Stellar protocol version. It runs a set of explicit, declared compatibility assertions (“fixtures”) against three surfaces a Stellar project typically depends on: XDR encoding/decoding, Stellar RPC responses, and Soroban transaction simulation.

It is developer infrastructure — a CLI and a GitHub Action, not a frontend application, not a smart contract, not a wallet, and not a hosted service. It does not certify that an arbitrary application is completely compatible with a Stellar protocol version; it reports whether the compatibility assertions currently implemented in its fixture packs pass against your configured dependencies and RPC endpoint.

What it checks

Who this is for

• New to Protocol Canary? Start with What Problem Does It Solve? and How It Works.
• Evaluating Protocol 28 compatibility? Go straight to Protocol 28.
• Wiring this into CI? See the GitHub Action guide.
• Adding a compatibility fixture or engine check? See Fixtures and Authoring a Fixture.
• Reviewing this project? Architecture, Security, and Limitations are the three pages written specifically for that.

Quick example

cargo install --git https://github.com/StellarCanary/Protocol-Canary --tag v0.1.1 --locked
stellar-canary check --fixtures-dir <checkout-of-ProtocolCanary-Fixtures>/protocol-28 --protocol 28

Stellar Protocol Canary
────────────────────────────────────────

Project: Protocol-Canary (unknown)
Target protocol: 28
Network: testnet (observed protocol 28)

XDR
  3/3 PASS

RPC
  1/1 PASS

Soroban
  1/1 PASS

────────────────────────────────────────

5/5 applicable checks passed.

Status: PASS

This is a real run’s output (not illustrative text) — see Your First Check to reproduce it yourself.

Current support

Ecosystem

Three repositories make up Protocol Canary:

See Architecture for how they fit together, CLI Reference for every command, and Contributing to work on any of the three.

What Problem Does It Solve?

Think of Protocol Canary as an early-warning system: like the caged birds once carried into mines to detect dangerous gas before it reached the miners, it is meant to detect a compatibility problem before it reaches production — not to guarantee that no problem exists.

The real problem

Stellar protocols evolve. A protocol upgrade can add new XDR types (for example, Protocol 28’s CAP-0083 StellarValue case), change RPC response shapes, or add new Soroban host behavior. Applications and infrastructure depend on the current shape of all three surfaces, often implicitly — a hand-written XDR decoder, an assumption about an RPC field, a Soroban call that happens to work today.

Normal unit tests primarily exercise application code against mocked or assumed protocol behavior. They do not, by themselves, test the boundary between that application code and the real, current behavior of the Stellar network and its official libraries. That boundary is exactly what changes when a protocol upgrades.

What Protocol Canary does about it

It makes that compatibility boundary explicit and repeatable:

• Explicit, because every check is a declared fixture — a specific XDR value, RPC assertion, or Soroban call, with a source reference (a CAP number or official spec) — not an implicit assumption baked into application code.
• Repeatable, because the same fixture pack and the same CLI produce the same result locally and in CI, against the real stellar-xdr crate and a real RPC endpoint, rather than a mock.

What it does not claim

• It does not claim that every protocol change causes breakage — most fixtures simply keep passing.
• It does not prove an arbitrary application is compatible with a protocol version in any general sense. A passing run means the fixtures that are currently implemented for the target protocol passed against your configured dependencies and RPC endpoint — see Limitations.
• It does not replace testing against a real testnet or mainnet deployment.

How It Works

A single stellar-canary check run goes through the same pipeline whether it runs on a developer’s machine or inside a GitHub Actions job:

1. Configuration — .stellar-canary.toml (or CLI flags, which take precedence) determines the target protocol, network, RPC endpoint, and which surfaces (xdr/rpc/soroban) are enabled.
2. Project detection — the CLI inspects the current directory and classifies it as soroban, rpc-consumer, stellar-sdk, generic-stellar, or unknown. This determines which fixtures’ required_capabilities are satisfied.
3. Fixture loading — every .toml file under --fixtures-dir is loaded recursively and validated as a whole set: unique IDs, resolvable file references, schema conformance. A structurally invalid fixture fails the whole load (exit code 4) — it is never silently skipped.
4. Planning — fixtures are filtered to the ones whose declared protocol matches the run’s target and whose required_capabilities the detected project satisfies. Everything else is recorded as skipped, with a stated reason, never silently dropped and never counted as failed.
5. Execution — each planned fixture runs through the surface crate that understands its surface field:
   • xdr — decodes/encodes against the official stellar-xdr crate. Offline, no network call.
   • rpc — makes a live call (getNetwork, getLatestLedger) against the configured RPC endpoint and checks the declared assertions.
   • soroban — builds an unsigned transaction and calls simulateTransaction. It never signs, never requires a private key, and never submits a transaction to any network.
6. Policy evaluation — the set of per-fixture results is reduced to one overall status: pass, warning, fail, or error. error (an execution problem, such as an unreachable RPC endpoint) takes precedence over what the pass/fail counts alone would suggest, and maps to its own exit code.
7. Reporting — the same result set renders as terminal output (default), --format json, or --format markdown.

Developer / CI
     │
     ▼
.stellar-canary.toml + CLI flags
     │
     ▼
project detection ──► fixture loading & validation
     │                        │
     │                        ▼
     │                  planner (protocol + capability match)
     │                        │
     │           ┌────────────┼────────────┐
     │           ▼            ▼             ▼
     │          XDR          RPC         Soroban
     │        (offline)  (live, read)  (live, simulate-only)
     │           └────────────┼────────────┘
     │                        ▼
     │                policy evaluation (status + exit code)
     │                        │
     ▼                        ▼
terminal / JSON / markdown report

Only the XDR surface is offline. A run with rpc or soroban enabled in .stellar-canary.toml (the default, and the case for the protocol-28 pack) genuinely requires network access to the configured --rpc-url — a failure to reach it is reported as an execution error (exit code 3), not silently skipped. See Architecture for the full component breakdown and Troubleshooting for how to tell an RPC outage apart from a real compatibility failure.

Architecture

This page summarizes the verified architecture. The full, exhaustive version — environment variables, external-dependency tables, and the complete failure-boundary matrix — lives in Protocol-Canary’s docs/architecture.md and is authoritative; this page reproduces its key diagrams rather than duplicating it in full, per this site’s own documentation change discipline.

Component responsibilities

Protocol-Canary depends on nothing outside itself to run. ProtocolCanary-Action depends on a released Protocol-Canary version and, in practice, a ProtocolCanary-Fixtures checkout. Neither Protocol-Canary nor ProtocolCanary-Fixtures depends on the Action — both work standalone from the command line.

Execution topology

Protocol Canary runs in exactly two places, plus one external network it talks to. There is no hosted backend and no database.

LOCAL (developer machine)                  EXTERNAL (public network)
--------------------------                 --------------------------
Developer
  │
  ▼
stellar-canary check  ------------------->  Stellar RPC (Testnet by default)
  │         │                                 │  getNetwork
  │         `--------------------------->      `  simulateTransaction
  │                                           (read-only / simulation-only,
  ▼                                            no submission, no key)
fixture directory
(local path, or a
 ProtocolCanary-Fixtures checkout)
  │
  ▼
report (terminal / JSON / Markdown)


CI (GitHub-hosted, ephemeral per run)
--------------------------------------
Developer's repository
  │
  ▼
GitHub Actions workflow
  │
  ▼
StellarCanary/ProtocolCanary-Action@v1
  │
  ▼
cargo install --git Protocol-Canary   -> same LOCAL flow above, run
  │                                      inside the runner's ephemeral VM
  ▼
JSON report
  │
  ▼
GitHub job summary + annotations + workflow artifact

Both the local and CI flows run the identical stellar-canary binary against the identical fixture format — the Action never reimplements or reinterprets a compatibility result; it passes or fails the job according to the CLI’s own exit code and status field.

Trust boundaries

• Fixtures are untrusted data, not code. A fixture file is declarative TOML; no field executes a shell command, script, or arbitrary code.
• No private keys. Nothing in this system ever reads, stores, or transmits a private key or seed phrase. The Soroban surface builds and simulates unsigned transactions only.
• No transaction submission. Every network interaction is read-only (getNetwork, getLatestLedger) or simulation-only (simulateTransaction).
• The Action never reinterprets a result. It parses the CLI’s JSON and passes or fails the job according to the CLI’s own exit code / status field.

Stateful vs. ephemeral components

No component in this system introduces its own persistent application storage: no database, no hosted API state, no server.

Cost / hosting model

No paid backend, persistent server, or database is required for any normal use of any of the three repositories. No Docker installation is required. GitHub-hosted Actions runners on the free tier are sufficient for the CI workflows in this ecosystem.

For the complete picture — including the environment-variable table, the external-dependency table, and every documented failure boundary — see docs/architecture.md in the Protocol-Canary repository.

Installation

Protocol Canary does not yet publish prebuilt binaries or checksums (see Releases). The documented install path is cargo install against a pinned release tag.

Prerequisites

• A Rust/Cargo toolchain. The repository pins 1.91.0 in its own rust-toolchain.toml; cargo install will use whatever toolchain is active on your machine (rustup will fetch a matching one automatically if you use it).
• Network access to crates.io (for dependency resolution) and GitHub (to fetch the source).

No other runtime dependency is required. There is no Docker requirement and no OS-specific instructions beyond “a working Rust toolchain” — this project does not claim support for a specific operating system beyond what the Rust toolchain itself supports.

Install the current release

cargo install --git https://github.com/StellarCanary/Protocol-Canary --tag v0.1.1 --locked

• --tag v0.1.1 pins the exact, currently verified release — never main, and never an unpinned “latest”.
• --locked uses the exact dependency versions in the repository’s own committed Cargo.lock, rather than whatever the latest compatible versions happen to be at install time.

Verify the install:

stellar-canary version

stellar-canary 0.1.1

Alternative: build from a local checkout

git clone https://github.com/StellarCanary/Protocol-Canary
cd Protocol-Canary
cargo install --path crates/canary-cli

Alternative: run without installing (development)

From a workspace checkout, every command can be run directly through Cargo without installing a binary at all:

cargo run -p canary-cli -- check
cargo run -p canary-cli -- inspect
cargo run -p canary-cli -- fixtures --protocol 28

Next step

Continue to Your First Check.

Your First Check

This walks through a real run against the real Protocol 28 fixture pack, with real output — nothing here is illustrative text.

1. Install Canary

See Installation:

cargo install --git https://github.com/StellarCanary/Protocol-Canary --tag v0.1.1 --locked
stellar-canary version
# stellar-canary 0.1.1

2. Clone the fixture pack

git clone https://github.com/StellarCanary/ProtocolCanary-Fixtures

3. Run the Protocol 28 check

stellar-canary check \
  --fixtures-dir ProtocolCanary-Fixtures/protocol-28 \
  --protocol 28

Stellar Protocol Canary
────────────────────────────────────────

Project: Protocol-Canary (unknown)
Target protocol: 28
Network: testnet (observed protocol 28)

XDR
  3/3 PASS

RPC
  1/1 PASS

Soroban
  1/1 PASS

────────────────────────────────────────

5/5 applicable checks passed.

Status: PASS

4. Understand the PASS result

• 3/3, 1/1, 1/1 — every fixture that applies to Protocol 28 in this pack ran and passed, grouped by surface (XDR, RPC, Soroban).
• Network: testnet (observed protocol 28) — the RPC and Soroban surfaces made a live call to the default testnet endpoint, and the network actually reported protocol 28, matching the run’s target.
• Status: PASS and exit code 0 — see Exit Codes for the full contract.

This is not a claim of complete Protocol 28 coverage — see Protocol 28 for exactly what these five fixtures do and do not prove.

5. Run JSON mode

stellar-canary check \
  --fixtures-dir ProtocolCanary-Fixtures/protocol-28 \
  --protocol 28 \
  --json

{
  "schemaVersion": 1,
  "toolVersion": "0.1.1",
  "targetProtocol": 28,
  "project": { "name": "Protocol-Canary", "type": "unknown" },
  "network": { "name": "testnet", "observedProtocol": 28 },
  "status": "pass",
  "counts": { "total": 5, "passed": 5, "failed": 0, "warnings": 0, "errors": 0, "skipped": 0 },
  "results": [
    { "testId": "p28-xdr-cap83-empty-tx-set", "protocol": 28, "surface": "xdr", "status": "pass", "summary": "StellarValue round-tripped byte-for-byte", "durationMs": 0, "fixtureId": "p28-xdr-cap83-empty-tx-set" },
    { "testId": "p28-xdr-cap85-external-ref-malformed", "protocol": 28, "surface": "xdr", "status": "pass", "summary": "ContractExecutable was correctly rejected", "details": "failed to fill whole buffer", "durationMs": 0, "fixtureId": "p28-xdr-cap85-external-ref-malformed" },
    { "testId": "p28-xdr-cap85-external-ref-roundtrip", "protocol": 28, "surface": "xdr", "status": "pass", "summary": "ContractExecutable round-tripped byte-for-byte", "durationMs": 0, "fixtureId": "p28-xdr-cap85-external-ref-roundtrip" },
    { "testId": "p28-rpc-network", "protocol": 28, "surface": "rpc", "status": "pass", "summary": "GetNetwork response matched all 3 assertion(s)", "durationMs": 574, "fixtureId": "p28-rpc-network" },
    { "testId": "p28-soroban-native-asset-name", "protocol": 28, "surface": "soroban", "status": "pass", "summary": "simulation succeeded as expected", "durationMs": 380, "fixtureId": "p28-soroban-native-asset-name" }
  ],
  "git": { "commit": "ffa072cb14c682bfa5ebf158ab9bbe6058962aef", "branch": "main", "isDirty": false }
}

This is a real, unedited run captured against soroban-testnet.stellar.org. Field-by-field meaning is in JSON Report.

6. Understand the exit code

echo $?
# 0

0 means every applicable fixture passed. See Exit Codes for the full table (0–5) and what each one means.

What’s next

• CLI Reference for every command and flag.
• Protocol 28 for exactly what these five fixtures do and do not cover.
• GitHub Action to run this in CI instead of locally.

CLI Reference

stellar-canary ships five subcommands. This reference matches the CLI shipped in v0.1.1, verified directly against stellar-canary --help and each subcommand’s own --help output.

Rehearse Stellar protocol upgrades before they reach your production stack.

Usage: stellar-canary <COMMAND>

Commands:
  check     Run compatibility checks against the current project
  inspect   Print offline project diagnostics
  fixtures  List fixtures available for a protocol
  report    Render a previously generated JSON report
  version   Print the tool version
  help      Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

Every command accepts -h/--help for the exact, current flag list. If this page and your installed binary’s --help output ever disagree, trust --help — it is generated directly from the CLI’s own argument parser (clap) and cannot drift from the shipped behavior the way a document can.

check

Runs compatibility checks against the current project. This is the command everything else in this documentation builds toward.

Run compatibility checks against the current project

Usage: stellar-canary check [OPTIONS]

Options:
      --protocol <PROTOCOL>          Target protocol version (overrides configuration)
      --network <NETWORK>            Network to run live checks against [default: testnet]
      --rpc-url <RPC_URL>            RPC endpoint to use for live checks
      --config <CONFIG>              Path to a configuration file (default: .stellar-canary.toml in the project root)
      --rpc-timeout <RPC_TIMEOUT>    Request timeout for the RPC client in seconds [default: 10]
      --fixtures-dir <FIXTURES_DIR>  Directory containing fixture files [default: fixtures]
      --format <FORMAT>              Output format [default: terminal] [possible values: terminal, json, markdown]
      --json                         Shorthand for --format json
      --output <PATH>                Write the rendered report to this path, in addition to stdout.
      --verbose                      Include skip reasons in Markdown/terminal output and populate the JSON report's verbose field.
      --quiet                        Shorten terminal-format output to a single status line.
      --max-concurrency <MAX_CONCURRENCY>  Maximum number of concurrent network requests for RPC/Soroban fixtures [default: 4]
  -h, --help                         Print help

Protocol selection

Precedence, highest first: --protocol flag, then .stellar-canary.toml’s protocol field, then the built-in default of 28.

Fixture directory

--fixtures-dir defaults to fixtures relative to the current directory. It accepts any local path — including a checkout of ProtocolCanary-Fixtures — and is loaded recursively. A directory that does not exist is treated as zero fixtures (a trivial 0/0 pass), not an error. See Fixtures.

Output formats

• terminal (default) — human-readable, colorized when the output is a TTY.
• --format json (or the --json shorthand) — machine-readable, the contract ProtocolCanary-Action consumes. See JSON Report.
• --format markdown — a Markdown table, suitable for a PR comment or job summary.

Note on verbosity: The --quiet flag only affects the terminal output format, condensing the output to a single Status: line instead of a full report. The --verbose flag affects the detail included: it adds skip reasons to both markdown and terminal outputs, and is passed through into the json report’s verbose field.

The report is printed to stdout regardless of exit code, including on a compatibility failure — a caller does not need to inspect stderr to get the report.

Writing the report to a file

--output <PATH> writes the rendered report to PATH in addition to printing it to stdout. Stdout behavior is unchanged, so an existing shell redirection (stellar-canary check --json > result.json) still works, and a caller that cannot capture stdout — for example one wrapped by a tool that does not pass stdout through cleanly — has a direct way to get the report into a file.

The file receives exactly the bytes stdout receives, in the format selected by --format/--json, and including the trailing newline println! adds. That also means --quiet writes its single Status: line rather than a full report. The file is created if PATH does not exist and truncated if it does, and it is written before anything is printed, so an unusable path fails before a report reaches stdout.

stellar-canary check --json --output result.json
stellar-canary report result.json --format markdown

The exit code for the run itself is unchanged by --output: it never turns a 0 into a 1 or the other way around. The one failure mode the flag adds is PATH itself — a directory that does not exist, or one the process cannot write — which is reported as a configuration error (exit code 2) and prints nothing to stdout, since the report could not be produced where it was asked for. See Exit Codes.

Network behavior

Not every check requires the network:

• XDR checks are offline. They decode/encode against the official stellar-xdr crate with no network call.
• RPC and Soroban checks require a live, reachable --rpc-url. --network selects which network name is reported (testnet by default); --rpc-url is the actual endpoint contacted. If RPC/Soroban checks are enabled (the default) and the endpoint is unreachable, that is reported as an execution error, not skipped — see Exit Codes and network troubleshooting.
• The endpoint’s identity is validated against the run’s assumptions. When getNetwork succeeds, its passphrase is compared to --network and its protocol version to --protocol: a passphrase mismatch aborts as a configuration error (exit 2) before any check runs, since results from the wrong network must not be attributed to the requested one; an observed protocol that differs from the target prints a warning: line on stderr (e.g. warning: the RPC endpoint reports protocol 27, but this run targets protocol 28) and the run continues — observing a not-yet-upgraded network while rehearsing the next protocol is a legitimate use case, but it must be visible rather than only an (observed protocol N) annotation in the report.

Disable a surface in .stellar-canary.toml ([tests] rpc = false / soroban = false) to run fully offline.

Example: PASS

stellar-canary check --fixtures-dir ProtocolCanary-Fixtures/protocol-28 --protocol 28

Stellar Protocol Canary
────────────────────────────────────────

Project: Protocol-Canary (unknown)
Target protocol: 28
Network: testnet (observed protocol 28)

XDR
  3/3 PASS

RPC
  1/1 PASS

Soroban
  1/1 PASS

────────────────────────────────────────

5/5 applicable checks passed.

Status: PASS

Exit code 0.

Example: JSON

stellar-canary check --fixtures-dir ProtocolCanary-Fixtures/protocol-28 --protocol 28 --json

See the full real example in Your First Check and the field-by-field reference in JSON Report.

Example: FAIL

A real fixture deliberately given a wrong expected value:

Stellar Protocol Canary
────────────────────────────────────────

Project: Protocol-Canary (unknown)
Target protocol: 28
Network: testnet (observed protocol 28)

XDR
  demo-xdr-fail                ❌ FAIL

────────────────────────────────────────

0/1 applicable checks passed.

Status: NOT READY

Failure:
demo-xdr-fail

failed to decode StellarValue input

failed to fill whole buffer

Exit code 1. Note that the terminal reporter’s headline for a failing run reads Status: NOT READY — the machine-readable status field in --json output is "fail"; see JSON Report for the exact string values a script should key off of instead of terminal text.

inspect

Prints offline project diagnostics. No fixtures run, and no network call is made.

Print offline project diagnostics

Usage: stellar-canary inspect [OPTIONS]

Options:
      --protocol <PROTOCOL>          Target protocol version (overrides configuration)
      --fixtures-dir <FIXTURES_DIR>  Directory containing fixture files to inspect [default: fixtures]
      --config <CONFIG>              Path to a configuration file (default: .stellar-canary.toml in the project root)
  -h, --help                         Print help

What it inspects

The same project-detection logic check uses internally, surfaced on its own: the project root, its detected type, which Stellar-related dependencies/artifacts were detected, the configured target protocol, and which surfaces (xdr/rpc/soroban) are currently enabled.

With --fixtures-dir and --protocol, inspect drives the same compatibility-planner preview described in the README: it lists fixtures that would run and fixtures skipped for a protocol mismatch, disabled surface, or missing project capability, without contacting Stellar RPC or Soroban endpoints.

Example

Run inside a checkout of Protocol-Canary itself:

stellar-canary inspect

Project root: /home/hollujay/Protocol-Canary
Project type: unknown

Detected Stellar SDK/XDR dependency: false
Detected Soroban contract usage: false
Detected RPC client dependency: false
Detected WASM artifact: false

Configured protocol: 28
Available compatibility surfaces:
  xdr:     enabled
  rpc:     enabled
  soroban: enabled

Project type: unknown here is correct — this repository is Canary’s own source code, not a Stellar application, so none of the detection heuristics (Soroban contract, RPC client, stellar-sdk dependency, WASM artifact) match. Run inspect inside your own project to see its actual detected type and which fixtures’ required_capabilities it will satisfy.

inspect is useful before check when you want to confirm project detection or configuration without waiting on a live network call.

fixtures

Lists the fixtures available for a given protocol, without running them.

List fixtures available for a protocol

Usage: stellar-canary fixtures [OPTIONS]

Options:
      --protocol <PROTOCOL>          Protocol version to list fixtures for (default: the configured protocol, or 28 if there is no configuration file)
      --fixtures-dir <FIXTURES_DIR>  Directory containing fixture files [default: fixtures]
      --config <CONFIG>              Path to a configuration file (default: .stellar-canary.toml in the project root)
  -h, --help                         Print help

This loads and validates --fixtures-dir exactly the way check does (so a malformed fixture is reported the same way, exit code 4), but stops before running anything — no network call, regardless of the fixtures’ surfaces.

As with check, a --fixtures-dir that does not exist is treated as zero fixtures, not an error: the command exits 0 and prints the same message as a real but empty directory, so check the path if you expected fixtures.

$ stellar-canary fixtures --fixtures-dir /does/not/exist --protocol 28
Protocol 28 fixtures

(no fixtures found in /does/not/exist)

Example

stellar-canary fixtures --fixtures-dir ProtocolCanary-Fixtures/protocol-28 --protocol 28

Protocol 28 fixtures

XDR
  p28-xdr-cap83-empty-tx-set
  p28-xdr-cap85-external-ref-malformed
  p28-xdr-cap85-external-ref-roundtrip

RPC
  p28-rpc-network

Soroban
  p28-soroban-native-asset-name

Fixtures are grouped by surface and listed in sorted-path order, matching check’s deterministic ordering. Use this to confirm which fixtures a --fixtures-dir/--protocol combination will actually run before spending a live network call on check — see Fixtures for what a fixture is and how this list maps to the file layout.

report

Renders a previously generated JSON report in another format, without touching the network.

Render a previously generated JSON report

Usage: stellar-canary report [OPTIONS] <PATH>

Arguments:
  <PATH>  Path to a JSON report produced by `stellar-canary check --json`

Options:
      --format <FORMAT>  Output format to render the stored report as [default: markdown] [possible values: terminal, json, markdown]
  -h, --help             Print help

This is a pure re-render: it parses the exact shape documented in JSON Report (JsonReporter::parse) and reproduces it in the requested format — it never re-runs a check or makes a network call. Note the default output format here is markdown, unlike check’s default of terminal.

Example

stellar-canary check --fixtures-dir ProtocolCanary-Fixtures/protocol-28 --protocol 28 --json > result.json
stellar-canary report result.json --format markdown

## Stellar Protocol Canary

Protocol 28 compatibility

| Surface | Result |
|---|---|
| XDR | ✅ Pass |
| RPC | ✅ Pass |
| Soroban | ✅ Pass |

**Result: PASS**

stellar-canary report result.json --format terminal

Stellar Protocol Canary
────────────────────────────────────────

Project: Protocol-Canary (unknown)
Target protocol: 28
Network: testnet (observed protocol 28)

XDR
  3/3 PASS

RPC
  1/1 PASS

Soroban
  1/1 PASS

────────────────────────────────────────

5/5 applicable checks passed.

Status: PASS

There is no export format beyond terminal/json/markdown — do not assume an HTML, PDF, or CSV output exists; it does not.

version

Prints the tool’s own version.

Print the tool version

Usage: stellar-canary version

Options:
  -h, --help  Print help

stellar-canary version

stellar-canary 0.1.1

Why version visibility matters here

toolVersion also appears in every JSON report, independent of schemaVersion and targetProtocol. Because a fixture pack can require a specific stellar-xdr/CLI capability (for example, ContractExecutable XDR support was added in 0.1.1, not 0.1.0 — see Releases), knowing exactly which CLI version produced a given report matters when comparing results across machines, across CI runs, or against the version compatibility table.

Exit Codes

stellar-canary’s process exit codes are a stable, documented contract — GitHub automation and other callers rely on them rather than parsing terminal text. They are defined in crates/canary-core/src/errors.rs as the ExitCode enum, and this table matches that source directly (each value is also asserted by that file’s own exit_codes_match_the_documented_contract unit test).

How the mapping works

Every error in the CLI is a CanaryError variant, which is classified into one of four categories, which in turn maps to an exit code:

Verified examples

Each of these was captured from a real run of the v0.1.1 binary, not written from memory:

Exit 0 — check against the real Protocol 28 pack: 5/5 applicable checks passed.

Exit 1 — check against a fixture with a deliberately wrong expected value:

error: (terminal output shows "❌ FAIL" and "Status: NOT READY";
the machine-readable status is "fail" — see JSON Report)

Exit 2 — check --config /does/not/exist.toml:

error: configuration error: failed to read configuration file /does/not/exist.toml: No such file or directory (os error 2)

Exit 3 — check with --rpc-url pointed at an unreachable host:

Failure:
p28-rpc-network

failed to call GetNetwork

network transport error calling getNetwork: error sending request for url (...)

Note that in this case the XDR checks still ran and passed (3/3 PASS) — an RPC outage does not block offline checks. status in JSON output is "error", which overrides whatever the pass/fail counts alone would otherwise suggest.

Exit 4 — check against a fixture missing a required field:

error: invalid fixture: failed to parse fixture file /path/bad.toml: TOML parse error at line 1, column 1
  |
1 | id = "demo-missing-field"
  | ^^^^^^^^^^^^^^^^^^^^^^^^^
missing field `description`

Exit 5 is reserved for internal errors (Git metadata or local-cache failures) — this documentation does not fabricate a specific reproduction for it, since triggering one requires an environment fault (e.g. a corrupt local cache file) rather than a normal CLI invocation. Its meaning above is taken directly from the source’s ErrorCategory::Internal mapping, not guessed.

Protocol 28

Protocol 28 is the only protocol version Protocol Canary currently supports. This page describes exactly what the currently implemented Protocol 28 checks verify, using the real fixture pack in ProtocolCanary-Fixtures/protocol-28 — and, just as importantly, what they do not.

Protocol 28 is implemented in Stellar Core 28.0.0, Stellar RPC 28.0.0, and the official stellar-xdr 28.0.0 crate. Among its changes, three CAPs are relevant to this project’s three compatibility surfaces:

Protocol 28
    ├── CAP-0083  (validators can vote to drop a transaction set)
    ├── CAP-0085  (externally managed contract executables / fleet upgrades)
    └── CAP-0086  (sparse-map host functions for storage migration)

Currently implemented Protocol 28 checks

Five fixtures, verified against https://soroban-testnet.stellar.org:

Running this pack currently produces 5/5 PASS. This is a factual result about these five specific fixtures — it does not mean full Protocol 28 compatibility.

What this pack does not check

• CAP-0085 host-function semantics beyond the wire format. The two CAP-0085 fixtures prove the encoding round-trips; they do not deploy a real externally-managed-executable contract fleet end-to-end (owner contract, ExecutableTagObject entry, an instance referencing it, and a resolved invocation). Building that verifiably requires upstream stellar CLI support for constructing such a deployment, which was not yet available (27.1.0) when this pack was authored.
• CAP-0086 sparse-map host functions — a documented gap, not an oversight. CAP-0086 adds no new top-level XDR type; it is host-environment behavior only observable by invoking a deployed contract that calls sparse_map_new_from_linear_memory / sparse_map_unpack_to_linear_memory. As of this pack’s release, the latest published soroban-sdk does not expose these functions at any stable, documented API surface, so no fixture exists for them yet. This gap closes once the SDK exposes them or a verified deployed contract using them can be pointed to — see Roadmap.
• Anything about a specific downstream application beyond whether these five fixtures pass against its own configured dependencies and RPC endpoint.
• Mainnet. Every fixture in this pack was verified against Testnet only. Running against mainnet requires --network mainnet --rpc-url <endpoint> explicitly and has not itself been separately verified.

Consuming this pack

stellar-canary check --fixtures-dir <checkout>/protocol-28 --protocol 28 --json
# or, scanning every protocol pack in a fixtures checkout at once:
stellar-canary check --fixtures-dir <checkout> --protocol 28 --json

Both forms work identically — see Fixtures for why: the loader filters by each fixture’s own declared protocol field, not by directory path.

Adding more Protocol 28 fixtures

See Authoring a Fixture. Every new fixture must cite a real upstream source (a CAP number, an XDR definition, or an official RPC/host-function reference) in its source_reference field — fixtures are never added merely to raise a coverage number.

Fixtures

A fixture is a single declarative compatibility assertion: one TOML file describing one concrete thing to check — an XDR value that must round-trip, an RPC response that must match a shape, or a Soroban call that must simulate successfully.

Why fixtures are separate from the engine

Fixtures live in their own repository, ProtocolCanary-Fixtures, deliberately separate from the Protocol-Canary engine that runs them:

• Fixtures are data, not code. A fixture file is plain TOML. There is no field that executes a shell command, script, or arbitrary code, and none will be added — this is a hard trust boundary, since fixtures are public and consumed by CI pipelines. See Security.
• Fixtures version independently of the engine. A new protocol pack can be added to ProtocolCanary-Fixtures without a Protocol-Canary release, as long as it fits the existing schema.
• Anyone can supply their own fixture directory. --fixtures-dir accepts any local path — a ProtocolCanary-Fixtures checkout is the canonical source, not the only possible one.

Directory structure

Any directory tree works. The loader recursively walks --fixtures-dir and collects every file ending in .toml, at any depth. Subdirectory names (xdr/, rpc/, soroban/, protocol-28/xdr/cap-0083/) are a convention for human navigation only — the loader does not read directory names to determine a fixture’s surface or protocol; only the fixture file’s own surface and protocol fields matter. Non-.toml files (READMEs, licenses) are silently ignored. Fixtures load in sorted-by-path order, which is what makes stellar-canary fixtures and stellar-canary check output deterministic.

To see which fixtures a directory actually contains before running them, use stellar-canary fixtures --fixtures-dir <path> --protocol <N> — it loads the directory exactly the way check does and lists everything it picked up, grouped by surface, without executing a single assertion. See fixtures for the full command reference.

The real protocol-28 pack looks like this:

protocol-28/
├── README.md
├── rpc/
│   └── p28-rpc-network.toml
├── soroban/
│   └── p28-soroban-native-asset-name.toml
└── xdr/
    ├── cap-0083/
    │   └── p28-xdr-cap83-empty-tx-set.toml
    └── cap-0085/
        ├── p28-xdr-cap85-external-ref-malformed.toml
        └── p28-xdr-cap85-external-ref-roundtrip.toml

Schema

Every fixture file has this common shape (from schemas/fixture-v1.schema.json in ProtocolCanary-Fixtures, which mirrors the loader’s actual implementation):

id = "unique-string"                # required, pattern ^[a-z0-9][a-z0-9-]*$
protocol = 28                       # required, integer
surface = "xdr"                     # required: "xdr" | "rpc" | "soroban"
category = "cap-0083"                # required, free-text (avoid "misc"/"other")
description = "..."                  # required
source_reference = "CAP-0083"        # strongly expected: an authoritative upstream reference
required_capabilities = ["soroban-contract"]  # optional, see below

# optional: paths to externally stored input/expected data, relative to
# this file's own directory — existence-checked by the validator
input_file = "large-payload.xdr.b64"
expected_file = "expected.xdr.b64"

# everything else is surface-specific — see the per-surface tables below

required_capabilities values are kebab-case canary_core::Capability names: soroban-contract, rpc-client, stellar-sdk-dependency, wasm-artifact, raw-ledger-access. A fixture requiring a capability the target project doesn’t have is skipped, not failed.

Per-surface body fields

ID resolution and protocol filtering

The id field is the fixture’s identity everywhere — the string shown in stellar-canary fixtures, the testId/fixtureId in results and the JSON report, and what duplicate-checking validates against. IDs must be unique across the entire loaded directory tree, not just one file.

Each fixture declares its own protocol. A run’s target protocol is compared against every loaded fixture’s protocol; a mismatch is skipped (with a stated reason), never failed. This means a single --fixtures-dir can safely hold fixtures for multiple protocol versions side by side.

Validation

ProtocolCanary-Fixtures ships a structural validator:

python3 tools/validate/validate.py

It checks schema conformance, unique IDs, protocol/surface enums, source references, and that every referenced file actually exists. It is structural only — it never executes a fixture’s assertion and never makes a network call. python3 -m unittest discover tests runs its own test suite.

Protocol-Canary’s own loader performs the load-time validation that actually gates a check/fixtures run: a duplicate ID, a dangling input_file/expected_file reference, or a structurally invalid file (bad TOML, missing required field, unrecognized surface) fails the whole load with exit code 4 — never silently treated as a project incompatibility.

Testing a fixture

stellar-canary fixtures --fixtures-dir <path> --protocol <N>   # confirm it's picked up
stellar-canary check --fixtures-dir <path> --protocol <N>      # actually run it

The fixtures command is the preview half of that pair — see fixtures for its options and output format.

Adding a fixture or a protocol pack

See Authoring a Fixture for the contributor workflow, and ProtocolCanary-Fixtures’s own CONTRIBUTING.md for repository-specific review expectations. A new protocol pack follows the same schema and directory convention as protocol-28/ — see protocol-27/ in the Fixtures repository for a real example of an intentionally not-yet-populated pack (fixtures are added only after their upstream behavior is independently verified, never as placeholders).

Authoring a Fixture

This is the contributor workflow for adding one new fixture to ProtocolCanary-Fixtures. It exists to raise the bar on what counts as a real compatibility assertion — fixtures are never added merely to increase a coverage number, and a reviewer must be able to answer all of the following from the fixture file and its header comment alone, without running anything:

1. What Stellar behavior does this test?
2. Why is that behavior important?
3. What protocol/CAP introduced it?
4. What is the expected result, and where does that expectation come from?
5. Is the test deterministic?

Steps

1. Identify a concrete compatibility assertion. Read the CAP text, the upstream XDR definition, the upstream implementation, or the official release/API docs — in that order of preference. Never cite a source you have not actually checked describes the specific behavior you are asserting.
2. Determine the applicable protocol version and pick a stable ID following p<protocol>-<surface>-<slug> (e.g. p28-xdr-cap85-external-ref-roundtrip). IDs are lowercase and unique across the entire repository — the loader validates this across every *.toml file under a given --fixtures-dir, not just one file. Never rename an ID just because an implementation detail changed; if the semantic assertion itself changes, add a new ID instead of repurposing an old one.
3. Create a fixture matching the real schema — see Fixtures for the exact common fields and per-surface body. Set source_reference to an authoritative URL or CAP identifier, and add a header comment (a # block above the TOML body) explaining, in prose, how the expected value was derived or observed — e.g. “built with the official stellar-xdr 28.0.0 crate against the CAP-0083 StellarValue type”, not “looks right”. Use deterministic input: no fixture may depend on ledger state that changes between runs (a current ledger sequence, “the latest anything”) unless the assertion is explicitly scoped and documented as a live-network check.
4. Validate it:

   python3 tools/validate/validate.py
   

5. Run it through the real CLI:

   stellar-canary fixtures --fixtures-dir <path> --protocol <N>
   stellar-canary check --fixtures-dir <path> --protocol <N>
   

6. Add or update documentation — the relevant docs/protocol-NN.md table in ProtocolCanary-Fixtures, and this site’s Protocol 28 page if you touched that pack.
7. Submit a pull request against ProtocolCanary-Fixtures, following its own CONTRIBUTING.md.

What does not belong in a fixture

• A shell command, script, or any other executable field — none exists in the schema, and a pull request introducing one is rejected regardless of stated purpose.
• A private key, seed phrase, or funded-account secret. A source_account is a public address only, used to build an unsigned transaction.
• A fixture whose expected value was guessed, “probably correct”, or not independently checked against an authoritative source — see the CAP-0086 gap on the Protocol 28 page for what happens instead when a real fixture cannot yet be built responsibly: it is documented as a gap, not filled with an unverifiable guess.

JSON Report

This documents stellar-canary check --json (equivalently --format json) output, as implemented in crates/canary-report/src/json.rs and verified by running the real v0.1.1 binary against the real ProtocolCanary-Fixtures protocol-28 pack. This is the exact contract ProtocolCanary-Action consumes to build its GitHub job summary and annotations.

schemaVersion is currently 1. A purely additive field (like git and counts were, historically) does not bump it; an incompatible shape change would.

Real example (PASS)

{
  "schemaVersion": 1,
  "toolVersion": "0.1.1",
  "targetProtocol": 28,
  "project": { "name": "Protocol-Canary", "type": "unknown" },
  "network": { "name": "testnet", "observedProtocol": 28 },
  "status": "pass",
  "counts": { "total": 5, "passed": 5, "failed": 0, "warnings": 0, "errors": 0, "skipped": 0 },
  "results": [
    { "testId": "p28-xdr-cap83-empty-tx-set", "protocol": 28, "surface": "xdr", "status": "pass", "summary": "StellarValue round-tripped byte-for-byte", "durationMs": 0, "fixtureId": "p28-xdr-cap83-empty-tx-set" }
  ],
  "git": { "commit": "ffa072cb14c682bfa5ebf158ab9bbe6058962aef", "branch": "main", "isDirty": false }
}

(Full five-result example in Your First Check.)

Field reference

Backward compatibility around counts

counts and git were added after schemaVersion 1 was first shipped, as purely additive fields. A consumer reading an older report that predates counts should derive it from results[].status rather than assume the field exists — this is exactly what ProtocolCanary-Action does (see its tests/unit/output.test.ts), so it works against both the 0.1.0 and 0.1.1 Protocol-Canary releases.

Real example (execution error)

Status: error

"results": [
  { "testId": "p28-rpc-network", "surface": "rpc", "status": "error",
    "summary": "failed to call GetNetwork",
    "details": "network transport error calling getNetwork: error sending request for url (...)" }
]

network.observedProtocol is correctly absent here — the live call never returned a value — while network.name is still present.

What a downstream consumer should key off of

• Overall pass/fail: status, not the process exit code, if you’re parsing a saved file rather than checking $? directly — remember error takes precedence.
• Per-check detail (a PR comment, an annotation): iterate results, using surface to group and status/summary/details for content.
• “N/M passed” summaries: counts.passed / counts.total directly — never compute a percentage without stating the denominator.
• Protocol mismatch detection: compare targetProtocol to network.observedProtocol when both are present.

Reading a saved report back

stellar-canary report result.json --format markdown

parses exactly this shape and re-renders it without touching the network — see report.

GitHub Action

StellarCanary/ProtocolCanary-Action is the official GitHub Actions integration. It is a thin wrapper around the real stellar-canary CLI — it does not know what a CAP means, does not reimplement XDR/RPC/Soroban testing, and never reinterprets a compatibility result the CLI didn’t report.

What it does

1. Installs the requested Protocol-Canary version (from source, pinned to an immutable commit — see Installation & integrity).
2. Runs stellar-canary check --format json with your inputs.
3. Publishes a GitHub job summary and (optionally) annotations from that JSON.
4. Optionally uploads the JSON report as a workflow artifact.
5. Passes or fails the job according to Canary’s own exit code — never a result the Action computed itself.

What it does not do

• It does not reimplement any compatibility check itself.
• It does not require a private key, and never submits a transaction — same guarantee as the CLI it wraps.
• It does not use Docker, a paid service, or a hosted backend.
• There is deliberately no format input: the Action always requests --format json (the only way it can build the summary and annotations), and never invokes Canary twice.

Quick start

- uses: actions/checkout@v4
- uses: StellarCanary/ProtocolCanary-Action@v1
  with:
    protocol: "28"

@v1 is the published, floating major-version tag — the documented consumer path. (uses: ./ only appears inside this Action’s own internal integration test workflow; it is not something a consumer should write.)

Inputs

Outputs

Fixtures checkout

The Action does not read fixture files itself — it only forwards --fixtures-dir to the CLI. To check against the real Protocol 28 pack, check it out as a separate step and point fixtures-dir at it:

- uses: actions/checkout@v4
- uses: actions/checkout@v4
  with:
    repository: StellarCanary/ProtocolCanary-Fixtures
    path: canary-fixtures
- uses: StellarCanary/ProtocolCanary-Action@v1
  with:
    protocol: "28"
    fixtures-dir: canary-fixtures/protocol-28
    network: testnet
    rpc-url: https://soroban-testnet.stellar.org

Installation & integrity

Protocol-Canary does not yet publish prebuilt release binaries or checksums (see Releases). This Action installs it with cargo install --git, pinned to the immutable commit the requested version’s tag resolved to at run time (falling back to the tag itself, with a visible warning, only if that resolution fails). This requires a Rust/Cargo toolchain on the runner — GitHub-hosted Ubuntu runners include one by default. A successful build is cached (best-effort, via actions/cache; never required for correctness).

Summary, annotations, and artifact

• Job summary — a per-surface pass/fail table rendered from the JSON report.
• Annotations (when annotations: true, the default) — each failing, warning, or errored fixture becomes a GitHub Actions annotation pointing at the specific testId.
• Artifact (when upload-report: true, the default) — the raw JSON report is uploaded as a workflow artifact named stellar-protocol-canary-report. Artifact upload is always auxiliary: if it fails, the underlying compatibility result is unaffected, and a warning is logged rather than the job failing on that account alone.

Failure behavior

See CI Workflows & Failure Behavior for the full distinction between a compatibility failure and an execution failure, and what each looks like in a real workflow run.

Timeout behavior

timeout-minutes (default 15) bounds how long the Canary process is allowed to run. If it is exceeded, the process is terminated and the Action reports status: execution-failed with a timeout diagnostic — never a fabricated compatibility result.

Supported Canary versions

This table grows as Protocol-Canary cuts new releases; a schemaVersion change to its JSON report would be a breaking change for this Action and would be called out here explicitly.

CI Workflows & Failure Behavior

A complete, minimal workflow

This is a real workflow, taken directly from ProtocolCanary-Action’s examples/protocol-28.yml — checking a project against the real Protocol 28 fixture pack, against a live Testnet RPC endpoint:

name: Stellar Compatibility (Protocol 28)

on:
  pull_request:

permissions:
  contents: read

jobs:
  compatibility:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout project
        uses: actions/checkout@v4

      - name: Checkout Protocol 28 fixtures
        uses: actions/checkout@v4
        with:
          repository: StellarCanary/ProtocolCanary-Fixtures
          path: canary-fixtures

      - name: Stellar Protocol Canary
        uses: StellarCanary/ProtocolCanary-Action@v1
        with:
          protocol: "28"
          fixtures-dir: canary-fixtures/protocol-28
          network: testnet
          rpc-url: https://soroban-testnet.stellar.org

permissions: contents: read is sufficient — the Action does not need write access to your repository to check compatibility. A project with no Soroban contracts, for example, can set [tests] soroban = false in its own .stellar-canary.toml without this workflow needing to change.

Using the Action’s outputs

- name: Stellar Protocol Canary
  id: canary
  uses: StellarCanary/ProtocolCanary-Action@v1
  with:
    protocol: "28"
    upload-report: "true"

- name: Show result
  if: always()
  run: |
    echo "status: ${{ steps.canary.outputs.status }}"
    echo "passed: ${{ steps.canary.outputs.passed }}"
    echo "failures: ${{ steps.canary.outputs.failures }}"

(From examples/pull-request.yml.)

Two different kinds of “red”

The Action distinguishes a compatibility failure from an execution failure — they are not the same thing, and it never conflates them:

A third, separate case — the job summary itself failing to publish — is reported as “Failed to publish Canary summary,” distinct from both.

What a compatibility failure looks like

When a fixture’s assertion genuinely fails, the underlying CLI exits 1 (see Exit Codes), and:

• The job fails — the workflow step reports non-zero.
• The job summary shows the per-surface pass/fail table with the specific failing testId(s).
• A GitHub annotation is created pointing at the failing fixture (when annotations: true, the default).
• The artifact (stellar-protocol-canary-report, when upload-report: true) still uploads — a compatibility failure is a real result, not an upload problem, so the report is preserved for inspection either way.

This has been verified end-to-end against the real, published @v1 Action: a deliberate FAIL run produced a failed job, an annotation naming the exact failing fixture, status: fail, and the JSON artifact matching the documented schema — the same pipeline as a PASS run, just with a different, honestly reported result.

Network vs. compatibility failures

A failure to reach the configured RPC endpoint is not a compatibility failure — it is an execution problem. status is error (not fail), the affected fixtures report "status": "error" with a transport-error message, and any offline (XDR) fixtures in the same run are unaffected and still report their real pass/fail result. See Network Troubleshooting for how to tell the two apart when triaging a red CI job.

Troubleshooting

stellar-canary: command not found

Symptom. Shell reports the binary doesn’t exist after cargo install.

Likely cause. ~/.cargo/bin isn’t on your PATH, or the install targeted a different Cargo home.

What to check. cargo install --list | grep canary-cli and echo $PATH.

Fix. Add ~/.cargo/bin (or your $CARGO_HOME/bin) to PATH, or invoke the binary by its full path.

Wrong version reported

Symptom. stellar-canary version prints something other than 0.1.1, or behavior doesn’t match this documentation.

Likely cause. An older install (e.g. 0.1.0) is still first on PATH, or cargo install was run without --tag v0.1.1.

What to check. stellar-canary version and which stellar-canary.

Fix. Reinstall with the exact tag: cargo install --git https://github.com/StellarCanary/Protocol-Canary --tag v0.1.1 --locked --force. 0.1.0 predates ContractExecutable XDR support two current Protocol 28 fixtures require, and its report predates the counts field — see Releases.

Fixture directory not found

Symptom. check/fixtures reports 0/0 applicable checks when you expected fixtures to run.

Likely cause. --fixtures-dir is missing, misspelled, or relative to the wrong working directory — a nonexistent --fixtures-dir is treated as zero fixtures, not an error, by design.

What to check. stellar-canary fixtures --fixtures-dir <path> --protocol <N> first, to see exactly what would be loaded, before running check.

Fix. Correct the path, or confirm you’re in the directory you think you’re in.

Malformed .stellar-canary.toml

Symptom. check exits with code 2 and a failed to parse configuration file error, for example with protocol = "28" (a string instead of a number):

error: configuration error: failed to parse configuration file .stellar-canary.toml: TOML parse error at line 2, column 12
  |
2 | protocol = "28"
  |            ^^^^
invalid type: string "28", expected u32

Likely cause. The config file exists but isn’t valid for the schema: a TOML syntax error (an unclosed quote or [table header), a value of the wrong type, or a missing required field (version and protocol are required). This applies both to a file passed with --config and to a .stellar-canary.toml picked up from the project root.

What to check. The line and column in the error; the caret marks the offending text.

Fix. Correct that line and rerun. Nothing runs until the file parses.

Unsupported config version

Symptom. check exits with code 2 and:

error: configuration error: unsupported configuration version 2 in .stellar-canary.toml: this build supports version 1

Likely cause. The file’s version field doesn’t match the config schema version (SUPPORTED_CONFIG_VERSION) of the installed CLI. This is typically a config written for a newer or older Protocol Canary release than the one you have installed. 0.1.1 supports version 1 only.

What to check. The version line in the config file, and stellar-canary version.

Fix. Set version = 1 if the rest of the file matches this release’s schema, or install the Protocol Canary release the config was written for.

Malformed fixture

Symptom. check/fixtures exits with code 4 and a TOML parse error or “missing field” message.

Likely cause. A required field is missing (every fixture needs id, protocol, surface, category, description), a surface-specific required field is missing (see Fixtures), or the TOML itself doesn’t parse.

What to check. The exact file path in the error message, and the schema table in Fixtures.

Fix. Correct the file. No fixtures from that load run until it’s fixed — this is deliberate, not a partial-load bug.

Protocol mismatch

Symptom. A fixture you expect to run is missing from stellar-canary fixtures output, or shows up in skipped in the JSON report.

Likely cause. The fixture’s declared protocol doesn’t match the run’s target protocol (--protocol, or .stellar-canary.toml’s protocol field, or the default of 28).

What to check. skipped[].reason in --json output, e.g. "fixture targets protocol 27, this run targets protocol 28".

Fix. Pass --protocol matching the fixture, or confirm you meant to target a different protocol than your fixture pack provides.

RPC unreachable

Symptom. check exits 3, status is error, and RPC/Soroban fixtures report a transport error.

Likely cause. The configured --rpc-url is wrong, the endpoint is down, or there’s no network access from the current machine/runner.

What to check. The exact error text (e.g. error sending request for url (...)), and whether XDR-only fixtures in the same run still passed (they should — see Network Troubleshooting below).

Fix. Correct --rpc-url/--network, confirm connectivity, or disable rpc/soroban checks in .stellar-canary.toml if you only need the offline XDR surface.

Soroban simulation failure

Symptom. A Soroban fixture reports fail or error.

Likely cause. Either a genuine simulation-behavior mismatch (fail — the fixture’s expect.kind didn’t match what simulateTransaction actually returned), or a transport/execution problem reaching the network (error).

What to check. results[].status (fail vs. error) and results[].details in the JSON report — they are reported differently on purpose; see JSON Report.

Fix. For fail: re-verify the fixture’s expected behavior against the real network and its own header comment. For error: treat it as a network problem — see RPC unreachable above.

GitHub Action failure

Symptom. The workflow step running ProtocolCanary-Action fails.

Likely cause. Either a real compatibility failure (status: fail) or an execution failure (status: execution-failed — Canary couldn’t install, run, or produce parseable output).

What to check. The job summary text: a compatibility failure names the specific failing testId(s); an execution failure says “Protocol Canary could not be executed” with the actual diagnostic. See CI Workflows & Failure Behavior.

Fix. For a compatibility failure, fix the underlying incompatibility. For an execution failure, check the diagnostic — commonly a missing Rust toolchain issue on a non-standard runner, a bad version input, or a timeout-minutes that’s too short for the runner’s cold-cache build time.

Action version/reference problem

Symptom. The Action can’t be resolved, or installs an unexpected Protocol-Canary version.

Likely cause. uses: references a nonexistent tag, or the version input doesn’t match a real Protocol-Canary release tag.

What to check. Releases for the currently verified combination (Action @v1 installing Protocol-Canary 0.1.1).

Fix. Use uses: StellarCanary/ProtocolCanary-Action@v1 and either omit version (defaults to 0.1.1) or set it to a real release tag without a leading v.

JSON parsing problem

Symptom. A script consuming --json output fails to parse a field it expected.

Likely cause. Assuming a field is always present when it’s actually conditional — network (absent for offline runs), skipped (absent when empty), or results[].details (absent, not null, when there’s nothing to add).

What to check. The “Always present?” column in JSON Report.

Fix. Guard for absence rather than assuming presence; treat “missing counts” as “derive it from results” rather than an error, matching what the Action itself does.

Exit code interpretation

See the dedicated Exit Codes reference — do not guess at a code’s meaning from a partial memory of it; the source (crates/canary-core/src/errors.rs) and that page are the two places to check.

Network Troubleshooting

Some compatibility checks (RPC, Soroban) genuinely require a live, reachable Stellar RPC endpoint — Protocol Canary does not claim to be fully offline; only the XDR surface is. This means a failure to reach RPC is a different kind of result than a real protocol compatibility failure, and the tool reports them differently on purpose:

Do not treat every red CI run or non-zero exit code as “Protocol Canary found an incompatibility” — check status/exit code first to know which kind of red you’re looking at.

Security

This page summarizes the security posture documented across all three repositories’ own SECURITY.md files, which remain authoritative: Protocol-Canary, ProtocolCanary-Fixtures, ProtocolCanary-Action.

What Protocol Canary executes

• The CLI decodes/encodes XDR against the official stellar-xdr crate, makes read-only RPC calls (getNetwork, getLatestLedger), and builds/simulates unsigned Soroban transactions (simulateTransaction).
• Fixtures are declarative TOML data, never executable code. No field in the fixture schema runs a shell command, script, or arbitrary code, and none will be added — a pull request introducing one is rejected regardless of stated purpose.
• The Action never executes a shell string built from fixture data, repository configuration, PR body/title, issue text, or commit messages. Arguments are always passed to the Canary process as an array (child_process.spawn(binary, [...args])), never through a shell.

What it does not execute

• No transaction submission, anywhere in the system. Every network interaction is read-only or simulation-only.
• No arbitrary code from a fixture file.
• No downloaded script executed by the Action — the toolchain performing the build is cargo, already present on the runner.

Network behavior

• Only outbound, read/simulate calls to a configured Stellar RPC endpoint (Testnet by default). Nothing in a normal check run mutates the target network.
• A failure to reach the endpoint is reported as an execution error (exit code 3), never silently ignored and never misreported as a compatibility result.

Private-key behavior

No component in this system ever reads, stores, or transmits a private key, seed phrase, or any other signing credential. The Soroban surface builds and simulates unsigned transactions only. This has been verified directly against the source: the only std::env calls in Protocol-Canary are temp_dir()/current_dir() — there is no key material code path.

Fixture trust boundary

ProtocolCanary-Fixtures fixtures — and any third-party fixture directory — must be treated as untrusted input by any consumer, including Protocol-Canary itself. The schema has no executable field. decode-failure fixtures deliberately feed malformed input to the XDR decoder to prove it rejects bad input correctly, rather than crashing or silently accepting it.

Action trust boundary

Pull request source code is treated as untrusted by the Action. It never interpolates PR-controlled text into a shell command, and it enforces a configurable timeout (timeout-minutes, default 15) that forwards workflow cancellation signals to the child process, so a hung or malicious process cannot outlive the job.

The Action’s minimum required permission is contents: read; it does not call the GitHub API on your repository at all for its core function (it does call the public, unauthenticated GitHub REST API against StellarCanary/Protocol-Canary itself, to resolve a release tag to a commit before installing it).

Report handling

The JSON report the CLI produces is plain data (see JSON Report). The Action parses it and never re-executes anything based on its content beyond deciding pass/fail and rendering text.

Reporting a vulnerability

Each repository documents the same private path: open a private report via GitHub’s “Report a vulnerability” feature on that repository, or contact the maintainers directly, rather than filing a public issue. Include a description of the issue and its impact, steps to reproduce, and the affected version or commit. Reports are acknowledged and worked through a fix and disclosure timeline before any public write-up.

Audit status

No formal third-party security audit has been performed on any of the three repositories. Confidence in the claims above comes from the design itself (no key-material code path exists) and each repository’s own automated test suite — not from an external review. This project does not claim, and has never claimed, any security certification.

Contributing

This page is a starting point across all three repositories. Each repository’s own CONTRIBUTING.md remains authoritative for its repository-specific process — this page does not duplicate them, only orients you toward the right one.

Three contribution paths

Core engine — Protocol-Canary

Where to work: crates/canary-* (the CLI, project detection, fixture loading, the XDR/RPC/Soroban runners, policy evaluation, reporting).

What belongs here: changes to how compatibility checks are executed or reported — a new surface capability, a new report format, a bug in project detection, a new configuration option. Protocol-specific assertions belong in ProtocolCanary-Fixtures instead, not here.

How to test:

cargo fmt --all --check
cargo check --workspace --all-targets
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings

All four must pass. Unit tests must not require network access; anything that talks to a real RPC endpoint belongs in tests/integration and must be explicitly opt-in.

How to submit: a pull request against StellarCanary/Protocol-Canary, following its CONTRIBUTING.md (workspace structure, coding standards, and commit style).

Fixtures — ProtocolCanary-Fixtures

Where to work: a protocol-pack directory (e.g. protocol-28/), organized by surface and CAP.

What belongs here: a new compatibility assertion for a real, verifiable piece of protocol behavior — never a fixture added just to raise a coverage number. See Authoring a Fixture for the full workflow.

How to test:

python3 tools/validate/validate.py
python3 -m unittest discover tests

No build system is required — the validator uses only the Python 3.11+ standard library.

How to submit: a pull request against StellarCanary/ProtocolCanary-Fixtures, following its CONTRIBUTING.md.

GitHub Action — ProtocolCanary-Action

Where to work: src/ (TypeScript), bundled to the committed dist/index.js.

What belongs here: changes to how CI consumes Canary’s report — the summary, annotations, artifact handling, installation/pinning logic. Compatibility logic itself belongs in Protocol-Canary, not here.

How to test:

npm run typecheck
npm run lint
npm test
npm run build

Unit and integration tests never require network access or a real stellar-canary binary — they run against a mock CLI (tests/fixtures/mock-canary.cjs) that simulates every documented result state via the MOCK_CANARY_SCENARIO environment variable. The repository’s own integration.yml workflow is the only place it talks to a real Canary build and a real Testnet endpoint, and it never gates a pull request.

How to submit: a pull request against StellarCanary/ProtocolCanary-Action, following its CONTRIBUTING.md.

Contributing to this documentation site

The site itself lives at docs-site/ in the Protocol-Canary repository. See docs-site/README.md for how to build and preview it locally with mdBook. Keep implementation detail in each repository’s own docs//CONTRIBUTING.md and link to it from here rather than duplicating it — this site is the developer journey, not a second source of truth.

Roadmap

This reflects Protocol-Canary’s own ROADMAP.md. It is practical, scoped work that fits the current architecture — not a schedule with committed dates. Items move up when someone (maintainer or contributor) picks them up.

Near term

• Wire CacheStore into check. The local, file-backed result cache in canary-core is implemented and unit-tested but not yet used by check’s execution path — every run currently calls RPC/Soroban fresh. Useful once fixture counts or CI frequency make rate limits a concern.
• Additional Protocol 28 fixtures. CAP-0086 sparse-map host functions have no fixture yet — the currently published Soroban SDK surface does not expose the host functionality the intended fixture would need. See Protocol 28. Track upstream SDK releases and add the fixture (in ProtocolCanary-Fixtures) once that changes.
• More RPC compatibility assertions. Only getNetwork has a fixture today; getLatestLedger is supported by canary-rpc but unused by any shipped fixture.

Medium term

• Protocol 29 support, once its CAPs are finalized and released upstream — a new protocol pack entry, plus fixtures in ProtocolCanary-Fixtures, following the same process as Protocol 28. No release currently supports Protocol 29 or any version beyond 28 — do not infer support from this roadmap entry.
• Expanded Soroban compatibility assertions beyond the current construct → simulate → result smoke fixture, covering more host functions as they stabilize.
• Additional CI integrations beyond GitHub Actions, if a real need arises, scoped as their own repository rather than bolted onto ProtocolCanary-Action.

Explicitly out of scope for now

• A hosted backend, database, or web frontend — this is a CLI and CI tool.
• Remote/versioned fixture fetching (as opposed to a local checkout).
• Transaction submission of any kind.

How to propose an addition

Open an issue describing the real upstream behavior (a CAP, an RPC method, a host function) the work would cover. See Contributing.

Releases

Protocol-Canary

ProtocolCanary-Action

ProtocolCanary-Fixtures

Version compatibility

This is the only combination that has actually been run together and verified end-to-end (5/5 PASS against live Testnet). v0.1.0 is installable but predates ContractExecutable XDR support (needed by two current Protocol 28 fixtures) and the counts field in its JSON report — the Action tolerates the missing field, but v0.1.1 is the version this table verifies against.

Tags vs. releases vs. binaries — what actually exists

Do not assume a Protocol-Canary binary download exists anywhere — every documented install path in Installation builds from source.

Limitations

Stated plainly, not hidden:

• Current fixture coverage is not complete Protocol 28 coverage. Five fixtures exist today, across three CAPs’ surfaces. See Protocol 28 for exactly what is and isn’t checked.
• CAP-0086 currently lacks a fixture. This is a documented, deliberate gap — the published soroban-sdk doesn’t yet expose the relevant host functions at a stable API surface, and this project does not fabricate a fixture it cannot independently verify. See Roadmap.
• Protocol Canary does not certify arbitrary applications. A passing run means the fixtures currently implemented for the target protocol passed against your configured dependencies and RPC endpoint — it is not a guarantee that an application cannot break in some other way, and it does not replace testing against a real testnet or mainnet deployment.
• Live RPC checks depend on network availability. The RPC and Soroban surfaces are not offline — see Network Troubleshooting. Only the XDR surface runs with no network call.
• The tool does not submit transactions as part of normal Soroban simulation checks. Every Soroban check builds and simulates an unsigned transaction; nothing in a normal check run mutates the target network.
• The current release does not publish prebuilt binaries. Every documented install path builds from source via cargo install. See Releases.
• CacheStore exists but is not currently wired into the primary check flow. The type is implemented and unit-tested in canary-core, but check calls RPC/Soroban fresh on every run rather than reusing a prior cached result. See Architecture.
• No formal third-party security audit has been performed on any of the three repositories. See Security.
• Mainnet has not been separately verified. Every fixture in the current pack was verified against Testnet only; running against mainnet requires explicit --network mainnet --rpc-url <endpoint> configuration.
• Only Protocol 28 is supported. There is no Protocol 29 (or earlier, beyond an unpopulated protocol-27/ placeholder pack) support in any released version.

FAQ

Why not just use unit tests?

Unit tests primarily exercise your application code against mocked or assumed protocol behavior. They don’t, by themselves, test the boundary between your code and the real, current behavior of the Stellar network and its official libraries — which is exactly what changes when a protocol upgrades. See What Problem Does It Solve?.

Does Canary send transactions?

No. Every Soroban check builds and simulates an unsigned transaction via simulateTransaction. Nothing in a normal check run submits a transaction to any network.

Does Canary need private keys?

No. No component in this system ever reads, stores, or transmits a private key or seed phrase — verified directly against the source. See Security.

Does Canary require Docker?

No. It’s a Rust CLI you cargo install, plus a Node-based GitHub Action for CI. Neither requires Docker.

Does Canary require a server?

No. There is no hosted backend, no database, and no persistent server anywhere in this system. See Architecture.

What happens if RPC is unavailable?

The affected checks report an execution error (status: error, exit code 3), not a compatibility failure. Offline (XDR) checks in the same run are unaffected and still report their real result. See Network Troubleshooting.

Does 5/5 mean full Protocol 28 support?

No. It means the five fixtures currently implemented for Protocol 28 passed. CAP-0086 has no fixture yet, and CAP-0085’s fixtures test wire encoding, not a full deployed contract fleet. See Protocol 28 for exactly what is and isn’t covered.

Why are fixtures separate from the engine?

So fixtures — public, declarative TOML data consumed by CI pipelines — stay a clean trust boundary: no field executes code, and the fixture pack can version independently of the CLI. See Fixtures.

How do I use Canary in GitHub Actions?

- uses: actions/checkout@v4
- uses: StellarCanary/ProtocolCanary-Action@v1
  with:
    protocol: "28"

See GitHub Action for inputs, outputs, and a complete workflow checking the real Protocol 28 fixture pack.

How do I add support for another protocol behavior?

If it’s a new compatibility assertion for an existing surface, add a fixture — see Authoring a Fixture. If it requires new engine capability (a new surface, a new XDR type the CLI doesn’t yet decode), that’s a change to Protocol-Canary itself — see Contributing. Either way, it must cite a real, verifiable upstream source — this project does not add speculative or guessed behavior.