Blog
Separate levels of architecture evidence
I try to say what kind of evidence supports an architectural claim before saying it works.
A specification states intent
A diagram or written contract helps me reason about responsibilities and expected boundaries. It is valuable because it can expose contradictions before implementation, not because it proves that implementation follows it. I keep intended behavior separate from observed behavior. When I review a proposal, I ask which assumptions must hold and which parts of the design are still decisions rather than facts.
Writing the intent down has a concrete benefit: it separates decisions from assumptions. A responsibility diagram can name who owns a queue, who may retry, and what happens to a message that fails twice. Those are choices I can disagree with before code exists, which is much cheaper than discovering them in review.
The same document ages. When implementation and description disagree, the difference is a question rather than proof that the code is wrong; the description may simply be older. I keep the note next to the change that alters it, so a later reader can see which statement was meant to hold at which point.
claims:
- statement: "a malformed request is rejected"
level: exercised
command: "bun test src/requests"
- statement: "the packaged service starts with the documented config"
level: observed
command: "./scripts/smoke.sh --config config/local.yaml"
- statement: "requests are routed to the owning module"
level: specified
reference: "docs/routing.md"Tests and builds answer narrower questions
A focused test can show that a selected behavior held for its chosen inputs. A successful build can show that pieces fit well enough to produce an artifact. Neither tells me that untested interactions are correct, nor that the artifact was configured or started as intended. I find it useful to label claims accordingly: specified, exercised, built. This prevents a passing check from silently becoming evidence for a different question.
A test answers the question its inputs pose, and little more. I try to make the test name state the claim, because a green suite is often read as a general assurance. If a test exercises one path through a retry policy, the label exercised fits that path; it does not fit the policy as a whole.
A build is a different kind of evidence. It shows that the pieces compose into an artifact, and a reproducible build adds that the artifact corresponds to the sources. It still says nothing about configuration or startup, which is why I keep built separate from observed even when the pipeline is green.
Operation requires its own observation
Runtime behavior depends on surrounding conditions that a design note cannot capture completely. I would verify an important claim at the level where it matters: observe the relevant behavior, record the conditions, and note what was not observed. A runtime check is not permanent proof either; conditions change. The useful habit is to keep the evidence chain visible so a later failure points to the level that needs renewed scrutiny.
When I observe the running system, I record the conditions along with the result: which revision, which configuration, which inputs, and what was left out. Without those notes, a successful observation is hard to repeat and hard to trust later; with them, someone can decide whether the same evidence still applies after the surroundings change.
The habit I want is a labeled chain. A claim that matters should point to the level of evidence that supports it, and no stronger. When something fails later, the label tells me where to look first, and it keeps a passing check from quietly standing in for a question it never asked.
#!/usr/bin/env bash
set -euo pipefail
level="${1:?usage: run-evidence.sh <level> <command>}"
shift
echo "level: $level"
echo "command: $*"
# Keep the conditions next to the result, so a passing run
# cannot be read as evidence for a wider claim.
"$@" 2>&1 | tee "evidence-$level-$(date +%Y%m%d-%H%M%S).log"