Blog
A predictable environment for agents
I find it easier to evaluate assisted work when the environment is explicit.
Reduce accidental variation
If the same task begins with different tools, dependencies, or working assumptions, differences in results are hard to interpret. I prefer a repeatable starting point: known tool versions, declared dependencies, and an explicit working context. This does not make a task deterministic; judgments and inputs still vary. It does remove avoidable uncertainty about whether a difference came from the work or from the surroundings.
Pinning is the concrete version of this preference. Versions live in a lockfile or a development shell, so the environment is a value someone can read. The goal is not to freeze the world; it is to make an update a visible change that can be reviewed like any other. An unpinned tool quietly updates and turns a comparison into a mystery.
The working context deserves the same treatment. Which directory is in scope, which variables are set, whether the task may reach the network: these are conditions of the work, and they should be declared before the work starts. When they are implicit, a result carries hidden assumptions that nobody can check later.
{
description = "Pinned tools for repeatable local work";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11";
outputs = { self, nixpkgs }:
let
system = "x86_64-linux";
pkgs = nixpkgs.legacyPackages.${system};
in {
devShells.${system}.default = pkgs.mkShell {
packages = [
pkgs.nodejs_22
pkgs.bun
pkgs.jq
pkgs.ripgrep
];
};
};
}Put lasting rules where they last
I separate durable guidance from task context. Long-lived instructions should describe stable boundaries and verification habits; a request should supply the current goal, scope, and exceptional constraints. If I bury a permanent rule in a one-off conversation, the next attempt may miss it. If I turn every passing preference into a permanent rule, guidance becomes noisy. The test is whether someone starting fresh could find the rules that still apply.
I ask what a fresh start would need to know. Boundaries that always apply, the commands that verify a change, and the habits that keep work reviewable belong in durable guidance. The current goal, the files in scope, and the exceptions belong to the request. Mixing them means the next task inherits yesterday's constraints.
Guidance also needs pruning. Rules accumulate, and an old preference that no longer matches the project turns into noise that hides the rules that still matter. I reread the durable notes when the work changes shape and remove what has stopped being true, because a stale rule is worse than no rule.
{
"version": 1,
"checks": [
{
"name": "typecheck",
"argv": ["bun", "run", "typecheck"],
"profiles": ["fast"]
},
{
"name": "build",
"argv": ["bun", "run", "build"],
"profiles": ["full"]
}
]
}Inspect the handoff
I would try the documented setup from a clean starting state and see whether the same tools, dependencies, and lasting guidance are available without relying on my earlier conversation. A reproducible setup does not certify the quality of a decision; it makes the conditions behind that decision easier to inspect. When a step depends on unstated local knowledge, I would document the dependency or remove it before trusting repeated execution.
My check is deliberately boring: start from a fresh checkout, run the documented setup, then run the declared fast checks. If a step needs something I set up weeks ago and never wrote down, the reproduction fails and I have found the gap. If it succeeds, I still know only that the environment is repeatable, not that the work is good.
The same test applies to the lasting rules. I imagine someone joining the work today and ask whether they could find the boundaries and the verification habits without reading my old conversations. Anything they could not find is either documented or removed.
#!/usr/bin/env bash
set -euo pipefail
# Run from a fresh checkout: nothing from an earlier session may help.
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
git clone --depth 1 "$1" "$tmp/work"
cd "$tmp/work"
nix develop --command ./scripts/check.sh --fast