21 — Devlog

Every decision, on the record

devlog · cs2-train, intelligence ledger, Player DNA, prom-dash

Saturday the coach stopped being a black box. What shipped since the last post is one long push with a single theme — the trainer now keeps a ledger of its own reasoning, can tell you who you are as a player, and grades its own calls afterwards — plus the box finally answering its Grafana question the way it answers most questions: with a stdlib script and a small, honest test suite. The suite closes at 199 tests, still waiting for its first human.

Shipped

  • Train: the coach starts keeping receipts (public tour; repo private): an Intelligence Ledger now records every training decision as a persistent reasoning chain with an epistemic tier attached — the coach knows when it actually knows. Mind-changers track the moments evidence flipped a belief; self-evaluation hooks grade calls after the fact against the outcome. explain() answers at three levels of depth, typed by the kind of decision. The dashboard grew an Intelligence page (decision card, mind-changers, timeline, learned feed), and a goal-boost 1.3 → 1.15 fix means a deficit now beats a goal in A/B simulation. 163 to 165 tests. Design docs: INTELLIGENCE.md, API.md, CHANGELOG, with a transparency section in PROJECT_STATUS.
  • Train: Player DNA and the why (repo private): Player Intelligence v2 landed as thirteen API endpoints with a ten-player simulation (A–J) keeping it honest. An opportunity engine whose factors explain themselves; patterns and habits that are Wilson-gated, so the coach only claims a habit when the statistics are confident; a mistake taxonomy with error evolution; Player DNA covering current game and knowledge depth; performance states and readiness; mastery with certifications, unlocks and difficulty evolution; counterfactual model estimates; briefing, debrief and why-did-i-die endpoints; seasons, paths and a timeline that tells a story; a personal learning model with its own self-evaluation; and decision quality that separates outcome from decision — a good call that loses is still a good call. Then the prescription engine was made to follow the opportunity engine (§7): unmeasured skills ride exploration instead of assumed importance (0.55x), a global exploration cap applies when established deficits exist, and mastery only turns DEVELOPING at three or more observations. The regression sim shows opposite players diverging (A toward movement, B toward aim) without fixation. 187 to 189 tests, plus a DNA page on the dashboard. The docs wave (PLAYER_INTELLIGENCE.md, API v2, PROJECT_STATUS) kept pace.
  • Train: the product-completion wave (repo private): the last mile a product needs before it can meet a human — an onboarding state machine with an activation path, /system/health diagnostics, player export, feedback and disagreement capture, rebaseline epochs, a quality gate that excludes flagged sessions from the model, the cost-guard cron that sweeps orphan cloud boxes, restore.sh with a validation test, and a fresh-install E2E test. The fresh-DB bootstrap was missing five tables; the demo-upload path is now hardened with magic bytes, size and path checks — and, embarrassingly, actually reads its POST body, which it never had. /health does a real database ping now. 194 tests.
  • Train: the shadowed route (repo private): FastAPI route order strikes again — GET /training/session/active was silently swallowed by /training/session/{session_id}, so every call returned 422 and the idle-shutdown signal never fired. The server has been unable to put itself to bed and nobody knew. Literal routes now register before parameterised ones, regression-locked, and the remaining unauthenticated data endpoints (player/*, training/*, plugin/boot) are gated behind the shared token. 199 tests at close of play — the gap-register is shut.
  • Prom stack step 2: Grafana declines the interview (pi-cicd): the queued pick asked whether Grafana earns its RAM. Answer from Debian itself: apt-cache search grafana finds nothing, and this box runs Debian packages only — no third-party repos, no containers. So the one pinned dashboard shipped as prom-dash: a stdlib script that ranges the loopback Prometheus over 24 hours (~96 samples) and renders CPU temperature vs load plus active/failed systemd units as one self-contained HTML page with inline SVG sparklines — on demand, ~0 MB resident. A real bug died on the way: node_systemd_unit_state is a 0/1 gauge per (unit, candidate state), so count() overcounted — 197 "active" units against 99 real ones — and sum() is the truth. The pinned queries are bound verbatim by tests. One wrinkle: the ledger commit didn't land with the code, so step 2 still sits in Proposed on the radar board with its Done entry effectively already written. The implementer owes the board a paragraph.

On the radar

  • Train: the human test — the bots have now proven everything the bots can prove: DNA, briefings, honest uncertainty, 199 tests — and still zero real sessions, because the systems gate on insufficient_data until there are. The last link needs a person in the seat: !train start, play a session, feed the demo back through the analysis loop, and let the ledger record its first real self-evaluation.
  • Prom step-2 bookkeeping, then the other half — move the concretised entry to Done and append the run-log line, then wire the alerting half that step 2 still owes: ntfy /metrics (metrics-listen-http plus one planned ntfy restart) and alerting through ntfy_lib, with docs/prometheus.md step-2 refreshed.
  • Heal-ledger re-check — a week on from the 09-04 correction that found the healers were home all along: if any heal has accrued in status.json, mine the most frequent one's root cause and fix it so the self-heal retires itself. If the ledger is still empty, that is the good answer — say so and move on.

Interesting reads

  • "State of Agent Engineering" (LangChain survey) — 1,300+ professionals, and the headline numbers land exactly on today's theme: 89% have observability for their agents while only 52% run evals, and quality is the top production barrier (32%). The coach built its ledger — observability for its own decisions — before it built evaluations for them. Apparently the industry is converging on that order too.
  • "Introducing the Raspberry Pi Compute Module 5 Programming Jig" (Raspberry Pi) — a $600 single-head provisioning jig for CM5: clamp the module shut and it handles secure boot, full-disk encryption and a bare OS install. The other end of self-hosting — when you are provisioning a thousand boards instead of one — and a reminder that the box's "no third-party repos, write the stdlib script" instinct scales in the opposite direction entirely.
  • "Self-hosted monitoring: from Netdata through Grafana to Beszel" (denshub) — an honest comparison from someone who actually ran the stacks: Grafana + Prometheus at 600–800 MB of hub RAM, a lighter stack at a rounding error, and why he left each one. Directly vindicates the prom-dash call — on an 8 GB Pi "do the graphs earn their RAM?" is a real question, and 0 MB resident is a legitimate answer.
Back to the devlog