Flying Probe Co-Pilot
A synthetic PCBA test-log pipeline, SPC that runs itself, and a citation-forced RAG co-pilot that refuses to answer when it isn’t sure — local-first, open source, one DuckDB file on a laptop.
Passing tests
at 97% coverage
Live citation eval
green on frozen 10-Q set
Weeks of build
evenings + weekends
DuckDB schema
5 dim · 1 run · 3 fact
Every PCBA line emits millions of measurement records per shift. The analytics layer usually isn’t there.
Logs sit in flat files
Correlating a shift's yield drop against a specific component family takes hours instead of minutes. There is no shared analytics spine.
SPC is understood on paper, rarely automated
Wheeler XmR charts, run-of-8, zone rules — all well-defined in theory, almost never running against live test data as soon as a run lands.
Per-component outliers slip past
The kind where one part number is suddenly failing four times the line average. Nobody's watching that slice unless the analytics layer surfaces it automatically.
Root-cause knowledge lives in Slack
"Why are these 0402s tombstoning?" ends up as a message to whoever remembers, instead of a query that returns a cited answer in seconds.
A thin analytics library, a hybrid-retrieval co-pilot, and a contract that refuses to guess.
- DuckDB — 9-table schema (5 dim · 1 run · 3 fact)
- Synthetic HP3070 / i3070 log generator (Keysight LRF)
- yield_over_time · failure_pareto · individuals_chart (Wheeler XmR + run-of-8) · z_score_anomalies
- Six-page Streamlit dashboard
- ChromaDB — cosine HNSW vector index
- rank-bm25 — lexical index for exact-match refdes queries
- Reciprocal Rank Fusion at k=60 — beats tuning weighted blends
- 8-document IPC / J-STD failure-mode knowledge base
- Gemini JSON mode with strict schema
- 4-condition anti-hallucination contract
- Refusal text is a fixed constant — not model-generated
- Failing lookup costs zero tokens (no LLM call on empty retrieval)
- Frozen 10 → 15-question live citation eval, env-gated
- Offline stubbed CI test — runs on every PR at zero cost
- 12-step agentic governance workflow with adversarial red-team subagents at the plan stage
- 659 passing pytests at 97% coverage
Every phase produced exactly one bug that changed how I think about a whole class of problem.
The shift-snap that lied about time
The synthetic generator drew a shift letter uniformly, then tried to snap the timestamp into that shift's window. The overnight-wrap correction was a literal `pass` — a 02:00 draw randomly assigned to shift C would silently jump to 22:00 the same day. SPC trend analysis on that dataset produced completely fictitious shift-to-shift yield deltas. The fix wasn't patching the wrap — it was deriving shift from time-of-day so the two values could no longer disagree.
The model that got retired mid-evaluation
First live eval attempt ran two minutes thirty-one seconds and failed with a 404 — Google had retired `gemini-2.0-flash` out from under me. The fix was three lines: model constant, test mirror, env example. The re-run passed 10 / 10 in 37 seconds. The follow-up — migrating off the already-deprecated `google-generativeai` SDK to its successor — landed the same day.
The test that only broke under concurrency
A parser test passed in isolation and failed ~60% of the time in the full suite. The culprit: a test helper writing to a fixed path at the repo root. Under parallel pytest workers, Windows file-watchers, and antivirus, the shared path became a race. Threading `tmp_path` through every call gave each test its own OS-temp file. A two-thread stress harness confirmed it: 484 / 800 failures on the shared path, 0 / 800 after isolation.
// Related reading