Gateway Runtime Spike — 5-Way Evaluation Plan

Build the same thin gateway adapter against 5 candidate runtimes, score each against the 6 decision gates from docs/gateway-contract.md, and produce a recommendation for human decision. No production changes — all spikes are throwaway prototypes in isolated worktrees.

The 5 Candidates

1. pi TypeScript

earendil-works/pi

Minimal agent toolkit with RPC mode / SDK for process embedding. "Adapt pi to your workflows without forking pi internals." First-class steering semantics. TypeScript (no #71 entanglement).

2. Open Agent Loops TypeScript

featherless-ai/open-agent-loops

Provider-agnostic agent loop SDK from Featherless. Plug-and-play seams: ModelClient, Memory, Tools, StopConditions. Single zod dep. Runs in Node/Bun/Deno/browser. Very new (v1.0.0, 2 weeks old).

3. Turnstone Python

turnstonelabs/turnstone

Self-hosted, local-first orchestration. Built-in shell/files/search/web tools. Explicit no-telemetry posture. Python (matches bridge language). Closest to Nagato's privacy posture. Previously read as "not a decision" — re-evaluating.

4. Stripped Hermes Fork Fork

Hermes 0.19.0 local fork

Take the existing version-locked Hermes fork and strip it to the Nagato gateway contract. Remove unused tools, harden defaults, delete surface area. Option B from issue #30 — own repo, patches as commits, periodic upstream merges.

5. Ground-Up New

Custom Python gateway

Build a minimal gateway from scratch that implements exactly the contract in docs/gateway-contract.md. No upstream to track. Maximum control, maximum maintenance. Python to match the bridge (avoids #71 language split).

The 6 Decision Gates

From docs/gateway-contract.md — every candidate must be scored against all 6.

Scoring Matrix

? = unknown until spike. ✅ = known pass. ❌ = known fail. The spike fills every ?.

Execution Plan

Gate	pi	OAL	Turnstone	Hermes Fork	Ground-Up
1. Trace fidelity	?	?	?	✅ (current)	✅ (by def)
2. Research tooling	?	?	?	✅ (current)	?
3. Session + memory	?	?	?	✅ (current)	✅ (by def)
4. Tool governance	✅ strong	✅ strongest	?	❌ maximal	✅ (by def)
5. Local-first	✅	✅	✅	❌ bleeding-edge	✅ (by def)
6. Maintenance	?	?	?	❌ 17 patches	all ours

Phase 0: Shared Adapter Contract (1 day)

Before touching any candidate, extract the thin adapter interface from docs/gateway-contract.md. This is the single contract every spike must implement.

File: spikes/shared/gateway_adapter.py (Python) or spikes/shared/gateway_adapter.ts (TypeScript)
Interface: start_run(session_id, prompt, tools) -> run_id, get_trace_events(run_id) -> [TraceEvent], steer(run_id, input), interrupt(run_id, input), get_artifact(artifact_id) -> bytes
Test fixture: 3 canned runs - (1) simple chat, (2) tool call with GTC mock, (3) multi-turn with steering. Same fixture for all 5 candidates.
Score sheet: spikes/shared/score_sheet.md - one row per gate per candidate, filled during spikes

Phase 1: pi Spike (2 days)

TypeScript earendil-works/pi

Worktree: nagato-game-assist/.worktrees/spike-pi
Clone: git clone https://github.com/earendil-works/pi.git ~/pi-spike
Adapter: spikes/pi/adapter.ts - implement the shared interface using pi RPC SDK
Test: Run the 3 canned fixtures through the pi adapter, verify trace events match the schema
Score: Fill all 6 gates for pi
Key question: Does pi RPC mode let us embed without forking? Can it emit trace events in our schema?

Phase 2: Open Agent Loops Spike (2 days)

TypeScript featherless-ai/open-agent-loops

Worktree: nagato-game-assist/.worktrees/spike-oal
Clone: git clone https://github.com/featherless-ai/open-agent-loops.git ~/oal-spike
Adapter: spikes/oal/adapter.ts - implement using OAL Agent/Dispatcher/ChannelBridge
Test: Same 3 fixtures
Score: Fill all 6 gates for OAL
Key question: Can OAL onEvent stream be shaped into our trace-event schema? Is it mature enough?

Phase 3: Turnstone Spike (2 days)

Python turnstonelabs/turnstone

Worktree: nagato-game-assist/.worktrees/spike-turnstone
Clone: git clone https://github.com/turnstonelabs/turnstone.git ~/turnstone-spike
Adapter: spikes/turnstone/adapter.py - implement using Turnstone orchestration API
Test: Same 3 fixtures
Score: Fill all 6 gates for Turnstone
Key question: Do Turnstone built-in tools overlap with GTC? Can it emit trace events? Does Python help (no #71 split)?

Phase 4: Stripped Hermes Fork (2 days)

Fork Hermes 0.19.0 local fork at head 86892b5c5

Worktree: nagato-game-assist/.worktrees/spike-hermes-stripped
Base: ~/hermes-agent-local-fork at nagato/hermes-0.19 branch
Task: Strip unused tools, profiles, platforms. Keep only: Runs API, chat completions, SSE trace stream, artifact fetch/verify, stop, session keys, memory context injection, GTC tool allowlist
Measure: Lines removed, lines remaining, patch count change
Score: Fill all 6 gates - this is the baseline
Key question: How much surface can we remove before things break? Does stripping reduce the 17-patch maintenance cost?

Phase 5: Ground-Up (3 days)

New Custom Python gateway

Worktree: nagato-game-assist/.worktrees/spike-ground-up
Build: spikes/ground-up/gateway.py - minimal Python gateway that implements exactly docs/gateway-contract.md
Components: OpenAI-compatible chat completions client (Featherless), SSE trace emitter, session store (SQLite), artifact store (content-addressed), GTC tool router, redaction boundary
Test: Same 3 fixtures
Score: Fill all 6 gates
Key question: How many lines of code? Is it less maintenance than patching Hermes? Can it pass the bridge contract tests?

Phase 6: Comparison + Recommendation (1 day)

Compile: Fill the final scoring matrix with all 5 candidates x 6 gates
Write: spikes/REPORT.md - findings, recommendation, trade-offs
Post: Summary as comment on Forgejo issue #30
Decision gate: Human picks: stay-pinned / formal fork / migrate to X

Parallelization Strategy

Phases 1-5 can run in parallel (up to 3 concurrent subagents per the ERR_NO_BUFFER_SPACE limit). Recommended batching:

Batch A (first): Phase 4 (Hermes stripped) + Phase 5 (ground-up) - both Python, both local, no external clones needed
Batch B (second): Phase 1 (pi) + Phase 2 (OAL) - both TypeScript, both need npm/bun
Batch C (third): Phase 3 (Turnstone) - Python, needs its own venv
Phase 0 must complete first - the shared adapter contract is the test harness for all spikes
Phase 6 runs last - needs all 5 score sheets

Kill wrangler/browser/computer_use before dispatching subagents. ERR_NO_BUFFER_SPACE with 3+ subagents + wrangler SSE + browser + computer_use for 20+ min.

Phase	Duration	Dependency
0. Shared adapter	1 day	None
1. pi spike	2 days	Phase 0
2. OAL spike	2 days	Phase 0
3. Turnstone spike	2 days	Phase 0
4. Hermes stripped	2 days	Phase 0
5. Ground-up	3 days	Phase 0
6. Comparison	1 day	All above
Total (parallel)	~5 days
Total (sequential)	~13 days