Changelog
Where we are.
What is done and what comes next, as it stands. We add to this list; we don’t rewrite it.
What is done
Done
- DoneA working Inferstack gateway and CLI: session affinity, routing, cost ledger, failover, evals, metrics
- DoneAgents, tasks and background runs with per-agent budgets, in the gateway, the CLI and the Console
- DoneThe Inferstack Console, live in the browser, with Linux and Android builds from the same code
- DoneMeasured on a laptop: 87% less served cost on the second turn of a session
- DoneA market study of 8.56M public posts and issues on inference
- DoneA baseline on real agent logs: 88.5% of prompt tokens repeat
- DoneA shipped product: Clutch, a basketball recruiting platform
What comes next
Next
- NextGPU engines. The prototype runs on CPU with llama.cpp; vLLM and SGLang are next.
- NextThe model-switch handoff
- NextWindows and macOS builds of the Console, and a signed Android release
- NextPaying customers
- NextAn external benchmark. That’s what a pilot is for.