bit → pico → nibble → byte → word
The model family.
One ladder, five generative rungs plus an embedding rung. Climbing it trades throughput for accuracy — a real speed ↔ smarts dial for tiered routing.
GENERATIVE RUNGS
Route cheap low, escalate hard high.
tok/s and accuracy below are the measured chat-template-path tournament numbers (5-item classify fixture, temp 0). Native single-shot throughput is lower and the nibble and byte rungs need the chat-template wrapper — see the notes.
Cheap high-volume triage
Fastest rung. Routes simple, high-throughput classification. Not for hard reasoning.
Lightweight classify / gate
Carries the LoRA adapter roster. Good for structured gating where a smart-enough call is cheap.
Value pick (chat-template path)
Best accuracy-per-token on the chat-template path. Ships LongCat-style layer-wise KV blending natively.
Mid escalation
Scored below nibble on the 5-item fixture — treat as unproven until a >=30-item fixture ranks it. Do not route here on current evidence.
Accuracy anchor
Slowest, smartest. The default model. Escalate hard or contested items here.
EMBEDDING RUNG
Vectors for retrieval.
The embedding rung is separate from the generative ladder. It is 400-rejected on chat and completion routes by design.
Retrieval / RAG vectors
Embeddings only — 400-rejected on chat/completions. Fixed 768 dims keep vector stores shape-stable.
The honest anomaly.
beam_loader.c): a small term fits one byte, larger terms take two or more. So a beam token is dynamically sized — its byte width, and its ratio to a standard AI token, varies per response (often ~2×). The live meter reports the real count and average byte size, never a hardcoded multiplier.bit owns cheap throughput and word is the only rung reaching hard delivery. The nibble and byte rungs require the template wrapper or degenerate single-shot.