A router that scores every prompt for difficulty without calling a model

Scoring difficulty with regexes instead of a model

Wayfinder Router is a proxy that drops in front of any OpenAI-compatible client and decides, per prompt, whether to hit a cheap local model or a paid hosted one. The trick is what it refuses to do: it never calls a model to make the call. Structural scoring counts word count, headings, lists and code blocks; an optional lexical pass looks for reasoning words, math symbols and hard constraints. The result is a 0.0-1.0 complexity score computed offline in sub-millisecond time, and a default threshold of 0.5 splits cheap from expensive. Point your base_url at the gateway, set the model field to auto, and x-wayfinder-router-score headers tell you where each call landed.

The author publishes the number that justifies the whole exercise: structural scoring alone reached 61% cost savings on RouterBench with calibration. The lexical features looked better — about 20% gains on held-out hard prompts — but didn't generalize, so they're off by default. The core scoring engine carries zero external dependencies.

"You cannot just route away from a situation at such a high level"

The sharpest pushback in the thread came from people who'd already built routers and run real agents, and they converged on one failure mode: the proxy is too far from the harness to be safe.

"You cannot just route away from a situation at such a high level especially when we are talking about models that are quite different in behaviour, with different context windows and tuned to different tool uses. The harness is doing all kind of funky things to compensate for issues … that a proxy that routes dynamically like this will work against." — a CISO building his own LLM gateway at cbk.ai

The author of an agent-sandboxing tool poked at the same seam from the context angle:

"If a prompt I give routes to one model, and then another prompt to another model, how does one tie the context together such that the next model knows what's going on? … it would always land hot (not in the cache)." — the author of the yoloai agent sandbox

Then the economics. The marquee example — routing "fix my typo" to a local model — barely moves the bill, because the cheap prompts are also the small ones.

"The usecase of saving money on simple prompt.. I think that has only a slight benefit. Fix my typo doesn't use many tokens anyhow." — an established commenter who also flagged the carry-over overhead of switching

One practitioner named a cost the README doesn't price in at all: switching models mid-conversation makes the abandoned model lose track of what it did. "It goes and introduces a regression or gets lost thinking it didn't do that," wrote a developer who'd watched agents misfire after out-of-band edits. The 61% benchmark is measured on independent prompts; the moment routing meets a stateful agentic session, the deterministic part stops being the hard part — keeping a handed-off context coherent does.