Research overview · 6 minute read
Route by difficulty.
Skip the essay.
VDAR-Router+ distills offline verbal difficulty analyses into a query-to-latent encoder that predicts four continuous latent states. One LLM prefill is enough to retrieve capability-similar queries and choose a model; readable analysis remains available off the routing path.
These are latent difficulty-analysis tokens—not latent chain-of-thought tokens. They describe what capabilities a query needs; they do not solve the query.
ONLINE ROUTING PATH
ONE PREFILL
QUERY
“Compare two algorithms and identify the hidden edge case.”
QWEN3.5-2B · LoRA
Difficulty encoder
prefill
CONTINUOUS ANCHOR HIDDEN STATES
CRITICAL PATH
Profile → retrieve → route
ON DEMAND
Readable analysis
4 × 128-d latents
1024-d routing profile
top-k = 30
01 / The problem
Good routing needs more than topic similarity.
Two queries can discuss the same topic yet demand very different levels of reasoning, knowledge, coding, or instruction following. VDAR showed that a verbalized difficulty analysis creates a useful capability-oriented retrieval space—but generating that analysis for every request dominates latency.
Research questionCan we preserve the routing signal of a verbal difficulty analysis without autoregressively decoding it on every request—and still recover a human-readable explanation when needed?
ORIGINAL VDAR
3.53 s
Verbalize first, then route
- 01Query
- 02Decode ~150 words
- 03Embed
- 04Retrieve
- 05Route
The useful representation arrives only after autoregressive generation.
VDAR-ROUTER+
92.5 ms
Form the representation directly
- 01Query
- 02
- 03Pool
- 04Retrieve
- 05Route
The routing profile is ready after one prefill; text generation is optional and off-path.
Latency is matched within protocol: batch 1, PyTorch generation stack, H200. VDAR-Router+ median is 92.5 ms (p90 138 ms) versus 3.53 s for the concise analyst.
02 / Core intuition
Keep the geometry.
Move language off-path.
The analysis text matters because it organizes queries by the capabilities needed to answer them. VDAR-Router+ learns that organization directly, then treats language as an interface to the representation—not the representation itself.
Difficulty is a geometry
Queries should be neighbors when they require similar model capabilities, even if their surface topics differ.
Four states can carry it
Trainable anchor embeddings collect query-conditioned hidden states in parallel. No discrete latent token IDs are generated.
Language is recoverable
A latents-only interpreter turns the same bottleneck into a readable difficulty analysis when inspection is worth the cost.
03 / Method
One representation.
Two useful views.
The promoted method—internally E-B-min—trains a Qwen3.5-2B backbone to match an offline teacher space while preserving enough information to explain its own difficulty profile.
OFFLINE SUPERVISION
Text generation happens here, once for the training corpus
A
TRAIN QUERY
Input q
QWEN3.5-2B
Concise analysis
~150 words
FROZEN EMBEDDER
Teacher profile yq
1024-d · normalized
ONLINE ENCODER
The routing path stops before any text decode
PROMPT + QUERY
q + four anchors
LoRA r16 · α32
Qwen3.5-2B
one causal prefill
PROJECTED STATES
z1z2
z3z4
4 × 128-d
ROUTING VIEW
critical path
Pool → retrieve → select
- 01 Mean-pool four latents; project and normalize a 1024-d profile.
- 02 Retrieve 30 historical queries by cosine similarity.
- 03 Estimate each model’s performance and cost from weighted neighbors.
SELECT
arg maxm [
α · expected performance − (1 − α) · normalized cost
]
LANGUAGE VIEW
on demand
Soft prefix → interpreter
- 01 Project each 128-d latent back to the 2048-d LLM embedding space.
- 02 Prepend the four vectors to a fixed “Difficulty analysis:” instruction.
- 03 Decode with the same LoRA backbone—without access to the query text.
ILLUSTRATIVE RECOVERED VIEW
Requires multi-step comparison, constraint tracking, and identifying a non-obvious failure condition…
TRAINING OBJECTIVE
L
=
1 − cos(pq, yq)
routing geometry
+
CE(analysis | z1:4)
information recovery
STAGE 1
12 epochs
teacher cosine only
STAGE 2
6 epochs
cosine + decode CE
“Two-term objective” refers to the two non-zero promoted losses; the trainer retains compatibility hooks for earlier auxiliary experiments.
ARCHITECTURE AT A GLANCE
- Backbone
- Qwen3.5-2B causal LM
- Adaptation
- LoRA r=16, α=32
- Anchor inputs
- 4 × 2048-d trainable embeddings
- Latent states
- 4 × 128-d projections
- Routing profile
- 1024-d, L2 normalized
- Retrieval
- cosine top-30
04 / Results
Routing parity,
without online prose.
Across two routing benchmarks, the promoted VDAR-Router+ method matches the strongest internal references within seed noise and outperforms the evaluated external routers under the VDAR evaluator, cost normalizer, and test splits. The evidence supports parity—not a claim that it beats F2.
2
benchmarks
3
E-B-min seeds · 13 / 7 / 29
30
retrieved neighbors
×100
reported reward scale
LRB
E-B · 3 seeds
LLMRouterBench
REWARD @ α = 0.8
43.23 ± 0.37
REWARD @ α = 0.6
31.25 ± 0.28
VDAR-Router+ main
43.23 / 31.25
F2 frozen variant
43.07 / 31.32
Teacher embedding reference
43.28 / 31.59
Best external ELO
41.23 / 30.55
RB
E-B · 3 seeds
RouterBench
REWARD @ α = 0.8
53.46 ± 1.13
REWARD @ α = 0.6
37.06 ± 0.53
VDAR-Router+ main
53.46 / 37.06
F2 frozen variant
54.36 / 36.42
Teacher embedding reference
54.02 / 36.75
Best external per α
42.15 / 25.23
Mean ± standard deviation and the three-seed protocol apply to promoted E-B-min. F2, teacher-embedding, and external-router rows are reported point estimates.
MATCHED H200 LATENCY
The essay was the bottleneck.
Concise analyst
3.53 s
VDAR-Router+
92.5 ms
38.1×
structural speedup on the same H200 protocol
LATENT RECOVERY PROXY
The bottleneck remains inspectable.
0.511
LLMRouterBench
decoded cosine · floor 0.143
decoded cosine · floor 0.143
0.526
RouterBench
decoded cosine · floor 0.210
decoded cosine · floor 0.210
This is evidence of representation recovery, not proof of word-for-word faithful explanation. The interpreter is off-path and has a meaningful miss-tail.
DO NOT CONFLATE THE VARIANTS
One research method, one lean routing option.
The main E-B-min model jointly learns routing geometry and latents-only interpretation. F2 is the cheaper frozen-encoder variant; H3 adds a separately trained interpreter.
| Variant | Role | Routing latency | Interpretation |
|---|---|---|---|
| E-B-min | Main method | 92.5 ms · H200 | Joint, latents-only · 6.27 s |
| F2 | Efficiency variant | 14.8 ms · A6000 | None on routing model |
| F2 + H3 | Optional explanation | 14.8 ms · A6000 | Post-hoc 2B · single seed |
Routing latencies are shown with hardware because H200 and A6000 numbers are not an absolute head-to-head comparison. E-B interpretation measured 6.27 s on H200; H3 invokes a separate Qwen3.5-2B decoder. F2’s 204× is a component-median comparison (200 encoder queries versus 50 generation queries), not paired per-query timing.
05 / What we learned
The ablations changed the story.
VDAR-Router+ is the result of several closed hypotheses. These findings are part of the method: they explain why the current design is deliberately smaller than the original experiment plan.
Teacher choice mattered more than encoder size.
Retargeting F2 from verbose to concise analyses lifted LLMRouterBench reward @0.8 from 39.37 to 43.07.
keptFidelity is not routing reward.
M2 produced stronger retrieval fidelity than F2, but that did not translate into a better routing policy.
lessonDecode supervision was reward-neutral.
Joint decoding improved representation recall without a measured routing-reward gain or regression.
kept for recoveryTwo losses matched the heavier stack.
Teacher cosine plus decoder CE matched the observed promoted runs without relying on six active auxiliary terms.
promoted
NEGATIVE RESULTS
Evaluated, then closed
Embedding-model retrieval
Added complexity and reduced direct routing reward; not adopted.
Segment alignment
Did not reduce latent redundancy and cost about 0.4 reward.
Six-term objective stack
Bought representation metrics without a corresponding reward gain.
Pooled-query projectors
Lost token-level difficulty cues before the learnable bottleneck.
06 / Project status
What is done—and what is next.
The text-only method is evaluated; the multimodal extension is a roadmap, not a reported result. This boundary is important for anyone joining the project.
COMPLETED
Text difficulty routing
- E-B-min main method
- Two benchmarks × three seeds
- Joint latents-only interpreter
- Matched H200 latency protocol
DEPLOYMENT OPTION
F2 + optional H3
- Lowest measured routing latency
- Reward parity with the main method
- Post-hoc decoder remains single-seed
- Separate hardware protocol
NEXT FRONTIER
Multimodal routing
- VL-RouterBench extension
- Interpreter miss-tail analysis
- Reward-aware retrieval objectives
- No multimodal result claimed yet
Start here
From overview to implementation.
This page gives the research story. Follow these four links to inspect the evidence, code path, and operational setup.