Work from
A model name isn’t a capability
Choosing a node that can actually serve the requested workload.
A router can know that a model exists somewhere in a mesh without knowing whether a particular node can serve the next request. That distinction becomes important when a runtime grows beyond chat to embeddings, reranking, and audio.
In mesh-llm, adding endpoints was only part of the work. Discovery, target selection, retries, and the native runtime needed to agree on what each loaded model could do.
Check the target, not just the name
Consider two nodes advertising the same model name. One advertises the capability needed by the request; the other does not. The first makes the model discoverable for that workload. It does not grant the second node the same capability.
The routing change filters concrete serving targets before ranking, affinity, reservations, and failover. If an elected local replica cannot handle the workload, the router can still consider a compatible remote peer. The local copy must not hide that peer merely because it shares the name.
Audio uploads need a stricter check: a single descriptor must establish both audio support and a compatible execution class. Combining an audio flag from one advertisement with a workload class from another would invent a capability that neither target actually has.
Preserve the old contract explicitly
Older peers have no workload metadata. They remain eligible for ordinary generation, but absence of metadata does not authorize embeddings, reranking, or speech synthesis. An explicitly unknown workload is rejected rather than treated as an old chat peer.
The native boundary has a separate compatibility rule. The host rejects an older library before resolving the new workload exports. Model metadata cannot compensate for a runtime that lacks the functions needed to execute the request.
Test the disagreement
Regressions cover reordered descriptors, incompatible local replicas, compatible remote peers, and mixed workload pools. These cases exercise a stronger property than successfully sending one request to one known model: changing discovery order must not change which targets are eligible.
The broader change also compares pinned workload representatives with separate monolithic executions. Those checks establish behavior for the tested models and inputs; they do not certify every model family or a fleet of physical GPU machines.
A useful discovery answer is “this model can be served somewhere.” Dispatch needs the narrower answer: “this target can serve this request.”