Model-agnostic by design: how Frond routes across models

When we were designing the model layer for Frond, the easiest choice would have been to pick a provider and build around it. Pick a strong model, get a favorable rate, optimize for its strengths. Most AI products take that path, and it works fine until it does not.

The problem with a single model

Building on one model is a dependency, and dependencies surface when something goes wrong. A rate-limit spike during a product launch. An API that degrades quietly for four hours on a Thursday afternoon. A model update that changes response behavior in ways that break your prompts. Each of these is an interruption to a product that teams depend on for answers throughout their day.

There is also the opportunity cost. The model landscape has not settled. New releases keep moving the capability frontier, and the gap between a fast small model and a frontier reasoning model changes on a cadence faster than any product cycle. A team built on one model watches each improvement arrive and stays on the sideline.

Route each step to the right model

Frond keeps a registry of models and routes each step of answering a question to the one that clears the quality bar for the least cost. The steps in producing an answer are not equal, and treating them as if they were wastes either money or quality.

• Classifying a question and deciding whether it can be answered from the available sources is lightweight work. A small, fast model handles it well and returns in milliseconds.
• Retrieving the right passages, reading them together, composing a coherent answer, and matching each claim to its source is the heavy step. It goes to a frontier model.
• If the primary provider is degraded or rate-limited, the router falls to the next qualified candidate in the registry automatically, with no change to the answer format.

The routing table is configuration, not code. Adding a new model when a better one is released means updating the registry, not rewriting the answer pipeline. When a lab ships something meaningfully better, Frond can take it in hours.

What the registry looks like

In practice, the registry is a list of models with metadata about each: what tasks it is optimized for, its typical latency, its cost per token, and how it should be ranked relative to others per task type. The router reads the registry and makes a routing decision per step, per question, in milliseconds. The decision is deterministic: if you want to know which model answered a given question, it is in the log.

step          model                    reason
classify      gpt-4.1-mini             fast, accurate routing
synthesize    gpt-4.1                  frontier, citation-quality
fallback      claude-3-5-haiku         provider diversification

Open models, for teams that need them

Model-agnostic also means not requiring questions to route through a hosted API. For teams with data-residency requirements or in regulated industries, sending knowledge through a third-party endpoint may not be an option.

On the Scale plan, Frond can route to open models hosted inside your own infrastructure. The routing table is unchanged. The answer arrives in the same format with the same cited source. Only the endpoint is different, and your knowledge stays inside your perimeter.

What this means going forward

The labs will keep shipping. Models will keep leapfrogging each other. Some questions that today require frontier capacity will eventually be answerable by small fast models at a fraction of the cost. A registry and a router mean Frond takes the best of each generation as it arrives, without a migration project and without asking teams to notice or care which lab answered.