A reliable automated process runs dozens of model calls to finish one task — cleanup, classification, retrieval planning, breadth analysis, depth analysis, adversarial evaluation, consolidation. Use one model for all of them and you are wrong in one of two directions at every step. A model strong enough for adversarial evaluation is wasted on input cleanup — you pay its price and its latency for a step that a small model finishes correctly. A model cheap enough for cleanup cannot carry the analysis — it fails the step that matters. Per-step success rate is the reliability budget for the whole process, so the wrong model in any slot erodes the end-to-end success rate the architecture exists to protect.

Model selection is the mechanism that puts the right model in each slot, automatically, and keeps the process running when the chosen model is unavailable. It does this across a frontier of cost against capability rather than by a fixed assignment, and that is the design claim worth stating plainly: the system pays for frontier capability only at the steps where frontier capability changes the output. Everywhere else it spends down the frontier toward free.

This is three problems, not one, and the architecture answers them in three layers:

  • Fit — can this model physically run on this hardware? (Resolved once, at install.)
  • Selection — given a catalog of hundreds of endpoints, which handful is worth considering for a slot, given what it costs, how capable it is, and — for one posture — how fast it answers? (The cost/capability frontier algorithm.)
  • Routing — which model fills each slot right now, and what happens when it is offline, rate-limited, or constrained? (Runtime resolution with graceful degradation.)

Each layer solves a distinct failure mode. I will take them in order.

Layer 1 — Hardware fit

The design claim: estimate a model’s memory requirement from its parameter count and quantization level, never from its file size.

A model that does not fit in RAM does not run — or it runs by swapping to disk at a latency that makes it useless. So fit is a hard gate before anything else. The naive approach is to read the file size on the model repository and compare it to available memory. It is wrong often enough to be dangerous, for two reasons: a repository may store weights behind XET pointers (the listed files are kilobyte stubs, not the multi-gigabyte weights), and file size on disk does not predict resident memory once the weights are dequantized and the inference engine’s working buffers are allocated.

The reliable estimate comes from two numbers the model always carries — its parameter count and its quantization level.

Quantization is the precision at which weights are stored. Lower precision trades a modest amount of quality for a large reduction in memory. The resident-memory estimate follows directly:

  • 4-bit: ≈ 0.5 GB per billion parameters
  • 8-bit: ≈ 1.0 GB per billion parameters
  • 16-bit (full precision): ≈ 2.0 GB per billion parameters
  • plus ≈ 2 GB of overhead for the inference engine and working buffers.

The budget the estimate is checked against is AVAILABLE_MODEL_RAM = total system RAM × 0.75 — the remaining quarter is left to the operating system and the application. A model whose estimate exceeds that budget is a hard block, not a warning.

Mixture-of-Experts (MoE) models need a second number. An MoE model has a large total parameter count but activates only a fraction of it per token. The memory estimate must use the total count — all weights load into RAM whether or not they fire on a given token — but the capability the model delivers tracks the active count. A 120B model that activates 4 of 128 experts reasons closer to a 4B dense model while occupying the RAM of a 120B one. The operative metric is active parameters per gigabyte of resident memory: dense models deliver roughly 1.5–2.0 B/GB; an MoE model can deliver as little as 0.06 B/GB. So the fit layer prefers dense models for reasoning work at any tier where a useful-size dense model fits in budget, and reaches for MoE only when RAM is too tight for any dense model worth running.

Fit also sorts hardware into capability tiers, which become the default ceiling on what auto-selection will pick:

TierAVAILABLE_MODEL_RAMDense model at 4-bit
16–11 GB3–7B
212–23 GB7–14B
324–47 GB30–70B
448–95 GB70B+
596+ GBtwo large models concurrently

The tradeoff this layer carries forward is the tool-calling reliability floor. Reliable tool calling — the model emitting a well-formed call the harness can execute — degrades below roughly 13B at 4-bit, or 7B at 8-bit. A Tier-1 machine can run a model that converses well and calls tools badly, and tool-call failure is a per-step reliability erosion that the upper layers cannot fully recover. Fit surfaces this rather than hiding it: a small-model selection ships with the tool-reliability caveat attached.

Alternative considered and rejected: size-from-file. It is simpler and needs no parameter parsing. It was rejected because it silently mis-estimates under quantization and breaks entirely on XET-pointer repositories — and the failure is a model that downloads, then will not load, after the user has waited for the download. A wrong answer that arrives late is worse than a slightly more expensive check up front.

Layer 2 — The cost/capability frontier

This is the layer the rest of the system is named for, and the one most worth getting right.

The problem: the catalog holds hundreds of candidate endpoints — at the time of writing, 358 models across 431 registered endpoints, spanning local weights and a dozen commercial providers. No human ranks those by hand, and re-ranking them by hand every time a provider ships a model does not scale. The system needs an algorithm that, given a slot and a budget posture, returns the few models worth using.

The data the algorithm runs on. Every catalog entry carries the fields the decision needs:

  • aa_intelligence_index — the capability axis: a single Artificial Analysis intelligence index (0–100). It is populated into the catalog from config/model-registry.json rather than by calling Artificial Analysis directly at install (most of it arrives embedded in the OpenRouter catalog). It is the only value the intelligence floor and the intelligence sort consume; a model that carries no benchmark ranks at the bottom rather than being estimated from a proxy such as Chatbot Arena Elo, whose correlation with the benchmark is too weak to rank on.
  • blended_per_m — blended cost in dollars per million tokens. This is the cost axis.
  • or_ttft_ms / latency_ttft_seconds and output_tokens_per_second — time-to-first-token and throughput. Time-to-first-token is the Speed preset’s latency gate (OpenRouter’s per-provider measurement preferred, the Artificial Analysis benchmark median as fallback); throughput is surfaced for the reader and the constraint checks.
  • parameters_b, size_bucket, is_free, vision_capable, context_window — structural filters.

The algorithm, in order. Budget, Premium, and Free share one pipeline — floor, then ceiling, then sort. (Speed adds a step of its own, described after the presets.)

1. Intelligence floor. Keep only models scoring at least floor_pct percent of the top intelligence in the bucket: threshold = top_intelligence × floor_pct / 100. The floor is how a slot says “do not go below this capability no matter how cheap.” A high floor keeps only near-top models; a low floor admits cheaper, weaker ones.

2. Cost ceiling. Drop models above a dollars-per-million ceiling, bounding spend independent of the capability band. Premium and Free set none. Budget sets an adaptive ceiling rather than a fixed one: it tracks a fraction of the current high-end market price — roughly 30% of the most expensive “normal” model still in the qualified pool — recomputed every time the catalog refreshes, with extreme outliers (a reasoning flagship priced many times the median) trimmed first so they cannot drag the ceiling upward. A fixed dollar cap goes stale as the market moves; an adaptive one holds the same posture while absolute prices fall.

3. Loosening. If fewer than the requested top_n models survive both bounds, Budget relaxes them in paired steps — lower the floor a few points and raise the ceiling a notch together, repeating until enough pass or the floor reaches its minimum (50%). Every loosening event is recorded in the output metadata, so a configuration that could not be satisfied at its stated posture says so rather than silently returning too few options.

4. Order and take. Sort the survivors and take top_n: one primary plus its fallbacks. The sort axis is the preset’s posture — cheapest-first for Budget, intelligence-descending for Premium and Free.

There is no Pareto pre-filter in this pipeline. The cheapest model above the floor and the highest-intelligence model are each already on the cost/capability frontier by construction, so an explicit dominance pass would change nothing about the primary pick — it would only prune the cheap same-tier alternatives a fallback chain wants to keep. The frontier is computed explicitly only where a preset needs a point in its interior rather than at an edge: Speed’s value knee, below.

The presets are named points on this frontier. Budget, Premium, and Free are the same algorithm under different parameters — floor, ceiling, and sort axis; Speed adds one selection step of its own (below). That near-uniformity is the property that makes them comprehensible:

PresetFloorCost ceilingLooseningEffect
PremiumnonenonenoBest-of-bucket by intelligence; cost ignored.
Budget70%, loosens to 50%adaptiveyesCheapest model that clears a moving quality floor under an adaptive price ceiling. The default; most configurations land here.
Speednone (latency gate)nonenoWithin a time-to-first-token gate, the best-value knee of the cost/intelligence frontier; reasoning models excluded. A fast first response, not maximum quality.
Free———Free-tier models only, intelligence-descending.

Budget is the design center, and it states the whole thesis in one line: keep every model within a band of the best — the top 70%, loosening toward 50% only if too few qualify — and buy the cheapest of those, under a ceiling that tracks the market. This is the cost optimum in the literal sense: the preset genuinely seeks the cheapest model that still clears a real quality bar, which is why it is where most configurations land. (It carried the name Optimum until that name was retired — “Optimum” quietly presumed the user’s optimum was always cost, when cost is only the optimum for the cost-conscious posture. “Budget” says what the preset is for without overclaiming; the algorithm is unchanged in spirit, now with a moving floor and an adaptive rather than fixed ceiling.) You accept a model scoring as little as 70% of the best — at most the top ~30% of measured capability given up, and lower only when too few models qualify — and in exchange you usually pay a small fraction of the top price, because the frontier is steep at the top, where the last increments of benchmark score cost the most.

Speed optimizes a different axis, and it is the one preset that uses the frontier directly. The other three pick the cheapest, or the smartest, model that clears a floor. Speed instead makes a promise about responsiveness and then buys the best value it can keep within that promise. The promise is a latency gate: each slot is restricted to models whose time-to-first-token — how long the user waits before the answer starts streaming — is under a ceiling (about 1.2 seconds at the time of writing, a provisional, tunable value). Within that gate, Speed computes the cost/intelligence frontier and picks its value knee — the point that maximizes intelligence gained per dollar spent, with intelligence and cost each normalized and cost taken on a log scale. On a frontier that bends, the knee is the diminishing-returns elbow: it captures the intelligence that is cheaply available and refuses the cost cliffs, where a model many times pricier buys only a few more points. There is no intelligence floor and no cost ceiling here — the knee replaces both. (Budget stays the pure cost-minimizer; Speed is the best-value pick under a latency budget.) Two further things define it. It excludes always-on reasoning models — models that emit a hidden chain of thought before their answer — because a reasoning model stalls exactly where Speed is trying to win: it spends seconds thinking before the first visible token, the opposite of a fast first response. Whether a model reasons in this way is read from a structured capability flag in the registry (sourced from OpenRouter), not guessed from its name, so the exclusion survives renames and new releases. And, stated plainly because it is the tradeoff being accepted: a latency gate admits mostly lower-intelligence models. Time-to-first-token is dominated by model size and serving stack, so even at its value knee Speed is buying responsiveness, not peak capability. It is the right preset for a chatbot turn or an early cleanup/classification step where the user is watching a cursor blink, and the wrong one for the analytical slots where the answer’s quality is the whole point. The latency itself comes from a per-provider scrape of OpenRouter’s published endpoint statistics — the time-to-first-token a model’s actual serving providers are delivering — with the Artificial Analysis benchmark median as a fallback for models OpenRouter has no live stats for. The latency ceiling and the knee’s cost weighting are provisional first-pass values, flagged as such pending calibration against live latency data, and are tunable.

Size buckets keep the frontier honest per slot. Each slot declares a size_bucket. Utility slots — input cleanup, classification, retrieval planning — declare small; analysis slots declare large. The selector filters to the bucket before it runs the frontier, so a cleanup slot never reaches for a frontier reasoning model (overkill: full price and latency for no quality gain on a step a small model does correctly) and an analysis slot never drops to a tiny one (underkill: a quality risk on the step that carries the task). top_n is set per slot too — three for utility, four for the Gear-4 analysis pair — so each slot gets a primary and enough fallbacks to survive provider outages. A third, speed-sensitive slot type — the models used for fast interactive turns — sits between utility and analysis: it is gated on measured throughput rather than parameter bucket and no longer carries a fixed floor of its own. It inherits the active preset’s own selection rule instead — Budget’s 70% floor, Premium’s 80%, Free’s none, or, under Speed, the same latency-gated value knee every other slot uses — so “fast” means fast at the preset’s own bar rather than at a single global one.

Adversarial diversity is a selection constraint, not a separate stage. The depth and breadth analysis slots in Gear 4 — the parallel adversarial pass, where two models work the same problem independently so each can catch the other’s errors — are most valuable when the two models come from different training lineages, because shared lineage means shared blind spots. The selector enforces this by excluding depth’s chosen primary from breadth’s candidate set, pushing breadth toward a different model family. (An earlier design made cross-company diversity its own pipeline stage — a notional “Gear 5.” That was retired: diversity is cheaper and cleaner as a constraint on selection than as a stage in execution.)

Alternatives considered and rejected:

  • One model for every slot. The status quo for most deployments. Rejected because it forces the single wrong-direction error at every step: priced for analysis, it overpays on utility; priced for utility, it fails analysis.
  • Manual per-slot assignment. Workable for five models; unworkable for 358, and it rots the moment a provider ships a new model or changes a price. The frontier algorithm re-derives the picks from the refreshed catalog with no human in the loop.
  • Rank by intelligence alone, ignore cost. This is exactly the Premium preset — offered, because some users want it, but rejected as the default because it pays frontier rents at every slot, including the slots where a model at 70% of the top score produces identical output.

The tradeoff. The frontier is only as good as the capability metric. Today that metric is a single Artificial Analysis benchmark index, drawn from the model registry. A single scalar still stands in for “how good is this model.” It can lag new releases, it cannot see task-specific strength, and any published benchmark is a target that can be optimized against. The system buys a large reduction in selection effort and spend, and accepts a dependency on external measurements it does not control. That dependency is the first entry in the open-problems list below.

Layer 3 — Routing and graceful degradation

Selection produces, per slot, a primary and an ordered fallback chain. Routing is what turns that chain into a running process when reality intervenes — when the primary is offline, rate-limited, already busy, or blocked by a hardware constraint.

A configuration maps each pipeline cell — a slot at a given gear — to its primary and fallbacks. (Gear is the tier of analytical machinery applied to a task: Gear 3 is a sequential pass; Gear 4 is the parallel adversarial pass. Heavier work runs at a higher gear.) The configuration in use is derived from context, not chosen by the user per query: interactive work resolves against the user-pipeline configuration; background work — the autonomous and agent execution contexts — resolves against background-default. Interactive work reserves its endpoints, so a background process never starves the human at the keyboard.

Single-slot resolution walks the chain and returns the first usable model:

route(cell):
  for model in cell.chain:          # primary, then fallbacks, in order
    if not model.enabled:      continue
    if not model.available:    continue   # offline / rate-limited / busy
    if violates_constraint(model): continue
    return model
  return DOWNGRADE

Process-level resolution wraps that in a gear-downgrade cascade. If every model in a slot’s chain is exhausted, the slot returns DOWNGRADE and the whole process drops a gear and retries — a parallel adversarial pass it cannot staff becomes a sequential pass it can:

execute(requested_gear):
  for gear in [requested, requested-1, ..., 1]:
    assignments = resolve every slot the gear needs via route()
    if any slot returned HALT:       return halted(status)
    if any slot returned DOWNGRADE:  continue        # try the next gear down
    return run(gear, assignments)
  return halted(status)

This is the reliability payoff of the whole selection stack: “my preferred model is unavailable” degrades to “the task ran at a lower gear,” not “the task failed.” Degradation is bounded and visible — the run records the gear it actually executed and why it dropped — rather than silent.

Two classes of rule guard the resolution:

  • Hard constraints block a model outright. Two large local models cannot run in parallel on one machine’s GPU (the MLX Metal constraint), and a model whose resident memory exceeds machine RAM cannot load. A STOP marker in a chain halts the process rather than letting it proceed on an unacceptable model.
  • Soft warnings are surfaced but do not block: a frontier model sitting in a utility slot (overkill), a weak or free model in an analysis slot (underkill), and the same provider on both halves of the adversarial pair (reduced independence). These are advisory because the user may have a reason; they are surfaced because the default reading of a configuration should make its compromises legible.

Why the frontier framing matters

State the economic property without dressing it up: when selection routes each step to the cheapest model that clears the step’s capability floor, frontier-model spend collapses to the few steps where frontier capability changes the answer. A thirty-call task does not pay thirty frontier prices; it pays a few, and spends the rest of its budget near the floor or at zero. The capability the task delivers is set by the slots that matter, which run at the top of the frontier, while the cost is set by the many slots that don’t, which run at the bottom. That decoupling of delivered capability from total cost is the point of doing selection as a frontier search rather than a fixed choice.

Open problems

Engineering honesty requires naming what this layer has not solved.

  • The capability metric is still too compressed. The registry combines benchmark and provider data, but the selector still consumes a single Artificial Analysis scalar capability score. It can lag new models, it does not capture task-specific or mode-specific capability (a model strong on general reasoning may be weak at a particular analytical mode), and public benchmarks are gameable. There is no per-task capability model; a slot that needs a specific competence cannot yet express it to the selector beyond the global floor.
  • Multi-objective selection exists for one preset, not as a general capability. The Speed preset now trades all three axes at once — it gates on latency, then picks the value knee of the cost/intelligence frontier — so “fast and good value” is a posture the selector can actually express. But Speed is the only preset that does it: Budget and Premium still optimize a single axis with the rest demoted to floors and ceilings, and there is no general frontier any slot can query for an arbitrary weighting of intelligence, cost, and latency. Throughput (tokens-per-second) remains only a gate, never part of any frontier. And Speed’s knee is only as good as its latency signal, which is point-in-time and provider-dependent — a scraped median over a rolling window, not a guarantee for the next call. A unified, per-slot multi-objective frontier is the obvious next step and is not built.
  • The intelligence floor is global, not per-mode. One floor_pct governs a whole bucket. Some modes tolerate a lower floor than others; the system cannot currently set the floor per mode.
  • Diversity is enforced by exclusion, not by measured independence. Breadth excludes depth’s primary, which forces a different model id, but two differently-named models trained on convergent data can still share blind spots. The system has no measure of genuine training-distribution independence, so the adversarial pair’s independence is assumed from family difference rather than verified.
  • Availability is point-in-time. route() checks whether an endpoint is reachable now; it does not model rate-limit recovery windows or predict which free-tier endpoint is about to throttle, so it can route to a model that fails on the next call. Predictive health, and modeling failure correlation across free endpoints, are unbuilt.

None of these blocks the architecture from doing its job — the selection stack runs in production every day — but each marks a place where the current mechanism is a defensible approximation rather than a finished answer.