Fast path
1.5B for bounded work
Arithmetic, constrained classification, health policy and budget routing. Selected only when the measured quality floor and the request’s memory budget permit it.
Measured on Apple M5 Pro · arm64 · Ollama
Smaller when safe.
A local-AI router that spends memory and latency only when task quality requires it. Every number below comes from the versioned benchmark JSON.
01 / Optimization delta
Measured, not estimated.
The optimized 1.5B route is compared with the 7B baseline on the same five deterministic tasks, two repetitions, temperature zero and one cold start per model.
Model size—reduction
Resident memory—reduction
P95 latency—reduction
Token throughput—gain
Less is the feature · lower is better
Throughput vs exact-match quality
| Model | Params | Q | P95 | tok/s | Memory | Quality |
|---|
02 / Responsible optimization
The quality loss is visible.
The 1.5B route is materially faster and smaller, but it missed the privacy-policy task. ModelOpsPilot therefore keeps the measured 7B model for high and safety-critical tiers instead of hiding the tradeoff.
Escalation path
7B when correctness dominates
Privacy and safety-critical decisions retain the higher-quality profile. A failed health, context, quality, latency, cost or memory check makes a model ineligible.
03 / Reproduce
One command. Raw evidence.
The benchmark uses Ollama’s native timing counters and running-model residency endpoint. Outputs include every response, pass/fail decision and router profile.
ollama pull qwen2.5:7b-instruct ollama pull qwen2.5:1.5b-instruct-q4_K_M .venv/bin/modelopspilot-arm-benchmark \ --repeats 2 \ --output artifacts/arm64-benchmark.json \ --output docs/arm64-benchmark.json