When discussing Mr F–track GPU inference, teams often collapse three things into one sentence: “we support model X.” A signable statement separates the stack: platform shell, Runtime, and model version.
- 1) Platform shell — pull models and expose HTTP (e.g. Ollama). Convenient, rarely the tok/s battleground.
- 2) Runtime — the engine that actually produces tokens: loader, kernels, KV/memory, scheduling, quant path. SparkInfer, llama.cpp, vLLM, genie-ai-runtime live here (different hardware stories).
- 3) Model version — name + weights + quant tag. Changing the tag means re-measure; never extrapolate public uplifts.
Phase-1 for many token shops: same GGUF + same GPU → Runtime A/B vs llama.cpp / current stack. That is a Runtime swap, not retraining. OpenAI-compatible serving can cut over with a base-URL change—after the customer-card A/B wins.
Ongoing adaptation hits new architectures, GPU generations, quant formats, workloads, and multi-GPU full-model tracks. Manage expectations with a Supported Model Matrix and paid re-bench milestones for new flagships.
Quotable: public numbers are reproducible only under stated model × quant × GPU × script. 5090 figures are not a 4090 promise.
See #/mrf and https://aipat.dev/reports/
Disclaimer: indicative ≠ SLA; open-source cite ≠ contracting party. contact@aipat.dev