GPU Runtime
High-performance inference on consumer and edge GPUs. Exclusively commercialized via aipat.dev—limited trials and PoCs available.
Indicative · retest required
Capability focus
Deliverable engineering capabilities—no unverifiable hard metrics on the public site.
GPU inference Runtime
MoE / LLM inference paths for consumer and edge GPUs; delivery subject to customer retest and SOW.
CUDA & memory scheduling
Kernel work, KV cache, and concurrency stability engineering; no hard percentage claims online.
Edge & offline scenarios
Deployment and privacy-agent directions on Jetson-class devices—suited to PoC and joint pilots.
Industry scenarios
Select a scenario for pains and solution boundaries. Performance wording is indicative · retest required—no ROI / payback promises.
Delivery robots
Jetson-class · autonomous nav · battery-sensitive
genie-ai-runtime · Jetson edge inference
Pains
- On-robot inference latency too high for real-time decisions
- Limited battery; GPU cycles wasted on generic frameworks
- Large binaries strain firmware space
- Cloud round-trips required; weak offline capability
Approach
- Edge-path latency and first-token focus—validated on customer SKUs
- Higher inference efficiency under the same hardware budget
- Lightweight binaries for firmware embed (size indicative · retest)
- Fully local offline run; data stays on device
Performance figures are indicative (indicative · retest required), not a guarantee across all hardware; customer-environment retest and SOW terms prevail.
OpenClaw and similar mentions refer to enterprise integration and delivery participation—not official commercial endorsement by any foundation. Details after NDA / SOW.
Architecture: general-purpose vs hand-written kernels
A structural comparison of trade-offs: broader compatibility vs a deeper single path. No hard speedup percentages.
General paths (e.g. llama.cpp / vLLM)
- 01Model layer (broad model coverage)
- 02Scheduler layer (multi-backend)
- 03CPU / Metal / Vulkan compatibility paths
- 04CUDA backend (generic kernels)
→ Compatibility over single-SKU peak · often a performance compromise
Dedicated genie / SparkInfer paths
- 01Model layer (deep optimization for target series, e.g. Qwen)
- 02Hand-written CUDA kernels
- 03Paths tuned for Jetson Unified Memory and similar
- 04Converged into one high-priority inference path
→ One path optimized in depth · results indicative · retest required on customer hardware
Three-step path from llama.cpp
Structurally usually three steps. Public timing and size comparisons are indicative; formal guidance follows the tech pack and integration.
- 1Obtain the Runtime deliverable
Download or receive the binary / tech pack. Public size comparisons are indicative; the actual package prevails.
- 2Load model weights
Load weights in the agreed format (often alignable with llama.cpp weight paths—see the tech pack).
- 3Switch the call entry
Point Serving / apps at the new API or Runtime entry; compatibility and change size depend on integration results.
Performance figures are indicative (indicative · retest required), not a guarantee across all hardware; customer-environment retest and SOW terms prevail.
I spent three years writing this engine. The goal is for every NVIDIA GPU to deliver its real performance. genie is not a framework—it is hand-written kernels, with as little extra code as possible. If you run a side-by-side on your own hardware, the numbers will speak.
Scenario picker
Three clicks—industry, hardware, pain—to a genie or SparkInfer suggestion. Copy is indicative · retest to confirm; no hard performance promises.
Step 1 · Your industry scenario?
Indicative performance comparison
Static front-end reference for discussion—not live scoring, no backend API. Any delta is indicative · retest required.
Indicative · retest required
Decode throughput (tok/s)
Indicative: under this config, SparkInfer throughput is higher than llama.cpp (~+197 tok/s). Indicative · retest required
| Metric | SparkInfer | llama.cpp | vLLM | Indicative delta |
|---|---|---|---|---|
| Decode tok/s | 473 | 276 | 278 | ~+197 tok/s vs llama.cppIndicative · retest required |
* Figures are indicative under specific configs (indicative · retest required), not guarantees across hardware; customer retest and SOW prevail.
Boundaries & who it is not for
Clear unfit cases build more trust than stacking metrics.
Request types we have declined
A prior request needed large-scale multi-GPU distributed inference with single-GPU latency targets far beyond the product boundary. genie / SparkInfer excel at single-GPU and edge-path depth—not full multi-GPU cluster scheduling. We stated the boundary, suggested a general multi-GPU stack first, and left the door open for edge / single-GPU PoC later.
Who genie / SparkInfer is not for
- Large-scale multi-GPU distributed inference as the main path (e.g. 8+ GPU clusters)
- Models outside currently adapted series with no near-term adaptation appetite
- Non-NVIDIA GPUs (no AMD / Intel support)
- Cloud-only inference with no owned or deployed hardware
About the next 3 months
Indicative schedule—subject to actual release; not a delivery commitment.
Dates are planning indicators and may change.
DeepSeek adaptation
Support for DeepSeek-series models
In progressGLM adaptation
Support for GLM-series models
PlannedMulti-GPU distributed
Multi-card inference (Alpha direction)
ResearchRTX 4090 optimization
Deeper Ada Lovelace-path tuning
PlannedEngagement path
- 1Submit trial intent
- 2Business review
- 3Guides / tech pack
- 4PoC & commercial talks
FAQ
Product differences, PoC, hardware boundaries, and commercial framing—no public pricing or efficiency refunds.
genie-ai-runtime targets NVIDIA Jetson-class edge devices with light embedding and Unified Memory–aware work. SparkInfer targets desktop / consumer GPUs (e.g. RTX). Both share a hand-written CUDA kernel direction, differentiated by platform; boundaries follow the tech pack and SOW.
Request benchmark reports
Public pages cover methods and boundaries only. Full score tables / PDFs are pre-sales materials—request by email (indicative ≠ SLA).
Jetson edge inference baseline report
Orin path selection focus and retest gates (no public score table)
SparkInfer desktop GPU performance report
Consumer-GPU Runtime A/B method and PoC prep (no public score table)
Channel & integration partners
Jetson solution vendors, SI partners, or industry advisors—welcome to introduce genie / SparkInfer to fit clients. Partnership details are handled by email; no public commission percentages.
Become a partner →Try Mr F’s GPU inference runtime
After review, aipat.dev issues a 14-day developer build. Benchmark on your hardware—metrics subject to customer-environment retest.
Company, contact, email, and GPU setup for a trial request
~2 minutesCorporate email preferred; activation code and download guide after approval
Usually within 1 business dayRun the benchmark script on your hardware, then commercial follow-up
~30–60 minutesLimited trial build
Indicative · retest requiredgenie-ai-runtime private GPU inference trial (enabled after approval).
- Memory efficiency & throughput: subject to customer retest
- Inference paths for RTX / Jetson-class scenarios
- Concurrency stability: load-test acceptance
- 14-day developer build (manual review)
Performance figures are indicative (indicative · retest required), not a guarantee across all hardware; customer-environment retest and SOW terms prevail.
Request a trial
Submit for review; activation codes are issued after approval (not instant).