MetalRT by RunAnywhere

Inference on the Apple Neural Engine.

An agentic kernel pipeline generates, compiles, and validates every kernel against the silicon it runs on.

How it works

One package, every chip.

Apple exposes no public API for Neural Engine placement, so a model compiles to one portable package and runs unchanged from an M1 to an M5 or an A-series iPhone. Every build reads the real per-op hardware placement and diffs it against the last build, so a silent CPU fallback cannot ship.

LLM decode

658tok/s

M4 Max · measured on device

TTFT

6.6ms

M4 Max · LLM time to first token

VLM decode

287tok/s

M4 Max · vision language model

STT latency

101ms

M4 Max · speech to text