Google open-sources ML Drift, its on-device GPU inference engine for phones, browsers and desktops, and freezes the TFLite GPU delegate

Why it matters

【Key points】

On Oct 8 the Google AI Edge team open-sourced ML Drift under Apache 2.0. It is the on-device GPU engine inside LiteRT, with one codebase covering OpenGL ES, OpenCL, Metal and WebGPU, and it also ships as a standalone C++ library for custom runtimes. Google says the legacy TensorFlow Lite GPU delegate will get no new features and recommends moving to ML Drift.

【Verifiable】

The post describes tensor virtualization that decouples logical tensors from physical GPU storage so all backends share one shader codebase; native 5D tensor support for 3D convolutions and spatiotemporal models; separate kernels and KV-cache layouts for LLM prefill and decode; and a desktop preview on Windows and Linux via Dawn, Chromium's WebGPU implementation. Google reports up to 12% lower memory overhead than other frameworks in Gemma benchmarks. Production users cited include YouTube Shorts (up to 40% lower frame latency), Chrome's built-in Gemini Nano APIs, Adobe Lightroom mobile (up to 30% faster) and Snapchat Lenses (30% lower latency). The repo includes a SKILL.md guide for coding agents writing custom shaders.

【Our read】

The hard part of on-device AI is less the model than the sprawl of phone, browser and PC GPUs and drivers. Open-sourcing an engine already running on millions of devices gives developers a shared low-level layer and puts Google in direct competition with vendor stacks. The immediate effect is a migration: the TFLite GPU path is now in maintenance. The desktop WebGPU backend is the piece to watch, since a solid one would give local coding agents a non-CUDA route for small models.