News
Google makes it easier for select LLMs to run fully on-device
Google announced a major upgrade for MediaPipe and TensorFlow Lite. The company’s new MediaPipe LLM Inference API enables select large language models to run fully on-device.
According to the press release, supported models include Gemma 2B, Falcon 1B, Phi 2, and Stable LM 3B, while supported platforms are the web, Android, and iOS.
Google says this new release enables LLMs to run fully on-device across platforms. Optimizations across the on-device stack make this possible, including new ops, quantization, caching, and weight sharing.
On Android, the MediaPipe LLM Inference API is intended for experimental and research use only. Production applications with LLMs can use the Gemini API or Gemini Nano on-device through Android AICore.
Stay up-to-date on Google Pixel, Android & Tech Stuffs by following I M Judge on X/Twitter where you can also discover the latest news, polls, reviews, and new features for Google Apps, Pixel Phones, and the Android operating system.
Do you like this post? Kindly, let us know on X/Twitter: we love hearing your feedback! If you prefer using other social platforms besides X, do follow/join us on Facebook and Telegram.