We Ship Seven On-Device Models. Here's How We Actually Think About Picking One

There's no single "best" model when you're running AI on a phone instead of a server. Every option is a trade-off between size, speed, and reasoning quality, and the right one depends entirely on what device your users actually carry. Our offline AI chat app ships seven LLM options. Here's the real lineup, and how we actually think about picking between them.

The four we recommend

Phi-4 Mini Light (1.4GB) is an ultra-compressed version of Microsoft's Phi-4 Mini, quantized down to fit tight memory budgets. It's our default recommendation, because most phones in the real world aren't the newest flagship. It loads fast and runs fast, and unlike the larger models on this list, it doesn't get hit with our conservative memory caps, so it keeps its full batch size and stays responsive.

Phi-4 Mini (2.49GB), the uncompressed version, is Microsoft's actual target quality for this model class: a genuine reasoning and math specialist, benchmarked at 94.6% on MATH-500. It needs meaningfully more RAM, and it falls into our "large model" bucket, which caps its context window and batch size to avoid out-of-memory crashes. Worth it if the device can afford it.

Gemma 4 E2B (2.62GB) is Google's smaller multimodal Gemma 4 variant, capable of text, image, and audio input, and covering eight languages (English, Spanish, French, German, Italian, Japanese, Korean, Chinese). It's a real step up in capability from the Phi models for multilingual or multimodal use cases, at a similar size to full Phi-4 Mini.

Gemma 4 E4B (5.41GB) is the largest and strongest model we ship, same multimodal and multilingual support as the E2B, with noticeably better output. It's also the most memory-hungry by far. On our Android builds, loading it with normal inference settings causes out-of-memory crashes on mid-range devices, so we cap its context window and batch size aggressively and disable memory locking just to keep it from taking the app down.

The three we ship but don't lead with

Gemma 2B (1.63GB) is a compact, fast, general-purpose chat model from Google. It's a reasonable middle-ground option, but it doesn't outperform Phi-4 Mini Light at a similar size, so we don't feature it by default.

Phi-3 Mini (2.23GB) was our original chat model before Phi-4 Mini existed in the app. It still works, but Phi-4 Mini Light beats it on both size and reasoning quality now, so there's rarely a reason to pick it over the newer option.

TinyLlama (638MB) is the smallest model in the app by a wide margin. It's cheap enough on memory that it doesn't need any of our context or batch caps at all, so it stays fully responsive even on old hardware. The trade-off is real, though: response quality is noticeably behind the rest of this list, and we don't recommend it as a default. It's there mainly for genuinely constrained, older devices where nothing else will run acceptably.

How we'd actually decide

If you're building something similar, the decision isn't really "which model is best." It's closer to: what's the minimum device spec you're willing to support, and how much reasoning or multimodal capability does your use case actually need?

A simple support chatbot rarely needs Gemma 4's ceiling; Phi-4 Mini Light will feel just as good to most users and run on far more devices. A tool doing real analysis, multilingual support, or image understanding benefits from paying the memory cost for Gemma 4. Letting the user choose, the way our app does, is often the honest answer: give people the trade-off and let them pick based on their own device and patience.

If you're scoping an on-device AI feature and need help figuring out which model class actually fits your target devices, we do this kind of evaluation at Riverborn. Happy to talk through it.