Yes, you can run an LLM on an Android phone with a free app. You do not need root or Termux. Download a model of about 2 to 5 GB and it runs offline.
What you need
- RAM: 6 GB at minimum for small models, 8 GB or more to be comfortable.
- Storage: room for the model, plus some spare.
- Android 7 or later for opchat. In practice, a recent phone gives far better speed.
Set it up in opchat
- Install opchat from Google Play.
- Open Settings → Models → On this device.
- Choose a recommended model, or use Browse & Download Models to search Hugging Face.
- Download it on Wi-Fi, then start a chat.
GGUF or LiteRT?
These are two formats for the same idea, and opchat runs both.
- GGUF runs on llama.cpp. It has the widest choice of models on Hugging Face.
- LiteRT is Google’s format, and it can use the phone’s GPU.
GPU support on Android varies by chip. opchat uses the GPU for LiteRT on Adreno graphics, found in Snapdragon phones. On other chips it defaults to the CPU, which is more stable there. You can change this setting yourself.
Which model to pick
Start with Gemma 4 E2B. If it is too slow, try LFM3 or a small Qwen 3.5 variant. If you have 12 GB of RAM or more, try Gemma 4 E4B.
What to expect
Expect slower replies than cloud AI, some heat on long sessions, and more mistakes than large models make. If you want to compare options, see the best apps for running AI models on a phone.