This time we will do a fun experiment of running on 1B LLM on a android phone.
Today’s smartphones are incredibly powerful – many times more so than the first computers from decades ago, yet they cost much less and can do a lot more. Recently, smaller but still very capable AI language models (called “SLMs” or Small Language Models) have been released. These models are relatively tiny, ranging from 1 billion parameters to under 4 billion, but they pack a punch. This raises an interesting question: Can we combine these two advancements – powerful smartphones and efficient small AI models? It seems the answer is yes! For those who are tech-savvy, running these AI models on your phone is becoming quite feasible. Tools like Termux packages make it relatively straightforward. In fact, some apps are already available on the Play Store that demonstrate this capability. (For example, there’s an app called PocketPal AI, though I’m not endorsing it, it’s just an interesting example of what’s out there.) This means that if you’re technically inclined, you could even set up your phone to act as an “API” (a way for other programs to talk to it) for an AI language model.
Let’s get you set up to run AI models on your phone! Here’s how you can get started:
Step 1: Install Termux 🔗
Termux is a special Android app that lets you run a Linux-like environment right inside your phone.
How does Termux do this? Imagine your Android phone as a big house. Termux creates a special, self-contained room within that house. This room has all the tools and setup of a Linux computer, completely separate from your Android system. This way, you can install and use Linux programs and commands without affecting your phone’s normal operation. It’s like having a mini-Linux computer running as an app!
You can install Termux directly from the Google Play Store:
Install Termux from the Play Store: https://play.google.com/store/apps/details?id=com.termux <@FIXME> Provide a simple screen recording/gif of installing Termux from the Play Store.
Step 2: Let’s Install Proot-Distro 🔗
First, we need to update our Termux packages and install proot-distro. This tool will help us set up a full Linux distribution like Debian inside Termux.
Open your Termux app and type the following commands, pressing Enter after each one:
apt update
apt install proot-distro
proot-distro is essential because it allows us to create and manage an isolated Debian Linux environment within Termux. This is like having a complete Linux operating system running as a separate container on your Android phone, without needing to root your device.
Step 3: Now, let’s install the Debian distribution: 🔗
Once proot-distro is installed, you can easily install Debian by running:
proot-distro install debian
This command will download and set up a minimal Debian system within your Termux environment.
For more details on proot-distro and its capabilities, you can visit its GitHub page: https://github.com/termux/proot-distro.
Step 4: Login to a Linux Distribution 🔗
For this tutorial, we will use Debian. Log in to your Debian environment with:
pd login debian
Step 5: Install Tmux 🔗
Screen is a terminal multiplexer that allows you to run multiple terminal sessions simultaneously:
apt update
apt upgrade
apt install screen
Step 6: Install Ollama 🔗
Within the Debian environment, download and install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
This script installs Ollama and its dependencies.
Step 8: 🔗
Start a GNU screen session with following command,
screen -R
Checkout this to know more about how to use GNU screen https://opensource.com/article/17/3/introduction-gnu-screen.
Step 9: Start chat with the model, 🔗
One window run,
ollama serve
And other window run
ollama run [model_name]
Running Gemma3-n with a Dedicated Flutter App 🔗
While the steps above demonstrate how to manually set up and interact with an LLM on your Android device, for a more user-friendly and integrated experience, we’ve developed a simple Flutter application. This app showcases how to leverage Gemma3-n directly within a mobile application, providing a more intuitive interface for interaction.
Key Features of the Flutter App: 🔗
- Simplified Interaction: No need for command-line tools or terminal sessions. Interact with Gemma3-n through a clean and intuitive user interface.
- Direct Integration: The app demonstrates how to directly integrate an LLM into a mobile application for various use cases.
- Cross-Platform Potential: Built with Flutter, this app can be extended to other platforms like iOS with minimal effort.
Get the App and Explore the Code: 🔗
We encourage you to explore the source code and even contribute to its development!
GitHub Repository: https://github.com/tinisoft/flutter_gemma/
We’re excited to see what you build with Gemma3-n on Android! Share your projects and ideas with us.
If you’re seeking skilled Flutter developers to bring your AI-powered mobile application to life, we’re here to help. [contact@tinisoft.tin]