Gemma 4: Google Just Dropped an AI That Runs on Your Phone — and It's Legitimately Powerful

Google just shipped Gemma 4 — a family of four open-source models ranging from tiny (runs on your phone with no internet) to beastly (competing with models 20x its size). Licensed under Apache 2.0 — take it, modify it, make money off it. This isn't "yet another big-tech AI release." This might be the biggest leap in AI accessibility in the past year.

Let's break down what's inside and why you should care.

Right now, something wild is happening in the open model space. A year ago, Gemma 3 was a benchwarmer — losing to Qwen in math, Llama in reasoning, impressing exactly nobody. Now Gemma 4 31B is the third-ranked model in the global Arena AI leaderboard, beating models that are 20x its size. On the AIME 2026 benchmark (competition-level math), its score jumped from 20.8% with Gemma 3 to 89.2% with Gemma 4 — that's not an improvement, that's a resurrection.

And here's the wildest part — the smaller versions run directly on an Android phone, offline. Fully local. With image and voice support. Picture this: a full-blown AI assistant in your pocket that needs no server, no subscription, and none of your data.

What Is Gemma 4 and Why Is Google Giving It Away for Free

Open-Weights vs. Closed Models: A 30-Second Primer

The AI world has two camps. Closed models are where you pay for API access and pray the company doesn't go bankrupt or jack up prices. Think GPT-4o from OpenAI, Claude from Anthropic, Gemini from Google itself. You can't download them, run them locally, or peek under the hood.

Open-weight models are a completely different game. You get the actual model weights (the "brain" of the neural network), and you do whatever you want: run it on your own hardware, fine-tune it, bake it into an app, monetize it. The major players here are Meta with Llama, Alibaba with Qwen, and now Google with Gemma.

Why Apache 2.0 Is a Big Deal

Gemma 4 ships under an Apache 2.0 license — one of the most permissive licenses in software. Previous Gemma versions were open-weight but came with a pile of restrictions: can't do this, can't do that. Now — just take it and build. The only condition is attribution. For comparison: Llama 4 from Meta has restrictions for apps with 700+ million users. Gemma 4 has no such limits.

So What's in It for Google?

The cynical answer: getting developers hooked on the Google Cloud and Vertex AI ecosystem. But there's a less cynical take too: Gemma 4 has been downloaded over 400 million times (counting all generations), and the community has created more than 100,000 model variations. It's a massive flywheel: the more people use Gemma, the more tools, features, and integrations spring up around it. Google wins even without direct sales.

Four Models — Which One Is for You

Gemma 4 isn't one model — it's a family of four, each built for different scenarios. Let's break them down like you're picking an RPG class.

Gemma 4 E2B — "The Pocket Prodigy"

• Parameters: ~2B effective (5.1B total, but some live in flash storage)
• Context: 128K tokens
• Modalities: text + images + audio
• VRAM: from 4 GB in 4-bit

Who it's for: Phones, Raspberry Pis, smart speakers. This model was built to run directly on-device — fully offline. It understands text, sees images, and hears voice. On Android, it'll power the next generation of Gemini Nano on flagship smartphones.

Real-world example: Point your camera at a restaurant menu abroad → the model reads the text from the photo and translates it. No internet. On your phone.

Gemma 4 E4B — "The Laptop All-Rounder"

• Parameters: ~4B effective (8B total)
• Context: 128K tokens
• Modalities: text + images + audio
• VRAM: from 5.5 GB in 4-bit

Who it's for: A laptop or a modest desktop. This is E2B on steroids — it also understands voice and images, but it's noticeably smarter. On LiveCodeBench (a code-writing benchmark), E4B scores 52%, while Gemma 3 27B (a model 7x larger!) manages just 29%.

Real-world example: A local coding assistant on your laptop — suggests code, explains errors, works with Android Studio right out of the box. All without internet and without API keys.

Gemma 4 26B A4B — "The Lazy Genius" (Mixture of Experts)

• Parameters: 26B total, but only ~4B active during generation
• Context: 256K tokens
• Modalities: text + images
• VRAM: from 16 GB in 4-bit

Who it's for: A desktop with a decent GPU (RTX 3060 and up). This is the craftiest model in the family. It knows as much as a 26-billion-parameter model, but during text generation it only uses 4 billion — the rest are chilling. Result: sixth place on the global Arena AI leaderboard while running at speeds comparable to a tiny 4B model.

Real-world example: Fire it up on a home PC with 24 GB RAM — the model works as an AI assistant for writing code, analyzing documents, and brainstorming. According to Google's Android blog, this model needs 24 GB of total RAM — totally doable.

Gemma 4 31B — "The Heavy Artillery"

• Parameters: 31B (all active)
• Context: 256K tokens
• Modalities: text + images
• VRAM: from 20 GB in 4-bit

Who it's for: A beefy desktop or server. This is the flagship. Third place globally among open models — the only things ahead of it are models that weigh several times more. On GPQA Diamond (grad-school-level questions) it scores 84.3%, nearly double what Gemma 3 managed.

Real-world example: A corporate AI assistant running on a single NVIDIA H100 server. Or — if you've got an RTX 4090 at home — a local brain trust for deep research, long documents, and complex reasoning.

Cheat Sheet: What Goes Where

@startuml
title Gemma 4 Model Family Architecture

package "Gemma 4 Model Family" {

  package "Edge Models (Mobile/Laptop)" {
    rectangle "Gemma 4 E2B\n~2B effective / 5.1B total\n128K context\nText + Image + Audio" as E2B #lightgreen
    rectangle "Gemma 4 E4B\n~4B effective / 8B total\n128K context\nText + Image + Audio" as E4B #lightblue
  }

  package "Desktop/Server Models" {
    rectangle "Gemma 4 26B A4B\n~4B active / 26B total (MoE)\n256K context\nText + Image" as A4B #gold
    rectangle "Gemma 4 31B\n31B active\n256K context\nText + Image" as B31 #lightcoral
  }
}

E2B -[hidden]down-> E4B
E4B -[hidden]down-> A4B
A4B -[hidden]down-> B31

note bottom of E2B
  Per-Layer Embedding (PLE)
  Flash storage for embeddings
  4GB VRAM (4-bit)
  Runs on Android phones
end note

note bottom of E4B
  Per-Layer Embedding (PLE)
  5.5GB VRAM (4-bit)
  Laptop/desktop use
end note

note bottom of A4B
  Mixture of Experts (MoE)
  8 experts + 1 shared
  16GB VRAM (4-bit)
  Best performance/speed ratio
end note

note bottom of B31
  Dense model (all params active)
  20GB VRAM (4-bit)
  Maximum capability
end note
@enduml

How It Actually Works: Architecture for Non-Engineers

Okay, the models are impressive. But why are they so fast, smart, and compact? Let's break down three key tricks Google uses in Gemma 4 — based on a deep dive by Maarten Grootendorst, a Google DeepMind engineer.

Trick #1: Sliding Window + Global Attention — "Read Fast, Occasionally Re-read"

Imagine you're reading a long book. You could read every page carefully, memorizing everything from page one to where you are now. That's global attention — accurate, but computationally brutal.

Or you could read with a "sliding window": only remember the last few pages and keep the rest in background memory. That's sliding window attention (or local attention) — fast and cheap, but sometimes you lose connections to earlier parts of the text.

Gemma 4 pulls a clever move: it alternates between these two modes. Most layers (4 out of 5 for the E2B model, or 5 out of 6 for the rest) operate in the fast "sliding" mode with a window of 512 tokens (for small models) or 1024 (for large ones). Every fifth or sixth layer "looks back" at the entire text.

Key fix from Gemma 3: the last layer is now always global. In the previous version, due to an unlucky division, the final layer could end up "blind" — seeing only the nearest tokens, which wrecked the final output. Now the model always finishes its reasoning with the full picture in view.

@startuml
title Sliding Window + Global Attention Mechanism

package "Gemma 4 Layer Processing" {
  rectangle "Layer 1\nSliding Window (512)\nLocal tokens only" as L1 #lightblue
  rectangle "Layer 2\nSliding Window (512)\nLocal tokens only" as L2 #lightblue
  rectangle "Layer 3\nSliding Window (512)\nLocal tokens only" as L3 #lightblue
  rectangle "Layer 4\nSliding Window (512)\nLocal tokens only" as L4 #lightblue
  rectangle "Layer 5\nGLOBAL ATTENTION\nFull context" as L5 #gold

  L1 --> L2
  L2 --> L3
  L3 --> L4
  L4 --> L5
}

package "Attention Patterns" {
  rectangle "Sliding Window\nSees 512 recent tokens\nFast, cheap" as sliding #lightcyan
  rectangle "Global Attention\nSees ALL tokens\nSlow, expensive" as global #lightyellow
}

sliding -[hidden]down-> global

note right of L1
  Small models (E2B):
  5 layers total
  4 sliding + 1 global
  
  Large models:
  6 layers total
  5 sliding + 1 global
end note

note right of L5
  Key Gemma 4 fix:
  Last layer ALWAYS global
  (bug fixed from Gemma 3)
  
  Ensures final output
  sees full context
end note
@enduml

Trick #2: Mixture of Experts — "Not All Neurons Work at the Same Time"

This is the 26B A4B's signature move, and it's genuinely elegant.

Imagine a university with 128 professors. When a student shows up with a question, a router identifies the topic and sends them to 8 relevant experts plus 1 "generalist" who knows a bit of everything (this "shared expert" is three times larger than the others — think of them as the department dean). The other 119 professors are off having coffee.

The result: the model "knows" as much as all 26 billion parameters combined, but when generating each token, only ~4 billion are actually working. It's like having the brain of a scientist but spending the energy of a grade-schooler solving a textbook problem.

In practice, 26B A4B runs at nearly the same speed as a 4B model, while scoring comparably to the 31B Dense. On AIME 2026 (competition-level math), the gap between them is just one percentage point: 89.2% vs 88.3%. That's, to put it mildly, impressive.

@startuml
title Mixture of Experts (MoE) Architecture - Gemma 4 26B A4B

package "Gemma 4 26B A4B Architecture" {

  rectangle "Input Token" as input #lightblue

  package "Expert Router (Gating Network)" as router #lightgreen
  rectangle "Router Decision:\nWhich experts to activate?" as routing_decision #orange

  package "Expert Layer (128 Experts Total)" as experts {
    rectangle "Expert 1" as E1 #lightgray
    rectangle "Expert 2" as E2 #lightgray
    rectangle "Expert 3" as E3 #lightgray
    rectangle "..." as dots #lightgray
    rectangle "Expert 128\n(Shared - 3× larger)" as E128 #gold
  }

  package "Active Experts per Token" as active {
    rectangle "Expert 1" as A1 #lightblue
    rectangle "Expert 2" as A2 #lightblue
    rectangle "Expert 7" as A7 #lightblue
    rectangle "Expert 128\n(Shared)" as A128 #gold
  }

  rectangle "Output Token" as output #lightcoral
}

input --> router
router --> routing_decision
routing_decision --> E1 : Route to\nrelevant experts
routing_decision --> E2
routing_decision --> E7
routing_decision --> E128 : Always included\n(shared expert)

E1 --> A1
E2 --> A2
E7 --> A7
E128 --> A128

A1 --> output : Combined output
A2 --> output
A7 --> output
A128 --> output

note bottom of router
  Top-K + 1 routing:
  Selects 8 most relevant experts
  + 1 always-on shared expert
  
  Result: ~4B active params
  out of 26B total
end note

note bottom of experts
  128 experts in FFN layer
  Only 8-9 active per token
  
  Shared expert is 3× larger
  than individual experts
end note

note bottom of output
  26B total knowledge
  4B computation per token
  
  Same speed as 4B dense
  Competitive with 31B dense
end note
@enduml

Trick #3: Per-Layer Embeddings — "A Cheat Sheet in Every Pocket"

This technique is used in the small E2B and E4B models, and it's the reason they can run on a phone.

Normally, a neural network stores a "dictionary" (embedding table) — a massive table where each of 262,144 tokens maps to a numeric vector. This dictionary is needed once — at the input stage, to turn words into numbers. Problem: it eats RAM alive.

Gemma 4 E2B does it differently. It has a small, dedicated vector for each token in each of its 35 layers (dimension 256, not the main 1536). These mini-dictionaries live not in RAM, but in the device's flash storage (the same place as your photos and apps). During inference, the model loads the necessary vectors for the input tokens once, then uses them as "cheat sheets" for each layer — a reminder of which token is being processed.

It's as if a student at an exam didn't keep the whole textbook in their head but glanced at small flashcards with key facts. The brain stays free for reasoning, and the cards sit in the backpack, not in your hands.

Thanks to this, the "E" in the name stands for "effective parameters." E2B needs only ~2B parameters in RAM, even though the full model weighs 5.1B. The difference — those PLE tables — happily lives in flash.

@startuml
title Per-Layer Embeddings (PLE) - Flash Storage Optimization

package "Gemma 4 E2B/E4B Memory Architecture" {

  package "Accelerator Memory (RAM/VRAM) - SMALL" {
    rectangle "Active Transformer\n~2-4B parameters\n(E2B/E4B effective)" as active #lightgreen
    rectangle "Layer Processing\n35 layers" as layers #lightblue
  }

  package "Flash Storage - LARGE" {
    rectangle "PLE Tables\nToken embeddings per layer\n(dim 256, not 1536)" as ple #lightyellow
    rectangle "Full Model Weights\n5.1B (E2B) / 8B (E4B)" as full_weights #lightgray
  }

  package "Embedding Flow" {
    rectangle "Input Tokens" as tokens #lightblue
    rectangle "Load PLE from Flash" as load #orange
    rectangle "Cache in RAM" as cache #lightcoral
    rectangle "Use across layers" as use #violet
  }
}

tokens --> load
load --> cache
cache --> use
use --> layers

ple --> cache : Load once,\ncache in RAM

active --> layers : Active computation

note bottom of active
  E2B: ~2B active params
  E4B: ~4B active params
  
  Only transformer layers
  stay in accelerator memory
end note

note bottom of ple
  PLE = Per-Layer Embeddings
  
  35 layers × 262K tokens × 256 dim
  Lives in FLASH, not RAM
  
  Loaded once per inference,
  used as "cheat sheets"
end note

note bottom of full_weights
  E2B total: 5.1B parameters
  E4B total: 8B parameters
  
  "E" = Effective parameters
  RAM usage = effective count
end note
@enduml

Bonus Optimizations (for the Curious)

Google didn't stop at three big tricks. Here are a few more hacks that make Gemma 4 faster and leaner:

K=V (Keys = Values). In global attention layers, the keys and values are the same thing. Normally, a neural network stores separate "keys" (what it searches with) and "values" (what it retrieves). Gemma 4 merged them: search and retrieve — same object. This cuts the KV-cache — the biggest memory hog during long contexts — in half.

Grouped Query Attention. In global layers, 8 query heads share a single key-value pair (instead of the usual 2 in local layers). Another way to compress the cache: with global attention, the context is enormous, so saving on each head adds up fast.

p-RoPE. Positional encoding (the thing that tells the model where each word sits in the text) is applied to only 25% of dimension pairs. Why? With very long contexts (256K tokens — roughly the length of War and Peace) the distances between tokens get so large that standard encoding starts to "hallucinate" positions. Trimming to 25% constrains the space of possible positions and helps the model generalize better over long texts.

Multimodality: A Model That "Sees" and "Hears"

Vision: How Gemma 4 Understands Images

All four Gemma 4 models are multimodal. That means they handle not just text, but images too.

Under the hood sits a Vision Transformer (ViT): the model takes an image, slices it into 16×16 pixel patches (like a mosaic), and converts each patch into a numeric vector. From there, those vectors are processed by the transformer — the exact same way words in text are.

What sets Gemma 4 apart from competitors:

Variable aspect ratio. The model doesn't crop images to a square. Thanks to 2D RoPE (two-dimensional positional encoding), it correctly handles horizontal panoramas, vertical stories, and every other proportion.

Soft token budget. You can choose how many "visual tokens" to spend on an image: 70, 140, 280, 560, or 1120. More tokens = higher resolution (up to ~1088 pixels) but slower processing. Fewer = faster but coarser. It's like a graphics quality slider in a game: want "ultra" — wait for it, want a "quick scan" — set it to 70 tokens.

The encoder size also varies by model: 150 million parameters for the small ones (E2B, E4B) and 550 million for the big ones (26B, 31B).

In practice: Gemma 4 31B scores 76.9% on MMMU Pro (a multimodal benchmark with university-level tasks) and 85.6% on MATH-Vision (solving math problems from images). That's serious.

@startuml
title Vision Processing - Gemma 4 Multimodal Pipeline

package "Vision Input Processing" {
  rectangle "Image Input\n(Any aspect ratio)" as image #lightblue
  rectangle "Patch Extraction\n16×16 pixel patches" as patches #lightgreen
  rectangle "ViT Encoder\n150M (E2B/E4B) / 550M (26B/31B)" as vit #orange
  rectangle "Visual Tokens\n70-1120 tokens (soft budget)" as visual_tokens #lightyellow
}

package "Transformer Processing" {
  rectangle "Gemma 4 Transformer\nLayers + Sliding Window" as transformer #lightcoral
  rectangle "Output Generation" as output #lightsteelblue
}

image --> patches
patches --> vit
vit --> visual_tokens
visual_tokens --> transformer
transformer --> output

note right of image
  2D RoPE encoding
  Preserves aspect ratio
  No cropping needed
end note

note right of visual_tokens
  Variable token budget:
  70 / 140 / 280 / 560 / 1120
  
  More tokens = higher resolution
  Up to ~1088 pixels
end note

note right of vit
  Vision Transformer (ViT)
  Small models: 150M params
  Large models: 550M params
end note
@enduml

Hearing: The Audio Encoder in E2B and E4B

Now this is genuinely unique. The small E2B and E4B models are the only ones in their class that natively understand audio. Neither Llama 4 nor Qwen 3.5 can do this.

Here's how it works under the hood:

1. Raw audio → mel spectrogram. The sound wave is converted into a 2D image where the x-axis is time and the y-axis is frequency. Think of the equalizer visualization in a music player.
2. Chunking + convolutions. The spectrogram is sliced into chunks that pass through two layers of 2D convolutions — shrinking the data down.
3. Conformer. A hybrid of a transformer and a convolutional network, purpose-built for audio. Turns the compressed data into contextual embeddings.
4. Linear projection into the Gemma 4 embedding space — and voilà, the model "hears."

All of this is optimized for speech recognition and translation. On the FLEURS benchmark, E4B achieves an error rate of just 0.08 — extremely low. And on CoVoST (speech translation) — 35.54, which is competitive for a model this size.

@startuml
title Audio Processing Pipeline - E2B/E4B Native Support

package "Audio Input Processing" {
  rectangle "Raw Audio\n(16kHz sampling)" as raw #lightblue
  rectangle "MEL Spectrogram\nTime × Frequency" as mel #lightgreen
  rectangle "2D Convolution\n(Chunked)" as conv #orange
  rectangle "Conformer\n(Transformer + CNN hybrid)" as conformer #lightcoral
  rectangle "Linear Projection\n→ Gemma embedding space" as projection #lightyellow
}

package "Audio Capabilities" {
  rectangle "Speech Recognition\n(ASR)" as asr #lightblue
  rectangle "Speech Translation\n(AST)" as ast #lightgreen
}

raw --> mel : Convert
mel --> conv : Chunk & compress
conv --> conformer : Contextual embeddings
conformer --> projection : Gemma tokens
projection --> asr
projection --> ast

note bottom of raw
  16kHz sampling
  Optimized for speech
end note

note bottom of mel
  Spectrogram =
  Audio as 2D image
  
  X-axis: time
  Y-axis: frequency
end note

note bottom of conformer
  Conformer architecture:
  Transformer + CNN hybrid
  
  Purpose-built for audio
end note

note bottom of projection
  Audio tokens projected
  into Gemma 4 embedding
  
  Enables audio + text
  multimodal processing
end note
@enduml

What This Means for You: Practical Use Cases

Enough theory. Here are concrete things you can do with Gemma 4 right now.

Scenario 1: Voice Assistant on Your Phone (E2B)

The E2B model runs on Android devices with 8 GB of RAM. No internet. With voice understanding. That means:

• Dictate a voice message → the model transcribes and edits the text
• Photograph a document → the model extracts text (OCR) and answers questions about it
• Speak in one language → the model translates into another

All of this — locally. No data leaves your device. For privacy-conscious people (and in 2026, that's not paranoia, it's common sense) — the perfect setup.

Scenario 2: Local Coding Assistant (E4B / 26B A4B)

Android Studio already supports Gemma 4 as a local model for Agent Mode. But it works outside Android Studio too. Install Ollama, download the model with one command — and you've got an AI coder sitting on your laptop that:

• Writes functions from descriptions
• Explains unfamiliar code
• Catches bugs
• Refactors legacy code

The 26B A4B model only needs 24 GB RAM — a standard modern desktop. And E4B fits in 12 GB — enough for any laptop with a discrete GPU.

Scenario 3: Document and Image Analysis (31B)

The context window goes up to 256K tokens — roughly 500 pages of text in a single prompt. The 31B model scores 66.4% on MRCR v2 (finding information in long documents), while Gemma 3 managed a pitiful 13.5%. Long context used to be a checkbox feature — now it actually works.

What you can do:

• Feed an entire code repository and ask the model to find vulnerabilities
• Throw in a lengthy contract and get a summary of key terms
• Send a batch of UI screenshots and get a detailed UX audit

Scenario 4: Build Your Own AI Product (for Developers and Startups)

Apache 2.0 means: you can take Gemma 4, fine-tune it on your own data, and ship it in a commercial product. No royalties, no user-count caps. The model is available on Hugging Face, via Ollama, Unsloth, NVIDIA NIM, and other platforms.

Google already highlights real use cases: a Bulgarian language model built on Gemma and Yale University's Cell2Sentence-Scale model for cancer research. These aren't marketing fantasies — this is what's actually being built on open models.

Gemma 4 vs. the Competition: Who Wins

A quick comparison against the main rivals — Llama 4 from Meta and Qwen 3.5 from Alibaba:

@startuml
title Open-Weight Model Comparison - Gemma 4 vs Llama 4 vs Qwen 3.5

package "Google" {
  rectangle "Gemma 4 31B\n#3 Arena AI\nApache 2.0 license" as G31 #lightgreen
  rectangle "Gemma 4 26B A4B\nMoE - 4B active\nBest efficiency" as G26 #lightblue
  rectangle "Gemma 4 E2B\nOn-device + Audio\nPhone-capable" as GE2 #gold
}

package "Meta" {
  rectangle "Llama 4\n~#10 Arena AI\nRestrictions apply" as L4 #lightgray
}

package "Alibaba" {
  rectangle "Qwen 3.5\n#2 Arena AI\nApache 2.0 license" as Q35 #lightyellow
}

GE2 ..> L4 : No small model
G26 ..> Q35 : Competes in reasoning
G31 ..> Q35 : Close in benchmarks

note bottom of G31
  Gemma 4 strengths:
  + Best reasoning/code/math
  + Native on-device audio
  + No usage restrictions
  
  Qwen 3.5 strengths:
  + 250K vocabulary
  + 201 languages
  + Longer context (quality)
end note
@enduml

The key takeaway from AI.rs: "A month ago, Gemma 3 was an underdog. Now Gemma 4 leads in reasoning, math, code, and visual tasks. A new leader has emerged in the open-model race."

Qwen 3.5 still has the edge in multilingual support (250K vocabulary, 201 languages vs. Gemma's 140), and Llama 4 is the context-length champion. But in terms of intelligence-per-parameter, Gemma 4 is ahead right now.

What Comes Next and Why This Changes Everything

A 100B+ Parameter Gemma Is Already in the Works

According to Interconnects.ai, Google is working on a MoE Gemma model with over 100 billion parameters. If the current 26–31B models are competing with the best — imagine what a hundred billion will do.

Open-Source AI Is Winning

A year ago, it seemed like the future of AI belonged to closed APIs. Today, open models are cracking the top 3 in global rankings, running on phones, and shipping under the most permissive licenses around. Gemma downloads have blown past 400 million, and the pace is only accelerating.

What does this mean for a regular person? AI is shifting from a service you pay a subscription for to a tool you own. The way Linux once transformed servers, and Android transformed phones, open models are transforming AI. You're no longer at the mercy of a single company, its pricing decisions, or its rules about what you can and can't do.

Gemma 4 isn't just another release. It's a signal: the era of AI that runs on your device, under your control, with no subscription and no compromises — is already here.

Sources: Google Blog, Maarten Grootendorst — A Visual Guide to Gemma 4, Hugging Face Blog, Engadget, Lushbinary Comparison Guide, AI.rs Updated Benchmarks, Android Developers Blog, Unsloth Documentation.

Published on 4/5/2026