Gemma 3 MoE 9B
Gemma 3 MoE variant. 9 B total, 2.5 B active. Strong fit for 12 GB cards.
About This Model
Gemma 3 MoE 9B is Google take on the open MoE recipe. 9 B total / 2.5 B active makes it the natural step-up from Gemma 3 4B for users with 12 GB cards. Same Gemma license terms apply, so commercial use is permitted with attribution but not unrestricted.
Check Your Hardware
See which quantizations of Gemma 3 MoE 9B your hardware can run.
Quantization Options
| Quantization | Bits | File Size | VRAM Needed | RAM Needed | Quality |
|---|---|---|---|---|---|
| Q4_K_M | 4.5 | 5.5 GB | 7 GB | 10 GB | 85% |
Context window & KV cache
Adds 1.00 GB to VRAMLong chats and RAG inputs cost real memory. Drag to see how 32K vs 128K context shifts your grade.
Model native max: 8K tokens. KV-cache estimate is approximate (±30 %); real usage depends on attention layout.
How to run Gemma 3 MoE 9B
Pick a runtime — copy & paste. Commands are pre-filled with this model’s repo.
GUI. Browse → download → chat. MLX on Apple Silicon.
LM Studio home →- 1
Open LM Studio
Go to the 🔍 Search tab.
- 2
Search for
bartowski/gemma-3-moe-9b-GGUF - 3
Download
Pick the Q4_K_M quant — best balance of size vs. quality.
- 4
Chat
Hit ▶ Load Model and start chatting. Toggle 'Local Server' to expose an OpenAI-compatible API on :1234.
Community benchmarks
Real tokens/sec reports from people running Gemma 3 MoE 9B on actual hardware.
No community runs yet for this model. Be the first to submit your numbers.
See It In Action
Real model outputs generated via RunThisModel.com — watch responses stream in real time.
Outputs generated by real AI models via RunThisModel.com. Generation speed shown is from cloud inference. Local speeds vary by hardware — check your device.
Frequently Asked Questions
How much VRAM do I need to run Gemma 3 MoE 9B?
Gemma 3 MoE 9B requires 7GB VRAM minimum with Q4_K_M quantization. For full precision, you need 7GB VRAM.
What is the best quantization for Gemma 3 MoE 9B?
Q4_K_M offers the best balance of quality and VRAM usage. Q8_0 is near-lossless if you have enough VRAM.