Can I run Dolphin 3.0 Llama 3.1 8B on my device?

Dolphin 3.0 Llama 3.1 8B requires a minimum of 5.3GB VRAM. Use RunThisModel to check your specific hardware compatibility and find the best quantization for your device.

How much VRAM does Dolphin 3.0 Llama 3.1 8B need?

Dolphin 3.0 Llama 3.1 8B needs 5.3GB VRAM at minimum (BF16 quantization). Higher quality quantizations need more: BF16: 16.5GB, Q4_K_M: 5.3GB.

How do I download Dolphin 3.0 Llama 3.1 8B?

You can download Dolphin 3.0 Llama 3.1 8B in GGUF format from HuggingFace (4.8GB minimum). Use the RunThisModel iOS app to download and run it directly on your device, or download manually from HuggingFace.

Can Dolphin 3.0 Llama 3.1 8B run on iPhone?

Dolphin 3.0 Llama 3.1 8B can run on iPhones with 8GB RAM (iPhone 15 Pro+) using smaller quantizations, though performance may be limited.

Cognitive Computations

Dolphin 3.0 Llama 3.1 8B

Name: Dolphin 3.0 Llama 3.1 8B
Author: Cognitive Computations

Eric Hartford's flagship uncensored fine-tune of Llama 3.1 8B. Steerable assistant with personality controls and no built-in refusals — strong general-purpose pick at 8B.

8B parametersllamallama3.1128K context5.3GB - 16.5GB VRAM

Check Your Hardware

See which quantizations of Dolphin 3.0 Llama 3.1 8B your hardware can run.

Quantization Options

Quantization	Bits	File Size	VRAM Needed	RAM Needed	Quality
BF16	16	16 GB	16.5 GB	17 GB	100%
Q4_K_M	4.5	4.8 GB	5.3 GB	5.8 GB	85%

Context window & KV cache

Adds 1.00 GB to VRAM

Long chats and RAG inputs cost real memory. Drag to see how 32K vs 128K context shifts your grade.

Model native max: 128K tokens. KV-cache estimate is approximate (±30 %); real usage depends on attention layout.

How to run Dolphin 3.0 Llama 3.1 8B

Pick a runtime — copy & paste. Commands are pre-filled with this model’s repo.

GUI. Browse → download → chat. MLX on Apple Silicon.

LM Studio home →

1
Open LM Studio
Go to the 🔍 Search tab.

Search for

bartowski/dolphin-2.9.4-llama3.1-8b-GGUF

3
Download
Pick the Q4_K_M quant — best balance of size vs. quality.
4
Chat
Hit ▶ Load Model and start chatting. Toggle 'Local Server' to expose an OpenAI-compatible API on :1234.

Community benchmarks

Real tokens/sec reports from people running Dolphin 3.0 Llama 3.1 8B on actual hardware.

No community runs yet for this model. Be the first to submit your numbers.

Self-host serving plan

Want to host Dolphin 3.0 Llama 3.1 8Bfor many users? Or run it on a card that’s technically too small? Slide the knobs.

Concurrent users: 1VRAM budget: 24 GBSystem RAM: 64 GBAllow NVMe spilloverPermit weights to spill to NVMe SSD (last resort, ~50× slower)

VRAM needed

6.5 GB

5.3 GB weights + 0.7 GB KV

Aggregate tok/s

across 1 user

Per-user tok/s

8 B dense

✅ Fits in 24 GB VRAM with 17.5 GB headroom. Pure-GPU inference — full speed.

Throughput is a sub-linear estimate: doubling users adds ~70 % of single-user TPS until ~8, then plateaus on memory bandwidth. MoE models scale concurrency much better because each user activates a different subset of experts.

Download & Run

HuggingFace

View model & download weights

Ollama

One-command install & run

See It In Action

Real model outputs generated via RunThisModel.com — watch responses stream in real time.

Llama 3.3 70B responding...

Outputs generated by real AI models via RunThisModel.com. Generation speed shown is from cloud inference. Local speeds vary by hardware — check your device.

Frequently Asked Questions

How much VRAM do I need to run Dolphin 3.0 Llama 3.1 8B?

Dolphin 3.0 Llama 3.1 8B requires 5.3GB VRAM minimum with BF16 quantization. For full precision, you need 16.5GB VRAM.

What is the best quantization for Dolphin 3.0 Llama 3.1 8B?

Q4_K_M offers the best balance of quality and VRAM usage. Q8_0 is near-lossless if you have enough VRAM.