Can RTX 4080 SUPER run Nomic Embed Text v1.5?

Yes — runs locally

~156 tok/sec · Instant — feels like typing. No noticeable delay.

Your VRAM

16 GB

Model size

0.137B

Best quant

FP16

VRAM needed

0.8 GB

The verdict

The RTX 4080 SUPER (16 GB VRAM) handles Nomic Embed Text v1.5 comfortably using the FP16 quantization, which fits in 0.8 GB. Expected throughput is around 156 tokens/second, which feels Instant — feels like typing. No noticeable delay. in interactive use. High quality text embedding model. 137M params. Good for RAG and search.

Setup tutorial: Nomic Embed Text v1.5 on RTX 4080 SUPER

AI-generated, GPU-specific. Verified commands for your exact hardware.

TL;DR

Run Nomic Embed Text v1.5 on an NVIDIA GeForce RTX 4080 SUPER with FP16 quantization for Grade S performance at ~955 tok/sec.

Prerequisites

Before starting, ensure you have at least 0.3GB of free disk space, a compatible operating system (Windows or Linux), and the latest NVIDIA drivers (version 525.60.12 or later) with CUDA 11.8 installed.

Expected performance

With the FP16 quantization, you can expect ~955 tok/sec throughput and 0.8GB VRAM usage, leaving 15.2GB of VRAM for context. This allows for a practical context window of up to 8192 tokens, depending on the complexity of the input.

1. Install runtimeOllama

curl -fsSL https://ollama.com/install.sh | sh
ollama config set device cuda

2. Download the model

Download the FP16 quantized model (0.3GB) from Hugging Face.

ollama pull nomic-ai/nomic-embed-text-v1.5-GGUF:nomic-embed-text-v1.5.f16.gguf

3. Run it

ollama run nomic-ai/nomic-embed-text-v1.5-GGUF:nomic-embed-text-v1.5.f16.gguf --device cuda --n-gpu-layers 32 --flash-attn

4. Optimize for RTX 4080 SUPER

For optimal performance on the NVIDIA GeForce RTX 4080 SUPER with 16GB VRAM, use the --n-gpu-layers 32 flag to offload layers to the GPU, and enable --flash-attn for faster attention computation. With 0.8GB VRAM used by the model, you have 15.2GB of VRAM available for context, allowing for a large practical context window.

Troubleshooting

Out of memory errors during inference

Reduce the --n-gpu-layers value or increase the batch size to better utilize the available VRAM.

Slow performance

Ensure that the CUDA backend is enabled with 'ollama config set device cuda'. Also, check if the latest NVIDIA drivers are installed.

Model fails to load

Verify that the model file was downloaded correctly and that there are no network issues. Try re-downloading the model using the 'ollama pull' command.

Alternative runtimes

Alternative runtimes like LM Studio, llama.cpp, and Jan can also run this model. Use LM Studio for a more user-friendly interface, llama.cpp for lightweight deployment, and Jan for cloud-based solutions. However, Ollama provides a balanced approach with good performance and ease of use on the NVIDIA GeForce RTX 4080 SUPER.

Full Nomic Embed Text v1.5 details →

Other models that run great on RTX 4080 SUPER

FAQ (20)

What GPU do I need to run Nomic Embed Text v1.5?

Nomic Embed Text v1.5 requires a GPU with at least 0.3 GB of VRAM for basic operation, but 0.8 GB is recommended for optimal performance, especially with higher quantization levels.

Is Nomic Embed Text v1.5 good for coding?

Nomic Embed Text v1.5 is primarily designed for text embedding tasks, which can be useful for code search and retrieval, but it may not be as specialized for coding as models specifically trained on programming data.

Nomic Embed Text v1.5 vs Llama 3.1 8B?

Nomic Embed Text v1.5 has 0.137 billion parameters, making it significantly smaller than Llama 3.1 8B, which has 8 billion parameters. This makes Nomic Embed Text v1.5 more lightweight and easier to run on less powerful hardware, but it may not perform as well on complex tasks.

Can I run Nomic Embed Text v1.5 on a Mac?

Yes, you can run Nomic Embed Text v1.5 on a Mac, provided your Mac meets the minimum VRAM requirements of 0.3 GB and has the necessary software dependencies installed.

How much VRAM does Nomic Embed Text v1.5 need?

Nomic Embed Text v1.5 requires between 0.3 GB and 0.8 GB of VRAM, depending on the quantization level used. Lower quantization levels require less VRAM but may impact performance.

Is Nomic Embed Text v1.5 censored?

Nomic Embed Text v1.5 is not explicitly censored. However, it adheres to ethical guidelines and best practices in AI development, which may influence its training data and output.

Is Nomic Embed Text v1.5 commercial-use allowed?

Yes, Nomic Embed Text v1.5 is licensed under the Apache-2.0 license, which allows for both commercial and non-commercial use without restriction.

Nomic Embed Text v1.5 context length?

Nomic Embed Text v1.5 supports a context length of up to 8192 tokens, which is quite generous for most text embedding tasks.

Want personalized recommendations for your exact setup? Detect my hardware →