Can RTX 3090 Ti run Nomic Embed Text v1.5?
Yes — runs locally
~132 tok/sec · Instant — feels like typing. No noticeable delay.
The verdict
The RTX 3090 Ti (24 GB VRAM) handles Nomic Embed Text v1.5 comfortably using the FP16 quantization, which fits in 0.8 GB. Expected throughput is around 132 tokens/second, which feels Instant — feels like typing. No noticeable delay. in interactive use. High quality text embedding model. 137M params. Good for RAG and search.
Setup tutorial: Nomic Embed Text v1.5 on RTX 3090 Ti
AI-generated, GPU-specific. Verified commands for your exact hardware.
Nomic Embed Text v1.5 runs at Grade S on an NVIDIA GeForce RTX 3090 Ti with FP16 quantization, achieving ~1432 tok/sec.
Prerequisites
Before starting, ensure you have at least 1GB of free disk space, a 64-bit version of Windows or Linux, NVIDIA driver version 470.82 or later, and CUDA 11.4 or later installed.
Expected performance
With the recommended FP16 quantization, you can expect ~1432 tok/sec performance while using only 0.8GB of VRAM. Given the 24GB VRAM of the RTX 3090 Ti, you will have 23.2GB of headroom for context, allowing for a practical context window of up to 8192 tokens without significant performance degradation.
1. Install runtimeOllama
pip install ollama
ollama init2. Download the model
Download the FP16 quantized model (0.3GB file) from Hugging Face.
ollama pull nomic-ai/nomic-embed-text-v1.5-GGUF:nomic-embed-text-v1.5.f16.gguf3. Run it
ollama run nomic-ai/nomic-embed-text-v1.5-GGUF:nomic-embed-text-v1.5.f16.gguf
ollama chat --model nomic-ai/nomic-embed-text-v1.5-GGUF:nomic-embed-text-v1.5.f16.gguf4. Optimize for RTX 3090 Ti
For optimal performance on the NVIDIA GeForce RTX 3090 Ti with 24GB VRAM, set --n-gpu-layers to 137 (the total number of layers in the model). Enable flash attention with --flash-attn and consider using tensor parallelism with --tensor-parallel-size 2 to further optimize throughput. This configuration will utilize approximately 0.8GB of VRAM, leaving ample headroom for larger context windows.
Troubleshooting
Out of memory error during inference
Reduce the batch size or context length. Alternatively, try reducing --n-gpu-layers or disabling flash attention.
Slow inference speed
Ensure that the CUDA toolkit is correctly installed and that the GPU drivers are up to date. Consider enabling tensor parallelism with --tensor-parallel-size 2.
Model not found
Verify that the model was successfully downloaded and is available in the Ollama model directory. Use `ollama list` to check.
Alternative runtimes
Alternative runtimes include LM Studio, llama.cpp, and Jan. LM Studio is suitable for a more user-friendly interface and easier deployment. llama.cpp offers more control over low-level optimizations and is ideal for advanced users. Jan is a lightweight runtime that is easy to integrate into existing applications. Choose based on your specific needs for performance, ease of use, and integration.
Other models that run great on RTX 3090 Ti
FAQ (20)
What GPU do I need to run Nomic Embed Text v1.5?
Nomic Embed Text v1.5 requires a GPU with at least 0.3 GB of VRAM for basic operation, but 0.8 GB is recommended for optimal performance, especially with higher quantization levels.
Is Nomic Embed Text v1.5 good for coding?
Nomic Embed Text v1.5 is primarily designed for text embedding tasks, which can be useful for code search and retrieval, but it may not be as specialized for coding as models specifically trained on programming data.
Nomic Embed Text v1.5 vs Llama 3.1 8B?
Nomic Embed Text v1.5 has 0.137 billion parameters, making it significantly smaller than Llama 3.1 8B, which has 8 billion parameters. This makes Nomic Embed Text v1.5 more lightweight and easier to run on less powerful hardware, but it may not perform as well on complex tasks.
Can I run Nomic Embed Text v1.5 on a Mac?
Yes, you can run Nomic Embed Text v1.5 on a Mac, provided your Mac meets the minimum VRAM requirements of 0.3 GB and has the necessary software dependencies installed.
How much VRAM does Nomic Embed Text v1.5 need?
Nomic Embed Text v1.5 requires between 0.3 GB and 0.8 GB of VRAM, depending on the quantization level used. Lower quantization levels require less VRAM but may impact performance.
Is Nomic Embed Text v1.5 censored?
Nomic Embed Text v1.5 is not explicitly censored. However, it adheres to ethical guidelines and best practices in AI development, which may influence its training data and output.
Is Nomic Embed Text v1.5 commercial-use allowed?
Yes, Nomic Embed Text v1.5 is licensed under the Apache-2.0 license, which allows for both commercial and non-commercial use without restriction.
Nomic Embed Text v1.5 context length?
Nomic Embed Text v1.5 supports a context length of up to 8192 tokens, which is quite generous for most text embedding tasks.
Want personalized recommendations for your exact setup? Detect my hardware →