Runs Perfectly (S/A/B)
142
With Offloading (C/D)
2
Cannot Run (F)
1
S
Excellent (138 models)
Runs great with room for large context windows
Mixtral 8x7B Instruct
Mistral AI
46.7BChat / LLMQ5_K_M30.5GB
Phi-3.5 MoE
Microsoft
41.9BChat / LLMQ4_K_M24.1GB
Qwen 2.5 32B
Alibaba
32BChat / LLMQ4_K_M19.0GB
Skyfall 31B v4.2
TheDrummer
31BChat / LLMQ4_K_M18.2GB
Qwen3 30B-A3B
Alibaba
30.5BChat / LLMQ4_K_M20.0GB
Gemma 3 27B
27BChat / LLMQ4_K_M15.9GB
Dolphin Mistral 24B (Venice Edition)
Cognitive Computations
24BChat / LLMQ8_025.9GB
Dolphin 3.0 R1 Mistral 24B
Cognitive Computations
24BChat / LLMQ4_K_M13.8GB
Cydonia 24B v4.3
TheDrummer
24BChat / LLMQ4_K_M13.8GB
Mistral Small 22B
Mistral AI
22BChat / LLMQ4_K_M12.9GB
Codestral 22B (abliterated)
failspy
22BCodingQ4_K_M12.9GB
Magnum v4 22B
Anthracite
22BChat / LLMQ4_K_M12.9GB
DeepSeek MoE 16B
DeepSeek
16.4BChat / LLMQ4_K_M11.0GB
Rocinante XL 16B v1
TheDrummer
16BChat / LLMQ4_K_M9.6GB
Phi-4
Microsoft
14BChat / LLMQ8_015.0GB
Qwen 2.5 14B
Alibaba
14BChat / LLMQ8_015.1GB
Qwen 2.5 Coder 14B
Alibaba
14BCodingQ8_015.1GB
Code Llama 13B Instruct
Meta
13BCodingQ4_K_M7.8GB
Gemma 3 12B
12BChat / LLMQ8_012.2GB
Mistral Nemo 12B
Mistral AI
12BChat / LLMQ8_012.6GB
FLUX.1 Schnell (GGUF)
Black Forest Labs
12BImage GenerationQ5_014.0GB
FLUX.1 Dev (GGUF)
Black Forest Labs
12BImage GenerationQ5_014.0GB
Magnum v4 12B
Anthracite
12BChat / LLMBF1624.5GB
Rocinante 12B v1.1
TheDrummer
12BChat / LLMBF1624.5GB
Mistral Nemo Base 12B
Mistral AI
12BChat / LLMBF1624.5GB
Solar 10.7B
Upstage
10.7BChat / LLMQ8_011.1GB
Falcon 3 10B
TII
10BChat / LLMQ8_010.7GB
Mochi 1 Preview
Genmo
10Bvideo-genFP1630.0GB
Gemma 2 9B Instruct
9.2BChat / LLMQ8_09.7GB
Yi 1.5 9B Chat
01.AI
9BChat / LLMQ8_09.2GB
Yi Coder 9B
01.AI
9BCodingQ8_09.2GB
Gemma 3 MoE 9B
9BChat / LLMQ4_K_M7.0GB
CodeGemma 7B
8.5BCodingQ8_08.9GB
DeepSeek R1 Distill 8B
DeepSeek
8BChat / LLMQ8_08.4GB
Llama 3.1 8B Instruct
Meta
8BChat / LLMFP1617.0GB
Granite 3.3 8B
IBM
8BChat / LLMQ8_08.6GB
Dolphin 3.0 Llama 3.1 8B
Cognitive Computations
8BChat / LLMBF1616.5GB
NeuralDaredevil 8B (abliterated)
mlabonne
8BChat / LLMBF1616.5GB
Llama 3.1 8B Instruct (abliterated)
mlabonne
8BChat / LLMBF1616.5GB
Stheno L3 8B v3.2
Sao10K
8BChat / LLMBF1616.5GB
Qwen3 8B Base
Alibaba
8BChat / LLMBF1616.5GB
EXAONE 3.5 7.8B
LG AI
7.8BChat / LLMQ8_08.2GB
InternLM 2.5 7B
Shanghai AI Lab
7.7BChat / LLMQ8_08.2GB
Qwen 2.5 7B Instruct
Alibaba
7.6BChat / LLMQ8_09.0GB
Qwen 2.5 Coder 7B
Alibaba
7.6BCodingQ8_08.0GB
Mistral 7B Instruct v0.3
Mistral AI
7.3BChat / LLMFP1615.5GB
LLaVA 1.6 7B
LLaVA
7BMultimodalQ8_08.5GB
Falcon 3 7B
TII
7BChat / LLMQ8_08.3GB
OLMo 2 7B
Allen AI
7BChat / LLMQ8_07.7GB
OpenChat 3.5 7B
OpenChat
7BChat / LLMQ8_07.7GB
StarCoder2 7B
BigCode
7BCodingQ8_07.6GB
Code Llama 7B
Meta
7BCodingQ8_07.2GB
OLMoE 1B-7B
AI2
6.9BChat / LLMQ8_07.3GB
DeepSeek Coder 6.7B
DeepSeek
6.7BCodingQ8_07.2GB
Yi 1.5 6B Chat
01.AI
6BChat / LLMQ8_06.5GB
Wan 2.2 TI2V 5B
Wan-AI
5Bvideo-genFP1616.0GB
CogVideoX 5B
THUDM
5Bvideo-genFP1616.0GB
Phi-3.5 Vision
Microsoft
4.2BMultimodalQ4_K_M3.2GB
Gemma 3 4B
4BChat / LLMQ8_04.3GB
Nemotron Mini 4B
NVIDIA
4BChat / LLMQ8_04.7GB
Danube 3 4B
H2O.ai
4BChat / LLMQ8_04.4GB
Phi-3.5 Mini 3.8B
Microsoft
3.8BChat / LLMQ8_04.3GB
Phi-4 Mini 3.8B
Microsoft
3.8BChat / LLMQ8_04.3GB
Stable Diffusion XL (CoreML)
Stability AI
3.5BImage GenerationCoreML3.3GB
SDXL Turbo (GGUF)
Stability AI
3.5BImage GenerationQ5_05.0GB
Granite 3.0 3B-A800M
IBM
3.4BChat / LLMQ4_K_M2.4GB
Llama 3.2 3B Instruct
Meta
3.2BChat / LLMQ8_03.7GB
Qwen 2.5 3B
Alibaba
3BChat / LLMQ8_03.9GB
Qwen 2.5 Coder 3B
Alibaba
3BCodingQ8_03.9GB
Falcon 3 3B
TII
3BChat / LLMQ8_03.8GB
StableLM Zephyr 3B
Stability AI
3BChat / LLMQ8_03.3GB
Rocket 3B
Pansophic
3BChat / LLMQ8_03.3GB
StarCoder2 3B
BigCode
3BCodingQ8_03.5GB
Stable Code 3B
Stability AI
3BCodingQ8_03.3GB
PaliGemma 3B
3BMultimodalQ4_K_M2.5GB
Gemma 2 2B
2.6BChat / LLMQ8_03.1GB
Stable Diffusion 3 Medium (GGUF)
Stability AI
2.5BImage GenerationQ8_09.2GB
Hunyuan3D 2
Tencent
2.5B3d-genFP1616.0GB
EXAONE 3.5 2.4B
LG AI
2.4BChat / LLMQ8_03.1GB
Qwen2-VL 2B
Alibaba
2.2BMultimodalQ8_02.0GB
CodeGemma 2B
2BCodingQ8_03.0GB
MiniCPM-V 2.6
OpenBMB
2BMultimodalQ8_03.0GB
Granite 3.3 2B
IBM
2BChat / LLMQ8_03.0GB
Moondream 2
Moondream
1.8BMultimodalQ4_K_M1.5GB
SmolLM2 1.7B
HuggingFace
1.7BChat / LLMQ8_02.2GB
Whisper Large v3
OpenAI
1.55BSpeech RecognitionQ8_03.4GB
Qwen 2.5 1.5B
Alibaba
1.5BChat / LLMQ8_02.3GB
DeepSeek R1 Distill 1.5B
DeepSeek
1.5BChat / LLMQ8_02.3GB
Qwen 2.5 Coder 1.5B
Alibaba
1.5BCodingQ8_02.3GB
Yi Coder 1.5B
01.AI
1.5BCodingQ8_02.0GB
ACE-Step 1.5XL
ACE Studio
1.5BAudio GenerationFP168.0GB
DeepSeek Coder 1.3B
DeepSeek
1.3BCodingQ8_01.8GB
Granite 3.0 1B-A400M
IBM
1.3BChat / LLMQ4_K_M1.3GB
Llama 3.2 1B Instruct
Meta
1.24BChat / LLMFP162.8GB
TRELLIS Image Large
Microsoft
1.2B3d-genFP1612.0GB
TinyLlama 1.1B
TinyLlama
1.1BChat / LLMQ8_01.6GB
Gemma 3 1B
1BChat / LLMQ8_01.5GB
Falcon 3 1B
TII
1BChat / LLMQ8_02.2GB
Stable Audio Open
Stability AI
1BAudio GenerationFP166.0GB
Stable Diffusion 2.1 Base (CoreML)
Stability AI / Apple
0.86BImage GenerationCoreML-Palettized1.6GB
Stable Diffusion 1.5 (CoreML)
Runway
0.86BImage GenerationCoreML-Palettized2.5GB
Stable Diffusion 1.5 (GGUF)
Runway / GPUStack
0.86BImage GenerationQ8_02.3GB
Stable Diffusion 2.1 (GGUF)
Stability AI
0.86BImage GenerationQ8_02.7GB
Whisper Large v3 Turbo
OpenAI
0.81BSpeech RecognitionQ8_02.0GB
Whisper Medium
OpenAI
0.77BSpeech RecognitionQ8_01.9GB
Distil-Whisper Large v3
HuggingFace
0.76BSpeech RecognitionQ8_01.9GB
BGE Reranker v2 M3
BAAI
0.568BRerankerFP161.6GB
Qwen 2.5 0.5B
Alibaba
0.5BChat / LLMQ8_01.1GB
Qwen 2.5 Coder 0.5B
Alibaba
0.5BCodingQ8_01.1GB
Danube 3 500M
H2O.ai
0.5BChat / LLMQ8_01.0GB
SmolLM2 360M
HuggingFace
0.36BChat / LLMQ8_00.9GB
BGE Large EN v1.5
BAAI
0.335BEmbeddingFP161.1GB
MusicGen Small
Meta
0.3BAudio GenerationONNX-Q4F160.8GB
Whisper Small
OpenAI
0.24BSpeech RecognitionQ8_00.9GB
Nomic Embed Text v1.5
Nomic AI
0.137BEmbeddingFP160.8GB
SmolLM2 135M
HuggingFace
0.135BChat / LLMFP160.8GB
Kokoro 82M TTS
Kokoro
0.082BText to SpeechONNX-Q8F160.6GB
Whisper Base
OpenAI
0.074BSpeech RecognitionQ8_00.3GB
Whisper Base English
OpenAI
0.074BSpeech RecognitionQ8_00.3GB
Whisper Tiny English (Quantized)
OpenAI
0.039BSpeech RecognitionQ5_10.1GB
Whisper Tiny
OpenAI
0.039BSpeech RecognitionQ8_00.2GB
BGE Small EN v1.5
BAAI
0.033BEmbeddingQ8_00.1GB
Snowflake Arctic Embed S
Snowflake
0.033BEmbeddingQ8_00.1GB
Jina Reranker Tiny EN
Jina AI
0.033BRerankerFP160.1GB
all-MiniLM-L6-v2
Sentence Transformers
0.023BEmbeddingQ8_00.1GB
Piper TTS - Amy (English)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Lessac (English)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - LibriTTS-R (English)
Rhasspy
0.02BText to SpeechONNX0.6GB
Piper TTS - Spanish (MLS)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - French (Siwis)
Rhasspy
0.02BText to SpeechONNX0.5GB
Piper TTS - German (Thorsten)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Chinese (Huayan)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Japanese (Kokoro)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Korean
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Russian (Irina)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Portuguese (Faber)
Rhasspy
0.02BText to SpeechONNX0.1GB
Piper TTS - Italian (Riccardo)
Rhasspy
0.02BText to SpeechONNX0.5GB
Piper TTS - Arabic (Kareem)
Rhasspy
0.02BText to SpeechONNX0.1GB
B
Good (4 models)
Runs but may be tight on memory
C
Possible (1 model)
Needs partial CPU offloading, slower performance
D
Struggling (1 model)
Heavy offloading required, very slow
F
Cannot Run (1 model)
Insufficient hardware to run this model
Similar GPUs
Frequently Asked Questions
What AI models can I run on the NVIDIA L40S?
The NVIDIA L40S with 48GB VRAM can run 142 AI models smoothly (grade B or better), including Mixtral 8x7B Instruct, Phi-3.5 MoE, Qwen 2.5 32B and 139 more.
How much VRAM does the NVIDIA L40S have?
The NVIDIA L40S has 48GB of VRAM, which is sufficient for running many local AI models.
Is the NVIDIA L40S good for AI?
With 48GB VRAM, the NVIDIA L40S can run 142 out of 145 models in our database at grade B or better. This is an excellent choice for local AI workloads.