Can RTX 3090 Ti run Stable Diffusion XL (CoreML)?
Yes — runs locally
~96 tok/sec · Instant — feels like typing. No noticeable delay.
The verdict
The RTX 3090 Ti (24 GB VRAM) handles Stable Diffusion XL (CoreML) comfortably using the CoreML quantization, which fits in 3.3 GB. Expected throughput is around 96 tokens/second, which feels Instant — feels like typing. No noticeable delay. in interactive use. Higher quality image generation. CoreML optimized for iOS. Requires 6GB+ usable memory (iPad/Mac recommended).
Setup tutorial: Stable Diffusion XL (CoreML) on RTX 3090 Ti
AI-generated, GPU-specific. Verified commands for your exact hardware.
Stable Diffusion XL (CoreML) runs at Grade S on an NVIDIA GeForce RTX 3090 Ti, achieving ~347 tok/sec with the CoreML quantization (2.8GB file).
Prerequisites
Before starting, ensure you have at least 10GB of free disk space, a 64-bit version of Windows or Linux, the latest NVIDIA drivers (version 525.60.13 or later), and CUDA 11.7 installed.
Expected performance
With the CoreML quantization, you can expect ~347 tok/sec, utilizing approximately 3.3GB of VRAM. This leaves a headroom of 20.7GB for context, allowing for a practical context window of several thousand tokens, depending on the specific requirements of your task.
1. Install runtimeOllama
pip install ollama
ollama init2. Download the model
Download the CoreML quantized model (2.8GB file) from Hugging Face.
ollama pull apple/coreml-stable-diffusion-xl-base-ios3. Run it
ollama run --model apple/coreml-stable-diffusion-xl-base-ios --device cuda
ollama interactive4. Optimize for RTX 3090 Ti
For optimal performance on the NVIDIA GeForce RTX 3090 Ti with 24GB VRAM, use the --n-gpu-layers flag to offload layers to the GPU. Enable flash attention (--flash-attn) to reduce memory usage and improve speed. With 24GB VRAM, you can set --n-gpu-layers to 32 or higher, leaving ample headroom for large context windows and batch sizes. Tensor parallelism can also be utilized to further distribute the workload across multiple GPUs if available.
Troubleshooting
Out of memory error during inference
Reduce the number of layers offloaded to the GPU using the --n-gpu-layers flag, or enable flash attention with --flash-attn.
Slow inference times
Ensure that the CUDA toolkit is correctly installed and that the GPU drivers are up to date. Consider increasing the batch size or enabling tensor parallelism if multiple GPUs are available.
Model fails to load
Verify that the model file has been downloaded correctly and is not corrupted. Try re-downloading the model using the 'ollama pull' command.
Alternative runtimes
Alternative runtimes include LM Studio, llama.cpp, and Jan. LM Studio is suitable for users who prefer a graphical interface and need advanced monitoring tools. llama.cpp is ideal for those who want a lightweight, highly customizable runtime, especially for smaller models. Jan is a good choice for distributed training and inference, particularly when working with multiple GPUs or clusters. For the NVIDIA GeForce RTX 3090 Ti, Ollama provides a well-optimized and user-friendly experience, making it the recommended choice for most users.
Other models that run great on RTX 3090 Ti
FAQ (20)
What GPU do I need to run Stable Diffusion XL (CoreML)?
Stable Diffusion XL (CoreML) is optimized for iOS devices and does not require a specific GPU. It leverages the Metal Performance Shaders (MPS) framework on iPad and Mac.
Is Stable Diffusion XL (CoreML) good for coding?
Stable Diffusion XL (CoreML) is primarily designed for generating high-quality images and is not optimized for coding tasks.
Stable Diffusion XL (CoreML) vs Llama 3.1 8B?
Stable Diffusion XL (CoreML) is an image generation model with 3.5B parameters, while Llama 3.1 8B is a text generation model with 8B parameters. They serve different purposes and are not directly comparable.
Can I run Stable Diffusion XL (CoreML) on a Mac?
Yes, Stable Diffusion XL (CoreML) can run on a Mac, but it requires at least 6GB of usable memory and is optimized for iOS devices like iPads.
How much VRAM does Stable Diffusion XL (CoreML) need?
Stable Diffusion XL (CoreML) requires 3.3 GB of VRAM, which is consistent across different quantization levels.
Is Stable Diffusion XL (CoreML) censored?
The model is not inherently censored, but it adheres to the CreativeML OpenRail-M license, which may have usage guidelines and restrictions.
Is Stable Diffusion XL (CoreML) commercial-use allowed?
Yes, Stable Diffusion XL (CoreML) can be used commercially, but you must comply with the terms of the CreativeML OpenRail-M license.
Stable Diffusion XL (CoreML) context length?
The context length for Stable Diffusion XL (CoreML) is not specified, as it is primarily an image generation model and does not handle text input in the same way as language models.
Want personalized recommendations for your exact setup? Detect my hardware →