Guides / 24 GB GPU
Best LLMs for a 24 GB GPU (RTX 4090, RTX 3090)
Updated November 2026
As of November 2026, the best option is Nanbeige4.3 33B by Nanbeige (Kanzhun), with a Small LM Index of 77.8, followed by Qwen3.8 27B (77.0) and Ternary Bonsai 2 27B (74.6).
Models that fit in a 24 GB graphics card at 4-bit quantization with an 8K-token context, ranked by the Small LM Index. Mixture-of-Experts models also run fast because only some parameters are active per token. Try other hardware and context lengths in the Can I run it? tool.
| # | Model | Lab | Small LM Index | Params | Memory (4-bit) | API / 1M | License |
|---|---|---|---|---|---|---|---|
| 1 | Nanbeige4.3 33B | Nanbeige (Kanzhun) | 77.8 | 33.1B | ~23 GB | self-host | Apache-2.0 |
| 2 | Qwen3.8 27B | Alibaba (Qwen) | 77.0 | 27B | ~17.9 GB | $0.13 | Apache-2.0 |
| 3 | Ternary Bonsai 2 27B | PrismML | 74.6 | 27.4B | ~18.2 GB | self-host | Apache-2.0 |
| 4 | Qwen3.6 27B | Alibaba (Qwen) | 74.0 | 27B | ~17.9 GB | $0.90 | Apache-2.0 |
| 5 | Muse Glimmer 30B | Meta | 72.6 | 29.6B | ~19.2 GB | $0.53 | Apache-2.0 |
| 6 | Qwen3.6 35B-A3B | Alibaba (Qwen) | 72.2 | 35B (3B active) | ~22.6 GB | $0.031 | Apache-2.0 |
| 7 | Qwen3.5 27B | Alibaba (Qwen) | 70.6 | 27B | ~17.9 GB | $0.69 | Apache-2.0 |
| 8 | Nanbeige4.2 3B | Nanbeige (Kanzhun) | 70.4 | 4.2B | ~3.8 GB | self-host | Apache-2.0 |
| 9 | Gemma 4 31B | 70.1 | 30.7B | ~21.9 GB | $0.16 | Apache-2.0 | |
| 10 | Qwen3.5 35B-A3B | Alibaba (Qwen) | 69.9 | 35B (3B active) | ~22.6 GB | $0.24 | Apache-2.0 |
| 11 | Spark-X2.6 4B | SparkLLM (XHToken) | 67.5 | 4.1B | ~3.4 GB | self-host | Apache-2.0 |
| 12 | Gemma 4 26B-A4B | 67.5 | 25.2B (3.8B active) | ~16.8 GB | $0.068 | Apache-2.0 | |
| 13 | Granite 4.2 30B | IBM (Granite) | 67.0 | 30B | ~21.4 GB | self-host | Apache-2.0 |
| 14 | GLM-4.7-Flash | Z.ai (GLM) | 66.1 | 31B (3B active) | ~20.4 GB | $0.15 | MIT |
| 15 | Spark-X2.5 4B | SparkLLM (XHToken) | 65.5 | 4.1B | ~3.4 GB | self-host | Apache-2.0 |
Memory covers the weights only; long contexts need more. Scores are mostly lab-reported, so gaps of 1–2 points are noise. See the methodology, or filter the full leaderboard yourself.
Head to head
Questions
What is the best LLM to run on an RTX 4090 or other 24 GB GPU?
As of November 2026, the best option is Nanbeige4.3 33B by Nanbeige (Kanzhun), with a Small LM Index of 77.8, followed by Qwen3.8 27B (77.0) and Ternary Bonsai 2 27B (74.6).
How is this list ranked?
Models that fit in a 24 GB graphics card at 4-bit quantization with an 8K-token context, ranked by the Small LM Index. Mixture-of-Experts models also run fast because only some parameters are active per token.
Which of the top five needs the least memory?
Qwen3.8 27B, at about 17.9 GB with 4-bit quantization.
Which is the cheapest to use through an API?
Qwen3.6 35B-A3B, at $0.031 per million tokens from the cheapest provider on OpenRouter.
Other guides
- Best small open models for coding
- Best small open models for math
- Best LLMs that run on an 8 GB GPU or laptop
- Best LLMs that run on a 16 GB GPU
- Best tiny LLMs for phones and edge devices
- Best cheap LLM APIs (open models)
- Best small open vision-language models
- Best open models with an Apache 2.0 or MIT license
- Best European open language models
- Best American open models under 100B
- Best fully open models (open data and code)