Compare / Spark-X2.5 4B vs Gemma 3 4B
Spark-X2.5 4B vs Gemma 3 4B
Updated November 2026
Verdict
Spark-X2.5 4B scores higher on the Small LM Index: 65.5 vs 17.5. On the 1 benchmark both labs report, Spark-X2.5 4B wins 1 and Gemma 3 4B wins 0. Both need about 3.5 GB of memory at 4-bit quantization. Only Gemma 3 4B has a hosted API on OpenRouter ($0.042 per million tokens); the other has to be self-hosted.
Pick Spark-X2.5 4B if…
- you want the strongest model overall
- you work with long documents
- you need a permissive license (Apache-2.0)
Pick Gemma 3 4B if…
- you want a hosted API rather than running it yourself
- you need image input
- you want the more widely used model (more quantized builds and community support)
Specifications
| Spark-X2.5 4B | Gemma 3 4B | |
|---|---|---|
| Small LM Index | 65.5 (likely 63–68) | 17.5 (likely 13–23) |
| Lab | SparkLLM (XHToken) · China | Google · United States |
| Released | 2026-08-24 | 2025-03-12 |
| Parameters | 4.1B (dense) | 4.3B (dense) |
| Memory at 4-bit | ~3.4 GB | ~3.5 GB |
| Context window | 1M tokens | 128K tokens |
| Input | text | text, image |
| Reasoning | Hybrid (thinking on/off) | No thinking |
| License | Apache-2.0 | Gemma |
| API price / 1M tokens | Self-host only | $0.042 ($0.040 in, $0.050 out) |
| Hugging Face downloads (30 days) | 30.3K | 982K |
Benchmarks
Higher is better. Bold marks the winner where both labs report a score. ≈ marks the index model's estimate for a score that wasn't reported, ✓ an independent measurement and * a score measured by another lab.
| Benchmark | Spark-X2.5 4B | Gemma 3 4B |
|---|---|---|
| MMLU-Pro Knowledge | ≈ 82 | 43.6 |
| GPQA Diamond Knowledge | 73.7 | 30.8 |
| Humanity's Last Exam Reasoning | 12.3 | ≈ 1 |
| AIME Math | 90.7 | ≈ 12 |
| LiveCodeBench Coding | ≈ 75 | 12.6 |
| SWE-bench Verified Coding | 41.6 | ≈ 5 |
| τ²-Bench Agentic | 75.1 | — |
| IFBench Instruction following | 75.0 | — |
| IFEval Instruction following | 93.0 | — |
| BFCL Agentic | 65.1 | — |
| HMMT Math | 81.2 | — |
| AA-LCR Long context | 56.3 | — |
| SciCode Coding | 34.7 | — |
Scores are mostly reported by each lab with its own settings, so small gaps are noise. See the methodology. Open in the interactive comparison to add more models.
Questions
Which is better, Spark-X2.5 4B or Gemma 3 4B?
Spark-X2.5 4B scores higher on the Small LM Index: 65.5 vs 17.5. On the 1 benchmark both labs report, Spark-X2.5 4B wins 1 and Gemma 3 4B wins 0.
Which is cheaper to run?
Only Gemma 3 4B has a hosted API on OpenRouter ($0.042 per million tokens); the other has to be self-hosted. Both need about 3.5 GB of memory at 4-bit quantization.
Which has the longer context window?
Spark-X2.5 4B has the longer context window: 1M tokens against 128K.
Can I use Spark-X2.5 4B and Gemma 3 4B commercially?
Spark-X2.5 4B is released under Apache-2.0 and Gemma 3 4B under Gemma. Apache 2.0 and MIT allow commercial use; other licenses can add conditions, so read the license before shipping a product.