German LLM Benchmark · Model Profile

Claude Haiku 4.5

Anthropic provider-internal run 2026-06-03

75 .6%

avg. German score

#9 of 26 models

+4.2pp above avg.

Benchmark breakdown

INCLUDE

68.3%

Native German exam and licensing questions covering region-specific knowledge — history, law, civics and culture. Written by humans in German, not translated.

4-option multiple choice native · Native German

via CohereLabs/include-base-44 ↗

MMLU-Pro

75.3%

Hard academic questions across 14 subjects — STEM, law, health, economics, philosophy and more. Professionally translated to German, with up to ten answer options per question.

10-option multiple choice translated · Professional translation

via li-lab/MMLU-ProX ↗

MMMLU

83.1%

OpenAI's multilingual MMLU, German split — general knowledge spanning STEM, the humanities, social sciences and other domains. Professionally translated to German.

4-option multiple choice translated · Professional translation

via openai/MMMLU ↗

Cost & speed

$3.248 per 1,000 questions

131 tokens / second

0.79s time to first token

Compare with other models → ← View leaderboard