Individual experiment result

INCLUDE (DE)

NVIDIA Nemotron 3 Ultra 550B-A55B NVIDIA

Exact match reasoning off

86.9%

Exact match

Run details

Date: 2026-08-15
Test cases: 107
Median latency: 3.91s
Total cost: —
Avg. prompt tokens: 124.1
Avg. answer tokens: 154.2
Avg. reasoning tokens: —
Quantization: Unverified

About this benchmark

INCLUDE-DE v1.2: 107 valid German questions after a blinded semantic audit excluded 32 duplicate, wrong-keyed, non-unique, corrupted, or underspecified source rows consistently; strict final-answer parsing; primary metric exact match.

4-option multiple choice native · Native German Source ↗

Provenance

Canonical model: nvidia/nemotron-3-ultra-550b-a55b
Run ID: 2026-08-15T03-47-39__openrouter__nvidia__nemotron-3-ultra-550b-a55b:free__include_de

All NVIDIA Nemotron 3 Ultra 550B-A55B results → ← New results