Empirical Intelligence
AI Model Benchmarks
Audited evaluations across frontier reasoning, code synthesis, and graduate scientific tasks. Every benchmark exposes test methodology, test dates, and environmental bounds.
Methodological Rule: No single benchmark metric represents overall artificial intelligence. Performance varies across temperature, thinking token allocation, prompt scaffolding, and test-time compute. Scores presented below reflect verified runs under standard conditions.
DeepSeek
Jan 20, 2025
DeepSeek R1
671B MoE (37B Active)
MATH-500
Accuracy % (Pass@1)
97.30%
Task Category: Mathematics
Hardware / Cluster: 8x NVIDIA H800 Cluster
Evaluation Method: Zero-shot evaluation with pure self-generated chain-of-thought verification.
Source: DeepSeek AI Lab
Reference →
Anthropic
Feb 24, 2025
Claude 3.7 Sonnet
2025-02-24 (Extended Thinking)
SWE-bench Verified
Resolved % (Pass@1)
70.30%
Task Category: Software Engineering
Hardware / Cluster: Cloud Cluster / 16K Thinking Budget
Evaluation Method: Tested on 500 verified real-world GitHub issues.
Source: Anthropic AI Research
Reference →