GPT-6-Astra Achieves 98% Accuracy on New TheoPhysEval Research Physics Benchmark

OpenAI's GPT-6-Astra, running in the Codex environment, solved 49 of 50 research-level theoretical physics problems (98 percent) on the new TheoPhysEval benchmark, a dataset of 50 unpublished problems developed by a team of physicists led by Suvrat Raju.

TheoPhysEval: A New Standard for AI in Physics

The TheoPhysEval benchmark was specifically designed to evaluate AI agents on routine, yet challenging, 'turn-the-crank' research tasks in theoretical physics. Unlike benchmarks that might focus on established knowledge, TheoPhysEval presents problems that require concrete computations and a deep understanding of physics principles to derive verifiable answers. The dataset, including all problems and the full test setup, is publicly available on Hugging Face, promoting transparency and further research.

Performance Across Leading AI Models

GPT-6-Astra's performance was exceptional, solving 49 out of 50 problems without any human intervention. This translates to a 98 percent accuracy rate, setting a high bar for AI in this domain. The average computational cost for GPT-6-Astra to solve each problem was approximately $2.40, totaling around $116 for the entire benchmark. This cost-efficiency, combined with high accuracy, highlights the potential for AI to significantly accelerate research workflows.

Other prominent AI models also participated, with varying degrees of success:

• Anthropic's Opus 5.5 in Claude Code: Achieved a 92 percent success rate.
• Google's Gemini 3.8 Flash: Scored 68 percent.
• Open-weight models: DeepSeek-V4.1-Flash and GLM 5.3 scored 54 percent and 42 percent, respectively.

Notably, the top-performing models, GPT-6-Astra and Opus 5.5, showed minimal drops in their scores even when web search capabilities were disabled. This suggests that these advanced AI agents have effectively internalized and memorized a substantial portion of existing physics literature, allowing them to draw upon a vast internal knowledge base.

Implications for Scientific Research and the Role of Human Experts

The results from TheoPhysEval indicate a significant milestone: leading AI agents can now match or even surpass expert human performance in concrete computational tasks within theoretical physics. This capability extends across 11 subfields of physics, demonstrating a broad applicability.

A particularly interesting finding emerged during the evaluation process: follow-ups with human experts revealed small errors in several of the original reference solutions. Once these human errors were corrected, Astra's answers were consistently found to be accurate. This not only underscores the AI's precision but also suggests its potential as a robust verification tool for human-generated solutions.

The authors of the benchmark propose that the primary remaining advantage for human physicists lies not in calculation ability, but in 'research taste'—the intuitive capacity to identify novel problems, formulate groundbreaking hypotheses, and steer the direction of scientific inquiry. While AI can efficiently execute complex calculations, the creative and strategic aspects of research still largely remain within the human domain.

Why This Matters Now

The performance of GPT-6-Astra on TheoPhysEval signals a significant shift in how theoretical physics research could be conducted. By automating complex calculations and problem-solving, AI tools can free up human researchers to focus on higher-level conceptual work, hypothesis generation, and experimental design. This could accelerate discovery, reduce research costs, and open new avenues for scientific exploration. The availability of the benchmark dataset on Hugging Face also provides a valuable resource for further development and benchmarking of AI in scientific domains.

Conclusion: What's Next for AI in Science

The TheoPhysEval benchmark clearly demonstrates that AI agents like GPT-6-Astra are becoming indispensable tools for scientific research, particularly in fields requiring extensive computation and problem-solving. While the 'research taste' and creative intuition of human scientists remain crucial, the ability of AI to handle routine yet complex tasks with high accuracy and efficiency will undoubtedly reshape the future of scientific discovery. We can anticipate further integration of such AI coding agents and productivity tools into research workflows, pushing the boundaries of what's possible in theoretical physics and beyond.

Sources

• [2610.16733] TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving
• RameshChandra1/TheoPhysEval · Datasets at Hugging Face
• TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving
• OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences
• Assessing AI agents at autonomous theoretical physics...