QORL: A 4B AI Model Outperforms Postgres Query Optimizer by 81%

AI-Powered Query Optimization: A New Benchmark for Postgres

Independent researcher Rohan Bansal's QORL experiment on September 16, 2026, compared the performance of Empero's Qwen3.8-4B-Distill, a 4-billion-parameter open-weights model, against Postgres's default optimizer in generating query plans, revealing significant speedups for complex, join-heavy queries. For broader context, explore our AI News.

Understanding the Challenge: Query Optimization

Query optimization, especially the intricate task of join ordering, is recognized as an NP-hard problem. Databases like Postgres employ sophisticated internal optimizers to determine the most efficient way to execute a query. However, these traditional methods can sometimes be suboptimal, leading to slower query execution times, particularly with increasingly complex data structures and larger datasets. The QORL project aimed to address this by leveraging advanced AI techniques to generate more effective query plans.

The QORL Experiment: Methodology and Models

The QORL experiment involved several key components and models. Initially, the untrained 4B base model struggled significantly, failing to produce valid query plans for 99 out of 113 benchmark queries. To overcome this, the training process for Empero's Qwen3.8-4B-Distill model involved two main stages:

1. Supervised Fine-Tuning (SFT): The model was fine-tuned on approximately 500 agent trajectories generated by GPT-6 Astra. This step provided the model with a foundational understanding of generating effective query plans.
2. Reinforcement Learning (RL): Following SFT, a custom "anchored" GRPO (Generalized Reinforcement Learning with Policy Optimization) variant was applied. This RL phase allowed the model to learn and refine its plan generation based on verifiable rewards, where the "answer checking" was as straightforward as timing the query execution.

The model steers Postgres using pg_hint_plan hints within a custom "qo-agent" harness, allowing it to influence the database's execution strategy directly.

Performance Comparison: Qwen3.8-4B-Distill vs. Postgres Default Optimizer

The results of the QORL experiment were notable. The Empero Qwen3.8-4B-Distill model achieved a 1.81x geometric-mean speedup and a 44.7% summed latency reduction across 113 join-heavy queries from the Join Order Benchmark. This translates to the AI-generated plans executing 81% faster than those produced by Postgres's default optimizer.

Feature Comparison: Query Optimization

Broader Context: Frontier Models and Accessibility

Before the 4B model was trained, frontier models such as GPT-6 Astra and Qwen 3.8 2.4T also demonstrated their ability to outperform default Postgres plans in the same testing harness. This indicates a broader trend of advanced AI models excelling in complex optimization tasks.

A significant aspect of the QORL project is its A significant aspect of the QORL project is its cost-effectiveness-effectiveness. The total compute cost for the entire project was approximately $1,200. This included $800 for renting a 2x H100 node for about 95 hours on Lambda and $400 in OpenAI API fees. This relatively modest budget suggests that meaningful post-training research in agentic RL is becoming increasingly accessible to individual researchers, potentially democratizing advanced AI development.

Implications and Future Directions

The success of QORL demonstrates that verifiable-reward RL can be effectively applied to database internals. The project's write-up garnered substantial attention on Hacker News, reaching over 450 points and sparking discussions about its implications and potential caveats. This interest underscores the community's recognition of AI's growing role in optimizing fundamental software systems.

The ability of a smaller, specialized AI model to significantly enhance database performance opens new avenues for improving efficiency in data-intensive applications. As AI models become more refined and accessible, we can expect further innovations in areas traditionally dominated by heuristic-based or manually optimized systems.

Conclusion

Rohan Bansal's QORL experiment with Empero's Qwen3.8-4B-Distill model marks a notable achievement in applying agentic reinforcement learning to database query optimization. By achieving an 81% speedup over the Postgres default optimizer for specific query types, the project not only showcases the practical benefits of AI in database performance but also highlights the increasing accessibility of advanced AI research for individual developers. This development suggests a future where AI plays a more integral role in optimizing core software infrastructure, offering substantial performance gains and potentially reducing operational costs.

Sources

• hello-agents/docs/chapter11/Chapter11-Agentic-RL.md at main · datawhalechina/hello-agents · GitHub
• Learning Agentic Policy from Action Guidance
• Libra: Efficient Resource Management for Agentic RL Post-Training
• README.md · empero-ai/Qwen3.8-4B-Distill at main - Hugging Face
• empero-ai/Qwen3.8-4B-Distill