LLM Routing for Text-to-SQL
An LLM routing orchestration framework for NL2SQL agentic workflows. My MEng research, released open source.
I built an LLM routing orchestration framework for academic research on NL2SQL agentic workflows, implementing three routing strategies: a Matrix Router based on a neural network, a RAG Router, and a Supervisor Agent Router.
The framework integrates MLflow for performance tracking with custom SQL accuracy metrics, and supports models from Vertex AI, Anthropic, OpenAI and local deployments.
It was evaluated across 13 LLMs. The RAG Router landed on the cost-performance Pareto front: on the 78 hold-out questions it scored higher on combined quality than the most expensive model, Claude Opus 4, at 56 times lower cost, $0.31 against $17.40, and ran three times faster. The framework is released as open source.
Highlights
- Three routing strategies: Matrix Router (neural network), RAG Router, Supervisor Agent Router
- MLflow performance tracking with custom SQL accuracy metrics
- Supports Vertex AI, Anthropic, OpenAI and local deployments
- Evaluated across 13 LLMs; RAG Router Pareto-optimal at 56 times lower cost
Built with
Other work