🏆 LLM-Dispatcher: Intelligent LLM Routing¶
LLM-Dispatcher is an intelligent Python package that automatically selects the best Large Language Model (LLM) for your specific task based on performance metrics, cost optimization, and real-time availability.
🚀 Key Features¶
- Intelligent Switching
Performance-based LLM selection using credible benchmarks from 2024-2025
- Real Metrics
Based on MMLU, HumanEval, GPQA, AIME, and other authoritative benchmarks
- Cost Optimization
Dynamic cost-quality balancing with intelligent routing decisions
- Smart Fallbacks
Automatic failover with intelligent routing and error recovery
- Multi-Modal
Support for text, vision, audio, and structured output across providers
- Real-time
Live performance monitoring and optimization with streaming support
📈 Supported Providers¶
| Provider | Models | Capabilities |
|---|---|---|
| OpenAI | GPT-4, GPT-4 Turbo, GPT-3.5 Turbo | Text, Vision, Function Calling, Structured Output |
| Anthropic | Claude-3 Opus, Sonnet, Haiku | Text, Vision, Reasoning, Code |
| Gemini 2.5 Pro, Flash, Ultra | Text, Vision, Multimodal, Fast | |
| xAI | Grok 3 Beta | Text, Reasoning, Math |
🎯 Performance Benchmarks¶
Based on latest 2024-2025 data:
| Model | MMLU | HumanEval | GPQA | AIME | HellaSwag | ARC | VQA |
|---|---|---|---|---|---|---|---|
| GPT-4 | 86.3% | 67.4% | 82.1% | 91.2% | 95.1% | 96.4% | 78.2% |
| Claude-3 Opus | 84.6% | 67.4% | 84.6% | 89.8% | 94.2% | 95.8% | 76.8% |
| Gemini 2.5 Pro | 84.0% | 65.2% | 84.0% | 87.3% | 93.8% | 94.2% | 74.5% |
| Grok 3 Beta | 82.1% | 63.8% | 84.6% | 93.3% | 92.1% | 93.8% | 71.2% |
🚀 Quick Start¶
Installation¶
Basic Usage¶
from llm_dispatcher import llm_dispatcher
@llm_dispatcher
def generate_text(prompt: str) -> str:
"""Automatically routed to the best LLM for text generation."""
return prompt
# Usage
result = generate_text("Write a story about a robot")
print(result)
Advanced Configuration¶
from llm_dispatcher import LLMSwitch, TaskType
# Initialize with custom configuration
switch = LLMSwitch(
providers={
"openai": {"api_key": "sk-..."},
"anthropic": {"api_key": "sk-ant-..."},
"google": {"api_key": "..."}
},
config={
"prefer_cost_efficiency": True,
"max_latency_ms": 2000,
"fallback_enabled": True
}
)
@switch.route(task_type=TaskType.CODE_GENERATION)
def generate_code(description: str) -> str:
"""Automatically uses the best model for code generation."""
return description
🔗 Integrations¶
- LangChain
Seamless integration with LangChain for complex workflows
- LangGraph
Advanced graph-based workflows with intelligent routing
- Coming Soon
More integrations and providers on the roadmap
📚 Documentation¶
- Getting Started - Installation and setup
- Quick Start - Your first steps
- Configuration - Advanced configuration
- API Reference - Complete API documentation
- Benchmarks - Performance data
- Contributing - How to contribute
🤝 Contributing¶
We welcome contributions! Please see our Contributing Guide for details.
Development Setup¶
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests
- Submit a pull request
📄 License¶
This project is licensed under the MIT License - see the LICENSE file for details.
🙏 Acknowledgments¶
- Benchmark data from Musaix, 51D.co, and Learnopoly
- LLM providers: OpenAI, Anthropic, Google, xAI
- Open source community
📞 Support¶
- 📧 Email: ashhadahsan@mail.com
- 🐛 Issues: GitHub Issues
- 💬 Discussions: GitHub Discussions
Built with ❤️ for the AI community