Skip to content

🏆 LLM-Dispatcher: Intelligent LLM Routing

Python 3.8+ License: MIT Code style: black Tests

LLM-Dispatcher is an intelligent Python package that automatically selects the best Large Language Model (LLM) for your specific task based on performance metrics, cost optimization, and real-time availability.

🚀 Key Features

  • Intelligent Switching

Performance-based LLM selection using credible benchmarks from 2024-2025

Learn more

  • Real Metrics

Based on MMLU, HumanEval, GPQA, AIME, and other authoritative benchmarks

View benchmarks

  • Cost Optimization

Dynamic cost-quality balancing with intelligent routing decisions

Cost analysis

  • Smart Fallbacks

Automatic failover with intelligent routing and error recovery

Error handling

  • Multi-Modal

Support for text, vision, audio, and structured output across providers

Multimodal guide

  • Real-time

Live performance monitoring and optimization with streaming support

Streaming guide

📈 Supported Providers

Provider Models Capabilities
OpenAI GPT-4, GPT-4 Turbo, GPT-3.5 Turbo Text, Vision, Function Calling, Structured Output
Anthropic Claude-3 Opus, Sonnet, Haiku Text, Vision, Reasoning, Code
Google Gemini 2.5 Pro, Flash, Ultra Text, Vision, Multimodal, Fast
xAI Grok 3 Beta Text, Reasoning, Math

🎯 Performance Benchmarks

Based on latest 2024-2025 data:

Model MMLU HumanEval GPQA AIME HellaSwag ARC VQA
GPT-4 86.3% 67.4% 82.1% 91.2% 95.1% 96.4% 78.2%
Claude-3 Opus 84.6% 67.4% 84.6% 89.8% 94.2% 95.8% 76.8%
Gemini 2.5 Pro 84.0% 65.2% 84.0% 87.3% 93.8% 94.2% 74.5%
Grok 3 Beta 82.1% 63.8% 84.6% 93.3% 92.1% 93.8% 71.2%

🚀 Quick Start

Installation

pip install llm-dispatcher

Basic Usage

from llm_dispatcher import llm_dispatcher

@llm_dispatcher
def generate_text(prompt: str) -> str:
    """Automatically routed to the best LLM for text generation."""
    return prompt

# Usage
result = generate_text("Write a story about a robot")
print(result)

Advanced Configuration

from llm_dispatcher import LLMSwitch, TaskType

# Initialize with custom configuration
switch = LLMSwitch(
    providers={
        "openai": {"api_key": "sk-..."},
        "anthropic": {"api_key": "sk-ant-..."},
        "google": {"api_key": "..."}
    },
    config={
        "prefer_cost_efficiency": True,
        "max_latency_ms": 2000,
        "fallback_enabled": True
    }
)

@switch.route(task_type=TaskType.CODE_GENERATION)
def generate_code(description: str) -> str:
    """Automatically uses the best model for code generation."""
    return description

🔗 Integrations

  • LangChain

Seamless integration with LangChain for complex workflows

LangChain integration

  • LangGraph

Advanced graph-based workflows with intelligent routing

LangGraph integration

  • Coming Soon

More integrations and providers on the roadmap

Roadmap

📚 Documentation

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests
  5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

  • Benchmark data from Musaix, 51D.co, and Learnopoly
  • LLM providers: OpenAI, Anthropic, Google, xAI
  • Open source community

📞 Support


Built with ❤️ for the AI community