Configuring Smart Routing

Configuring Smart Routing

This guide will walk you through configuring smart routing in UniCraft to optimize your AI model selection based on your specific requirements.

Prerequisites

Before configuring smart routing, ensure you have:

• UniCraft account set up
• At least one AI provider configured
• Basic understanding of your use case requirements

Step 1: Access Smart Routing Configuration

1. Log into your UniCraft dashboard
2. Navigate to Settings → Smart Routing
3. Click Configure Smart Routing

Step 2: Choose Routing Strategy

Select the routing strategy that best fits your needs:

Cost Optimized

Best for: Budget-conscious applications

{
  "strategy": "cost_optimized",
  "max_cost_per_request": 0.01,
  "quality_threshold": 0.8
}

Performance Optimized

Best for: Real-time applications

{
  "strategy": "performance_optimized",
  "max_response_time": 2000,
  "quality_threshold": 0.7
}

Quality Optimized

Best for: Content generation, analysis

{
  "strategy": "quality_optimized",
  "min_quality_score": 0.9,
  "max_cost_per_request": 0.05
}

Balanced

Best for: General-purpose applications

{
  "strategy": "balanced",
  "cost_weight": 0.4,
  "performance_weight": 0.3,
  "quality_weight": 0.3
}

Step 3: Configure Provider Priorities

Set the priority order for your providers:

const providerPriorities = {
  openai: {
    priority: 1,
    models: ["gpt-4", "gpt-3.5-turbo"],
    weight: 0.5,
  },
  anthropic: {
    priority: 2,
    models: ["claude-3-sonnet", "claude-3-haiku"],
    weight: 0.3,
  },
  google: {
    priority: 3,
    models: ["gemini-pro"],
    weight: 0.2,
  },
};

Step 4: Set Up Fallback Configuration

Configure automatic failover when primary providers are unavailable:

{
  "fallback_enabled": true,
  "fallback_strategy": "next_available",
  "retry_attempts": 3,
  "retry_delay": 1000,
  "circuit_breaker": {
    "enabled": true,
    "failure_threshold": 5,
    "recovery_timeout": 30000
  }
}

Step 5: Configure Load Balancing

If using multiple providers, set up load balancing:

{
  "load_balancing": {
    "enabled": true,
    "strategy": "weighted_round_robin",
    "weights": {
      "openai": 0.5,
      "anthropic": 0.3,
      "google": 0.2
    }
  }
}

Step 6: Set Up Monitoring and Alerts

Configure monitoring for your smart routing setup:

{
  "monitoring": {
    "enabled": true,
    "metrics": [
      "response_time",
      "success_rate",
      "cost_per_request",
      "provider_availability"
    ],
    "alerts": [
      {
        "metric": "response_time",
        "threshold": 5000,
        "action": "email"
      },
      {
        "metric": "success_rate",
        "threshold": 0.95,
        "action": "slack"
      }
    ]
  }
}

Step 7: Test Your Configuration

Test your smart routing configuration:

// Test with different request types
const testRequests = [
  {
    messages: [{ role: "user", content: "Simple question" }],
    expected_model: "gpt-3.5-turbo", // Cost-optimized
  },
  {
    messages: [{ role: "user", content: "Complex analysis" }],
    expected_model: "gpt-4", // Quality-optimized
  },
  {
    messages: [{ role: "user", content: "Quick response needed" }],
    expected_model: "claude-3-haiku", // Performance-optimized
  },
];

for (const request of testRequests) {
  const response = await unicraft.chat.completions.create(request);
  console.log(`Request: ${request.messages[0].content}`);
  console.log(`Model used: ${response.model}`);
  console.log(`Response time: ${response.response_time}ms`);
  console.log(`Cost: $${response.cost}`);
}

Advanced Configuration

Custom Routing Rules

Create custom routing rules for specific use cases:

const customRules = {
  content_generation: {
    condition: "request_type === 'content_generation'",
    preferred_models: ["gpt-4", "claude-3-sonnet"],
    max_cost: 0.05,
  },
  code_generation: {
    condition: "request_type === 'code_generation'",
    preferred_models: ["gpt-4", "claude-3-sonnet"],
    max_cost: 0.03,
  },
  simple_qa: {
    condition: "request_type === 'simple_qa'",
    preferred_models: ["gpt-3.5-turbo", "claude-3-haiku"],
    max_cost: 0.01,
  },
};

A/B Testing Configuration

Set up A/B testing for different routing strategies:

{
  "ab_testing": {
    "enabled": true,
    "tests": [
      {
        "name": "cost_vs_quality",
        "variants": [
          {
            "name": "cost_optimized",
            "weight": 0.5,
            "strategy": "cost_optimized"
          },
          {
            "name": "quality_optimized",
            "weight": 0.5,
            "strategy": "quality_optimized"
          }
        ],
        "duration": "7d",
        "metrics": ["cost", "quality_score", "user_satisfaction"]
      }
    ]
  }
}

Troubleshooting

Common Issues

1. No models available: Check provider configuration and API keys
2. High costs: Adjust cost thresholds and model preferences
3. Slow responses: Review performance settings and provider priorities
4. Frequent fallbacks: Check provider health and network connectivity

Debug Mode

Enable debug mode to troubleshoot routing decisions:

const response = await unicraft.chat.completions.create({
  messages: [{ role: "user", content: "Hello" }],
  debug: true,
});

console.log("Routing decision:", response.debug.routing_decision);
console.log("Available models:", response.debug.available_models);
console.log("Provider status:", response.debug.provider_status);

Best Practices

1. Start Simple: Begin with a basic configuration and gradually add complexity
2. Monitor Performance: Regularly review routing decisions and performance metrics
3. Test Thoroughly: Test your configuration with various request types
4. Set Realistic Thresholds: Ensure your cost and performance thresholds are achievable
5. Plan for Failures: Always configure fallback options
6. Regular Reviews: Periodically review and update your routing configuration

Next Steps

After configuring smart routing:

1. Monitor performance metrics in the dashboard
2. Set up alerts for important thresholds
3. Review and optimize based on usage patterns
4. Consider implementing custom routing rules for specific use cases