Using model routing to lower inference costs

Running frontier models for coding and other purposes can be very expensive, but also, sometimes, very effective. It’s tempting to use smaller or older models instead of frontier models because they’re cheaper, although they may not work as well. The ideal model for any given task tends to depend on the complexity of the task. Picking the most appropriate model automatically based on the request and the context is called model routing. How model routing works There are several ways of implementing model routing. In general, model routing depends on the pool of available models, the criteria us