AI Engineering Program — go from software engineer to production AI engineer · Live training with Kirill Eremenko · Watch the program breakdown→AI Engineering Program — go from software engineer to production AI engineer · Live training with Kirill Eremenko · Watch the program breakdown→AI Engineering Program — go from software engineer to production AI engineer · Live training with Kirill Eremenko · Watch the program breakdown→

Q: How do I choose the right LLM for my application?

Work through the criteria in order, and the right choice usually becomes obvious.

  1. How hard is the task? Routine work (summarizing, classification, chat) runs fine on cheap, fast models like the mini tiers. Hard reasoning needs a frontier model.

  2. Speed vs accuracy. Smaller models respond faster and cost less. Bigger models think better but add latency. Decide which your app needs more.

  3. Cost at your volume. A fraction of a cent per call is nothing at 100 calls a day, and a fortune at 10 million.

  4. Context window. This matters if your app sends large documents with each request.

  5. Data constraints. If data can't leave your environment, you're looking at open source models or cloud-hosted options inside your own tenancy.

Two warnings. Don't pick from leaderboards alone: benchmarks are widely gamed, and a model that tops a leaderboard can lose on your specific task. And don't over-deliberate: switching models is usually a one-line code change. The practical method is to build a small eval for your task, test two or three candidates on it, and let the results decide. Start cheap, and upgrade only where the eval shows the cheap model failing.

← Back to the full FAQ