Together AI Explained

Together AI is a cloud platform that runs open-weight AI models for developers who do not want to manage their own GPUs. It hosts over 200 models, including Llama, DeepSeek, Qwen, and Kimi K3, behind a single API. Teams use it for serverless inference, custom fine-tuning, and dedicated GPU clusters.

The platform sits in the same category as Groq and Fireworks AI: it does not build its own foundation models. Instead, it hosts other labs' open models and charges per token, per GPU-hour, or per fine-tuning run.

This guide covers what Together AI offers, what it costs, and when it beats the alternatives for a real production workload.

What Is Together AI

Together AI is a hosted inference and fine-tuning platform for open-weight AI models. It runs models like Llama, DeepSeek, and Qwen on its own GPU fleet so developers never touch hardware.

The company started as a research lab focused on decentralized AI training before pivoting to commercial infrastructure. Today it competes directly with Groq, Fireworks AI, and Replicate for the open-model hosting market.

Developers connect through an OpenAI-compatible API, which means most existing code that talks to OpenAI's API works against Together AI with just a base URL and API key change.

• 200+ open-weight models across chat, vision, image, video, audio, and embeddings
• OpenAI-compatible REST API
• Serverless, dedicated, and self-managed GPU cluster options
• Built-in fine-tuning (full and LoRA) for supported model families

What Models Does Together AI Host

Together AI's catalog spans eight categories: chat, vision, image, video, audio, embeddings, rerank, and moderation. Chat models are the deepest category, with entries from DeepSeek, Qwen, Meta, Google, and Moonshot's Kimi lineup.

Context windows vary a lot by model. Some flagship chat models support up to 1 million tokens, while smaller efficient models top out much lower. Always check the specific model page before assuming a context length.

Image generation runs through FLUX, Ideogram, and Stable Diffusion variants, priced per image rather than per token. Together AI has no rerank models in its serverless catalog, which is a real gap if your pipeline needs reranking.

• Chat: DeepSeek, Qwen, Llama, GLM, MiniMax, Kimi K3 (Moonshot)
• Vision: multimodal understanding on select chat models
• Image: FLUX, Ideogram, Stable Diffusion XL
• Video: text-to-video up to 1080p
• Audio: Whisper-based transcription
• Embeddings: one model, multilingual-e5-large-instruct
• Moderation: Llama Guard for content safety

How Much Does Together AI Cost

Together AI charges per million tokens for serverless inference, with input and output priced separately. Smaller, efficient models run under $0.20 per million input tokens, while large reasoning models like Kimi K3 run several dollars per million tokens for output.

Fine-tuning is billed per million training tokens and scales with model size. Small models start under $0.50 per million tokens; models in the 70-100B range run several dollars per million tokens for a full fine-tune, with LoRA typically cheaper.

Dedicated GPU endpoints are billed hourly. An H100 GPU on a dedicated endpoint costs more per hour than the same GPU in an on-demand cluster, and reserved clusters (30-day or 180-day commitments) drop the per-hour rate further.

Verify exact current rates on Together AI's own pricing page before budgeting - per-model prices change as new models launch and older ones get repriced.

• Serverless chat inference: roughly $0.14-$15.00 per million tokens depending on model and input/output
• Image generation: roughly $0.002-$0.06 per image depending on model
• Fine-tuning: roughly $0.48-$8.00+ per million training tokens depending on model size and method
• Dedicated H100: priced per GPU-hour, higher than on-demand cluster rate
• On-demand GPU clusters: per GPU-hour, with reserved-term discounts at 30 and 180+ days

How Does Fine-Tuning Work on Together AI

Fine-tuning on Together AI means training a base open model on your own data to change its behavior, without training a model from scratch. It supports both full fine-tuning and LoRA (a lighter-weight method) on major Llama, Mistral, and Qwen model sizes.

LoRA fine-tuning trains faster and costs less than a full fine-tune, and it is the default choice for most teams that just want the model to adopt a tone, format, or domain vocabulary. Full fine-tuning makes sense when you need deeper behavior changes.

Inference on a LoRA adapter runs at close to standard serverless rates plus a small overhead. That makes it cheaper to run a custom fine-tune in production than to stand up a dedicated GPU endpoint, as long as request volume stays moderate.

• Full fine-tuning: deeper behavior change, higher training cost
• LoRA fine-tuning: faster, cheaper, the default for most production use cases
• Fine-tuned LoRA adapters serve at near-standard serverless inference rates

Together AI vs Groq: Which Should You Use

Together AI wins on model breadth; Groq wins on raw speed. Groq runs a small, curated set of open models on custom LPU chips built for extremely low output latency, while Together AI runs 200+ models on standard GPU infrastructure.

Pick Groq when your product's bottleneck is time-to-first-token or tokens-per-second - live voice agents, real-time chat, anything where users notice a half-second delay. Pick Together AI when you need a specific model Groq does not host, or you need fine-tuning and dedicated GPU clusters in the same platform.

Some teams run both: Groq for the latency-critical chat path, Together AI for fine-tuning and less time-sensitive batch workloads. That split avoids paying Groq's premium for jobs that do not need it.

• Groq: faster inference, narrower model catalog, no fine-tuning
• Together AI: 200+ models, built-in fine-tuning, dedicated GPU clusters
• Latency-sensitive product → Groq; broad catalog or custom fine-tune → Together AI

Together AI vs Fireworks AI vs Replicate

Together AI, Fireworks AI, and Replicate all host open models as a service, but they differ on catalog depth, deployment control, and pricing model. Fireworks AI focuses more heavily on production-grade speed optimizations for a curated model set. Replicate leans toward one-off API calls for a huge range of community models, including many niche or research checkpoints.

Together AI sits in the middle: broader catalog than Fireworks AI, but more infrastructure control (dedicated clusters, reserved GPU terms) than Replicate typically offers. If you need long-term GPU capacity planning with a reserved discount, Together AI's reserved-cluster pricing is a real differentiator.

The honest failure mode with any of these three: picking one before you know your actual token volume. A team running light experimentation should start serverless everywhere and only move to dedicated or reserved capacity once usage is predictable.

• Fireworks AI: narrower catalog, strong latency optimization
• Replicate: widest long-tail catalog, most pay-per-call friendly for occasional use
• Together AI: broad catalog + fine-tuning + reserved GPU pricing in one platform

What Are Together AI Rate Limits

Together AI enforces per-model rate limits that scale with account tier and usage history, similar to most inference API providers. New accounts start on lower limits and can request increases as usage grows.

Exact requests-per-minute and tokens-per-minute limits vary by model and change over time, so check the current limits in your Together AI account dashboard rather than relying on a cached number. Dedicated endpoints avoid shared rate limits entirely, since you are paying for exclusive GPU capacity.

Together AI vs Routing Through OpenRouter

Together AI is a direct hosting provider; OpenRouter is a router that sits in front of many providers, including Together AI, and picks the cheapest or fastest available route per request. Going direct to Together AI cuts out the router's margin and gives you fine-tuning access OpenRouter does not offer.

Use OpenRouter when you want automatic failover across providers without managing multiple API keys. Use Together AI directly when you need fine-tuning, dedicated GPUs, or the lowest possible per-token cost on a model you have already committed to.

When Should You Use Together AI

Use Together AI when you need a wide range of open models under one account, plan to fine-tune a model on your own data, or want reserved GPU pricing for predictable long-term workloads. It is a strong default for teams that have not settled on one model yet and want to test several without switching providers.

Skip it if your only requirement is the absolute lowest latency on a single popular model - Groq will usually beat it there. Skip it too if you only need occasional, low-volume calls to obscure community models - Replicate's pay-per-call model fits that better.

When we run our competitor-monitor routine across the AI-tooling space, the pattern that keeps showing up is that teams pick an inference provider once and rarely revisit the decision, even after their token volume or latency needs change. Together AI's reserved-cluster pricing is worth relooking at any time usage roughly doubles.