What Is an AI Agent?

Question

What is an AI agent and what are its main cost components?

Quick answer

Formula: cost_per_workflow = generation + retrieval + reranking + vector_db + cache + infra

• Assumption: not every agent uses every block, so only model the steps that actually execute.
• Assumption: current calculators explicitly model generation, retrieval, reranking, vector, cache, and general infra costs.
• Assumption: provider-specific tool-call, search, memory, or session-fee lines are not first-class calculator rows today, so fold them into infra only if they are a small enough simplification.

Example: if generation=$0.024, retrieval=$0.002, reranking=$0.001, vector_db=$0.001, cache=-$0.003, and infra=$0.002, cost_per_workflow=$0.027 before monthly aggregation.

Common Building Blocks

• Planning or routing call: decides which step happens next.
• Retrieval: fetches relevant context from a vector store or search index.
• Reranking: improves relevance before synthesis when recall is noisy.
• Tool calling: hits APIs, databases, or actions outside the model.
• Memory or state: stores prior workflow data or fetches prior context.
• Synthesis: final answer generation after the earlier steps complete.

How Agents Differ From Prompt-only Apps

• They usually execute multiple paid steps instead of one model call.
• Branching logic makes cost less intuitive because not every request follows the same path.
• Retries, fallbacks, and tool failures can widen the spread between average and p90 cost.

What The Current Calculator Does Not Break Out Yet

• Dedicated provider fees for search, web, or tool-call products.
• Memory/state storage billed outside the model-token path.
• Session-fee or action-fee products that are not well represented as token or per-request infra cost.

Why Unit Economics Matter

• Small changes in retrieval depth or model choice compound across multi-step workflows.
• Gross margin can look healthy on light traffic and collapse when retries or long sessions increase.
• Explicit cost blocks make pricing and rollout decisions defensible before launch.