Average Tokens per AI Request: p50/p90 Estimates

Estimate average token load with practical p50 and p90 defaults for agent and retrieval workflows, then replace the assumptions with measured request logs.

Question

How many input and output tokens should I assume per AI request?

Quick answer

Formula: effective_input_tokens = base_prompt_tokens + retrieved_chunks * tokens_per_chunk

Assumption: input and output token estimates should use p50 and p90 ranges.
Assumption: retrieved chunk tokens are modeled separately from base prompt tokens.
Assumption: the same token profile is used when comparing models.

Example: base prompt 600 + (4 chunks * 180) gives 1,320 effective input tokens per request.

Quick Starting Heuristics

Support agent: 400-900 effective input tokens, 120-300 output tokens.
Internal AI assistant: 700-1400 effective input tokens, 200-450 output tokens.
Research assistant: 1200-3000 effective input tokens, 300-900 output tokens.

Worked p50/p90 Token Scenario

Use two planning cases for the same workflow: p50 as the median request and p90 as the heavier tail. These are starting assumptions, not universal benchmarks; replace them with measured request logs when available.

Planning case	Base prompt	Retrieved chunks	Tokens / chunk	Effective input	Output
Illustrative p50	600	4	180	1,320	250
Illustrative p90	1,000	6	220	2,320	350

Illustrative p50

Base prompt

600 tokens

Retrieved context

4 chunks × 180 tokens

Effective input

1,320 tokens

Output

250 tokens

Illustrative p90

Base prompt

1,000 tokens

Retrieved context

6 chunks × 220 tokens

Effective input

2,320 tokens

Output

350 tokens

The effective-input column applies the formula above. Keep the model fixed while comparing these cases so the cost change comes from token load rather than a simultaneous pricing change.

Price This Token Profile

Open AI Workflow Cost with the same starter inputs shown above. Switch to Conservative to pressure-test the p90 case, then replace both with measured logs.

Open Workflow Cost Calculator

How To Improve Accuracy

Sample production requests and compute p50/p90 token usage.
Model retrieval context separately from prompt template tokens.
Track changes after prompt, retrieval, or model updates.

Back to calculators: Prompt Overhead, AI Workflow Cost