{"head":{"title":"What Is Prompt Caching? LLM Cost Definition | OceanLab","description":"Prompt caching reuses the processed start of a prompt across calls, so repeated input is billed at a lower rate. How it works and when it saves money.","canonical":"https://oceanlab.in/glossary/prompt-caching"},"crumbs":[{"name":"Home","path":"/"},{"name":"Glossary","path":"/glossary"},{"name":"prompt caching","path":"/glossary/prompt-caching"}],"slug":"prompt-caching","term":"Prompt caching","aka":["context caching"],"category":"LLM costs","short":"Reusing the processed start of a prompt across calls, so repeated input is billed at a lower rate.","body":[["When many requests start with the same long prefix, such as instructions, tool definitions or a reference document, the provider can keep the processed prefix for a short time. Later calls that start with exactly the same prefix read it from the cache instead of processing it again."],["Cached reads are billed below normal input ",{"href":"/glossary/token","text":"tokens"},". Writing to the cache can cost more than normal input on some providers, and the cache expires after a period without use. Some providers cache automatically; others need you to mark where the cached part ends."]],"why":["For chat assistants, agents and document tools, most input is the same on every call. Caching can make that repeated part much cheaper and often faster to start answering. Whether it pays off depends on how often the same prefix is reused before it expires."],"example":["An ",{"href":"/glossary/ai-agent","text":"AI agent"}," sends the same system prompt and twenty tool definitions on every step of a task. Moving the changing parts, such as the user's request and tool results, after that fixed block lets every step after the first read the fixed block from the cache."],"inBuild":null,"mistakes":[["Putting changing content, like a timestamp or user name, at the very start. Any change in the prefix breaks the cache."],["Caching prompts that are rarely reused before the cache expires, so you pay the write cost without the savings."],["Comparing providers on the list input price alone when most of your input would be cached."]],"related":[{"href":"/glossary/token","label":"Token (LLM)","detail":"The unit an AI language model reads and writes, and the unit most LLM APIs bill by."},{"href":"/glossary/context-window","label":"Context window","detail":"The most tokens a model can take in and produce in one request."},{"href":"/glossary/batch-api","label":"Batch API","detail":"A way to send many LLM requests to run later, usually at a discount."},{"href":"/glossary/model-routing","label":"Model routing","detail":"Sending each AI request to the model best suited to it, instead of using one model for everything."}],"tools":[{"href":"/tools/prompt-caching-savings","label":"Prompt caching savings","detail":"Free tool"},{"href":"/tools/llm-api-cost-calculator","label":"LLM API cost calculator","detail":"Free tool"}],"links":[{"href":"/pricing/categories/llm-api","label":"LLM API pricing","detail":"Model prices side by side, with sources."},{"href":"/pricing","label":"Pricing library","detail":"Software and API prices, each with its source and date."}],"sources":[],"legal":false,"updatedAt":"2026-10-25"}