When TTLT drops on a Microsoft Foundry deployment, three explanations often arrive at once, and reading any one alone tells the wrong story. In many teams, dashboards pin one chart, one metric, or one deployment type — and attribute cache work to throughput savings (or vice versa), or load-shedding to a model upgrade. This article maps the signals three Microsoft documentation surfaces carry, and the correlation rules none of them call out.
The three surfaces are deployment types, latency, and prompt caching.
Axis one: deployment type changes what the other axes expose
Foundry offers three billing families — Standard, Provisioned, and Batch — whose sub-tiers change which metrics are worth plotting. Global Standard has the lowest price and broadest region coverage. Provisioned types reserve capacity for more consistent latency than Global Standard. Batch gives a 50% discount for a 24-hour async turnaround with no real-time SLA. Developer is fine-tune evaluation only, with a 24-hour lifetime and no SLA. Data Zone tiers restrict processing to a Microsoft-defined US/EU/APAC zone.
Dashboards built on Global Standard are not directly comparable to one built on Provisioned. Build per account type.
Axis two: the latency formula is half the answer
The Azure OpenAI latency article commits to a single formula — total response time equals time to first token plus time between tokens multiplied by tokens generated. TTLT scales with output tokens, so always pair a latency metric with a token-count metric before calling something a regression. The same five-second TTLT across a 2,000-token generation and a 50-token generation is two different signals.
The metrics that matter are five of the six documented latency signals: Time to Last Byte (end-to-end non-streaming), Time to Response (streaming first-byte), Time Between Tokens, Processed Prompt Tokens, and Generated Completion Tokens. Skip Normalized Time to First Byte for absolute diagnosis — the source documents it as a cross-prompt comparison tool.
Axis three: prompt caching only counts when the prefix qualifies
Cache eligibility is narrow: a request must contain at least 1,024 tokens, and the first 1,024 tokens must be identical across requests. A single character difference produces a cached_tokens value of zero. Hits surface as cached_tokens inside prompt_tokens_details.
On GPT-5.6 and later model families, Provisioned managed (PTU-M) deployments continue serving cached content, but neither breakpoints nor cache_write_tokens are reachable on this path. Standard pay-as-you-go exposes cache_write_tokens alongside cached_tokens. A dashboard that mixes both account types will misread the cost and pressure story.
Three correlation rules — design judgment, not source fact
- TTLT down,
cached_tokensup. A real cache hit. Verify the saved TTLT is consistent with the cached prefix length and re-check recent prefixes against the 1,024-token identity rule before crediting the cache. - TTLT down,
cached_tokensflat, GeneratedTokens flat. Not caching — capacity pressure eased. On PTU accounts, watch Provisioned-managed Utilization V2. On pay-as-you-go, watch Azure OpenAI Requests for 429 throttling and concurrent volume. - TTLT down,
cached_tokensflat, GeneratedTokens rising under load. Cache-eligible traffic is missing the cache — prompts arrive but fail the 1,024-token identity rule, or sit below the minimum. The latency article warns that mixing workloads on the same endpoint hurts both latency and cache hit rate.
What to instrument first
- Time to Last Byte split by ModelDeploymentName, paired with Generated Completion Tokens on the same window.
- For streaming products: Time to Response plus Time Between Tokens and Processed Prompt Tokens.
cached_tokenssampled from chat completions responses.- On PTU accounts: Provisioned-managed Utilization V2. On Standard pay-as-you-go:
cache_write_tokensalongsidecached_tokens.
Which deployment type earns the cache
Cache mechanics are largely stable across Global Standard, Data Zone Standard, and Regional Provisioned; what varies is the discount on cache reads (Provisioned managed reads can be billed at up to 100% discount on input tokens versus a smaller Standard discount) and the breakpoint or write-token exposure on PTU-M. Data Zone tiers only become a real choice under residency constraints; the deployment types article points to the model availability matrix for region-by-region support.
Rule 3 is the scenario worth acting on: TTLT falling, cached_tokens flat, GeneratedTokens rising under load signals cache-eligible traffic missing the cache — a workload worth migrating to a Provisioned tier, where cache reads carry a stronger discount and reserved capacity removes the capacity-easing explanation from Rule 2. No chart in any of the three surfaces names that combination.