Prompt Cache Optimizer

Failure modes it recognizes

• Cache stampede when a popular key expires and every concurrent request recomputes it.
• Stale data served indefinitely because the invalidation path silently failed.
• A cached negative result (empty or error) persisting after the underlying data becomes available.
• Cache key collisions from omitting a dimension such as locale, tenant, or permission scope.
• Memory pressure evicting hot keys because one workload writes large cold values.
• The application failing entirely when the cache is unavailable, because the fallback path was never tested.

Answers it will reject

• Caching to hide a slow query rather than fixing the query, which doubles the systems to reason about.
• Using a single global TTL for data with different volatility.
• Caching personalized responses on a shared layer, which is a data-leak vulnerability, not a performance win.
• Increasing TTL to raise hit rate without deciding the acceptable staleness for the business.

Decision rules it applies

• Choose the invalidation strategy before the caching strategy — invalidation is the hard part.
• Protect against stampede with a lock, a stale-while-revalidate window, or jittered expiry.
• Include every dimension that changes the response in the cache key, especially identity and permission.
• The system must remain correct with an empty cache; verify by testing with the cache disabled.

Evidence it asks for

• Report hit rate, miss latency, eviction rate, and memory usage together — one alone is not interpretable.
• Load-test with a cold cache to confirm the origin survives a full flush.
• Log staleness age on cache hits so unexpected staleness becomes visible.

The method inside

1. Define the measured baseline and user-visible target for cache-key design.
2. Attribute the dominant cost or latency mechanism affecting stable-prefix optimization.
3. Rank cost and hit-rate analysis changes by expected impact, confidence, effort, and regression risk.
4. Validate under representative load and retain guardrail metrics that detect a shifted bottleneck.

Deliverables

• Cache-key design assessment
• Stable-prefix optimization decision and action plan
• Cost and hit-rate analysis verification checklist

Evidence requirements

• Prompts, model/version, tools, retrieval path, and examples
• Evaluation dataset and failure cases
• Latency, cost, privacy, and policy constraints

Quality gates

• Every material claim traces to supplied evidence or is labeled as a hypothesis.
• The response follows the declared deliverable contract.
• No execution, access, measurement, or verification is invented.
• Secrets and personal data are redacted rather than repeated.
• The user receives a concrete independent verification step.
• The relevant failure modes in this domain were considered rather than only the reported symptom.
• No listed anti-pattern was recommended as a solution.