Posts
All the articles I've posted.
Token Economics on Amazon Bedrock: Prompt Caching and Tool Catalogs
Advice on LLM cost is usually advice on token counts. On Bedrock that is the wrong objective: a request spends in two currencies that disagree, and in an agentic loop what governs cost is whether the prefix is still cacheable. Two AWS recommendations applied together cost 18-20% more than either alone, the crossover tracks catalog tokens rather than tool count, and prompt caching makes output more deterministic, not less.
Tenancy, Rate Limits and PII Masking on AgentCore Gateway
Three articles of per-tenant controls — budgets, caches, rate limits — all keyed on a header any caller could set. Replacing it with a validated token costs 0.53 ms. Along the way: why a request-rate limit cannot protect a token quota, why a cache hit was silently stealing quota it never spent, and why the PII screen that costs 50x less is the one you must not use.
Semantic Caching for LLMs on AWS
An exact LLM cache almost never hits, and the obvious fix — serve the answer to the nearest stored prompt — is worse than no cache: the near-misses that change the answer sit closer in embedding space than the paraphrases that don't. A verification stage fixes the ranking and leaves a 5% floor of confidently wrong answers that you cannot measure in production. Built on DynamoDB vector search, measured three ways, and mostly an argument for not running it.
Sizing Aurora Connections for Autoscaled Services on AWS
A service that scales on request rate scales its connection count with it, and Aurora's ceiling is fixed by instance memory. The arithmetic, the formula behind max_connections, why lowering the pool limit is a trap, and what RDS Proxy does and does not fix.