Tag: amazon-bedrock
All the articles with the tag "amazon-bedrock".
Token Economics on Amazon Bedrock: Prompt Caching and Tool Catalogs
Advice on LLM cost is usually advice on token counts. On Bedrock that is the wrong objective: a request spends in two currencies that disagree, and in an agentic loop what governs cost is whether the prefix is still cacheable. Two AWS recommendations applied together cost 18-20% more than either alone, the crossover tracks catalog tokens rather than tool count, and prompt caching makes output more deterministic, not less.
Tenancy, Rate Limits and PII Masking on AgentCore Gateway
Three articles of per-tenant controls — budgets, caches, rate limits — all keyed on a header any caller could set. Replacing it with a validated token costs 0.53 ms. Along the way: why a request-rate limit cannot protect a token quota, why a cache hit was silently stealing quota it never spent, and why the PII screen that costs 50x less is the one you must not use.
Semantic Caching for LLMs on AWS
An exact LLM cache almost never hits, and the obvious fix — serve the answer to the nearest stored prompt — is worse than no cache: the near-misses that change the answer sit closer in embedding space than the paraphrases that don't. A verification stage fixes the ranking and leaves a 5% floor of confidently wrong answers that you cannot measure in production. Built on DynamoDB vector search, measured three ways, and mostly an argument for not running it.
Customizing inference on AgentCore Gateway: a plugin pipeline for routing, budgets and guardrails
Every model call through Amazon Bedrock AgentCore Gateway passes one place where you can check a budget, clamp tokens, run a guardrail, pick the model, cache the answer and meter the cost. A reference architecture that makes each of those a swappable plugin, deployed and measured, plus seven things about interceptors the documentation does not tell you.