Archives

Three articles of per-tenant controls — budgets, caches, rate limits — all keyed on a header any caller could set. Replacing it with a validated token costs 0.53 ms. Along the way: why a request-rate limit cannot protect a token quota, why a cache hit was silently stealing quota it never spent, and why the PII screen that costs 50x less is the one you must not use.

An exact LLM cache almost never hits, and the obvious fix — serve the answer to the nearest stored prompt — is worse than no cache: the near-misses that change the answer sit closer in embedding space than the paraphrases that don't. A verification stage fixes the ranking and leaves a 5% floor of confidently wrong answers that you cannot measure in production. Built on DynamoDB vector search, measured three ways, and mostly an argument for not running it.

A service that scales on request rate scales its connection count with it, and Aurora's ceiling is fixed by instance memory. The arithmetic, the formula behind max_connections, why lowering the pool limit is a trap, and what RDS Proxy does and does not fix.

Every model call through Amazon Bedrock AgentCore Gateway passes one place where you can check a budget, clamp tokens, run a guardrail, pick the model, cache the answer and meter the cost. A reference architecture that makes each of those a swappable plugin, deployed and measured, plus seven things about interceptors the documentation does not tell you.

Prompt-only LLM routers post strong in-distribution scores that are almost entirely recognition of which group a prompt came from. A reproduction of RouteLLM on Amazon Bedrock and AgentCore Gateway, a diagnostic suite that exposes it, and a zero-cost deferral rule that works where prompt-only routing does not.

MorphOS never shipped a telnet daemon. Building one meant confronting the fact that the Amiga lineage has no pseudo-terminals — and that the answer is stranger and more interesting than a port. A short tour of the design, the war stories, and why it is really groundwork for an sshd.

A reference architecture and operating guide for running LiteLLM as an enterprise LLM gateway on AWS with Amazon Bedrock, plus the FinOps, governance and security work that scales with it, and a managed alternative on AgentCore Gateway.