Tag: evaluation
All the articles with the tag "evaluation".
Token Economics on Amazon Bedrock: Prompt Caching and Tool Catalogs
Advice on LLM cost is usually advice on token counts. On Bedrock that is the wrong objective: a request spends in two currencies that disagree, and in an agentic loop what governs cost is whether the prefix is still cacheable. Two AWS recommendations applied together cost 18-20% more than either alone, the crossover tracks catalog tokens rather than tool count, and prompt caching makes output more deterministic, not less.
Replicating RouteLLM on Amazon Bedrock
Prompt-only LLM routers post strong in-distribution scores that are almost entirely recognition of which group a prompt came from. A reproduction of RouteLLM on Amazon Bedrock and AgentCore Gateway, a diagnostic suite that exposes it, and a zero-cost deferral rule that works where prompt-only routing does not.