Sentry Alerts for AI Agent Cost Anomalies (30-Minute Setup)

If you're running AI agents in production, the worst Friday afternoon is the one where you find out at 5 PM that an agent has been looping since 2 AM. The bill is thousands you weren't planning on. The agent is still running.

This 30-minute setup makes that Friday afternoon impossible.

The pattern

Sentry's "Issues" feature can ingest custom events. Most teams use it for errors. It works just as well for cost anomalies.

Wire every AI API call to log to Sentry with cost metadata. Set alerts on:

• Per-function cost rate (alerts if any function exceeds $X/hour)
• Per-agent-instance cost (alerts if a single agent run exceeds $X)
• Daily total spend (alerts if daily spend exceeds $X)

When an alert fires, Sentry pages whoever is on-call. The runaway agent gets a human eye within minutes instead of hours.

The build

Step 1: Set up a Sentry project for AI events. (5 min)

You probably already have Sentry. Create a new project specifically for AI cost events (or use a tag on your existing project — either works).

Step 2: Wrap your AI client. (10 min)

If you're using the Anthropic SDK:

import Sentry from "@sentry/nextjs";
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const PRICING = {
  "claude-opus-4-7": { input: 15, output: 75 },     // per million tokens
  "claude-sonnet-4-6": { input: 3, output: 15 },
  "claude-haiku-4-5": { input: 0.25, output: 1.25 },
};

export async function callClaude(opts) {
  const start = Date.now();
  const result = await client.messages.create(opts);
  const elapsed = Date.now() - start;

  const pricing = PRICING[opts.model] ?? PRICING["claude-sonnet-4-6"];
  const costUsd =
    (result.usage.input_tokens / 1_000_000) * pricing.input +
    (result.usage.output_tokens / 1_000_000) * pricing.output;

  Sentry.captureMessage("ai.call", {
    level: costUsd > 0.5 ? "warning" : "info",
    tags: {
      model: opts.model,
      agent: opts.metadata?.agentName ?? "unknown",
      function: opts.metadata?.functionName ?? "unknown",
    },
    extra: {
      cost_usd: costUsd,
      input_tokens: result.usage.input_tokens,
      output_tokens: result.usage.output_tokens,
      elapsed_ms: elapsed,
    },
  });

  return result;
}

Every Anthropic call now lands in Sentry with cost metadata.

Step 3: Set up the alerts. (10 min)

Three alerts in Sentry's UI:

Alert 1: Per-call cost. Trigger when any ai.call event has cost_usd > $2. Most calls cost cents. A $2 call is an outlier worth seeing.

Alert 2: Per-agent rate. Trigger when an agent tag generates more than 100 events in 5 minutes. That's loop territory.

Alert 3: Daily spend. A Sentry alert based on aggregate cost_usd over 24 hours. Threshold: 2x your expected daily spend.

Step 4: Route alerts to PagerDuty or Slack. (5 min)

Connect Sentry to your existing on-call routing. Anyone with eyes on the system gets paged when an alert fires.

What it catches

The failures these alerts are built for:

• An agent stuck in a loop, caught by the rate alert within minutes instead of at the end of the month
• A prompt that balloons its context, for example by including too much chat history, caught by the per-call alert
• A fan-out bug that multiplies the call count, caught by the rate alert
• A model change that nobody carried into the cost calculation, caught by the daily spend alert

Without alerts, each of these shows up on the monthly bill.

What tends to break

Thresholds set too low. A per-call threshold set too low floods the channel in the first hour. Start conservative and tune it over the first week against your real call costs.

Spend alerts nobody sees. Route cost alerts to a dedicated Slack channel so finance and engineering both see them.

Missing metadata. If some agent code skips the agent name, the alerts can't say which agent misbehaved. Make the wrapper require an agent name and throw if it's missing. That is easier to fix at the wrapper boundary than to chase later.

What this isn't

This isn't a replacement for hard cost ceilings. You should ALSO have:

• Per-function budget caps that hard-abort
• Recursion limits
• Time limits per task
• Rate limits per agent

Sentry alerts are the last line of defense. The hard ceilings are the first line.

What to extend

Once basic alerts are in place, add:

Per-customer cost tracking. If you're charging customers for AI usage, you need per-customer cost data, not just per-function.

Quality vs cost correlation. Tag each call with a quality outcome (success / failure / retry). Sentry can chart cost-per-success across models.

Forecasting. Daily aggregate gets fed into a weekly forecast. When the forecast diverges from budget, alert finance.

But set up the basic three alerts first. They cover the common runaway patterns.

Total time

About 30 minutes for the basic setup, plus some threshold tuning over the first week.