LLM Provider

The LLM is the optional, highest-cost layer: it runs as a fork for events the hot path and the local model could not settle — score ≥ only_above (30) or a Quarantine verdict. The core never calls an LLM API directly: everything goes through the LlmProvider trait, and selection happens by config. The default is off ([llm] provider = "none").

The LlmProvider trait

#[async_trait]
pub trait LlmProvider: Send + Sync {
    fn name(&self) -> &'static str;   // "openrouter", "ollama", …
    fn model_id(&self) -> &str;       // e.g. "anthropic/claude-3.5-sonnet"
    async fn classify(&self, req: ClassifyRequest) -> anyhow::Result<ClassifyResponse>;
    async fn explain(&self, req: ExplainRequest) -> anyhow::Result<String>;
}

Defined in sentry-ai/src/llm.rs. The contracts around it:

• ClassifyRequest — the payload (ProtocolData, works for Http/Tcp/Tls/…), a truncated human-readable summary and the mandatory JSON schema for the response.
• ClassifyResponse — verdict, risk_score (0–100), risk_level, signals, confidence (0.0–1.0), optional explanation and usage (LlmUsage: input/output tokens + cost, when the provider reports it).
• ExplainRequest — free-form explanation of an already-classified event (CLI and webhooks).

Shared prompt and parsing

Every adapter sends the same system prompt and JSON schema (llm/prompt.rs):

• SYSTEM_PROMPT — frames the model as a WAF triage analyst and tells it to be conservative: ordinary traffic → allow with a low score; known attack patterns → block/challenge with a high score.
• classify_schema — strict schema: closed verdict enum, risk_score 0–100, signals, confidence; extra fields forbidden (additionalProperties: false).
• context_from_event — the event summary for the user message (method, path, query, status, UA, referer, body preview; SNI/JA3/JA4 for TLS), truncated to 2048 characters for predictable token cost.
• parse_classify — tolerant parsing: extracts the JSON object even when surrounded by prose or markdown fences, normalizes tokens ("Rate-Limit" = rate_limit, "deny" = block), clamps out-of-range score/confidence and re-derives risk_level from the score.

Adapters

Constructed from config in make_llm_provider (also used by sentry bench llm). An unknown name or a missing key = warning

• stage disabled for the run — never a crash.

There is no dedicated Anthropic/OpenAI-SDK adapter: use openrouter (which routes to those models) or openai with base_url pointing at a compatible endpoint.

The daemon fork (LlmFork)

LlmFork mirrors the local AI fork:

• Trigger: score ≥ only_above (default 30) or a Quarantine verdict.
• Semaphore [llm] concurrency (default 4) + TTL cache keyed by payload hash (cache_ttl_secs, default 300 s) — identical payloads do not re-call the LLM within the window.
• Signal mapping (llm_signals): a benign answer (verdict allow with score < 20) produces no signal at all — the fork can only raise risk, so a zero-weight signal would be noise. A malicious answer → the LlmMalicious signal with weight = risk_score × confidence and detail = explanation (or the signal list).
• Re-entry through rescore_from (raise only); in mode = "fork" the event is updated in Postgres and applicable actions re-dispatched; in mode = "shadow" the outcome is only logged (would change verdict) — nothing acts.

[llm] configuration — minimal example

[llm]
provider = "openrouter"
model = "anthropic/claude-3.5-sonnet"
mode = "fork"                # or "shadow" to observe without acting
only_above = 30              # escalate from this score (or on Quarantine)
concurrency = 4
cache_ttl_secs = 300

export SENTRY_LLM_KEY="sk-or-v1-…"

Keys and defaults in LlmConfig; the default provider is none (off). To run fully local, switch to provider = "ollama" with model = "llama3.1" — no key, no code change.

Benchmarking providers

sentry bench llm --providers mock,ollama,openrouter --n 200

The sentry bench llm harness compares providers on latency, agreement with the heuristic pipeline, output quality and cost — using the built-in labeled synthetic kit or replaying an access.log (--events) — before you commit money to a provider.