Skip to main content

Error Handling

Reliable error handling protects your users, prevents data leakage, and keeps systems stable under load. This guide focuses on safe patterns for LLM proxy and security middleware workflows.

Error Categories​

  • Input validation errors: invalid request shape, missing fields
  • Security policy errors: blocked content or unsafe requests
  • Provider errors: upstream model timeouts or failures
  • Rate limit errors: exceeded quotas or throttling
  • Infrastructure errors: network, storage, or dependency failures

1) Fail Closed for Unsafe Requests​

If a security check fails, block the request and return a safe message. Do not forward unsafe content to providers.

2) Use Stable Error Codes​

Return consistent error types so clients can handle them predictably. Keep messages human-readable but non-sensitive.

3) Retry with Backoff​

For transient provider or network errors, use exponential backoff. Do not retry on policy violations.

4) Timeouts and Circuit Breakers​

Set timeouts for upstream calls and protect your system from cascading failures.

5) Structured Logging​

Log errors in a structured format and exclude sensitive content:

logging:
	json_logs: true
	container_mode: true

Store trace IDs so you can correlate requests without storing full prompts.

User-Facing Messages​

  • avoid exposing internal prompts or system details
  • provide actionable guidance for users (rephrase, retry, contact support)
  • route sensitive issues to human review

Monitoring and Alerting​

  • alert on spikes in blocked requests or provider errors
  • track latency and timeout rates
  • set thresholds for rate-limit events