Skip to main content

AWS Deployment

This guide shows two common ways to run KoreShield on AWS: a simple EC2 Docker host or a managed ECS service.

Use Cases​

  • Centralized LLM security proxy for multiple apps and teams
  • Compliance workloads that require audit trails and private networking
  • High-throughput production traffic with autoscaling and managed ingress

Architecture Choices​

  • EC2 + Docker for low-cost, low-ops environments
  • ECS Fargate for managed compute and simpler scaling
  • Optional: ALB in front of the service for TLS termination and health checks

Option A: EC2 + Docker​

1. Provision an EC2 instance​

  • Choose Amazon Linux 2023 or Ubuntu 22.04
  • Open inbound ports for your app and KoreShield (default 8000)
  • Prefer placing the instance in a private subnet with an ALB in front

2. Install Docker​

# Amazon Linux 2023
sudo dnf update -y
sudo dnf install -y docker
sudo systemctl enable --now docker
sudo usermod -aG docker ec2-user

3. Configure KoreShield​

Self-hosted customers receive a deployment bundle or a prebuilt image from KoreShield. Configure JWT and provider keys via environment variables or a managed secret store.

4. Run with Docker​

Run the KoreShield image provided for your account and pass environment variables (JWT + provider keys). Consult your deployment bundle for exact commands.

Option B: ECS (Fargate)​

1. Push the KoreShield image to ECR​

aws ecr create-repository --repository-name koreshield

# Authenticate and push
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <account-id>.dkr.ecr.us-east-1.amazonaws.com

docker tag koreshield:latest <account-id>.dkr.ecr.us-east-1.amazonaws.com/koreshield:latest
docker push <account-id>.dkr.ecr.us-east-1.amazonaws.com/koreshield:latest

2. Create an ECS task definition​

  • Container port: 8000
  • Environment variables: provider keys
  • Provide configuration via environment variables or managed secrets
  • Attach an IAM role for Secrets Manager or SSM access

3. Create a service​

  • Use an ALB or NLB
  • Configure health checks at /health
  • Enable autoscaling on CPU or request count

Secrets and Config​

  • Store provider keys in AWS Secrets Manager or SSM Parameter Store
  • Pass secrets to the task as environment variables
  • Use CONFIG_FILE if you mount a custom config path

Networking and TLS​

  • Terminate TLS at the ALB
  • Restrict inbound access to known CIDR ranges or VPC endpoints
  • Use a WAF if the endpoint is internet-facing

Observability​

  • Enable json_logs: true and ship logs to CloudWatch
  • Scrape /metrics with Prometheus or use a sidecar exporter
  • Create CloudWatch alarms for high error rates and latency

Security Notes​

  • Store API keys in AWS Secrets Manager or SSM Parameter Store.
  • Use json_logs: true in production.
  • Restrict inbound access to the proxy endpoint.

Troubleshooting​

  • 502/504 from ALB: confirm target health checks hit /health
  • 401 responses: verify your KORESHIELD_API_KEY header on clients
  • Timeouts: increase ALB idle timeout and client timeouts for streaming
  • Provider errors: confirm provider keys and allowed egress from the VPC

Next Steps​