How It Works: API Gateway & Traffic Control with Envoy


Updated: 2025-08-24

Summary

Envoy is a programmable edge proxy. Terminate TLS, route by host/path, enforce auth via ext_authz, handle WebSockets, shape traffic with retries/timeouts, and attach rate‑limit/backpressure.

Edge Responsibilities

- TLS termination (cert manager issues; Envoy mounts certs)
- Host/path routing and redirects
- Centralized auth (sessions/JWT via ext_authz to an auth service)
- WebSocket upgrades and idle timeouts
- Rate limiting (external service) and connection limits
- Header normalization, gzip/br encoding, HSTS

Minimal Listener + Virtual Hosts

static_resources:
  listeners:
  - name: https
    address: { socket_address: { address: 0.0.0.0, port_value: 443 } }
    filter_chains:
    - filter_chain_match: { server_names: ["api.example.com","auth.example.com"] }
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:
            - certificate_chain: { filename: "/etc/envoy/tls/tls.crt" }
              private_key:       { filename: "/etc/envoy/tls/tls.key" }
      filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          upgrade_configs: [{ upgrade_type: websocket }]
          normalize_path: true
          request_timeout: 15s
          route_config:
            virtual_hosts:
            - name: api
              domains: ["api.example.com"]
              routes:
              - match: { prefix: "/v1" }
                route: { cluster: api_v1 }
            - name: auth
              domains: ["auth.example.com"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: auth_svc }
          http_filters:
          - name: envoy.filters.http.router

  clusters:
  - name: api_v1
    connect_timeout: 0.5s
    type: STRICT_DNS
    load_assignment:
      cluster_name: api_v1
      endpoints:
      - lb_endpoints:
        - endpoint:
            address: { socket_address: { address: api.apps.svc.cluster.local, port_value: 8080 } }
  - name: auth_svc
    connect_timeout: 0.5s
    type: STRICT_DNS
    load_assignment:
      cluster_name: auth_svc
      endpoints:
      - lb_endpoints:
        - endpoint:
            address: { socket_address: { address: auth.apps.svc.cluster.local, port_value: 80 } }

Ext AuthZ (sessions/JWT) — allow/deny at the edge

# Add before router filter
http_filters:
- name: envoy.filters.http.ext_authz
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
    http_service:
      server_uri:
        uri: auth.apps.svc.cluster.local
        cluster: auth_svc
        timeout: 1s
      path_prefix: /oauth2/auth          # oauth2-proxy style
      authorization_request:
        allowed_headers:
          patterns: [{exact: cookie}, {exact: authorization}, {exact: x-forwarded-host}]
      authorization_response:
        allowed_upstream_headers:
          patterns: [{exact: x-auth-request-user}, {exact: x-auth-request-email}]
- name: envoy.filters.http.router

Retries, Timeouts, and Circuit Breakers

# Per-route overrides
route:
  timeout: 2s
  retry_policy:
    retry_on: 5xx,reset,connect-failure
    num_retries: 2
    per_try_timeout: 500ms

# Connection limits (protect backends)
circuit_breakers:
  thresholds:
  - max_connections: 1024
    max_pending_requests: 512
    max_requests: 2048

WebSocket Keep‑alives & Limits

# On the HTTP connection manager
stream_idle_timeout: 0s        # don't kill long WS connections
idle_timeout: 300s              # but cap idle HTTP streams

Cookie & Security Headers

# Example HSTS and frame options via header appender
response_headers_to_add:
- header: { key: "Strict-Transport-Security", value: "max-age=31536000; includeSubDomains" }
- header: { key: "X-Frame-Options", value: "DENY" }
- header: { key: "X-Content-Type-Options", value: "nosniff" }

Kubernetes Mount for TLS secrets (cert-manager)

# Deployment snippet
volumes:
- name: tls
  secret: { secretName: envoy-tls }
containers:
- name: envoy
  image: envoyproxy/envoy:v1.30.2
  volumeMounts: [{ name: tls, mountPath: /etc/envoy/tls, readOnly: true }]

Observability

- Access log JSON with request_id, user, route, duration.
- Prometheus metrics: requests, 4xx/5xx, p95 latency, open connections.
- Tracing: propagate W3C headers; sample thoughtfully.

Security

- mTLS for upstreams if feasible; strictly validate host headers.
- Lock down admin interface; never expose it publicly.
- Regularly rotate TLS keys/certs; use SDS for zero‑downtime rotation.

Pitfalls

- Global timeouts too long → slow failures cascade.
- Missing fall‑through 404 vhost → accidental default routing.
- Over‑broad ext_authz → auth service outage becomes total outage.

Taylor Swift

“You need to calm down.”

Comments

Post a Comment

Popular posts from this blog

Learning to Automate My Side Projects with SWE-agent + GitLab

Image
I wanted to share something I built. I managed to connect SWE-agent with my self-hosted GitLab so that Issues can drive automation—and I think it’s kind of cool. (Currently, it isn’t directly supported in SWE-Agent, but there is an issue ( https://github.com/SWE-agent/SWE-agent/issues/760 ) on GitHub to add support. Why wait? ^_^) I’ve been tinkering with SWE-agent in the homelab while between roles. SWE-agent doesn’t natively support GitLab, but that wasn’t going to stop me—I wired it into my self-hosted GitLab instance so Issues drive the automation end-to-end. The Flow I Built 1. Start with an Issue. When I create an Issue, I describe the task in plain language. For example: “create `/taylor-swift` route that responds with JSON key lucky=13.” 2. Label the intent. Add the label `run:swe` and tag the model (in my case: `model:qwen3-coder-30b-a3b-instruct` from LM Studio). There is also a label swe:in-progress so that the pipeline does not pick up in-progress issues. If the pipeline ne...
Read more

Ship-Ready Web Essentials: Search, Sitemap, Metadata & Icons (SvelteKit)

This guide is a pragmatic, production-minded checklist and how-to for hardening a SvelteKit landing site so it’s discoverable by search engines, understandable by AI crawlers, and delightful on modern devices. It includes file locations, example contents, and verification steps. Everything here works even if you only have a single landing page today. Branding note: this document uses a fictional brand (“Example Store”) and domain (example.com). Replace these with your own values when you implement. Quick To-Do Checklist • Serve a real XML sitemap at /sitemap.xml and reference it in robots.txt. • Add a lightweight, server-rendered /search?q=… endpoint (backs JSON-LD SearchAction). • Place ai.txt (allow/deny AI crawlers) and .well-known/security.txt (vuln-reporting). • Wire canonical Open Graph/Twitter tags and og.png for social previews. • Add platform icons (apple-touch-icon.png, safari-pinned-tab.svg) and site.webmanifest. • Harden app.html with meta theme-color (light/dark), color-sc...
Read more

Kubernetes Secrets Management — SOPS + age (GitOps‑friendly)

Image
  Owner: Platform Engineering Audience: Devs & Ops Status: Adopted This document captures the exact process we use to manage Kubernetes secrets with Mozilla SOPS and the age encryption tool. It includes why we chose this approach, how to work with secrets day‑to‑day, and trade‑offs versus other methods. Executive Summary We store Kubernetes Secret manifests in Git encrypted at rest using SOPS with an age public key.  SOPS encrypts your secret values with AES‑256‑GCM , which is industry gold standard. In SOPS you can not swap AES‑256‑GCM out. The age private key lives only in protected CI variables. CI decrypts the manifests at deploy time and applies them with kubectl. No in‑cluster controllers, CRDs, or webhooks are required. Why this approach Top benefits: • Zero in‑cluster installs (no CRDs/controllers to run or break). • GitOps‑native: encrypted YAML sits in the repo; diffs remain readable (metadata & keys stay visible). • Cloud‑agnostic: age works anywhere; no ...
Read more