Networking

Everything between your users and the API: the DNS records to create, where TLS terminates, what a proxy has to forward for real-time collaboration to work, where a load balancer sits when you scale past one replica, which health-check path to point it at, and the full ingress/egress port matrix.

One origin, four variables

TruePPM runs on a single hostname. The SPA, the REST API, and the WebSocket endpoint are all served from the same origin, and every shipped topology routes them by path on one host:

Path prefix	Served by
/	the compiled React SPA (nginx)
/api/	the Django API
/ws/	the Django Channels WebSocket endpoint (/ws/v1/projects/<uuid>/)
/static/	Django’s collected static files (admin CSS, Swagger UI)

Four settings all name that one origin. They are read by different subsystems and nothing cross-checks them, so a mismatch surfaces later as a 400, a broken email link, or an OIDC redirect_uri your identity provider rejects — never as a startup error:

Variable	Format	Read by	If it disagrees
DOMAIN	bare hostname, no scheme — trueppm.example.com	Docker Compose only: envsubst into the nginx server_name and the certificate paths, and certbot -d	nginx serves the default server or fails to find the certificate; ACME issues for the wrong name
ALLOWED_HOSTS	comma-separated bare hostnames	Django, on every request	HTTP 400 on every request whose Host header is not listed — including health checks that connect by IP (see Health checks)
TRUEPPM_FRONTEND_BASE_URL	full origin with scheme, no trailing slash, no path — https://trueppm.example.com	notification email deep-links	emails link to the wrong host, or omit links entirely when unset
TRUEPPM_PUBLIC_API_BASE_URL	full origin with scheme, no trailing slash — same origin as above	OIDC only: builds {base}/api/v1/auth/oidc/callback/	your IdP refuses the login with a redirect_uri mismatch

TRUEPPM_PUBLIC_API_BASE_URL is empty by default, which falls back to the incoming request’s absolute URL. That is correct only when the SPA and the API are genuinely reached at one origin — the production single-server Compose topology above, or any deploy proxied so Host reaches Django unchanged — and wrong behind a proxy that rewrites Host, where the value then depends on forwarded headers rather than on anything you set. Set it explicitly on any proxied deploy so the value your IdP allow-lists is deterministic. See Single sign-on.

This is not automatically true of the bundled local development stack (docker-compose.yml, make up) — that stack runs the SPA (Vite, :5173) and the API (:8000) on two different ports, and Vite’s dev proxy rewrites Host on the way through exactly like a reverse proxy would. docker-compose.yml therefore pins both TRUEPPM_PUBLIC_API_BASE_URL and TRUEPPM_FRONTEND_BASE_URL explicitly rather than relying on the empty default — see Testing SSO against the local development stack.

Split hostnames are not supported

Serving the SPA at app.example.com and the API at api.example.com does not work. TruePPM ships no CORS support — there is no django-cors-headers dependency and no CORS_* setting, so Django never emits an Access-Control-Allow-Origin header and the browser blocks every cross-origin XHR and WebSocket upgrade from the SPA. CSRF_TRUSTED_ORIGINS, CSP_CONNECT_SRC, and AUTH_REFRESH_COOKIE_SAMESITE address CSRF, CSP, and cookie scope — three different layers, none of them CORS — so relaxing them changes nothing about the block. Use one hostname.

A wildcard certificate and a wildcard DNS record are not required. One name is enough.

DNS

Records to create

One A record (and one AAAA if you serve IPv6) pointing at whatever terminates TLS — the Compose host’s public IP, your load balancer, or the ingress controller’s external address:

; example.com zone file
$ORIGIN example.com.
$TTL 300

trueppm     IN  A       203.0.113.42
trueppm     IN  AAAA    2001:db8::2a

; Optional but recommended: pin which CAs may issue for this name.
; Required if you use ACME (Let's Encrypt) — issuance fails if a CAA
; record exists and does not list the CA.
trueppm     IN  CAA 0 issue "letsencrypt.org"
trueppm     IN  CAA 0 iodef "mailto:ops@example.com"

Keep the TTL low (300s) until the install is verified, then raise it.

On Kubernetes, do not hand-maintain the record if you can avoid it — point external-dns at the same hostname you put in ingress.hosts[0].host, so the record and the Ingress cannot drift.

Verify before installing. ALLOWED_HOSTS rejects anything else, and ACME HTTP-01 validation needs the name to already resolve:

dig +short trueppm.example.com A
dig +short trueppm.example.com AAAA

Split-horizon and internal-only names

If TruePPM resolves to a private address on the inside and a public one on the outside, nothing in the app cares — it only ever sees the Host header. But TruePPM’s outbound SSRF guard does care: every host TruePPM connects to must resolve to a globally routable address, or the request is refused. That blocks an in-cluster identity provider (keycloak.sso.svc), an internal SMTP relay, or a MinIO or SeaweedFS endpoint on a private VIP.

TRUEPPM_EGRESS_ALLOWLISTED_HOSTS is the escape hatch — a comma-separated list of exact, case-insensitive hostnames exempted from that check:

TRUEPPM_EGRESS_ALLOWLISTED_HOSTS=keycloak.sso.svc.cluster.local,smtp.internal.example.com

No wildcards. The allow-list is consulted by every egress caller (SSO, webhooks, SMTP, integration credential verification), so only list a host you trust at least as much as your identity provider. See Outbound requests.

Internal (cluster) DNS

The Helm web tier’s nginx proxies /api/ and /ws/ to the API Service by name. The chart’s fullname helper collapses the release name when it already contains trueppm, so helm install trueppm ... yields trueppm-api and helm install foo ... yields foo-trueppm-api:

Object	In-cluster name	Port
API Service	<release>-api (or <release>-trueppm-api)	8000 (service.port)
Web Service	<release>-web (or <release>-trueppm-web)	80 (service.web.port), container 8080

nginx resolves upstreams once, at startup

nginx resolves a proxy_pass hostname when it parses the config, not per request. If the name does not resolve at that moment, nginx exits — the pod crash-loops with host not found in upstream, and no request is ever served.

Two consequences:

1. Never reuse the Compose nginx config on Kubernetes. nginx/app.conf.template proxies to http://api:8000 — a Compose network alias that does not exist in a cluster. The chart’s own config uses the release-scoped Service name, which is why it works and a copied compose config does not.
2. If you replace the web tier with your own nginx, the API Service must exist before that nginx starts. A helm upgrade that recreates both at once can lose the race; give your nginx a resolver directive and a variable proxy_pass if you need it to tolerate a missing upstream at boot.

TLS

TruePPM never terminates TLS itself. The API container speaks plain HTTP on :8000, always. Something in front of it — the bundled nginx, an ingress controller, or a cloud load balancer — owns the certificate.

Topologies

Topology	TLS terminates at	Must forward	TRUEPPM_SECURE_SSL_REDIRECT	HSTS owner
Compose + bundled nginx (TLS_MODE=letsencrypt / selfsigned)	the stack’s nginx, on :443	nothing to configure — nginx/app.conf.template already sets Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto and the WebSocket upgrade headers	leave false — nginx already 301s :80 → :443	the nginx template (max-age=63072000; includeSubDomains)
Compose behind an external LB (TLS_MODE=none)	your load balancer	LB → nginx: Host, X-Forwarded-For, X-Forwarded-Proto, Upgrade, Connection. See the caveat below — an old copy of the HTTP template may still overwrite X-Forwarded-Proto	leave false — the LB does the redirect	the load balancer
Helm + ingress-nginx	the Ingress (ingress.tls[].secretName)	ingress-nginx sets X-Forwarded-For / X-Forwarded-Proto and preserves Host itself; add the WebSocket timeout annotations from WebSockets	leave false — set nginx.ingress.kubernetes.io/ssl-redirect: "true" instead (it is the controller default when a TLS block exists)	ingress-nginx (hsts: true in its ConfigMap) and web.securityHeaders.strictTransportSecurity for the SPA document
Helm behind a cloud LB (ALB / GCLB), Ingress in HTTP mode	the cloud load balancer	the LB must send X-Forwarded-Proto: https and preserve Host. ALB and GCLB both do by default. Keep the backend protocol on HTTP/1.1 so the WebSocket upgrade survives	leave false	the cloud LB, plus web.securityHeaders.strictTransportSecurity
Helm + cert-manager	the Ingress, with the Secret provisioned automatically	as ingress-nginx above	leave false	as ingress-nginx above

Every row needs the original Host preserved, not just the two Compose ones. X-Forwarded-Host is ignored — see the caution below for what to do when your edge rewrites Host.

TRUEPPM_SECURE_SSL_REDIRECT stays false in every supported topology. Turn it on only when TruePPM receives the original scheme reliably and nothing in front of it already redirects — otherwise you get a redirect loop. The probe paths (/api/v1/health/, /api/v1/readyz, /api/v1/edition/) are exempt from the redirect regardless, so enabling it never breaks a health check. See TLS redirect posture.

The scheme comes from the proxy; the host does not

TruePPM takes the scheme from your proxy and the host from the request, and the asymmetry is deliberate. settings.prod sets SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https") and USE_X_FORWARDED_HOST = False.

1. X-Forwarded-Proto is trusted unconditionally — TruePPM does not check who sent it, because the container only ever speaks plain HTTP on :8000 and has no other way to learn the original scheme. So the API Service must not be reachable directly: anything that can open a socket to the API pod can assert X-Forwarded-Proto: https and be believed. The chart’s NetworkPolicy restricts ingress to the datastore pods only; it deliberately does not restrict ingress to the API pods, so this is yours to enforce — see Ports and firewall.

2. X-Forwarded-Host is ignored, so your edge must preserve Host. Every proxy TruePPM ships already does (proxy_set_header Host $host), and none sets X-Forwarded-Host. Trusting it would mean believing a header that, on every supported topology, only the client could have written — Host is the routing key at the edge, so a value that does not route never arrives, whereas X-Forwarded-Host routes fine and would be believed anyway. There is deliberately no setting to turn this on.

If your edge rewrites Host to an internal service name and you cannot change it, set TRUEPPM_PUBLIC_API_BASE_URL to your public origin. That pins the two absolute URLs TruePPM would otherwise derive from the request — the OIDC redirect_uri and the inbound Git-webhook URL an admin pastes into GitHub/GitLab. Without it, both render with the internal name and fail silently: the webhook simply never arrives. (Email links are unaffected — they come from TRUEPPM_FRONTEND_BASE_URL, never from the request.)

Whatever Host does arrive must be in ALLOWED_HOSTS — see Host names you must include.

Every plain-HTTP path now preserves the scheme

Any config whose listener is plain HTTP — because TLS terminates somewhere in front of it, not at this nginx — needs the inbound X-Forwarded-Proto preserved, not overwritten: nginx/app-http.conf.template (TLS_MODE=none, behind an external load balancer), nginx/demo.conf.template (the Compose public demo), and the two demo branches of the Helm web tier’s rendered ConfigMap (demo.enabled / demo.interactive — this is the path try.trueppm.com uses, behind a Cloudflare Tunnel). nginx/app.conf.template and the Helm production branch are the opposite case: they either terminate TLS themselves or document no X-Forwarded-Proto override at all, so they are unaffected.

If you copied one of the affected configs before this was fixed (#3191 finding #4, closed for all four paths by #4426), your copy may still contain proxy_set_header X-Forwarded-Proto $scheme, which replaces your proxy’s https with this nginx’s own (always http) scheme — so TruePPM emits no HSTS, builds http:// absolute URLs, would 301-loop if TRUEPPM_SECURE_SSL_REDIRECT were on, and rejects a same-origin token refresh with a 403 on every page reload. The shipped templates now honor an inbound value through a map and fall back to $scheme only when none is present:

map $http_x_forwarded_proto $forwarded_proto {
    default $scheme;
    ~.      $http_x_forwarded_proto;
}

Re-copy the template (or upgrade the chart), or add the map yourself and use $forwarded_proto on every proxy_set_header X-Forwarded-Proto line. Either way this is only safe when the proxy in front is the only way in — a client that reaches nginx directly can set the header too.

X-Forwarded-For has a quieter failure, and it has two causes that look identical from the outside. DRF’s per-IP throttles key on the client address TRUEPPM_NUM_PROXIES hops back in X-Forwarded-For (default 1). Get either the header or the hop count wrong and every request arrives with the same apparent source address, so the per-IP login and anonymous throttles collapse into one shared bucket — which does not merely fail to throttle an abuser, it lets one client spend everybody’s budget.

Cause 1 — the header is absent. The production branch of the Helm web tier’s nginx sets Host, X-Real-IP and Authorization but not X-Forwarded-For, so routing /api through the web tier leaves DRF reading REMOTE_ADDR — the web pod’s own peer. The default ingress.hosts sends /api and /ws straight to the API Service, where this does not arise. If you change that, add the header at your edge.

Cause 2 — the header is present and the hop count is too low. The demo branch of that same nginx does set X-Forwarded-For, with $proxy_add_x_forwarded_for, which appends its own peer to whatever arrived. So the header being there is not enough: NUM_PROXIES has to match how many proxies actually sit in front of Django, and it is the last NUM_PROXIES entries that get skipped.

Count the hops on the path the request really takes:

Path to Django	Proxies	TRUEPPM_NUM_PROXIES
Ingress → API Service (the default ingress.hosts), nothing in front of the ingress controller	1	1
Cloud load balancer or CDN → ingress-nginx → API Service	see below	1 or 2
Ingress → web tier → API Service	2	2
Cloudflare Tunnel → web tier → API Service (demo mode)	2	2
Docker Compose production stack (nginx → api)	1	1
Development docker-compose.yml (:8000 directly, or the Vite proxy on :5173)	0	0, set in the file
Client → API Service, nothing in front	0	0

A proxy only counts if it appends to X-Forwarded-For. The Vite dev server’s proxy passes the header through without adding to it, so the development stack counts as zero proxies even when you browse through :5173.

Demo mode is always in the two-hop rows: it refuses an Ingress path that reaches the API Service directly, so /api/v1/share/ necessarily crosses the web tier. values-demo.yaml therefore overrides this to 2. At 1 the demo’s only public endpoint shares a single 60/min bucket across every visitor on the internet, and one crawler answers 429 to everyone else.

Set it too high and you have the opposite problem: a client can prepend its own X-Forwarded-For and choose its own throttle bucket. With nothing in front of Django and the default of 1, rotating that one header gives a client a fresh login brute-force budget per value. The per-account login_account limit still applies, but the per-IP limit stops working. Match the number to the deployment, and re-check it whenever you change what sits in front of the API.

TruePPM refuses to start on a negative TRUEPPM_NUM_PROXIES, because DRF would then read the header from its client-written end.

A load balancer in front of the ingress controller

This is the ordinary production shape on a managed cloud (AWS ALB or NLB, Google Cloud Load Balancing, Azure Application Gateway, Cloudflare proxying the hostname). It is also where the default of 1 quietly stops being right. ingress-nginx is configured by its controller ConfigMap, not by this chart:

• use-forwarded-headers: "false" (the ingress-nginx default). The controller discards the incoming X-Forwarded-For and sends only the address it saw, which is the load balancer’s. No value of TRUEPPM_NUM_PROXIES recovers the visitor, so every client shares one bucket. This is fine only for an L4 load balancer that preserves the source address (an NLB or type: LoadBalancer Service with externalTrafficPolicy: Local, or PROXY protocol with use-proxy-protocol). In that case the controller really does see the visitor, and 1 is right.
• use-forwarded-headers: "true" and proxy-real-ip-cidr set to the load balancer’s address range. The controller works out the client address itself, trusting the header only from the range you named, and forwards that single address. Keep TRUEPPM_NUM_PROXIES at 1. This is the setup we recommend, because the trust decision is made by address range at the edge rather than by counting hops. Do not leave proxy-real-ip-cidr at its default of 0.0.0.0/0. With every address trusted, the controller walks the header back to its leftmost entry, which the client wrote, and you are in the too-high case again.
• compute-full-forwarded-for: "true". The controller appends to the chain instead of replacing it, so Django sees one more entry per proxy. Set TRUEPPM_NUM_PROXIES to the full count, usually 2, and make sure nothing except the load balancer can reach the controller.

A CDN in front of a cloud load balancer adds a hop again. Don’t work the count out from a diagram. Read it back from the running deployment, as described in the next section.

Check the value

The address the throttles actually used can’t be seen from outside, so check it directly. A workspace operator can call the System Health endpoint, and its security.client_address block shows how their own request was resolved:

curl -s -H "Authorization: Bearer $TOKEN" \
  https://pm.example.com/api/v1/health/system/ | jq .security.client_address

{
  "num_proxies": 1,
  "resolved": "198.51.100.23",
  "remote_addr": "10.42.1.7",
  "forwarded_for": ["198.51.100.23"],
  "forwarded_for_entries": 1,
  "status": "ok",
  "detail": "Nothing wrong is provable from this request. …"
}

resolved is the address every per-IP throttle keyed this request on. Run two checks against it:

1. It is your address. Compare it with the public address you are connecting from. If it is a private or load-balancer address, the value is too low.
2. You cannot change it. Repeat the call with -H "X-Forwarded-For: 192.0.2.1". If resolved becomes 192.0.2.1, the value is too high.

status flags the cases one request is enough to prove:

status	Meaning
ok	Nothing provably wrong. Still run both checks above.
fewer_hops_than_configured	Fewer X-Forwarded-For entries arrived than TRUEPPM_NUM_PROXIES says proxies exist. The value is too high for this path, or a proxy on it does not set the header.
forwarded_for_ignored	The value is 0, but the header arrived. If a proxy wrote it, every client shares that proxy’s bucket.
non_public	resolved is a private, loopback or link-local address. That is correct from inside your network, but if you came in over the internet it is a proxy’s address.
unparsable	The position the value selects doesn’t hold an IP address.

The block describes the path your request took. If /api reaches Django by more than one path (for example straight from the ingress and also through the web tier), check each path.

The client_address block is added after v0.4.0-beta.3, and is absent on that tag and earlier.

cert-manager

ClusterIssuer is cluster-scoped, so one object serves every namespace. Pick HTTP-01 if the hostname is reachable from the public internet on port 80, and DNS-01 if it is not (or if you want a wildcard).

HTTP-01 — cert-manager creates a temporary Ingress to answer the challenge, so ingress.class must match the controller that owns your hostname:

clusterissuer-http01.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ops@example.com
    # cert-manager creates this Secret and stores the ACME account key in it.
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            class: nginx

kubectl apply -f clusterissuer-http01.yaml
kubectl get clusterissuer letsencrypt-prod -o wide   # READY should be True

DNS-01 — no inbound port 80 needed, works for internal-only hostnames and wildcards. Route 53 shown; every provider differs only in the solver block:

kubectl create secret generic route53-credentials \
  --namespace cert-manager \
  --from-literal=secret-access-key='<AWS secret access key>'

clusterissuer-dns01.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ops@example.com
    privateKeySecretRef:
      name: letsencrypt-dns-account-key
    solvers:
      - dns01:
          route53:
            region: us-east-1
            hostedZoneID: Z0123456789ABCDEFGHIJ
            accessKeyID: AKIAIOSFODNN7EXAMPLE
            secretAccessKeySecretRef:
              name: route53-credentials
              key: secret-access-key
        # Only solve for names in this zone; other names fall through to
        # another solver. Drop the selector if this issuer serves everything.
        selector:
          dnsZones:
            - example.com

Pair either issuer with the chart’s Ingress. The annotation names the issuer; cert-manager watches the Ingress, sees the tls block, and creates the trueppm-tls Secret for you — you never run kubectl create secret tls:

my-values.yaml

ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod   # or letsencrypt-dns
    # Keep the chart's own defaults; Helm deep-merges this map, but naming a
    # key here replaces it, so restate the three below rather than dropping
    # them. All three ship as chart defaults already (values.yaml) — shown
    # here only because adding cert-manager.io/cluster-issuer to this map
    # would otherwise silently keep them (deep-merge), which is correct, but
    # worth restating explicitly since this file overrides the whole
    # `ingress.annotations` key by example elsewhere on this page.
    nginx.ingress.kubernetes.io/proxy-body-size: "110m"
    # WebSocket lifetime — see the timeout table below.
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
  hosts:
    - host: trueppm.example.com
      paths:
        - path: /api
          pathType: Prefix
          service: api
        - path: /ws
          pathType: Prefix
          service: api
        - path: /
          pathType: Prefix
          service: web
  tls:
    - secretName: trueppm-tls
      hosts:
        - trueppm.example.com

Watch the issuance; a stuck certificate almost always names its own cause:

kubectl -n trueppm get certificate trueppm-tls
kubectl -n trueppm describe certificate trueppm-tls
kubectl -n trueppm get challenges          # empty once validation completes

Bring your own certificate — Kubernetes

If your certificate comes from an internal CA or a commercial vendor, create the Secret yourself and skip cert-manager entirely. The Secret must be in the same namespace as the Ingress:

kubectl create secret tls trueppm-tls \
  --namespace trueppm \
  --cert=fullchain.pem \
  --key=privkey.pem

fullchain.pem must be the leaf certificate followed by every intermediate, in that order. A leaf-only file validates in a browser that already caches the intermediate and fails everywhere else.

Then reference it, with no cert-manager.io/cluster-issuer annotation:

ingress:
  enabled: true
  className: nginx
  tls:
    - secretName: trueppm-tls
      hosts:
        - trueppm.example.com

Renewal is yours: replace the Secret (kubectl create secret tls ... --dry-run=client -o yaml | kubectl apply -f -) before expiry. ingress-nginx picks up the change without a restart.

Bring your own certificate — Docker Compose

TLS_MODE has three values (letsencrypt, selfsigned, none) and none of them means “use the certificate I already have”. The HTTPS nginx template reads a fixed pair of paths, so the supported way in is to put your files there:

certbot/conf/live/${DOMAIN}/fullchain.pem
certbot/conf/live/${DOMAIN}/privkey.pem

Order matters, because init-prod.sh overwrites both files on the selfsigned path and replaces nginx/active.conf.template on every path:

# 1. First boot only. TLS_MODE=selfsigned selects the HTTPS nginx template and
#    leaves the certbot renewal profile OFF — both of which is what you want.
#    It also writes a throwaway self-signed pair, which step 2 replaces.
./init-prod.sh

# 2. Install the real certificate over it.
cp /path/to/fullchain.pem certbot/conf/live/"${DOMAIN}"/fullchain.pem
cp /path/to/privkey.pem   certbot/conf/live/"${DOMAIN}"/privkey.pem

# 3. Reload nginx in place — no downtime, no container restart.
docker compose -f docker-compose.prod.yml exec nginx nginx -s reload

Caution

Do not re-run init-prod.sh afterwards. With TLS_MODE=selfsigned it regenerates the self-signed pair and silently overwrites your certificate; with TLS_MODE=none it swaps the config back to the plain-HTTP template. On renewal, repeat steps 2 and 3 only.

The nginx container already reloads itself every 6 hours, so replacing the files and waiting is also valid — the explicit reload just makes it immediate.

WebSockets behind a proxy

Real-time collaboration runs over wss://<host>/ws/v1/projects/<uuid>/. The connection is authenticated by a single-use ticket on the query string (?ticket=...), minted by POST /api/v1/ws/ticket/.

Required headers

Any proxy in the path must speak HTTP/1.1 upstream and pass the upgrade handshake through. This is the shipped block from nginx/app.conf.template, annotated:

# Maps the inbound Upgrade header to the right Connection value. Without the
# map, a `Connection: upgrade` hard-coded on non-WebSocket requests breaks
# keepalive; with it, normal HTTP gets `Connection: close`.
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

location /ws/ {
    proxy_pass             http://api:8000;

    # REQUIRED. The Upgrade mechanism does not exist in HTTP/1.0, which is
    # nginx's default to upstreams — without this the handshake never happens
    # and the client sees a plain 200 or 400 instead of a 101.
    proxy_http_version     1.1;

    # REQUIRED. These two carry the handshake itself.
    proxy_set_header       Upgrade           $http_upgrade;
    proxy_set_header       Connection        $connection_upgrade;

    # REQUIRED. Django validates this against ALLOWED_HOSTS on the WebSocket
    # path too; nginx would otherwise send the upstream's name ("api").
    proxy_set_header       Host              $host;

    # Client identity for the per-IP throttles (see TRUEPPM_NUM_PROXIES).
    proxy_set_header       X-Real-IP         $remote_addr;
    proxy_set_header       X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header       X-Forwarded-Proto $scheme;

    # A WebSocket carries no traffic while a project is quiet. The default
    # 60s would close it; this is a connection lifetime, not a request timeout.
    proxy_read_timeout     86400s;
}

Idle timeouts

Every proxy closes a connection that looks idle. The knob is different on each one, and on some of them it is not an idle timeout at all:

Proxy	Knob	Default	Set to
nginx (standalone)	proxy_read_timeout / proxy_send_timeout	60s	3600s or more
ingress-nginx	nginx.ingress.kubernetes.io/proxy-read-timeout / proxy-send-timeout (seconds, unquoted number as a string)	60	"3600"
AWS ALB	idle_timeout.timeout_seconds, via alb.ingress.kubernetes.io/load-balancer-attributes	60	3600 (ALB maximum 4000)
GCP external HTTP(S) LB	backend service timeoutSec, via a BackendConfig	30	3600
HAProxy	timeout tunnel (haproxy.org/timeout-tunnel on the HAProxy Ingress Controller)	inherits timeout client/timeout server	3600s

Recommended floor: 3600 seconds. A user with the schedule open in a background tab over lunch is a normal case, and a reconnect storm from a whole team timing out together is not.

On GCP, timeoutSec is a hard lifetime, not an idle timeout

For WebSocket backends the Google Cloud load balancer treats the backend service’s timeoutSec as the maximum duration the connection may stay open, whether or not traffic is flowing. Keepalive pings do not extend it. Every client will be disconnected at that interval; size it accordingly and make sure the reconnect path is exercised.

Why idle sockets survive a 60s default today

Nothing in the TruePPM client sends a heartbeat — the project socket is receive-only, and the presence keepalive re-arms a Redis TTL without putting a byte on the wire. What actually keeps an idle connection alive is uvicorn’s protocol-level ping:

• ws_ping_interval defaults to 20.0 — uvicorn sends a WebSocket PING frame every 20 seconds on every open socket.
• ws_ping_timeout also defaults to 20.0 — if the matching PONG does not come back within 20 seconds, uvicorn closes the connection.

The API image runs bare uvicorn with no override for either, so both defaults apply. That 20-second ping is the only reason a 60-second proxy default does not drop every socket, and it is load-bearing rather than deliberate. Two rules follow:

1. Never set --ws-ping-interval above your proxy’s idle timeout. Raising it to “reduce chatter” reintroduces exactly the disconnect it is preventing.
2. Do not buffer the WebSocket location. A proxy that delays PONGs by more than 20 seconds makes the server drop the client — a disconnect that looks like a network fault and is not.

No sticky sessions

Session affinity is not required, and you should not configure it. Any replica can serve any socket:

• The channel layer is channels_redis.core.RedisChannelLayer, backed by the same Valkey every replica shares. There is no in-memory layer anywhere in the settings tree.
• Group names derive from the project UUID (project_<uuid>), not from pod identity, so a broadcast reaches subscribers on every replica.
• The WebSocket ticket is written to Valkey and consumed with GETDEL, so a ticket minted by the pod that answered POST /api/v1/ws/ticket/ is redeemable on any other pod — and exactly once.
• Celery-originated broadcasts (cpm_complete and friends) go through the same channel layer, so a worker’s result reaches clients regardless of which replica they are attached to.

The one requirement is that all replicas share one logical Valkey. Point them at the same instance (or the same Sentinel set); do not give each replica its own cache.

HTTP/2 and HTTP/3

WebSockets over HTTP/2 need the RFC 8441 Extended CONNECT protocol, which most ingress controllers and cloud load balancers do not implement, and HTTP/3 needs RFC 9220. In practice:

• Browser → edge over HTTP/2 is fine. A browser that cannot upgrade over h2 falls back to HTTP/1.1 for the WebSocket request specifically.
• Edge → TruePPM must be HTTP/1.1. That is what proxy_http_version 1.1 guarantees on the nginx path. On a cloud LB, leave the backend protocol at HTTP/1.1 for the /ws target group or backend service — forcing HTTP/2 or gRPC there breaks the upgrade with no useful error.

Symptoms

What you see	Cause	Fix
Closes immediately, code 4003	The authenticated user’s role on that project is Viewer (or they are not a member). Not a networking fault. The web app does not normally dial for a Viewer at all — it shows No live updates, and tasks and the forecast refresh every 5 minutes.	Grant Team Member or above; see RBAC.
Closes immediately, code 4001	No valid credential: the ticket is missing, expired, or already redeemed. Tickets are single-use, so a client that reuses one across reconnects fails on the second attempt.	Mint a fresh ticket per connection.
Closes immediately, code 4404 (a browser shows 1006: this one is refused before the handshake is accepted)	The path matched no route.	Check the edge is not rewriting or stripping the /ws prefix — /ws/v1/projects/<uuid>/ must arrive intact.
Handshake never completes; you get HTTP 200 with the SPA’s HTML, or a 400	The proxy stripped Upgrade/Connection, or spoke HTTP/1.0 upstream, or /ws fell through to the SPA’s try_files catch-all.	Add the three required headers and proxy_http_version 1.1; route /ws before /.
Connects fine, then 1006 at a regular interval, all users together	A proxy idle or lifetime timeout below the connection’s real lifetime. On GCP this is timeoutSec and pings will not save you.	Raise the timeout from the table above.
Connects, then the server drops after 20–40 seconds of quiet	PONGs are being buffered past ws_ping_timeout (20s).	Disable response buffering on the WebSocket location.
Works on one replica, breaks after scaling out	Replicas do not share a Valkey, or two different Valkey instances are configured.	Point every replica at one instance. Do not reach for session affinity.

Scaling out

Where the load balancer sits

The edge is the only externally-facing object. Both Services stay ClusterIP; nothing below the edge should be reachable from outside the cluster — see the X-Forwarded-Proto caveat above for why that is a security property and not just tidiness.

Prerequisites for more than one API replica

Everything in the API tier is stateless except what is listed here. Work through it before raising replicaCount:

• One hostname, one origin. All replicas answer on the same name; the four variables above agree.
• A shared PostgreSQL. Use a managed instance; the bundled sub-chart is dev/demo only. Watch the connection ceiling — see Sizing.
• One shared Valkey. It is the Celery broker, the Channels layer, the WebSocket ticket store, and the per-project scheduling lock. Splitting it breaks real-time collaboration and duplicates CPM runs.
• Shared storage for attachments — object storage (S3, MinIO, SeaweedFS, Ceph) or a ReadWriteMany volume. This is a hard requirement above one replica, not a durability nicety: with attachments on each pod’s local disk, a file uploaded through replica A returns 404 when the download lands on replica B. Set TRUEPPM_DEFAULT_FILE_STORAGE and TRUEPPM_S3_BUCKET_NAME — see object storage.
• Celery beat stays at exactly one replica. It fires the periodic drains; two overlapping beats double-dispatch every job. The chart pins it to one replica with a Recreate strategy.
• No session affinity. See above — configuring it only concentrates load.
• A PodDisruptionBudget (podDisruptionBudget.enabled=true) so a node drain cannot take every API replica at once.
• WebSocket timeouts raised at the edge (previous section) before the reconnect load of a larger user base finds them.

Migrations are not a concern: they run in an initContainer before the API container starts, and Django’s migration table makes concurrent runs safe.

Health checks for an external load balancer

Path	Auth	What it checks	Codes	Use it for
/api/v1/health/	none (AllowAny)	Nothing. It returns {"status": "ok"} if the process is running and can route a request.	200 only	Liveness. A load balancer target check where you want “is the process up”.
/api/v1/readyz	none (AllowAny)	Database reachable, cache reachable, and every migration shipped in the image applied.	200 ready, 503 not ready	Readiness — the right choice for an external LB’s target-group health check. It removes a pod whose datastore is down or whose schema and code disagree mid-upgrade.
/api/v1/edition/	none (AllowAny)	Nothing. Returns edition, version, and build SHA.	200 only	Build identification, not health.
/api/v1/health/beat/	workspace operator only (IsWorkspaceOperator)	Celery Beat heartbeat freshness.	200 / 503	Authenticated monitoring (Prometheus with a token). Not usable as an LB probe — it 403s without credentials.
/api/v1/health/system/	workspace operator only (IsWorkspaceOperator)	Deep system status for the admin UI.	200	Admin diagnostics. Not an LB probe.
/api/v1/health/dead-letter/, /api/v1/health/email/	workspace operator only (IsWorkspaceOperator)	Dead-letter and email gauges, Prometheus text exposition.	200	Scrape targets, with credentials. Not LB probes.

/api/v1/readyz has no trailing slash

Every sibling path on this page ends in /. This one does not, and Django will not redirect to it — a probe configured as /api/v1/readyz/ gets a 404 and marks every target unhealthy. Copy the path exactly.

A health check that connects by IP gets an HTTP 400

Neither probe path is exempt from ALLOWED_HOSTS. A load balancer configured with a target of 203.0.113.42:8000 and no Host override sends Host: 203.0.113.42, which is not in ALLOWED_HOSTS, so Django returns 400 Bad Request — and the check fails on a perfectly healthy pod. This is the most common cause of a first install that looks broken.

Fix it in whichever way your LB supports, in this order of preference:

1. Send the right Host header on the check. AWS ALB target groups take a Host matcher on the health check; GCP health checks take a host field; HAProxy uses http-check send hdr Host <name>; nginx proxy_set_header Host. Set it to the same hostname as ALLOWED_HOSTS. On Kubernetes, the chart handles the equivalent problem for kubelet probes with probes.api.hostHeader, which sets httpGet.httpHeaders on both API probes.
2. Add the address to ALLOWED_HOSTS. Works, but widens what the app will answer on, and needs updating whenever the address changes.

Do not set ALLOWED_HOSTS=*. It disables Django’s Host-header validation entirely, which is a real defense against cache-poisoning and password-reset poisoning attacks.

A worked ALB example, on the chart’s Ingress:

ingress:
  annotations:
    alb.ingress.kubernetes.io/healthcheck-path: /api/v1/readyz
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600

Verify a probe path by hand before wiring it up — from inside the cluster, with the Host header set the way the LB will send it:

kubectl -n trueppm run curl --rm -it --image=curlimages/curl --restart=Never -- \
  curl -sS -o /dev/null -w '%{http_code}\n' \
  -H 'Host: trueppm.example.com' \
  http://trueppm-api:8000/api/v1/readyz

200 is ready. 503 is a real dependency failure — read the body, which names which check failed. 400 means the Host header did not match ALLOWED_HOSTS.

Ports and firewall

The chart’s NetworkPolicy locks down the bundled datastore pods and deliberately leaves the API and worker pods unrestricted in both directions — their outbound endpoints (your IdP, your SMTP relay, your object store) are deployment-specific, so a chart-imposed allow-list would break real installs. That work is yours to do at the platform layer, and this is the list to do it from. See Datastore network isolation for what the chart already covers.

One exception: with demo.interactive: true, the chart does restrict the api and celery-worker pods’ egress, to DNS and the bundled datastores only — this deployment shape needs no other outbound access at all. See Interactive demo mode.

Inbound

Source	Destination	Port / protocol	Required?	Why
Browsers, mobile clients	edge (ingress controller / cloud LB / Compose nginx)	443/tcp	Yes	HTTPS and WSS. The only port that must be open to users.
Browsers	edge	80/tcp	Yes for ACME HTTP-01; otherwise optional	Serves /.well-known/acme-challenge/ and the 301 to HTTPS. Close it only if you use DNS-01 and accept that http:// gives no redirect.
Let’s Encrypt validation servers	edge	80/tcp, from anywhere	Only with ACME HTTP-01	The CA connects from unpredictable addresses; it cannot be source-restricted.
Edge	web Service / web tier pods	80/tcp (container 8080)	Yes when the web tier is enabled	Serves the SPA.
Edge	API Service / API pods	8000/tcp	Yes	REST and WebSocket. Must be reachable only from the edge — see the X-Forwarded-Proto caveat.
kubelet (node)	API pods	8000/tcp	Yes (Kubernetes)	Liveness and readiness probes.
API, Celery worker, Celery beat, backup CronJob, demo-seed Job	PostgreSQL	5432/tcp	Yes	Primary data store. Never expose to the internet.
API, Celery worker, Celery beat, demo-seed Job	Valkey	6379/tcp	Yes	Broker, channel layer, WebSocket tickets, scheduling locks. Never expose to the internet.
Prometheus / your monitoring	API	8000/tcp	Optional	Scrapes /api/v1/health/beat/, /health/dead-letter/, /health/email/ — all staff-authenticated, so the scraper needs a token.
Operators	API pod, via kubectl port-forward	8000/tcp (local)	Optional	The supported route to Django admin; no firewall rule needed.

Outbound

TruePPM’s SSRF guard already refuses any destination that resolves to a non-globally-routable address unless it is in TRUEPPM_EGRESS_ALLOWLISTED_HOSTS. That is an application control, not a network one — you still need the firewall rules below.

Source	Destination	Port / protocol	Required?	Why
API, worker, beat	DNS resolver	53/udp, 53/tcp	Yes	Resolving every destination below.
Kubernetes nodes	GitLab Container Registry (registry.gitlab.com)	443/tcp	Yes, at pull time (Helm default)	Pulling the API and web images from the chart’s default image.repository / image.webRepository. Opening only ghcr.io leaves a stock helm install in ImagePullBackOff. Mirror the images internally for an air-gapped install and point those values at the mirror.
Docker host (Compose)	GHCR (ghcr.io)	443/tcp	Yes, at pull time (Compose)	docker-compose.prod.yml pulls the API and web images from ghcr.io/trueppm/{api,web}. Kubernetes nodes need this only if you override the chart’s image repositories to GHCR.
Whoever runs helm install / helm upgrade	GHCR (ghcr.io)	443/tcp	If you install the chart from its OCI artifact	Pulling oci://ghcr.io/trueppm/charts/trueppm. This is the workstation or CI runner running Helm, not the cluster nodes. Not needed when you install from a local or mirrored chart.
API, worker	SMTP relay	EMAIL_PORT, default 587/tcp (465 implicit TLS, 25 plain)	If email notifications are enabled	Notification and invitation email. EMAIL_USE_TLS defaults to true.
API	OIDC / OAuth2 identity provider	443/tcp	If SSO is configured	Discovery document, authorization, token exchange, JWKS. An in-cluster IdP needs TRUEPPM_EGRESS_ALLOWLISTED_HOSTS.
Browsers	OIDC identity provider	443/tcp	If SSO is configured	The login redirect happens in the user’s browser, not server-side — the client network needs this too.
API, worker	S3 endpoint, MinIO, or SeaweedFS	443/tcp (MinIO commonly 9000/tcp, SeaweedFS S3 gateway commonly 8333/tcp)	If object storage is enabled — required above one replica	Attachment upload, download, and presigned URLs (TRUEPPM_S3_ENDPOINT_URL).
API, worker	OTLP collector	4317/tcp (gRPC) or 4318/tcp (HTTP)	If OTEL_EXPORTER_OTLP_ENDPOINT is set	Traces and metrics. Off by default — no endpoint, no egress.
API, worker	Slack, and your own webhook receivers	443/tcp	If webhooks or the Slack integration are configured	Outbound webhook delivery from the outbox.
API, worker	Jira / GitLab / self-hosted trackers	443/tcp	If a user connects an external task source	Read-only pull of a contributor’s assigned items.
certbot (Compose)	acme-v02.api.letsencrypt.org	443/tcp	With TLS_MODE=letsencrypt	Issuance and the 12-hourly renewal.
cert-manager (Kubernetes)	ACME directory	443/tcp	With cert-manager	Issuance and renewal.
cert-manager (Kubernetes)	DNS provider API (e.g. Route 53)	443/tcp	With a DNS-01 solver	Writing and deleting the _acme-challenge TXT record.
Backup CronJob	your backup destination	destination-specific	If backups ship off-cluster	See Backup & restore.

An install with no email, no SSO, no telemetry, no webhooks, and object storage inside the cluster needs no outbound internet access at all once the images are pulled.

See also

• Deployment — Compose, Helm, and single-server walkthroughs
• OpenShift Deployment — Ingress→Route conversion and SCC compatibility
• Configuration — the full environment-variable reference
• Helm values — every chart key
• Sizing — replica counts and where the bottlenecks are
• Security — CSP, headers, secrets, and the NetworkPolicy
• Single sign-on — the OIDC redirect URI and in-cluster identity providers