For application-based deployment access from Azure DevOps, follow Entra workload identities. Configure installation trust under app.env; the worker inherits it unless explicitly overridden. Register each release application and its workspace permissions in the organization dashboard.
Prerequisites
For content promotion between installations, see Organization Releases. On a target installation, enable the organization release sync CronJob to follow the selected Git branch and apply the configured deployment policy. Kubernetes traffic cutover is separate.
- Kubernetes 1.19+
- Helm 3.16.3 (baseline tested in CI)
- PV provisioner support
- A Temporal server reachable from the cluster (see Temporal)
Installation
The chart enables the Temporal Worker by default and renders it for production. helm install fails until you set the Worker's Temporal settings and one Worker credential source. postgresql.auth.password is required when the bundled PostgreSQL is enabled. Keep the non-secret values in a file. Pass secrets with --set.
# Clone repo
git clone https://github.com/steelengineai/steelengine.git && cd steelengine
# Generate secrets
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
POSTGRES_PASSWORD=$(openssl rand -base64 16 | tr -d '/+=')
# Registry credentials from your subscription
kubectl create namespace steelengine
kubectl create secret docker-registry steelengine-ghcr \
--namespace steelengine \
--docker-server=ghcr.io \
--docker-username=<github-user> \
--docker-password=<token>
# Broker secret shared only by the app and the Worker
kubectl create secret generic steelengine-temporal-worker-broker \
--namespace steelengine \
--from-literal=TEMPORAL_WORKER_BROKER_SECRET="$(openssl rand -hex 32)"Verify images before mirroring
Each image published to ghcr.io/steelengineai is signed with Sigstore and carries an SPDX software bill of materials (SBOM). Signatures are keyless: they are issued to the SteelEngine publishing workflow's GitHub identity, so there is no public key to distribute. Verify by digest with cosign after docker login ghcr.io:
set -euo pipefail
IMAGE=ghcr.io/steelengineai/steelengine@sha256:<digest>
IDENTITY='^https://github.com/steelengineai/steelengine/\.github/workflows/ghcr-publish\.yml@'
ISSUER=https://token.actions.githubusercontent.com
# Signature
cosign verify "$IMAGE" \
--certificate-identity-regexp "$IDENTITY" --certificate-oidc-issuer "$ISSUER"
# SBOM (SPDX JSON)
cosign verify-attestation "$IMAGE" --type spdxjson \
--certificate-identity-regexp "$IDENTITY" --certificate-oidc-issuer "$ISSUER" \
| jq -r '.payload | @base64d | fromjson | .predicate' > sbom.spdx.jsonSignatures and SBOMs are stored in the registry next to the image. They are attached to the image as OCI referrers, which docker pull and docker push leave behind. To keep them when you mirror into your own registry, copy the image together with its referrers, for example with oras cp --recursive. Images published before this was introduced are unsigned.
Create steelengine-values.yaml:
global:
imagePullSecrets:
- name: steelengine-ghcr
app:
env:
NEXT_PUBLIC_APP_URL: "https://steelengine.yourdomain.com"
BETTER_AUTH_URL: "https://steelengine.yourdomain.com"
# HTTPS URL the Worker uses to reach the app; required with the production broker
INTERNAL_API_BASE_URL: "https://steelengine.yourdomain.com"
NEXT_PUBLIC_SOCKET_URL: "https://steelengine-ws.yourdomain.com"
realtime:
env:
NEXT_PUBLIC_APP_URL: "https://steelengine.yourdomain.com"
BETTER_AUTH_URL: "https://steelengine.yourdomain.com"
NEXT_PUBLIC_SOCKET_URL: "https://steelengine-ws.yourdomain.com"
ALLOWED_ORIGINS: "https://steelengine.yourdomain.com"
worker:
env:
TEMPORAL_ADDRESS: "temporal.yourdomain.com:7233"
TEMPORAL_NAMESPACE: "steelengine-production"
TEMPORAL_SERVER_NAME: "temporal.yourdomain.com"
TEMPORAL_AUTH_ISSUER: "https://steelengine.yourdomain.com"
TEMPORAL_AUTH_JWKS_URL: "https://steelengine.yourdomain.com/api/auth/jwks"
TEMPORAL_BUILD_ID: "<image digest or git sha>"
auth:
credentialBroker:
enabled: true
secret:
name: steelengine-temporal-worker-broker
key: TEMPORAL_WORKER_BROKER_SECRET# Install
helm install steelengine ./helm/steelengine \
--namespace steelengine \
--values steelengine-values.yaml \
--set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
--set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
--set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
--set realtime.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
--set postgresql.auth.password="$POSTGRES_PASSWORD"realtime.env.BETTER_AUTH_SECRET must equal the app's value. The realtime server uses it to verify socket tokens. If you leave it empty, realtime reads BETTER_AUTH_SECRET from the app Secret instead. The PostgreSQL password may only contain letters, digits, ., _, and -. The chart copies the Worker's TEMPORAL_* values into the app pod, so the app and the Worker always agree on the Temporal settings.
Cloud-Specific Values
The example files set storage classes, ingress annotations, and node selectors for each cloud. They ship a placeholder postgresql.auth.password and no worker: block. Create the namespace and Secrets from Installation first. Then layer your own values file after the cloud file and keep the --set flags.
helm install steelengine ./helm/steelengine \
--namespace steelengine \
--values ./helm/steelengine/examples/values-aws.yaml \
--values steelengine-values.yaml \
--set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
--set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
--set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
--set realtime.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
--set postgresql.auth.password="$POSTGRES_PASSWORD"helm/steelengine/examples/values-production.yaml shows the complete production shape, including the Worker values, ingress, autoscaling, and telemetry.
After the install, open /signup on the app URL and create the first account. Only an empty database accepts a sign-up without an invitation, so that account is the first admin, and everyone else joins by invitation.
Key Configuration
# Custom values.yaml
app:
replicaCount: 2
env:
NEXT_PUBLIC_APP_URL: "https://steelengine.yourdomain.com"
OPENAI_API_KEY: "sk-..."
postgresql:
persistence:
size: 50Gi
ingress:
enabled: true
className: nginx
tls:
enabled: true
app:
host: steelengine.yourdomain.comSee helm/steelengine/values.yaml for all options.
App shutdown
When a rollout or scale-down stops an app pod, the pod can first sleep for app.preStopSleepSeconds, a native preStop sleep that needs Kubernetes 1.30 or later. It is 0 (off) by default; on Kubernetes 1.30 or later, set it to about 10 so the Service and ingress stop sending the pod new requests before it shuts down. The server then gets SIGTERM and stops accepting connections. It closes idle keep-alive connections, and every response after that carries Connection: close. It waits up to app.shutdownDrainSeconds for in-flight requests and after() work, gives its own shutdown handlers up to 5 seconds and exits. Requests still open when the drain window ends are cut off. Unset, the drain window is app.terminationGracePeriodSeconds (30 by default) minus the preStop sleep minus 5, so 25 seconds. The chart refuses values where the preStop sleep plus the drain window does not fit inside the grace period, because Kubernetes kills the pod when the grace period ends.
Synchronous workflow executions (POST /api/workflows/<id>/execute) keep their request open until the run finishes. If callers wait for long runs, raise all three values. For example, to cover a 300-second execution timeout:
app:
preStopSleepSeconds: 10
shutdownDrainSeconds: 300
terminationGracePeriodSeconds: 330A stopping pod can then take up to five and a half minutes to go away. Outside Kubernetes, the server reads the drain window from APP_SHUTDOWN_DRAIN_MS (25000 by default).
Temporal Worker
worker.enabled defaults to true. For a production Worker, the chart requires:
worker.env.TEMPORAL_ADDRESS,TEMPORAL_NAMESPACE,TEMPORAL_SERVER_NAME,TEMPORAL_AUTH_ISSUER, andTEMPORAL_BUILD_ID- exactly one of
worker.env.TEMPORAL_AUTH_JWKS_URLorworker.env.TEMPORAL_AUTH_JWKS_FILE - exactly one credential source:
worker.auth.credentialBroker.enabled: trueorworker.auth.apiKeySecret.name
Each missing value stops rendering with a message that names it. For example: worker.env.TEMPORAL_ADDRESS is required for a production Temporal Worker.
Credential broker (used above). The Worker refreshes a short-lived Temporal token from the app. It needs app.enabled: true, worker.env.TEMPORAL_IDENTITY (default steelengine-temporal-worker), an INTERNAL_API_SECRET of at least 32 bytes, and an https:// app.env.INTERNAL_API_BASE_URL. Provide the broker secret as secret.name (an existing Secret) or secret.value (the chart creates one), never both. The chart mounts it into the app and Worker pods only.
Projected Secret. You issue a worker-scoped Temporal token yourself and store it in a Secret. The Worker reads it through TEMPORAL_API_KEY_FILE and picks up rotations automatically.
kubectl create secret generic steelengine-temporal-worker-auth \
--namespace steelengine \
--from-literal=TEMPORAL_API_KEY="$TEMPORAL_WORKER_JWT"worker:
auth:
credentialBroker:
enabled: false
apiKeySecret:
name: steelengine-temporal-worker-auth
key: TEMPORAL_API_KEYThe Worker pod defaults to 4 Gi memory and 1000m CPU limits, with worker.env.NODE_OPTIONS capping the Node heap at 3072 MiB. If you lower the memory limit, lower the heap cap to about 75% of it, or the Worker can crash with a heap out-of-memory error. It gets a 90-second termination grace period so pollers can drain. Its health server listens on worker.healthPort (3003). Per-queue concurrency and database pool settings live under worker.env. See Docker → Temporal Worker for the variable names. Set worker.enabled: false only for an installation that will never run workflows. worker.autoscaling.enabled adds a CPU/memory autoscaler in place of worker.replicaCount. The autoscaler measures usage against worker.resources.requests, so keep the CPU request near real usage. Prefer CPU alone (worker.autoscaling.targetMemoryUtilizationPercentage: null): Node keeps its heap after a burst, so a memory target can scale out and never scale back in. Set worker.env.TEMPORAL_ACTIVITY_MEMORY_TARGET (for example "0.85") alongside it. A busy Worker then leaves extra Activities queued for new replicas instead of running out of memory. CPU must stay above target for worker.autoscaling.scaleUpStabilizationWindowSeconds (300 by default) before replicas are added, so the CPU burst of a starting Worker during a rollout does not scale it out.
The Workers and the tool search Job also receive the non-secret storage and model settings you set in app.env: keys starting with AZURE_STORAGE_, AZURE_OPENAI_, AZURE_ANTHROPIC_, S3_ or VLLM_, plus AZURE_ACCOUNT_NAME, AWS_REGION, KB_OPENAI_MODEL_NAME, BLACKLISTED_PROVIDERS, BLACKLISTED_MODELS and NEXT_PUBLIC_AZURE_CONFIGURED. A value in worker.env takes precedence.
The Worker and document worker have a startup probe on /health/live (worker.startupProbe, documentWorker.startupProbe; 180 and 240 seconds). A Worker loads and bundles its Workflows before its health server starts, which takes about a minute, and the probe keeps liveness from restarting it during a rollout. Keys, secrets and connection strings are never copied this way; keep them in the app Secret.
Autoscaling on running executions (KEDA)
Most Activities wait on HTTP calls and model responses, so a Worker can fill all its Activity slots while its CPU stays low, and a CPU autoscaler adds Workers too late. With KEDA 2.x installed, the chart can also scale on the number of running workflow executions. KEDA is off by default. Without it, installs keep the CPU/memory HPAs.
KEDA must run in the cluster before you turn this on, or the install fails on the unknown ScaledObject kind. On AKS, enable the managed add-on (az aks update --enable-keda; Kubernetes 1.35 ships KEDA 2.17). Elsewhere, install the kedacore/keda Helm chart.
worker:
autoscaling:
enabled: true # required: KEDA reuses minReplicas, maxReplicas and the CPU target
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 70
keda:
enabled: true
targetQueryValue: 15 # running executions per Worker
activationTargetQueryValue: 0
pollingInterval: 15 # seconds between queries
cooldownPeriod: 300 # only used when scaling to zero
app:
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 80
behavior:
scaleDown:
stabilizationWindowSeconds: 600
keda:
enabled: true
targetQueryValue: 40 # running executions per app podWith keda.enabled, the chart renders a KEDA ScaledObject instead of the HPA, never both. It has two triggers: the CPU target (and the memory target, if set) and a postgresql trigger. KEDA adds replicas once the query's count divided by targetQueryValue is more than the current replica count. The app counts running executions (keda.postgresql.query): paused runs are pending, so they don't count, and runs started more than six hours ago are skipped. Workers count block Activities running now (worker.autoscaling.keda.query), which includes child workflows that run inside a parent, such as a Parallel Window calling a worker workflow; a running-executions count would see only the parents. Finished blocks waiting on a fold and rows a crashed Worker abandoned are left out, and preparation, planning and window Activities aren't counted, so keep the CPU target as the second trigger. Set app.autoscaling.keda.query or worker.autoscaling.keda.query to use another query. The Worker keeps its HPA behavior: scale-up waits worker.autoscaling.scaleUpStabilizationWindowSeconds, and scale-in removes one Worker every five minutes after a 600-second window. The app uses app.autoscaling.behavior.
The trigger connects to the database the Workers use (postgresql or externalDatabase) through a TriggerAuthentication named <release>-keda-postgresql. That reads the password from the same Secret: the chart-managed one, or postgresql.auth.existingSecret / externalDatabase.existingSecret with its passwordKey. With the in-cluster PostgreSQL and networkPolicy.enabled, the chart lets the KEDA operator's namespace (keda.operatorNamespace, kube-system for the AKS add-on, keda for the upstream chart) reach the database.
Scaling Workers on Task Queue backlog
The running-executions query counts work that has already started, so it only grows once Workers are busy. worker.autoscaling.keda.backlog adds Workers as soon as tasks wait. Temporal hands a task straight to an open poll, so a task only waits in the backlog when every Worker's slots are full.
worker:
autoscaling:
scaleUpStabilizationWindowSeconds: 15
scaleUpPercent: 100 # one 15-second step may double the fleet...
scaleUpPods: 10 # ...or add 10 Workers, whichever is more
keda:
enabled: true
backlog:
enabled: true
targetPerReplica: 10 # waiting tasks per Worker
targetAgeSeconds: 15 # keep adding Workers while the oldest task waits longerEach Worker serves the backlog of its eight Task Queues, Workflow and Activity tasks together, as JSON at /metrics/task-queue-backlog on worker.healthPort. Temporal reports a queue's backlog across all its pollers, so any one Worker answers for the fleet. The chart adds a <release>-worker Service for that port and one or two KEDA metrics-api triggers:
task-queue-backlog(AverageValue): KEDA asks for the backlog divided bytargetPerReplicaWorkers. A backlog of 200 at 10 asks for 20.task-queue-backlog-age(Value, only whentargetAgeSecondsis above 0): while the oldest task has waited longer thantargetAgeSeconds, each step multiplies the replica count by age divided by target. At twice the target the fleet doubles. Unlike the count, it keeps growing a fleet that is already large.
The Worker reads the backlog with Temporal's DescribeTaskQueue and caches it for five seconds. The SteelEngine Temporal server must let Worker tokens call DescribeTaskQueue, which it does only for the Task Queues the token names. Roll out that server image and a Worker image that serves the endpoint before you turn the backlog on. Until then the endpoint answers 503. The HPA keeps scaling up on the CPU and query triggers, but it won't scale in while a trigger fails.
worker.autoscaling.scaleUpPercent and scaleUpPods set how many Workers one 15-second step may add, the larger of the two. The defaults, 100% and 4, are the Kubernetes defaults. With backlog scaling, keep scaleUpStabilizationWindowSeconds short. A waiting backlog already means every Worker is full, and the HPA ignores the CPU of Workers that are still starting.
KEDA's own temporal scaler, added in KEDA 2.17, isn't used. It authenticates with a static API key or a client certificate. The SteelEngine frontend accepts only short-lived transport JWTs, so KEDA would need a long-lived token or a rotation job of its own.
The ScaledObject keeps the chart's HPA name (<release>-worker, <release>-app) and takes over the existing HPA, so you can turn KEDA on with a normal upgrade. To turn it off again, delete the ScaledObject first (kubectl delete scaledobject <release>-worker) so the chart can recreate its own HPA.
What scales the app
The app scales on CPU and running executions, not on in-flight requests. Each app pod is one Node.js process, so its event loop saturates as its CPU climbs, and CPU already tracks request load. Workflows that start child runs through the execute API also raise the running-executions count. The chart can't count in-flight requests across pods by itself. That needs a metrics server such as Prometheus, scraping the ingress controller. With one in the cluster, add a request-rate trigger through app.autoscaling.keda.extraTriggers. The chart appends each entry to the ScaledObject as written, and worker.autoscaling.keda.extraTriggers does the same for Workers:
app:
autoscaling:
keda:
extraTriggers:
- type: prometheus
metadata:
serverAddress: http://prometheus.monitoring.svc:9090
query: sum(rate(nginx_ingress_controller_requests{ingress="steelengine-app"}[1m]))
threshold: "50" # requests per second per podWhen app pods run hot, check whether the autoscaler is already at app.autoscaling.maxReplicas before you add a trigger. A pod at 150% CPU already asks for about twice as many pods. To add pods faster, set a scale-up policy in app.autoscaling.behavior:
app:
autoscaling:
behavior:
scaleUp:
stabilizationWindowSeconds: 0
selectPolicy: Max
policies:
- {type: Percent, value: 100, periodSeconds: 15}
- {type: Pods, value: 8, periodSeconds: 15}Each app pod opens up to DATABASE_POOL_MAX database connections, so maxReplicas × DATABASE_POOL_MAX must stay below the database's max_connections unless PgBouncer sits in front of it.
app.autoscaling scales the app on its own. It works with or without KEDA. When it is on, the top-level autoscaling block scales only realtime, so realtime can stay at realtime.replicaCount. When it is off, the top-level block scales the app and realtime together, as before. While app.autoscaling.enabled is on, helm upgrade sets the app Deployment's replicas to its current live count (or minReplicas on a new install), so an upgrade never resets the app to one pod. Tools that render offline with helm template, such as Argo CD or Flux, cannot read the live count and render minReplicas on every sync; the autoscaler then adds pods back. For those, set minReplicas to the count you want to keep through a sync.
Document Worker
Generate document blocks run on their own Task Queue, which only the document worker polls. Turn it on with documentWorker.enabled: true; without it those blocks wait until they time out. It uses the Worker image and worker.env, minus the Worker's Temporal identity and port, OTEL_SERVICE_NAME, E2B_ENABLED, EGRESS_ALLOWLIST, EGRESS_ALLOWLIST_HOSTED_DATABASES and NODE_OPTIONS (its heap cap is documentWorker.env.NODE_OPTIONS, sized to its own memory limit), and runs a Gotenberg sidecar that listens only inside the pod for PDF and Office rendering.
The document worker requires Kubernetes 1.29 or later, because Gotenberg runs as a native sidecar container. On older clusters the chart refuses to render with documentWorker.enabled: true.
In production the document worker uses the credential broker with its own identity (documentWorker.identity) and its own broker secret, so it can only poll the document queue:
documentWorker:
enabled: true
auth:
brokerSecret:
name: steelengine-document-worker-broker # or leave empty and set value
key: DOCUMENT_GENERATION_TEMPORAL_WORKER_BROKER_SECRETUse a dedicated Secret for the broker secret, so only the app and the document worker receive it. If you point it at the app Secret, the Worker, which loads that Secret whole, receives it too; the chart then blanks the key in the Worker's environment, but the value is still in a Secret mounted into the Worker pod.
The chart creates the Gotenberg credentials unless you set documentWorker.gotenberg.auth.existingSecret. The chart keeps the generated password across upgrades by reading the live Secret, which GitOps tools that render offline (Argo CD, Flux) cannot do. With those tools, create the Secret yourself and set existingSecret.
External Database
postgresql:
enabled: false
externalDatabase:
enabled: true
host: "your-db-host"
port: 5432
username: "postgres"
password: "your-password"
database: "steelengine"
sslMode: "require"Database Secrets
By default, the chart builds DATABASE_URL from postgresql or externalDatabase and the corresponding password secret. Enabling app.secrets.existingSecret for your authentication and encryption keys does not require a DATABASE_URL key in that Secret.
To use a complete connection URL from your app Secret for the app and its migration init container, explicitly map the key:
app:
secrets:
existingSecret:
enabled: true
name: my-app-secrets
keys:
DATABASE_URL: connection-uriHere, connection-uri is the name of a key in my-app-secrets. Leave keys.DATABASE_URL empty to keep the constructed URL. The Worker and realtime service still use postgresql or externalDatabase, so keep those settings and their password secret pointed at the same database as the app.
Connection pooling (PgBouncer)
Every app pod and Worker keeps its own pool of up to DATABASE_POOL_MAX connections, so without a pooler, app replicas × 20 plus Worker replicas × 25 has to stay below the database's max_connections. pgbouncer.enabled puts PgBouncer in transaction pooling mode between them and the database. Pods then connect to PgBouncer, which runs each transaction on one of a fixed set of server connections, so replica counts no longer depend on max_connections.
pgbouncer:
enabled: true
replicaCount: 2 # each PgBouncer process is single-threaded
appPoolSize: 50 # server connections per replica for the app...
workerPoolSize: 100 # ...and for the Workers
reservePoolSize: 10 # extra per pool once a client waits reservePoolTimeout seconds
maxClientConn: 5000 # client connections per replica
tls:
enabled: true # clients connect with sslmode=require
issuerRef:
name: my-selfsigned-issuer # optional: cert-manager issues certificatesSecretThe database sees at most replicaCount × (appPoolSize + workerPoolSize + 2 × reservePoolSize) connections from PgBouncer, 340 with the defaults. Keep that, plus the direct connections below, under max_connections. A transaction holds a server connection only while it runs, so a pool far smaller than the client count is normal. Raise a pool only when clients wait in PgBouncer (cl_waiting in SHOW POOLS). replicaCount is fixed, not autoscaled, because each replica adds its own server connections.
Evicting a PgBouncer pod drops the database connections of every client using it, so a running workflow can fail mid-query. On clusters with the cluster autoscaler, set pgbouncer.podAnnotations to cluster-autoscaler.kubernetes.io/safe-to-evict: "false" so node scale-down leaves PgBouncer alone. Node drains for upgrades still evict it, one pod at a time when podDisruptionBudget.enabled is set. A stopping PgBouncer pod keeps serving its existing clients for pgbouncer.terminationGracePeriodSeconds (default 60), but busy connections never sit idle long enough to close on their own. Set DATABASE_MAX_LIFETIME_SECONDS in app.env and worker.env well below that window, so every connection rotates to a running pod before the stopping one is killed. A connection rotates only after its running query, so size the window for the preStop delay (10 seconds), the lifetime, pgbouncer.queryWaitTimeout and the longest statement timeout: for example 45 seconds with a 300-second window when the queue wait is 120 seconds and the Worker's statement timeout is 60 seconds.
The app and the Workers each connect to their own pool, the <database>_app and <database>_worker aliases, through <release>-pgbouncer. These stay on direct connections:
- the migrations init container, which sets session timeouts and builds indexes concurrently
- the app's session-level advisory locks, which serialize cron jobs and source-control syncs. The chart gives the app
DATABASE_SESSION_URL, the direct URL. Behind a transaction pooler, a lock and its unlock could run on different server sessions. - realtime, the document worker, the tool search Job and the KEDA trigger
The app and Worker clients use unnamed statements (prepare: false) and keep SET LOCAL and transaction-level advisory locks inside transactions, so they work with transaction pooling. They send DATABASE_LOCK_TIMEOUT_MS and DATABASE_STATEMENT_TIMEOUT_MS as the options startup parameter, which PgBouncer can't pass on. PgBouncer ignores it and sets the same values, from app.env and worker.env, on each server connection of that pool with connect_query.
The hop from PgBouncer to the database follows externalDatabase.sslMode, or postgresql.tls.enabled. Client TLS is off by default. With pgbouncer.tls.enabled, PgBouncer reads tls.crt and tls.key from pgbouncer.tls.certificatesSecret, and setting issuerRef.name asks cert-manager for that certificate. A self-signed issuer is enough, because clients encrypt without verifying the certificate. Use an RSA key if you supply your own: the app runs on Bun, whose TLS client fails the handshake with PgBouncer on an ECDSA certificate. With networkPolicy.enabled, only the app and the Workers reach PgBouncer, and PgBouncer reaches only DNS and the database.
Secret scoping
Chart 1.4.0 supports app.extraEnvVars for Kubernetes value and valueFrom entries in the app container. For an installation Admin API key, reference a dedicated Secret using valueFrom.secretKeyRef and remove any previous shared reference. The chart resolves overlapping names in the app with app.extraEnvVars first, followed by shared extraEnvVars, app.env, and generated values. Check the rendered Temporal CA paths against your mounts during the upgrade. When the installed release already contains duplicate env names, use Helm 4's --server-side=true or validate a Helm 3 replacement upgrade in QA. Verify the CA and URL settings on the live Deployments after the upgrade.
The app, the Worker and the tool search Job load the app Secret, and every Secret in app.secrets.additionalSecretNames, as environment variables. Realtime and the document worker don't. They reference single keys from the app Secret, listed in realtime.secrets.appSecretKeys and documentWorker.secrets.appSecretKeys:
- Realtime:
BETTER_AUTH_SECRET,INTERNAL_API_SECRET,REDIS_URLand the OpenTelemetry auth keys. - Document worker:
ENCRYPTION_KEY,API_ENCRYPTION_KEY,INTERNAL_API_SECRET, model provider keys and endpoints,REDIS_URL, object storage credentials and container names, and the OpenTelemetry auth keys.
Each reference is optional, so a missing key stays unset and the service reports what it needs at startup. A value set in the component's env or in extraEnvVars wins. Add keys to those lists, or whole Secrets to realtime.secrets.extraSecretNames and documentWorker.secrets.extraSecretNames, when you need more. Neither service receives OAuth client secrets, CRON_SECRET or the Worker broker secret.
Other Chart Features
- Cron jobs:
cronjobs.enableddefaults totrue. The jobs call the app for schedule reconciliation, subscription renewal, inactivity alerts, connector sync, model retirement notices, and cleanup of stale executions and idempotency keys. Every job sendsAuthorization: Bearer $CRON_SECRET, taken fromapp.env.CRON_SECRETwhen set and otherwise from the app Secret, so it works with an existing Secret or External Secrets too. IfCRON_SECRETis missing, the app rejects every job with 401. The stale-execution job also deletes old Temporal journals and finished checkpoint chains. Execution log cleanup (cronjobs.jobs.cleanupExecutionLogs) is off by default because it permanently deletes run history older thanLOG_RETENTION_DAYS; see Run History Retention. - Tool search index:
toolEmbeddings.enabledruns a Job after each install or upgrade that rebuilds the index the Chat assistant uses to find integrations and blocks. It embeds with Azure OpenAI (AZURE_OPENAI_API_KEYfrom the app Secret,AZURE_OPENAI_ENDPOINTfromapp.env) orOPENAI_API_KEYfrom the app Secret. Without it, tool search falls back to name matching. A failed run doesn't fail the deploy. - Autoscaling and availability:
autoscaling(HPA for the app and realtime),app.autoscaling(the app alone),worker.autoscaling, optional KEDA scaling on running executions (above),podDisruptionBudget(app, realtime, Worker and document worker), andmonitoring.serviceMonitorfor Prometheus. App probes use/api/health. - Network policy and storage:
networkPolicyrestricts pod-to-pod traffic and allows managed Postgres, Redis and Temporal onnetworkPolicy.externalServicePorts, plus DNS, 443 and (with telemetry) OTLP. Realtime, the Workers and the chart's cron jobs reach the app. SetnetworkPolicy.ingressControllerNamespaceSelector(for examplematchLabels: {kubernetes.io/metadata.name: ingress-nginx}) so only your ingress controller reaches the app and realtime; empty allows any source. App egress is limited to those ports, so addnetworkPolicy.egressrules for anything else. Worker and document worker egress stays open, since integrations and OpenTelemetry use arbitrary ports; setnetworkPolicy.restrictWorkerEgress: trueto limit it to DNS, 443, OTLP (4317/4318), the app andexternalServicePorts.sharedStorageprovisions ReadWriteMany volumes for workflows that share files between pods. - Branding:
branding.enabledmounts a ConfigMap of text and base64 binary files at/app/public/branding. TheNEXT_PUBLIC_BRAND_*variables inapp.envreference them. See Environment Variables. - Landing pages:
app.env.NEXT_PUBLIC_LANDING_ENABLEDoverrides whether marketing pages are shown or redirect to/login. See Landing Pages for hostname-based defaults. - Telemetry:
telemetry.enableddeploys an OpenTelemetry Collector with Jaeger, Prometheus, and OTLP backends. The chart routes the Worker's traces and metrics to it. To use an external backend such as HyperDX instead, setotlp.endpointand put the auth header in the app Secret asOTEL_EXPORTER_OTLP_HEADERS(orHYPERDX_API_KEY). - Laya Decision:
laya.enabledruns the Laya decision server in-cluster on CPU (about 1 vCPU and 4 GiB; both checkpoints stay resident) and gives the app and Temporal WorkerLAYA_URLandLAYA_API_KEY, which makes the Laya Decision block appear. The chart generates the key once and keeps it across upgrades; setlaya.apiKey.existingSecretto supply your own. A NetworkPolicy lets only the app and Worker reach it, and it has no egress. The image issteelengineai/laya, tagged by laya version and model commit. - Code runner:
codeRunner.enabledruns Python Function blocks, and JavaScript that imports modules, in your cluster instead of E2B, and gives the Temporal WorkerCODE_RUNNER_URLandCODE_RUNNER_TOKEN. Each call runs in a fresh process as its own Unix user with a private directory and memory, process and time limits. The image (steelengineai/code-runner, tagged by a hash of its inputs) has Python's standard library plusrequestsand Node's built-in modules. Run it under a VM-isolating runtime: setcodeRunner.runtimeClassName(for examplekata-vm-isolationwith AKS Pod Sandboxing) and, on a dedicated tainted pool,codeRunner.nodeSelectorandcodeRunner.tolerations. With a runtime class set,codeRunner.hardLimits: trueenforces each call's memory and process limits through cgroups. The pod then runs privileged inside its own VM, and the chart refuses that setting without a runtime class. The runner also blocks the per-call users from private, loopback, link-local and cloud metadata addresses (codeRunner.blockedEgressCidrs) with iptables inside the pod, because a NetworkPolicy alone does not stop a Kata pod reaching the metadata endpoint, and it refuses to start if that block is not in effect. That needsNET_ADMIN, which Pod Securitybaselineforbids, so run it in its own namespace (codeRunner.namespace) that allows it, holds the image pull Secret, and contains nothing else. A NetworkPolicy lets only the release's Workers connect. If each runner pod fills most of a dedicated node and the pool cannot grow, setcodeRunner.strategyto{type: RollingUpdate, rollingUpdate: {maxSurge: 0, maxUnavailable: 1}}so upgrades replace one pod at a time instead of waiting for room for an extra pod. On shutdown the runner closes each keep-alive connection after its response and waits up tocodeRunner.drainSecondsfor running calls before it exits. SetcodeRunner.preStopSleepSecondsso the Service stops sending it new calls first, and keepcodeRunner.terminationGracePeriodSecondsabove the two combined. Roll out withcodeRunner.routeWorkers: false, check the runner pods are Ready, then turn it on to give the WorkerCODE_RUNNER_URL. Keep E2B enabled if you use Copilot's table files. - Ollama:
ollama.enabledruns Ollama in-cluster with optional NVIDIA GPU scheduling (ollama.gpu.strategyistime-slicingormig), and points the app and Workers at it. For an external server, setapp.env.OLLAMA_URL. With neither, the pods get noOLLAMA_URLand Ollama stays off. - Secrets:
app.secrets.existingSecretandexternalSecrets(External Secrets Operator) replace inline secret values. See Secret scoping for which pods receive which keys. - Pod security: containers run as non-root with
allowPrivilegeEscalation: falseand all capabilities dropped, in pods with theRuntimeDefaultseccomp profile, and no pod mounts a Kubernetes API token (serviceAccount.automountServiceAccountToken: false). Override through each component'ssecurityContextandpodSecurityContext. - Local rehearsal:
bun run selfhost:validatebrings up kind, ingress-nginx with TLS, this chart, and a TLS-fronted Temporal server on one machine.
Commands
# Port forward for local access
kubectl port-forward deployment/steelengine-app 3000:3000 -n steelengine
# View logs
kubectl logs -l app.kubernetes.io/component=app -n steelengine --tail=100
kubectl logs deployment/steelengine-worker -n steelengine --tail=100
# Upgrade
helm upgrade steelengine ./helm/steelengine --namespace steelengine --values steelengine-values.yaml
# Uninstall
helm uninstall steelengine --namespace steelengineResource names are <release>-app, <release>-worker, <release>-realtime, and <release>-postgresql. Helm drops the chart name when the release name already contains it. A release called steelengine yields steelengine-app, not steelengine-steelengine-app.