Secrets
The repository is public, so no secret is ever stored in Git, not even encrypted.
Secrets live in AWS SSM Parameter Store (region eu-west-1) and
external-secrets copies them into the cluster as
normal Kubernetes Secrets.
AWS SSM /homelab/<namespace>/<name> (SecureString)
│ read by the IAM user homelab-external-secrets
▼
ClusterSecretStore aws-ssm (kubernetes/infrastructure/external-secrets/)
│
▼
ExternalSecret in the app's folder (in Git: only names and paths, no values)
│
▼
Secret in the app's namespace (created and refreshed by external-secrets)Bootstrap (done by hand, not in Git)
- IAM user
homelab-external-secretswith an inline policy that allows onlyssm:GetParameter,ssm:GetParametersandssm:GetParametersByPathonparameter/homelabandparameter/homelab/*. SecureStrings use the AWS managed keyaws/ssm, which needs no extra KMS permission. - Its access key as the Secret
external-secrets/aws-ssm-credentials(keysaccess-key-idandsecret-access-key).
Same pattern as cert-manager’s Route53 key. Later this moves to Terraform.
Adding a secret
Store the value in AWS (it never touches the repo):
aws ssm put-parameter --region eu-west-1 --type SecureString \ --name /homelab/<namespace>/<name> --value '<value>'To change it later, add
--overwrite.Add an
externalsecret.yamlto the app’s folder:apiVersion: external-secrets.io/v1 kind: ExternalSecret metadata: name: <app>-secrets spec: refreshInterval: 1h secretStoreRef: kind: ClusterSecretStore name: aws-ssm target: name: <app>-secrets # the Kubernetes Secret that gets created data: - secretKey: DB_PASSWORD # key in the Kubernetes Secret remoteRef: key: /homelab/<namespace>/db-passwordCheck
kubectl -n <namespace> get externalsecret:STATUSshould beSecretSynced.
A changed value in AWS reaches the cluster within refreshInterval. Apps that read the
Secret only at start-up need a restart to see it.
Cost
Standard parameters are free. Reading SecureStrings uses KMS, which is free up to
20 000 requests a month. With refreshInterval: 1h that is room for about 25
ExternalSecrets; raise the interval if there are many more.