install 1password connect server

https://developer.1password.com/docs/connect/get-started/?deploy=kubernetes&method=1password-com#manual-step-2-deploy-1password-connect-server

Step 1: Create a Secrets Automation workflow

(skip) Step 2: Deploy 1Password Connect Server

skip in favour of installing together with operator [#Kubernetes Operator](#Kubernetes Operator)

helm repo add 1password https://1password.github.io/connect-helm-charts/

helm install connect 1password/connect --set-file connect.credentials=1password-credentials.json

Kubernetes Operator

https://developer.1password.com/docs/k8s/k8s-operator/?deployment-type=helm

helm repo add 1password https://1password.github.io/connect-helm-charts/

helm install connect 1password/connect --set-file connect.credentials=1password-credentials.json --set operator.create=true --set operator.token.value=OP_CONNECT_TOKEN

usage

1password: vaults/pet.homeserver.k3s/items/test-k8s-operator username: foo password: bar

1p-secret-test.yaml

apiVersion: onepassword.com/v1  
kind: OnePasswordItem  
metadata:  
  name: test-k8s-operator  
spec:  
  itemPath: "vaults/pet.homeserver.k3s/items/test-k8s-operator"

kubectl get secret test-k8s-operator -o yaml

output

apiVersion: v1
data:
  password: YmFy
  username: Zm9v
kind: Secret
metadata:
  annotations:
    operator.1password.io/item-path: vaults/swsnjyxgiso7q3xbq3vapmq5za/items/utttuh32nnng5bubn6r3atuvhu
    operator.1password.io/item-version: "1"
  creationTimestamp: "2024-09-06T11:47:28Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: test-k8s-operator
  namespace: default
  ownerReferences:
  - apiVersion: onepassword.com/v1
    kind: OnePasswordItem
    name: test-k8s-operator
    uid: 1df0c719-2d80-423e-8f11-228f45d90dbc
  resourceVersion: "1811892"
  uid: de9f154d-48b7-4054-b8d1-fe83b8117994
type: Opaque

echo "YmFy" | base64 -d
# bar%

echo "Zm9v" | base64 -d
# foo%

so it contains the entire 1password item, with all keys (eg. password) & values (eg. foo)

replace hardcoded secrets everywhere

BEFORE: hardcoded in secrets.yaml, but .gitignored

cons:

  • plaintext credentials on my machine
  • can only helm install from my machine (or at least need to copy secrets.yaml)

.gitignore

[...]
secrets.yaml
1password-credentials.json
[...]

secrets.yaml

[...]
services:
	myservice:
		secrets:
			DB_PASSWORD: "some-password" # plain
[...]

values.yaml

[...]
services:  
    myservice: 
        enabled: true  
        replicaCount: 1  
        image:  
            repository: rclone/rclone  
            tag: latest  
            pullPolicy: Always  
        secrets:  
            DB_PASSWORD:  "CHANGE_ME" # overwrite in secrets.yaml
[...]

templates/myservice/db-secrets.yaml

{{- if .Values.services.myservice.enabled }}  
apiVersion: v1  
kind: Secret  
metadata:  
  name: myservice-db  
  namespace: default  
  labels:  
    app.kubernetes.io/name: myservice-db  
stringData:  
  POSTGRES_PASSWORD: "{{ .Values.services.myservice.secrets.DB_PASSWORD }}"  
{{- end }}

and deployed with

helm upgrade -f values.yaml -f secrets.yaml  homeserver .

AFTER with 1password operator, this changed to

pro:

  • no plaintext credentials on my machine
  • no credentials on my machine (other than kubectontext file thingy to connect to cluster) cons:
  • more moving pieces
  • deleted old secrets, but did not replace them!

.gitignore

[...]
secrets.yaml
1password-credentials.json
[...]

secrets.yaml

values.yaml

[...]
services:  
    myservice: 
        enabled: true  
        replicaCount: 1  
        image:  
            repository: rclone/rclone  
            tag: latest  
            pullPolicy: Always  
[...]

templates/myservice/db-secrets.yaml

{{- if .Values.services.myservice.enabled }}  
apiVersion: onepassword.com/v1  
kind: OnePasswordItem  
metadata:  
  name: myservice-db  
  namespace: default  
  labels:  
    app.kubernetes.io/name: myservice-db  
spec:  
  itemPath: "vaults/myvault/items/myservice-db-secrets"  
{{- end }}

and deployed with

helm upgrade -f values.yaml  homeserver .

problem: secrets are gone

when doing the helm upgrade, it seemed that the secrets were destroyed (as expected), but never re-created…

first guess: POLLING_INTERVAL defaults to 10 minutes (600 seconds) src, so just need to wait 10 minutes -> nope, still nothing after 10 minutes

second guess: order of execution screwed me over

  1. helm creates new OnePasswordItem with 1password -> failed silently because already exists
  2. 1password operator noticed new OnePasswordItem , tries to create new secret, fails silently because already exists
  3. helm deletes old secret solution: make a change to the OnePasswordItem yaml file, and redeploy -> worked, now the secrets exist again

could be, because adding a new OnePasswordItem that is not replacing an existing secret, did not cause any problems…

I wish I had looked into kubectl logs onepassword-connect-operator-xxx-yyy while debugging, because there are a bunch of interesting logs:

ERROR Reconciler error {“controller”: “onepassworditem”, “controllerGroup”: “onepassword.com”, “controllerKind”: “OnePasswordItem”, “OnePasswordItem”: {“name”:“myservice-db”,“namespace”:“default”}, “namespace”: “default”, “name”: “myservice-db”, “reconcileID”: “[redacted]”, “error”: “cannot update status: Operation cannot be fulfilled on onepassworditems.onepassword.com "myservice-db": the object has been modified; please apply your changes to the latest version and try again”} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

INFO Secret with name myservice-db and version 1 already exists

INFO Updating Secret myservice-db at namespace ‘default’

though the ERROR Reconciler error does not seem to map to the error, because I updated more secrets that failed to be created, than I have errors, and those errors mention 1 secret specifically