Add support for rotating docker secrets (#293)

* Add parameters for rolling updates to `docker_secret`

* Extract `remove_secrets` to its own function in `docker_secret`

* Store existing secrets in a list instead of a single secret

With this change `docker_secret` now supports the case where we store
multiple versions of a secret with the `_v123` postfix.

`absent` state implicitly handles removing these this way.

* When using `rolling_versions` don't automatically remove current secret

To make rolling updates actually work instead of failing on trying to
remove a secret that is attached to a service, use the
`versions_to_keep` parameter to remove old versions of the secret after
creating the new one. This way the secret with the new data is created
with a different name and can be attached to the service by its ID
without having to delete the previous one first which would fail if it
is already attached to a service.

* Add version numbers to newly created secrets

Attach the incremental version number to the secret name as a `_v123`
postfix where `123` is replaced with an incremental counter starting
from 1.
A label with the numeric version is also attached to the secret to ease
calculating the new version number upon change with the name
`ansible_version`.

* Return `secret_name` for docker secrets as well

* Add integration test for rolling secrets

* Update `docker_secret` documentation as per review comments

* Correctly return `docker_secret` version number as int

* Use template string for naming `docker_secrets` instead of concatenation

* Return the correct secret name on deletion failure

* Simplify `docker_secret` creation

* Add missing comma for `docker_secret` schema

* Only remove old docker secrets if `rolling_versions` is set

* Add check in `docker_secret` version parsing to handle NaNs

* Add newly created `docker_secret` to internal secret list to avoid additional deletions

* Add changelog fragment for `docker_secret` `rolling_versions` feature

* Update changelogs/fragments/270-rolling-secrets.yml

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: Felix Fontein <felix@fontein.de>

changelogs/fragments/270-rolling-secrets.yml

@@ -0,0 +1,2 @@
+minor_changes:
+  - docker_secret - add support for rolling update, set ``rolling_versions`` to ``true`` to enable (https://github.com/ansible-collections/community.docker/pull/293, https://github.com/ansible-collections/community.docker/issues/21).

plugins/modules/docker_secret.py

@@ -51,6 +51,21 @@ options:
       - If C(true), an existing secret will be replaced, even if it has not changed.
     type: bool
     default: no
+  rolling_versions:
+    description:
+      - If set to C(true), secrets are created with an increasing version number appended to their name.
+      - Adds a label containing the version number to the managed secrets with the name C(ansible_version).
+    type: bool
+    default: false
+    version_added: 2.2.0
+  versions_to_keep:
+    description:
+      - When using I(rolling_versions), the number of old versions of the secret to keep.
+      - Extraneous old secrets are deleted after the new one is created.
+      - Set to C(-1) to keep everything or to C(0) or C(1) to keep only the current one.
+    type: int
+    default: 5
+    version_added: 2.2.0
   name:
     description:
       - The name of the secret.
@@ -155,6 +170,13 @@ secret_id:
   returned: success and I(state) is C(present)
   type: str
   sample: 'hzehrmyjigmcp2gb6nlhmjqcv'
+secret_name:
+  description:
+    - The name of the created secret object.
+  returned: success and I(state) is C(present)
+  type: str
+  sample: 'awesome_secret'
+  version_added: 2.2.0
 '''
 
 import base64
@@ -204,15 +226,36 @@ class SecretManager(DockerBaseClass):
                 self.client.fail('Error while reading {src}: {error}'.format(src=data_src, error=to_native(exc)))
         self.labels = parameters.get('labels')
         self.force = parameters.get('force')
+        self.rolling_versions = parameters.get('rolling_versions')
+        self.versions_to_keep = parameters.get('versions_to_keep')
+
+        if self.rolling_versions:
+            self.version = 0
         self.data_key = None
+        self.secrets = []
 
     def __call__(self):
+        self.get_secret()
         if self.state == 'present':
             self.data_key = hashlib.sha224(self.data).hexdigest()
             self.present()
+            self.remove_old_versions()
         elif self.state == 'absent':
             self.absent()
 
+    def get_version(self, secret):
+        try:
+            return int(secret.get('Spec', {}).get('Labels', {}).get('ansible_version', 0))
+        except ValueError:
+            return 0
+
+    def remove_old_versions(self):
+        if not self.rolling_versions or self.versions_to_keep < 0:
+            return
+        if not self.check_mode:
+            while len(self.secrets) > max(self.versions_to_keep, 1):
+                self.remove_secret(self.secrets.pop(0))
+
     def get_secret(self):
         ''' Find an existing secret. '''
         try:
@@ -220,10 +263,17 @@ class SecretManager(DockerBaseClass):
         except APIError as exc:
             self.client.fail("Error accessing secret %s: %s" % (self.name, to_native(exc)))
 
-        for secret in secrets:
+        if self.rolling_versions:
-            if secret['Spec']['Name'] == self.name:
+            self.secrets = [
-                return secret
+                secret
-        return None
+                for secret in secrets
+                if secret['Spec']['Name'].startswith('{name}_v'.format(name=self.name))
+            ]
+            self.secrets.sort(key=self.get_version)
+        else:
+            self.secrets = [
+                secret for secret in secrets if secret['Spec']['Name'] == self.name
+            ]
 
     def create_secret(self):
         ''' Create a new secret '''
@@ -232,12 +282,17 @@ class SecretManager(DockerBaseClass):
         labels = {
             'ansible_key': self.data_key
         }
+        if self.rolling_versions:
+            self.version += 1
+            labels['ansible_version'] = str(self.version)
+            self.name = '{name}_v{version}'.format(name=self.name, version=self.version)
         if self.labels:
             labels.update(self.labels)
 
         try:
             if not self.check_mode:
                 secret_id = self.client.create_secret(self.name, self.data, labels=labels)
+                self.secrets += self.client.secrets(filters={'id': secret_id})
         except APIError as exc:
             self.client.fail("Error creating secret: %s" % to_native(exc))
 
@@ -246,11 +301,19 @@ class SecretManager(DockerBaseClass):
 
         return secret_id
 
+    def remove_secret(self, secret):
+        try:
+            if not self.check_mode:
+                self.client.remove_secret(secret['ID'])
+        except APIError as exc:
+            self.client.fail("Error removing secret %s: %s" % (secret['Spec']['Name'], to_native(exc)))
+
     def present(self):
         ''' Handles state == 'present', creating or updating the secret '''
-        secret = self.get_secret()
+        if self.secrets:
-        if secret:
+            secret = self.secrets[-1]
             self.results['secret_id'] = secret['ID']
+            self.results['secret_name'] = secret['Spec']['Name']
             data_changed = False
             attrs = secret.get('Spec', {})
             if attrs.get('Labels', {}).get('ansible_key'):
@@ -260,25 +323,26 @@ class SecretManager(DockerBaseClass):
                 if not self.force:
                     self.client.module.warn("'ansible_key' label not found. Secret will not be changed unless the force parameter is set to 'yes'")
             labels_changed = not compare_generic(self.labels, attrs.get('Labels'), 'allow_more_present', 'dict')
+            if self.rolling_versions:
+                self.version = self.get_version(secret)
             if data_changed or labels_changed or self.force:
                 # if something changed or force, delete and re-create the secret
+                if not self.rolling_versions:
                     self.absent()
                 secret_id = self.create_secret()
                 self.results['changed'] = True
                 self.results['secret_id'] = secret_id
+                self.results['secret_name'] = self.name
         else:
             self.results['changed'] = True
             self.results['secret_id'] = self.create_secret()
+            self.results['secret_name'] = self.name
 
     def absent(self):
         ''' Handles state == 'absent', removing the secret '''
-        secret = self.get_secret()
+        if self.secrets:
-        if secret:
+            for secret in self.secrets:
-            try:
+                self.remove_secret(secret)
-                if not self.check_mode:
-                    self.client.remove_secret(secret['ID'])
-            except APIError as exc:
-                self.client.fail("Error removing secret %s: %s" % (self.name, to_native(exc)))
             self.results['changed'] = True
 
 
@@ -290,7 +354,9 @@ def main():
         data_is_b64=dict(type='bool', default=False),
         data_src=dict(type='path'),
         labels=dict(type='dict'),
-        force=dict(type='bool', default=False)
+        force=dict(type='bool', default=False),
+        rolling_versions=dict(type='bool', default=False),
+        versions_to_keep=dict(type='int', default=5),
     )
 
     required_if = [
@@ -313,7 +379,8 @@ def main():
     try:
         results = dict(
             changed=False,
-            secret_id=''
+            secret_id='',
+            secret_name=''
         )
 
         SecretManager(client, results)()

tests/integration/targets/docker_secret/tasks/test_secrets.yml

@@ -135,6 +135,82 @@
       that:
         - output.failed
 
+# Rolling update
+
+  - name: Create rolling secret
+    docker_secret:
+      name: rolling_password
+      data: opensesame!
+      rolling_versions: true
+      state: present
+    register: original_output
+
+  - name: Create variable secret_id
+    set_fact:
+      secret_id: "{{ original_output.secret_id }}"
+
+  - name: Inspect secret
+    command: "docker secret inspect {{ secret_id }}"
+    register: inspect
+    ignore_errors: yes
+
+  - debug: var=inspect
+
+  - name: assert secret creation succeeded
+    assert:
+      that:
+         - "'rolling_password' in inspect.stdout"
+         - "'ansible_key' in inspect.stdout"
+         - "'ansible_version' in inspect.stdout"
+         - original_output.secret_name == 'rolling_password_v1'
+    when: inspect is not failed
+  - assert:
+      that:
+         - "'is too new. Maximum supported API version is' in inspect.stderr"
+    when: inspect is failed
+
+  - name: Create secret again
+    docker_secret:
+      name: rolling_password
+      data: newpassword!
+      rolling_versions: true
+      state: present
+    register: new_output
+
+  - name: assert that new version is created
+    assert:
+      that:
+         - new_output.changed
+         - new_output.secret_id != original_output.secret_id
+         - new_output.secret_name != original_output.secret_name
+         - new_output.secret_name == 'rolling_password_v2'
+
+  - name: Remove rolling secrets
+    docker_secret:
+      name: rolling_password
+      rolling_versions: true
+      state: absent
+
+  - name: Check that secret is removed
+    command: "docker secret inspect {{ original_output.secret_id }}"
+    register: output
+    ignore_errors: yes
+
+  - name: assert secret was removed
+    assert:
+      that:
+        - output.failed
+
+  - name: Check that secret is removed
+    command: "docker secret inspect {{ new_output.secret_id }}"
+    register: output
+    ignore_errors: yes
+
+  - name: assert secret was removed
+    assert:
+      that:
+        - output.failed
+
   always:
   - name: Remove Swarm cluster
     docker_swarm:

tests/integration/targets/docker_swarm_service/tasks/tests/secrets.yml

@@ -5,6 +5,7 @@
     service_name: "{{ name_prefix ~ '-secrets' }}"
     secret_name_1: "{{ name_prefix ~ '-secret-1' }}"
     secret_name_2: "{{ name_prefix ~ '-secret-2' }}"
+    secret_name_3: "{{ name_prefix ~ '-secret-3' }}"
 
 - name: Registering container name
   set_fact:
@@ -24,6 +25,14 @@
   register: "secret_result_2"
   when: docker_api_version is version('1.25', '>=') and docker_py_version is version('2.1.0', '>=')
 
+- docker_secret:
+    name: "{{ secret_name_3 }}"
+    data: "secret3"
+    state: "present"
+    rolling_versions: true
+  register: "secret_result_3"
+  when: docker_api_version is version('1.25', '>=') and docker_py_version is version('2.1.0', '>=')
+
 ####################################################################
 ## secrets #########################################################
 ####################################################################
@@ -131,6 +140,40 @@
   register: secrets_8
   ignore_errors: yes
 
+- name: rolling secrets
+  docker_swarm_service:
+    name: "{{ service_name }}"
+    image: "{{ docker_test_image_alpine }}"
+    resolve_image: no
+    command: '/bin/sh -v -c "sleep 10m"'
+    secrets:
+      - secret_name: "{{ secret_name_3 }}_v1"
+        filename: "/run/secrets/{{ secret_name_3 }}.txt"
+  register: secrets_9
+  ignore_errors: yes
+
+- name: update rolling secret
+  docker_secret:
+    name: "{{ secret_name_3 }}"
+    data: "newsecret3"
+    state: "present"
+    rolling_versions: true
+  register: secrets_10
+  when: docker_api_version is version('1.25', '>=') and docker_py_version is version('2.1.0', '>=')
+  ignore_errors: yes
+
+- name: rolling secrets service update
+  docker_swarm_service:
+    name: "{{ service_name }}"
+    image: "{{ docker_test_image_alpine }}"
+    resolve_image: no
+    command: '/bin/sh -v -c "sleep 10m"'
+    secrets:
+      - secret_name: "{{ secret_name_3 }}_v2"
+        filename: "/run/secrets/{{ secret_name_3 }}.txt"
+  register: secrets_11
+  ignore_errors: yes
+
 - name: cleanup
   docker_swarm_service:
     name: "{{ service_name }}"
@@ -147,6 +190,9 @@
       - secrets_6 is not changed
       - secrets_7 is changed
       - secrets_8 is not changed
+      - secrets_9 is changed
+      - secrets_10 is not failed
+      - secrets_11 is changed
   when: docker_api_version is version('1.25', '>=') and docker_py_version is version('2.4.0', '>=')
 - assert:
     that:
@@ -405,6 +451,7 @@
   loop:
     - "{{ secret_name_1 }}"
     - "{{ secret_name_2 }}"
+    - "{{ secret_name_3 }}"
   loop_control:
     loop_var: secret_name
   ignore_errors: yes