When using combined tag@digest references (e.g., nginx:1.21@sha256:abc...),
Docker does NOT store the tag in RepoTags. It only stores the digest in
RepoDigests. The previous implementation required BOTH to match, which
always failed because RepoTags was empty.
This caused docker_container to pull the image on every run even when
the image with the correct digest already existed locally, breaking
idempotency.
The fix: When a digest is specified, match by digest only since it's the
authoritative identifier. The tag is informational for human readability.
Real-world example from docker image inspect:
"RepoTags": [], # Empty when pulled by digest\!
"RepoDigests": ["portainer/portainer-ee@sha256:7ecf2008..."]
Updated tests to reflect the correct behavior:
- test_empty_repo_tags_matches_by_digest (the critical fix case)
- test_combined_tag_digest_matches_even_if_tag_differs
- test_multiple_tags_irrelevant_for_combined
Extends the tag@digest fix to also cover push operations in docker_image.py.
The push_image() method was passing combined tag@digest format directly to
the Docker API's /images/{name}/push endpoint, which fails with
"invalid tag format" errors.
This fix:
1. Imports build_pull_arguments() into docker_image.py
2. Uses the helper in push_image() before calling the API
3. When tag contains @ (but isn't a pure digest), passes the full
reference as the image name and omits the tag parameter
This complements the previous fix to pull_image() methods, ensuring
both pull and push operations handle tag@digest correctly.
When using combined image:tag@digest references, parse_repository_tag
returns (repo, "tag@digest"). The Docker SDK and API don't accept
tag@digest in the tag parameter, causing "invalid tag format" errors.
This fix:
1. Adds build_pull_arguments() helper function to _util.py
2. Uses the helper in both pull_image implementations
3. When tag contains @ (but isn't a pure digest), passes the full
reference as the repository/fromImage parameter instead of splitting
Tested formats:
- portainer/portainer-ee:2.35.0-alpine@sha256:abc...
- ghcr.io/gethomepage/homepage:v1.7@sha256:abc...
- localhost:5000/myapp:v2.0@sha256:abc...
The existing filter_images_by_tag already handles tag@digest for
lookups, so find_image continues to work correctly.
Includes comprehensive unit tests for build_pull_arguments().
The parse_repository_tag fix alone is not sufficient because Docker stores
RepoTags and RepoDigests separately. When looking up an image with combined
tag@digest (e.g., nginx:1.21@sha256:abc...), the _image_lookup function must
split the combined format and match BOTH RepoTags (for the tag) AND
RepoDigests (for the digest).
Docker stores:
- RepoTags: ["nginx:1.21"]
- RepoDigests: ["nginx@sha256:abc..."]
But NEVER stores the combined format. The previous code would construct:
- lookup = "nginx:1.21@sha256:abc..." (never matches RepoTags)
- lookup_digest = "nginx@1.21@sha256:abc..." (never matches RepoDigests)
This fix:
1. Adds filter_images_by_tag() helper function to _util.py to avoid code
duplication between _common.py and _common_api.py
2. Detects combined tag@digest format in the tag parameter
3. Splits into tag_part and digest_part
4. Constructs proper lookups for both RepoTags and RepoDigests
5. Requires BOTH to match for successful image lookup
Without this fix, image_label_mismatch: ignore fails because the image
cannot be found, resulting in no image labels being included in expected
labels comparison.
Includes comprehensive unit tests in test__util.py covering all scenarios
including edge cases for multiple @ symbols and empty tag parts.
The parse_repository_tag() function was incorrectly parsing Docker image
references that contained both a tag and a digest (e.g., nginx:1.0@sha256:abc).
Previously, when splitting by '@' first, the tag would be included in the
repository name, resulting in incorrect parsing:
- Input: "nginx:1.0@sha256:abc123"
- Old output: ("nginx:1.0", "sha256:abc123")
- Expected: ("nginx", "1.0@sha256:abc123")
The fix now:
1. Checks for digest (@) separator first
2. Examines the part before the digest for a tag (:) separator
3. Combines tag and digest as "tag@digest" when both are present
Added test cases:
- test_index_image_tag_and_sha
- test_index_user_image_tag_and_sha
- test_private_reg_image_tag_and_sha
* Re-enable typing and improve config.
* Make mypy pass.
* Improve settings.
* First batch of types.
* Add more type hints.
* Fixes.
* Format.
* Fix split_port() without returning to previous type chaos.
* Continue with type hints (and ignores).
* Remove __metaclass__ = type.
for i in $(grep -REl '__metaclass__ = type' plugins/ tests/); do
sed -e '/^__metaclass__ = type/d' -i $i;
done
* Remove super arguments, and stop inheriting from object.
* Adjust all __future__ imports:
for i in $(grep -REl "__future__.*absolute_import" plugins/ tests/); do
sed -e 's/from __future__ import .*/from __future__ import annotations/g' -i $i;
done
* Remove all UTF-8 encoding specifications for Python source files:
for i in $(grep -REl '[-][*]- coding: utf-8 -[*]-' plugins/ tests/); do
sed -e '/^# -\*- coding: utf-8 -\*-/d' -i $i;
done
* Reformat.
* Make all doc fragments, module utils, and plugin utils private.
* Remove some unused and no longer needed imports.
This hopefully also fixes the CI issues, which do not happen locally for me...
* Fix formatting.
* Try to make CI happy, again.
* Fix imports.
* Lint.
* Remove unicode text prefixes.
* Replace str.format() uses with f-strings.
* Replace % with f-strings, and do some cleanup.
* Fix wrong variable.
* Avoid unnecessary string conversion.
* Vendor parts of the Docker SDK for Python
This is a combination of the latest git version
(db7f8b8bb6)
with some fixes to make it compatible with Python 2.7
and adjusting some imports.
* Polishing.
* Fix bug that prevents contexts to be found when no Docker config file is present.
Ref: https://github.com/docker/docker-py/issues/3190
* Linting.
* Fix typos.
* Adjust more to behavior of Docker CLI.
* Add first iteration of docker_context_info module.
* Improvements.
* Add basic CI.
* Add caveat on contexts[].config result.
* Add inventory filter capability.
* Use community.library_inventory_filtering_v1 collection.
* Bump dependency to 1.0.0.
* Mention the new dependency in the changelog.
* vendored Docker SDK for Python code: volume: added support for bind propagation
https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation
Cherry-picked from bea63224e0
Co-authored-by: Janne Jakob Fleischer <janne.fleischer@ils-forschung.de>
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* vendored Docker SDK for Python code: fix: eventlet compatibility
Check if poll attribute exists on select module instead of win32 platform check
The implementation done in #2865 is breaking usage of docker-py library within eventlet.
As per the Python `select.poll` documentation (https://docs.python.org/3/library/select.html#select.poll) and eventlet select removal advice (eventlet/eventlet#608 (comment)), it is preferable to use an implementation based on the availability of the `poll()` method that trying to check if the platform is `win32`.
Fixes https://github.com/docker/docker-py/issues/3131
Cherry-picked from 78439ebbe1
Co-authored-by: Mathieu Virbel <mat@meltingrocks.com>
* vendored Docker SDK for Python code: fix: use response.text to get string rather than bytes
Adjusted from 0618951093
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* vendored Docker SDK for Python code: Fix missing asserts or assignments
Cherry-picked from 0566f1260c
Co-authored-by: Aarni Koskela <akx@iki.fi>
---------
Co-authored-by: Janne Jakob Fleischer <janne.fleischer@ils-forschung.de>
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
Co-authored-by: Mathieu Virbel <mat@meltingrocks.com>
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
Co-authored-by: Aarni Koskela <akx@iki.fi>
* Move copying functionality to module_utils.
* Add docker_container_copy_into module.
* Use new module in other tests.
* Fix copyright and attributes.
* Improve idempotency, improve stat code.
* Document and test when a stopped container works.
* Improve owner/group detection error handling when container is stopped.
* Fix formulation.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Improve file comparison.
* Avoid reading whole file at once.
* Stream when fetching files from daemon.
* Fix comment.
* Use read() instead of read1().
* Stream files when copying into container.
* Linting.
* Add force parameter.
* Simplify library code.
* Linting.
* Add content and content_is_b64 options.
* Make force=false work as for copy module: only copy if the destination does not exist.
* Improve docs.
* content should be no_log.
* Implement diff mode.
* Improve error handling.
* Lint and improve.
* Set owner/group ID to avoid ID lookup (which fails in paused containers).
* Apply suggestions from code review
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
