* socket: fix for errors on pipe close in Windows (https://github.com/docker/docker-py/pull/3099)
Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.
Fixes https://github.com/docker/docker-py/issues/3098.
Cherry-picked from f84623225e
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* socket: use poll() instead of select() except on Windows (https://github.com/docker/docker-py/pull/2865)
Fixes https://github.com/docker/docker-py/issues/2278, which was originally addressed in https://github.com/docker/docker-py/pull/2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.
Cherry-picked from a02ba74333
Co-authored-by: Tyler Westland <tylerofthewest@gmail.com>
* api: respect timeouts on Windows named pipes (https://github.com/docker/docker-py/pull/3112)
Cherry-picked from 9cadad009e
Co-authored-by: Imogen <59090860+ImogenBits@users.noreply.github.com>
* Add URL to changelog.
* api: avoid socket timeouts when executing commands (https://github.com/docker/docker-py/pull/3125)
Only listen to read events when polling a socket in order
to avoid incorrectly trying to read from a socket that is
not actually ready.
Cherry-picked from c5e582c413
Co-authored-by: Loïc Leyendecker <loic.leyendecker@gmail.com>
---------
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
Co-authored-by: Tyler Westland <tylerofthewest@gmail.com>
Co-authored-by: Imogen <59090860+ImogenBits@users.noreply.github.com>
Co-authored-by: Loïc Leyendecker <loic.leyendecker@gmail.com>
* Make compatible with requests 2.29.0.
* This fix should also work with urllib3 2.0 according to urllib3 maintainer.
* Add changelog fragment.
* We still need the constraint for CI until Docker SDK for Python has a new release with a fix.
* Make modifications to response_class as small as possible.
* Revert "We still need the constraint for CI until Docker SDK for Python has a new release with a fix."
This reverts commit 698d544a1e08308e8bf8b4e56ab78c5079f9a17b.
* The pip coming with the ansible-core 2.11 alpine3 image seems to be too old.
Requests with stream=True MUST be closed or else the connection will
never be returned to the connection pool. Both ContainerApiMixin.attach
and ExecApiMixin.exec_start were leaking in the stream=False case.
exec_start was modified to follow attach for the stream=True case as
that allows the caller to close the stream when done (untested).
Tested with:
# Test exec_run (stream=False) - observe one less leak
make integration-test-py3 file=models_containers_test.py' -k test_exec_run_success -vs -W error::ResourceWarning'
# Test exec_start (stream=True, fully reads from CancellableStream)
make integration-test-py3 file=api_exec_test.py' -k test_execute_command -vs -W error::ResourceWarning'
After this change, one resource leak is removed, the remaining resource
leaks occur because none of the tests call client.close().
Fixes https://github.com/docker/docker-py/issues/1293
(Regression from https://github.com/docker/docker-py/pull/1130)
Cherry-picked from 34e6829dd4
Co-authored-by: Peter Wu <pwu@cloudflare.com>
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* fix: fix tmpfs_size and tmpfs_mode not being set
* fix: wrong file
* fix: add changelog fragment
* fix: update changelog fragment to match formatting
Co-authored-by: Felix Fontein <felix@fontein.de>
---------
Co-authored-by: Felix Fontein <felix@fontein.de>
* Do not crash when plugin doesn't exist.
* Improve style.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Move copying functionality to module_utils.
* Add docker_container_copy_into module.
* Use new module in other tests.
* Fix copyright and attributes.
* Improve idempotency, improve stat code.
* Document and test when a stopped container works.
* Improve owner/group detection error handling when container is stopped.
* Fix formulation.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Improve file comparison.
* Avoid reading whole file at once.
* Stream when fetching files from daemon.
* Fix comment.
* Use read() instead of read1().
* Stream files when copying into container.
* Linting.
* Add force parameter.
* Simplify library code.
* Linting.
* Add content and content_is_b64 options.
* Make force=false work as for copy module: only copy if the destination does not exist.
* Improve docs.
* content should be no_log.
* Implement diff mode.
* Improve error handling.
* Lint and improve.
* Set owner/group ID to avoid ID lookup (which fails in paused containers).
* Apply suggestions from code review
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Add more debug output.
* Add basic integration test.
* Split into lines.
* Fix docker detection, add podman detection.
ci_complete
* Improve regular expression.
* Document that this module is trying its best, but might not be perfect.
* Update comment.
* Add better tests for env and env_file.
* Make sure that non-container options are also passed to preprocessing code.
* Add changelog fragment.
* Add env_file override test.
This is related to
42789818be
in the sense that for Python > 2, we also exclusively use shutil.which now,
but we do not remove the helper function since we need it for Python 2 on Windows.
Co-authored-by: Daniel Möller <n1ngu@riseup.net>
Co-authored-by: Daniel Möller <n1ngu@riseup.net>
* utils: fix IPv6 address w/ port parsing
This was using a deprecated function (`urllib.splitnport`),
ostensibly to work around issues with brackets on IPv6 addresses.
Ironically, its usage was broken, and would result in mangled IPv6
addresses if they had a port specified in some instances.
Usage of the deprecated function has been eliminated and extra test
cases added where missing. All existing cases pass as-is. (The only
other change to the test was to improve assertion messages.)
Cherry-picked from
f16c4e1147
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* client: fix exception semantics in _raise_for_status
We want "The above exception was the direct cause of the following exception:" instead of "During handling of the above exception, another exception occurred:"
Cherry-picked from
bb11197ee3
Co-authored-by: Maor Kleinberger <kmaork@gmail.com>
* tls: use auto-negotiated highest version
Specific TLS versions are deprecated in latest Python, which
causes test failures due to treating deprecation errors as
warnings.
Luckily, the fix here is straightforward: we can eliminate some
custom version selection logic by using `PROTOCOL_TLS_CLIENT`,
which is the recommended method and will select the highest TLS
version supported by both client and server.
Cherry-picked from
56dd6de7df
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* transport: fix ProxyCommand for SSH conn
Cherry-picked from
4e19cc48df
Co-authored-by: Guy Lichtman <glicht@users.noreply.github.com>
* ssh: do not create unnecessary subshell on exec
Cherry-picked from
bb40ba051f
Co-authored-by: liubo <liubo@uniontech.com>
* ssh: reject unknown host keys when using Python SSH impl
In the Secure Shell (SSH) protocol, host keys are used to verify the identity of remote hosts. Accepting unknown host keys may leave the connection open to man-in-the-middle attacks.
Do not accept unknown host keys. In particular, do not set the default missing host key policy for the Paramiko library to either AutoAddPolicy or WarningPolicy. Both of these policies continue even when the host key is unknown. The default setting of RejectPolicy is secure because it throws an exception when it encounters an unknown host key.
Reference: https://cwe.mitre.org/data/definitions/295.html
NOTE: This only affects SSH connections using the native Python SSH implementation (Paramiko), when `use_ssh_client=False` (default). If using the system SSH client (`use_ssh_client=True`), the host configuration
(e.g. `~/.ssh/config`) will apply.
Cherry-picked from
d9298647d9
Co-authored-by: Audun Nes <audun.nes@gmail.com>
* lint: fix deprecation warnings from threading package
Set `daemon` attribute instead of using `setDaemon` method that
was deprecated in Python 3.10.
Cherry-picked from
adf5a97b12
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
* api: preserve cause when re-raising error
Use `from e` to ensure that the error context is propagated
correctly.
Cherry-picked from
05e143429e
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
* build: trim trailing whitespace from dockerignore entries
Cherry-picked from
3ee3a2486f
Co-authored-by: Clément Loiselet <clement.loiselet@capgemini.com>
* Improve formulation, also mention the security change as a breaking change.
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
Co-authored-by: Maor Kleinberger <kmaork@gmail.com>
Co-authored-by: Guy Lichtman <glicht@users.noreply.github.com>
Co-authored-by: liubo <liubo@uniontech.com>
Co-authored-by: Audun Nes <audun.nes@gmail.com>
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
Co-authored-by: Clément Loiselet <clement.loiselet@capgemini.com>
* Move licenses to LICENSES/, use SPDX-License-Identifier, mention all licenses in galaxy.yml.
* ignore.txt lines cannot be empty or contain only a comment.
* Cleanup.
* This particular __init__.py seems to be crucial.
* Try extra newline.
* Markdown comments are a real mess. I hope this won't break Galaxy...
* More licenses.
* Add sanity test.
* Skip some files, lint.
* Make sure there is a copyright line everywhere.
* Also check for copyright line in sanity tests.
* Remove colon after 'Copyright'.
* Normalize lint script.
* Avoid colon after 'Copyright' in lint script.
* Improve license checker.
* Update README.md
Co-authored-by: Maxwell G <9920591+gotmax23@users.noreply.github.com>
* Remove superfluous space.
* Referencing target instead of symlink
Co-authored-by: Maxwell G <9920591+gotmax23@users.noreply.github.com>
* Begin experiments for docker_container rewrite.
* Continued.
* We support API >= 1.25 only anyway.
* Continued.
* Fix bugs.
* Complete first basic implementation.
* Continuing.
* Improvements and fixes.
* Continuing.
* More 'easy' options.
* More options.
* Work on volumes and mounts.
* Add more options.
* The last option.
* Copy over.
* Fix exposed ports.
* Fix bugs.
* Fix command and entrypoint.
* More fixes.
* Fix more bugs.
* ci_complete
* Lint, fix Python 2.7 bugs, work around ansible-test bug.
ci_complete
* Remove no longer applicable test.
ci_complete
* Remove unnecessary ignore.
ci_complete
* Start with engine driver.
* Refactoring.
* Avoid using anything Docker specific from self.client.
* Refactor.
* Add Python 2.6 ignore.txt entries for ansible-core < 2.12.
* Improve healthcheck handling.
* Fix container removal logic.
* ci_complete
* Remove handling of older Docker SDK for Pyhon versions from integration tests.
* Avoid recreation if a pure update is possible without losing the diff data.
* Cover the case that blkio_weight does not work.
* Update plugins/module_utils/module_container/docker_api.py
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Improve memory_swap tests.
* Fix URLs in changelog fragment.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Rewrite the docker_plugin module to use the low-level client from Docker SDK for Python.
* Rewrite to no longer use the Docker SDK for Python.
* Remove Docker SDK for Python version from tests.
* Remove support for Ansible 2.9 and ansible-base 2.10.
* Remove Ansible 2.9 compatiblity code.
* Remove docker-compose from EE.
* Drop support for Python 2.6. Stop advertising docker-py for Python 2.6.
* Drop support for API versions 1.20 to 1.24.
* Fix condition.
* Vendor parts of the Docker SDK for Python
This is a combination of the latest git version
(a48a5a9647)
and the version before Python 2.7 support was removed
(650aad3a5f),
including some modifications to work with Ansible module_utils's
system (i.e. third-party imports are guarded, and errors are
reported during runtime through a new exception
MissingRequirementException).
* Create module_utils and plugin_utils for working with the vendored code.
The delete call cannot be called delete() since that method already exists from requests.
* Vendor more code from Docker SDK for Python.
* Adjust code from common module_utils.
* Add unit tests from Docker SDK for Python.
* Make test compile with Python 2.6, but skip them on Python 2.6.
* Skip test that requires a network server.
* Add changelog.
* Update changelogs/fragments/398-docker-api.yml
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Minimum API version is 1.25.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Add image_label_mismatch to docker_container
* Apply suggestions from code review
Co-authored-by: Felix Fontein <felix@fontein.de>
* Test image_label_mismatch error message
* Add change fragment for image_label_mismatch
* Break long line in docker_container.py for pep-8 compliance
* pep8 compliance
* Update changelogs/fragments/370-add-image-label-mismatch.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* fix: add expected_labels to parameters_map in docker_container
* Apply suggestions from code review
* Apply suggestions from code review
* Update tests/integration/targets/docker_container/tasks/tests/options.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* adds data_path_addr option for swarm init and swarm join. (ansible-collections/community.docker#339)
* adds changelog fragment (ansible-collections/community.docker#339)
* adds formatting to changelog entry
Co-authored-by: Felix Fontein <felix@fontein.de>
* adds version to doc entry in ansible_swarm
Co-authored-by: Felix Fontein <felix@fontein.de>
* rewrites doc entry to formal language
Co-authored-by: Felix Fontein <felix@fontein.de>
* Correct docker api version (src: docker api docs)
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
The `local` [Docker logging driver](https://docs.docker.com/config/containers/logging/local/) is a new-ish, optimized local logging driver that improves upon json-file. Since logs written using `local` can still be accessed through the Docker daemon API, all that is needed here is to add to the supported list.
* Try adding EE support.
* Use GHA instead of AZP for EE tests.
* Update changelog fragment, extend tests.
* Disable current_container_facts test.
* Increase verbosity.
* 2.9 compatibility.
* Use docker instead of podman for building EE and running tests in it.
* Output some more information (helpful for debugging).
* Fix GHA handling for current_container_facts.
* Try to fix permissions.
* fix config docs and update to use config system
wean off play_context which did not have the correct data in all cases
* moar fixes
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* updated for backwards compat
* badmergeresolution
* makeitworks
* attempt to fix unit test
* mocking it# No more than 50 chars. #### 50 chars is here: #
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update plugins/connection/docker.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* nomock
* remove bad paste
* properly load connection to initialize config
* initizlie docker args
* Fix bugs.
* Call _set_conn_data() when needed.
* Cache result of _get_docker_remote_user() now that it is called multiple times per task.
* Fix unit tests.
* list.clear() is Python 3...
* Add changelog.
* Call _set_conn_data() also in _connect().
Co-authored-by: Felix Fontein <felix@fontein.de>
* Implement `cap_add` and `cap_drop` handling for `docker_swarm_service`
* Fix typos in changelog fragment
* Add missing `version_added` docstrings to `docker_swarm_service`
* Check api version too in `docker_swarm_service` capability tests
* Fix API version checking in tests for docker_swarm_service
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add `cap_add` and `cap_drop` to sample output for `docker_swarm_service`
* Use proper diff checking for `docker_swarm_service` capabilities
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add parameters for rolling updates to `docker_secret`
* Extract `remove_secrets` to its own function in `docker_secret`
* Store existing secrets in a list instead of a single secret
With this change `docker_secret` now supports the case where we store
multiple versions of a secret with the `_v123` postfix.
`absent` state implicitly handles removing these this way.
* When using `rolling_versions` don't automatically remove current secret
To make rolling updates actually work instead of failing on trying to
remove a secret that is attached to a service, use the
`versions_to_keep` parameter to remove old versions of the secret after
creating the new one. This way the secret with the new data is created
with a different name and can be attached to the service by its ID
without having to delete the previous one first which would fail if it
is already attached to a service.
* Add version numbers to newly created secrets
Attach the incremental version number to the secret name as a `_v123`
postfix where `123` is replaced with an incremental counter starting
from 1.
A label with the numeric version is also attached to the secret to ease
calculating the new version number upon change with the name
`ansible_version`.
* Return `secret_name` for docker secrets as well
* Add integration test for rolling secrets
* Update `docker_secret` documentation as per review comments
* Correctly return `docker_secret` version number as int
* Use template string for naming `docker_secrets` instead of concatenation
* Return the correct secret name on deletion failure
* Simplify `docker_secret` creation
* Add missing comma for `docker_secret` schema
* Only remove old docker secrets if `rolling_versions` is set
* Add check in `docker_secret` version parsing to handle NaNs
* Add newly created `docker_secret` to internal secret list to avoid additional deletions
* Add changelog fragment for `docker_secret` `rolling_versions` feature
* Update changelogs/fragments/270-rolling-secrets.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Prepare for distutils.version being removed in Python 2.12.
* Fix copy'n'paste error.
* Fix import.
* Re-add Loose prefix.
* Fix Python version typo.
* Improve formulation.
* Move message into own line.
* Fix casing, now that the object is no longer called Version.
* Minor fixes to nsenter connection plugin
- Ensure the nsoption_pid option is retrieved in _connect instead of
__init__ to prevent a crasher due to initialization order
- Replace the use of --all-namespaces with specific namespaces to
support compatibility with Busybox nsenter (for example, Alpine)
* minor tidy
* Fix PEP8 violation
* Changelog fragment
* Update changelogs/fragments/249-nsenter-fixes.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Jeff Goldschrafe <jeff.goldschrafe@flatiron.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Only pass chdir on when it is provided, and prevent this option from being used for Docker SDK for Python < 3.0.0.
* Also fix docker_api connection plugin.
* Remove docker_login's email option.
* container_default_behavior now has default value no_defaults.
* Disallow 'all' next to other ports. Deprecate 'all' in favor of publish_all_ports.
* Change default for network_mode.
* Add changelog fragment.
* Fix tests.
* Next expected release is 2.0.0.
* Fix handling of command and entrypoint in a backwards-compatible way.
* Fix copy'n'paste error.
* Fix some more.
* Improve documentation.
* Keep command and entrypoint as lists and not as strings.
* Simplify code, since we're already emitting the deprecation warning in this case during parameter processing.
* Change default only in community.docker 3.0.0.
* Update tests/integration/targets/docker_container/tasks/tests/options.yml
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Apply suggestion to more places.
Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
* Move variable handling to doc fragment, and make them known to module_utils.
* Pass Daemon connection options to connection plugin.
* Add changelog fragment.
* Fix syntax error.
* Forgot 'options:'.
* Add test to tag image with ID.
* Document that source=local also works with image IDs.
* Improve 'repository' documentation.
* Fix typo.
* Looks like a fix is needed to make this work.
* ...
* Avoid unnecessary re.search() calls.
* Add changelog fragment.
* Improve main description.
* Remove import.
* feat: allow unlimited memory_swap in docker_container
summary: in docker_container, allow `memory_swap: values
`unlimited` and `-1` for unlimited container swap usage.
`--memory-swap=-1` is a valid docker run value allowing
unlimited swap usage. `docker_container` should allow
setting this value via `memory_swap` values `unlimited`
and `-1` as it sometimes must explicitly be set to prevent
docker from attempting to limit swap usage to double the
`--memory` value by default. In rootless docker environments,
attempting to set `memory-swap` usage limits will result in an error
if the user doesn't have the ability to do so. Allowing explicit
unlimited swap usage via docker_container will prevent these errors.
https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details
* Update tests/integration/targets/docker_container/tasks/tests/options.yml
* Apply suggestions from code review
Co-authored-by: Felix Fontein <felix@fontein.de>
* Start removing default.
* Add deprecate() function to AnsibleDockerClientBase.
* Deprecate default value for tls_hostname.
* Add changelog.
* Mention deprecation in documentation.
* adds missing changelog fragment file
changelog fragment file for #125
* Update changelogs/fragments/125-correct-error-message-for-docker-sdk-version.yaml
Co-authored-by: Felix Fontein <felix@fontein.de>
* Change docker_swarm_service option publish.published_port from required to optional. If not specified random high port is assigned by docker.
* Update changelogs/fragments/101-make-service-published-port-optional.yaml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Mark join_token as no_log.
* Apply suggestions from code review
Co-authored-by: Piotr Wojciechowski <23406016+WojciechowskiPiotr@users.noreply.github.com>
Co-authored-by: Piotr Wojciechowski <23406016+WojciechowskiPiotr@users.noreply.github.com>
* Allow to specify pull platform.
* Add basic test and document that the value is not used for idempotency at the moment.
* Fix pulling.
* Simplify code.
* Add API version for pull_platform.
* Move pull_platform into new pull option. Use apply_defaults=True to avoid some special logic.
* Add example.
* Remove apply_defaults=True.
* docker swarm - Add no_log to the signing_ca_key argument
This will prevent accidental disclosure.
See: CVE-2021-20191
* Update changelogs/fragments/CVE-2021-20191_no_log.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
* docker.py: allow docker versions with a leading 'v'
* Update changelogs/fragments/76-leading-v-support-in-docker-version.yml
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
* Avoid crash for docker-py < 2.5.0.
* Add warnings when load_image does not return a generator.
* Add test.
* Update plugins/modules/docker_image.py
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Support IPv6 zones (RFC 4007).
* Add changelog fragment.
* Remove change for docker_network.
* Add IPv6 zone test.
It looks like an arbitrary zone name works. If Docker daemon ever starts
validating it (against what?) we either have to try to fix this test by
a valid value, or remove it again.
