52 Commits

Author SHA1 Message Date
Felix Fontein a426232523 Fix imports. (#585) 2023-02-12 22:09:02 +01:00
Felix Fontein 983b2b4783 exec: fix file handle leak with container.exec_* APIs (https://github.com/docker/docker-py/pull/2320) (#582)
Requests with stream=True MUST be closed or else the connection will
never be returned to the connection pool. Both ContainerApiMixin.attach
and ExecApiMixin.exec_start were leaking in the stream=False case.
exec_start was modified to follow attach for the stream=True case as
that allows the caller to close the stream when done (untested).

Tested with:

    # Test exec_run (stream=False) - observe one less leak
    make integration-test-py3 file=models_containers_test.py' -k test_exec_run_success -vs -W error::ResourceWarning'
    # Test exec_start (stream=True, fully reads from CancellableStream)
    make integration-test-py3 file=api_exec_test.py' -k test_execute_command -vs -W error::ResourceWarning'

After this change, one resource leak is removed, the remaining resource
leaks occur because none of the tests call client.close().

Fixes https://github.com/docker/docker-py/issues/1293
(Regression from https://github.com/docker/docker-py/pull/1130)

Cherry-picked from https://github.com/docker/docker-py/commit/34e6829dd40e99e9ba47ea02fcfabda09e08d36e

Co-authored-by: Peter Wu <pwu@cloudflare.com>
Co-authored-by: Milas Bowman <milas.bowman@docker.com>
2023-02-12 08:29:28 +01:00
Kristof Mattei d2f551fc5d fix: fix tmpfs_size and tmpfs_mode not being set (#580)
* fix: fix tmpfs_size and tmpfs_mode not being set

* fix: wrong file

* fix: add changelog fragment

* fix: update changelog fragment to match formatting

Co-authored-by: Felix Fontein <felix@fontein.de>

---------

Co-authored-by: Felix Fontein <felix@fontein.de>
2023-02-10 14:05:09 +01:00
Felix Fontein 54a3dc151d Remove unneccessary imports (#575)
* Remove unneccessary imports.

* Add noqas.
2023-02-09 15:25:45 +01:00
Felix Fontein 4e6ac335f3 Improve envvar fallback handling. (#554) 2023-01-13 06:37:04 +01:00
Felix Fontein e198e4ab43 Add docker_container_copy_into module (#545)
* Move copying functionality to module_utils.

* Add docker_container_copy_into module.

* Use new module in other tests.

* Fix copyright and attributes.

* Improve idempotency, improve stat code.

* Document and test when a stopped container works.

* Improve owner/group detection error handling when container is stopped.

* Fix formulation.

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>

* Improve file comparison.

* Avoid reading whole file at once.

* Stream when fetching files from daemon.

* Fix comment.

* Use read() instead of read1().

* Stream files when copying into container.

* Linting.

* Add force parameter.

* Simplify library code.

* Linting.

* Add content and content_is_b64 options.

* Make force=false work as for copy module: only copy if the destination does not exist.

* Improve docs.

* content should be no_log.

* Implement diff mode.

* Improve error handling.

* Lint and improve.

* Set owner/group ID to avoid ID lookup (which fails in paused containers).

* Apply suggestions from code review

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
2023-01-09 11:52:29 +01:00
Felix Fontein 3a40112a76 Remove timeout when waiting for container to finish. (#527) 2022-12-08 22:15:42 +01:00
Felix Fontein 2957138153 latest docker-py bugfix (npipe) (#513)
* socket: handle npipe close on Windows (https://github.com/docker/docker-py/pull/3056)

Fixes https://github.com/docker/docker-py/issues/3045

Cherry-picked from https://github.com/docker/docker-py/commit/30022984f6445fbc322cbe97bb99aab1ddb1e4fd

Co-authored-by: Nick Santos <nick.santos@docker.com>

* Add changelog fragment.

Co-authored-by: Nick Santos <nick.santos@docker.com>
2022-12-01 06:59:05 +01:00
iamjpotts 166d485216 Make image archive/save idempotent, using image id and repo tags as keys (#500) 2022-11-30 23:45:36 +01:00
Felix Fontein edf0d3ec99 Make kill_signal accept strings. (#506) 2022-11-28 22:10:07 +01:00
Felix Fontein 5b31f17016 Add image_name_mismatch option. (#488) 2022-11-01 19:48:58 +00:00
Felix Fontein 51d5744cb0 docker_container: deprecate ignore_image and purge_networks (#487)
* Deprecate ignore_image and purge_networks.

* Fix YAML.

* Simple replacement doesn't work in this case.
2022-11-01 19:57:56 +01:00
Felix Fontein f7cf12555c docker_container: fix env_file option (#452)
* Add better tests for env and env_file.

* Make sure that non-container options are also passed to preprocessing code.

* Add changelog fragment.

* Add env_file override test.
2022-08-15 07:45:59 +02:00
Felix Fontein 74134eda33 Fix docker_container tests (#441)
* Add diff output to figure out a bit more why the test fails.

* Make sure that both images have been pulled in advance.

* Dump the correct image.

* Allow tty test to fail in certain circumstances.
2022-08-08 23:23:23 +02:00
Felix Fontein 1e4633a606 For Python > 2, always use shutil.which instead of custom Windows helper code. (#438)
This is related to
https://github.com/docker/docker-py/commit/42789818bed5d86b487a030e2e60b02bf0cfa284
in the sense that for Python > 2, we also exclusively use shutil.which now,
but we do not remove the helper function since we need it for Python 2 on Windows.

Co-authored-by: Daniel Möller <n1ngu@riseup.net>

Co-authored-by: Daniel Möller <n1ngu@riseup.net>
2022-08-08 20:58:12 +02:00
Felix Fontein ae708a7333 Vendored Docker SDK for Python updates (#434)
* utils: fix IPv6 address w/ port parsing

This was using a deprecated function (`urllib.splitnport`),
ostensibly to work around issues with brackets on IPv6 addresses.

Ironically, its usage was broken, and would result in mangled IPv6
addresses if they had a port specified in some instances.

Usage of the deprecated function has been eliminated and extra test
cases added where missing. All existing cases pass as-is. (The only
other change to the test was to improve assertion messages.)

Cherry-picked from
https://github.com/docker/docker-py/commit/f16c4e1147c81afd822fe72191f0f720cb0ba637

Co-authored-by: Milas Bowman <milas.bowman@docker.com>

* client: fix exception semantics in _raise_for_status

We want "The above exception was the direct cause of the following exception:" instead of "During handling of the above exception, another exception occurred:"

Cherry-picked from
https://github.com/docker/docker-py/commit/bb11197ee3407798a53c50e43aa994fe8cd9c8e7

Co-authored-by: Maor Kleinberger <kmaork@gmail.com>

* tls: use auto-negotiated highest version

Specific TLS versions are deprecated in latest Python, which
causes test failures due to treating deprecation errors as
warnings.

Luckily, the fix here is straightforward: we can eliminate some
custom version selection logic by using `PROTOCOL_TLS_CLIENT`,
which is the recommended method and will select the highest TLS
version supported by both client and server.

Cherry-picked from
https://github.com/docker/docker-py/commit/56dd6de7dfad9bedc7c8af99308707ecc3fad78e

Co-authored-by: Milas Bowman <milas.bowman@docker.com>

* transport: fix ProxyCommand for SSH conn

Cherry-picked from
https://github.com/docker/docker-py/commit/4e19cc48dfd88d0a9a8bdbbe4df4357322619d02

Co-authored-by: Guy Lichtman <glicht@users.noreply.github.com>

* ssh: do not create unnecessary subshell on exec

Cherry-picked from
https://github.com/docker/docker-py/commit/bb40ba051fc67605d5c9e7fd1eb5f9aa3e0fb501

Co-authored-by: liubo <liubo@uniontech.com>

* ssh: reject unknown host keys when using Python SSH impl

In the Secure Shell (SSH) protocol, host keys are used to verify the identity of remote hosts. Accepting unknown host keys may leave the connection open to man-in-the-middle attacks.

Do not accept unknown host keys. In particular, do not set the default missing host key policy for the Paramiko library to either AutoAddPolicy or WarningPolicy. Both of these policies continue even when the host key is unknown. The default setting of RejectPolicy is secure because it throws an exception when it encounters an unknown host key.

Reference: https://cwe.mitre.org/data/definitions/295.html

NOTE: This only affects SSH connections using the native Python SSH implementation (Paramiko), when `use_ssh_client=False` (default). If using the system SSH client (`use_ssh_client=True`), the host configuration
(e.g. `~/.ssh/config`) will apply.

Cherry-picked from
https://github.com/docker/docker-py/commit/d9298647d91c52e1ee9ac448e43a7fea1c69bdbe

Co-authored-by: Audun Nes <audun.nes@gmail.com>

* lint: fix deprecation warnings from threading package

Set `daemon` attribute instead of using `setDaemon` method that
was deprecated in Python 3.10.

Cherry-picked from
https://github.com/docker/docker-py/commit/adf5a97b1203623ae47bf7aa1367b6bb7c261980

Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>

* api: preserve cause when re-raising error

Use `from e` to ensure that the error context is propagated
correctly.

Cherry-picked from
https://github.com/docker/docker-py/commit/05e143429e892fb838bbff058391456ba3d0a19c

Co-authored-by: Milas Bowman <milas.bowman@docker.com>

* build: trim trailing whitespace from dockerignore entries

Cherry-picked from
https://github.com/docker/docker-py/commit/3ee3a2486fe75ed858f8a3defe0fc79b2743d5df

Co-authored-by: Clément Loiselet <clement.loiselet@capgemini.com>

* Improve formulation, also mention the security change as a breaking change.

Co-authored-by: Milas Bowman <milas.bowman@docker.com>
Co-authored-by: Maor Kleinberger <kmaork@gmail.com>
Co-authored-by: Guy Lichtman <glicht@users.noreply.github.com>
Co-authored-by: liubo <liubo@uniontech.com>
Co-authored-by: Audun Nes <audun.nes@gmail.com>
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
Co-authored-by: Clément Loiselet <clement.loiselet@capgemini.com>
2022-07-31 17:09:18 +02:00
Felix Fontein 9c5d562c0e Fix bug when TLS is used (#432)
* Fix bug when TLS is used.

* Add HTTP/HTTPS connection test.
2022-07-26 08:25:53 +02:00
Felix Fontein a4539a309e Move licenses to LICENSES/, use SPDX-License-Identifier, mention all licenses in galaxy.yml (#430)
* Move licenses to LICENSES/, use SPDX-License-Identifier, mention all licenses in galaxy.yml.

* ignore.txt lines cannot be empty or contain only a comment.

* Cleanup.

* This particular __init__.py seems to be crucial.

* Try extra newline.

* Markdown comments are a real mess. I hope this won't break Galaxy...

* More licenses.

* Add sanity test.

* Skip some files, lint.

* Make sure there is a copyright line everywhere.

* Also check for copyright line in sanity tests.

* Remove colon after 'Copyright'.

* Normalize lint script.

* Avoid colon after 'Copyright' in lint script.

* Improve license checker.

* Update README.md

Co-authored-by: Maxwell G <9920591+gotmax23@users.noreply.github.com>

* Remove superfluous space.

* Referencing target instead of symlink

Co-authored-by: Maxwell G <9920591+gotmax23@users.noreply.github.com>
2022-07-20 07:45:33 +02:00
Felix Fontein e26890a909 Implement platform parameter for docker_container, first version. (#426) 2022-07-15 17:14:57 +02:00
Felix Fontein 5d0a036819 docker_container: add image_comparison parameter (#428)
* Add image_comparison parameter.

* Forgot version_added.
2022-07-15 17:14:40 +02:00
Felix Fontein 37c868e192 Add support for cgroupns_mode parameter. (#427) 2022-07-15 17:14:23 +02:00
Felix Fontein 77e63e2cca Rewrite docker_container to use Docker API directly (#422)
* Begin experiments for docker_container rewrite.

* Continued.

* We support API >= 1.25 only anyway.

* Continued.

* Fix bugs.

* Complete first basic implementation.

* Continuing.

* Improvements and fixes.

* Continuing.

* More 'easy' options.

* More options.

* Work on volumes and mounts.

* Add more options.

* The last option.

* Copy over.

* Fix exposed ports.

* Fix bugs.

* Fix command and entrypoint.

* More fixes.

* Fix more bugs.

* ci_complete

* Lint, fix Python 2.7 bugs, work around ansible-test bug.

ci_complete

* Remove no longer applicable test.

ci_complete

* Remove unnecessary ignore.

ci_complete

* Start with engine driver.

* Refactoring.

* Avoid using anything Docker specific from self.client.

* Refactor.

* Add Python 2.6 ignore.txt entries for ansible-core < 2.12.

* Improve healthcheck handling.

* Fix container removal logic.

* ci_complete

* Remove handling of older Docker SDK for Pyhon versions from integration tests.

* Avoid recreation if a pure update is possible without losing the diff data.

* Cover the case that blkio_weight does not work.

* Update plugins/module_utils/module_container/docker_api.py

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>

* Improve memory_swap tests.

* Fix URLs in changelog fragment.

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
2022-07-15 07:24:14 +02:00
Felix Fontein 9e57f29b3b Refactoring. (#415) 2022-07-03 13:28:11 +02:00
Felix Fontein 623786c659 Implement all remaining deprecations for 3.0.0 (#400)
* Remove support for Ansible 2.9 and ansible-base 2.10.

* Remove Ansible 2.9 compatiblity code.

* Remove docker-compose from EE.

* Drop support for Python 2.6. Stop advertising docker-py for Python 2.6.

* Drop support for API versions 1.20 to 1.24.

* Fix condition.
2022-07-02 17:13:53 +02:00
Felix Fontein 4d508b4c37 Vendor API connection code from Docker SDK for Python (#398)
* Vendor parts of the Docker SDK for Python

This is a combination of the latest git version
(https://github.com/docker/docker-py/tree/a48a5a9647761406d66e8271f19fab7fa0c5f582)
and the version before Python 2.7 support was removed
(https://github.com/docker/docker-py/tree/650aad3a5fb84059b392ad450f760ed08143ae3f),
including some modifications to work with Ansible module_utils's
system (i.e. third-party imports are guarded, and errors are
reported during runtime through a new exception
MissingRequirementException).

* Create module_utils and plugin_utils for working with the vendored code.

The delete call cannot be called delete() since that method already exists from requests.

* Vendor more code from Docker SDK for Python.

* Adjust code from common module_utils.

* Add unit tests from Docker SDK for Python.

* Make test compile with Python 2.6, but skip them on Python 2.6.

* Skip test that requires a network server.

* Add changelog.

* Update changelogs/fragments/398-docker-api.yml

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>

* Minimum API version is 1.25.

Co-authored-by: Brian Scholer <1260690+briantist@users.noreply.github.com>
2022-07-02 16:40:44 +02:00
Felix Fontein 21d112bddb Unvendor distutils.version (#271)
* Unvendor distutils.version.

* Document breaking change.

* Update comment, add StrictVersion.
2022-07-02 15:06:58 +02:00
Felix Fontein bc64aef5ca Revert "Revert "Remove deprecated functionality. (#363)""
This reverts commit e6d597b539.
2022-07-02 14:28:31 +02:00
Felix Fontein e6d597b539 Revert "Remove deprecated functionality. (#363)"
This reverts commit 209aeb57fd.
2022-07-02 14:18:53 +02:00
Felix Fontein 209aeb57fd Remove deprecated functionality. (#363) 2022-07-01 19:02:22 +02:00
Felix Fontein f41d7ac1b8 Move common utility code to plugins.module_utils.util (#390)
* Move common utility code to plugins.module_utils.util.

* Also adjust plugins.

* Fix import.
2022-06-20 18:39:13 +02:00
Felix Fontein a44e0736cf Improve API version handling. (#389) 2022-06-20 07:34:38 +02:00
Felix Fontein 7e710f743d Bump deprecations we forgot to remove in 2.0.0. (#362) 2022-05-20 14:45:52 +02:00
Felix Fontein 6adc7bddf3 Add PSF-license.txt file. (#350) 2022-05-16 07:35:23 +02:00
Felix Fontein 9cd46a7d41 Try to make more compatible with podman-docker. (#292) 2022-02-11 12:23:36 +01:00
Felix Fontein d4b7ec0a9a Fix comment. (#272) 2022-01-06 12:56:12 +00:00
Felix Fontein 264f5238c7 Fix unintended breaking change by vendoring distutils.version. (#269) 2022-01-05 20:08:59 +00:00
Felix Fontein 4b235a2607 Prepare for distutils.version being removed in Python 3.12 (#258)
* Prepare for distutils.version being removed in Python 2.12.

* Fix copy'n'paste error.

* Fix import.

* Re-add Loose prefix.

* Fix Python version typo.

* Improve formulation.

* Move message into own line.

* Fix casing, now that the object is no longer called Version.
2021-12-24 11:28:35 +01:00
Felix Fontein 4070dab9a8 Remove deprecated features, bump version to 2.0.0 (#210)
* Remove docker_login's email option.

* container_default_behavior now has default value no_defaults.

* Disallow 'all' next to other ports. Deprecate 'all' in favor of publish_all_ports.

* Change default for network_mode.

* Add changelog fragment.

* Fix tests.

* Next expected release is 2.0.0.
2021-10-05 17:39:42 +02:00
Felix Fontein 930f9d1bcd Improve error message on import errors. (#188) 2021-08-02 21:04:25 +02:00
Felix Fontein badcd61a74 Replace ansible.module_utils._text by ansible.module_utils.common.text.converters. (#164) 2021-06-26 13:39:51 +02:00
Ajpantuso 49c8fd0aa5 docker_host_info - Allow filters which are passed as lists (#160)
* Initial Commit

* Adding integration tests

* Adding example in docs

* Adding changelog fragment

* Applying initial review suggestions
2021-06-22 22:27:38 +02:00
Felix Fontein 902bcc6193 Pass Docker daemon connection params from inventory to connection plugin (#157)
* Move variable handling to doc fragment, and make them known to module_utils.

* Pass Daemon connection options to connection plugin.

* Add changelog fragment.

* Fix syntax error.

* Forgot 'options:'.
2021-06-22 06:29:56 +02:00
Felix Fontein c20186901b Only emit tls_hostname deprecation warning if TLS is actually used (#143)
* Only emit tls_hostname deprecation warning if TLS is actually used.

* Linting.
2021-05-17 13:24:50 +02:00
Felix Fontein cffba7b15d Deprecate default for tls_hostname (#134)
* Start removing default.

* Add deprecate() function to AnsibleDockerClientBase.

* Deprecate default value for tls_hostname.

* Add changelog.

* Mention deprecation in documentation.
2021-05-04 07:50:33 +02:00
daeseokyoun 18ccec6b10 common: correct error message for docker sdk version (#125)
Annouced that 5.0.0 in Docker SDK for Python do not support Python 2.7 anymore.
When the system use Python 2.7, and run "pip install docker" command to
install Docker SDK for Python module. Then the 5.0.0 will be installed.

When occurred the error for mismatching version for Python 2.7, the user
could not feature it out why it failed.

Co-authored-by: Daeseok Youn <daeseok.youn@navercorp.com>
2021-04-16 23:53:20 +02:00
Felix Fontein ff503d9bd7 Add docker_container_exec module (#105)
* Move some code from plugin_utils to module_utils.

* First version of docker_container_exec module.

* Linting.

* Avoid using display.

* Move common socket_handler code to module_utils.

* Add module support, use it in docker_container_exec.

* Add tests.

* Fix copyright lines.

* Add this and other 'new' modules to README.

* Apply suggestions from code review

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>

* Update plugins/modules/docker_container_exec.py

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
2021-04-06 23:53:17 +02:00
Felix Fontein e9b10ee0a4 Add use_ssh_client flag (#114)
* Add use_ssh_client option.

* Unrelated cleanup (unneeded imports).

* Add SSH connection tests.
2021-04-02 10:14:09 +02:00
Felix Fontein f2d16da643 docker_image: improve/fix handling of image IDs (#87)
* Improve/fix handling of image IDs in docker_image.

* Fix syntax error.

* Linting.

* Fix name collision.

* Add various tests.

* Fix tests.

* Improve image finding by ID, and fix various related bugs.

* accept_not_there -> accept_missing_image.

* Remove unnecessary dummy variable.
2021-02-28 10:40:11 +01:00
Felix Fontein ed9bf1117f docker_image: allow to specify pull platform (#89)
* Allow to specify pull platform.

* Add basic test and document that the value is not used for idempotency at the moment.

* Fix pulling.

* Simplify code.

* Add API version for pull_platform.

* Move pull_platform into new pull option. Use apply_defaults=True to avoid some special logic.

* Add example.

* Remove apply_defaults=True.
2021-02-24 23:45:34 +01:00
Felix Fontein 8bd33b06c4 docker_api connection plugin (#40)
* Add basic docker_api connection test.

* Split AnsibleDockerClient into module-independent and module-specific part.

* Mention new connection plugin.

* Fix tests.

* Add first version of docker_api connection plugin.

* Linting.

* Fix references.

* Improve connection tests.

* Fix put_file for all Python versions.

* Fix fetch_file.

* Linting.

* Update plugins/connection/docker_api.py

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>

* Move potential common code to module_utils / plugin_utils.

* Move socket_handler to plugin_utils.

* Fix typo.

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
2020-12-15 20:07:55 +00:00