[
https://issues.apache.org/jira/browse/AIRFLOW-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288739#comment-16288739
]
Russell Pierce commented on AIRFLOW-944:
----------------------------------------
Potentially (at least partially) duplicated by Airflow-1057, Airflow-1159, and
Airflow-1380
> Docker operator does not work with Docker >= 1.19
> -------------------------------------------------
>
> Key: AIRFLOW-944
> URL: https://issues.apache.org/jira/browse/AIRFLOW-944
> Project: Apache Airflow
> Issue Type: Bug
> Components: docker
> Affects Versions: Airflow 1.7.1.3, 1.8.0rc4
> Environment: Ubuntu 16.04
> Reporter: Ludovic Claude
>
> Docker operator does not work when mem_limit is set and Docker version 1.19
> or more recent is used.
> Here are the logs, I have seen this issue with Airflow 1.7.1.3 and Airflow
> 1.8.0 rc4.
> [2017-03-06 11:37:54,895] {base_task_runner.py:95} INFO - Subtask:
> [2017-03-06 11:37:54,895] {docker_operator.py:132} INFO - Starting docker
> container from image hbpmip/mipmap
> [2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask:
> [2017-03-06 11:37:54,902] {models.py:1417} ERROR - mem_limit has been moved to
> host_config in API version 1.19
> [2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask: Traceback
> (most recent call last):
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python3.5/dist-packages/airflow/models.py", line 1369, in run
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: result
> = task_copy.execute(context=context)
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/tmp/src/airflow-imaging-plugins/airflow_pipeline/operators/docker_pipeline_operator.py",
> line 191, in execute
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: logs =
> super(DockerPipelineOperator, self).execute(context)
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python3.5/dist-packages/airflow/operators/docker_operator.py",
> line 172, in execute
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:
> user=self.user
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 133,
> in create_container
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:
> volume_driver, stop_signal, networking_config,
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 138,
> in create_container_config
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: return
> utils.create_container_config(self._version, *args, **kwargs)
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File
> "/usr/local/lib/python3.5/dist-packages/docker/utils/utils.py", line 1041, in
> create_container_config
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:
> 'mem_limit has been moved to host_config in API version 1.19'
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:
> docker.errors.InvalidVersion: mem_limit has been moved to host_config in API
> version 1.19
> [2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:
> [2017-03-06 11:37:54,903] {models.py:1433} INFO - Marking task as UP_FOR_RETRY
> [2017-03-06 11:37:54,912] {base_task_runner.py:95} INFO - Subtask:
> [2017-03-06 11:37:54,912] {models.py:1462} ERROR - mem_limit has been moved
> to host_config in API version 1.19
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
