[
https://issues.apache.org/jira/browse/AIRFLOW-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300592#comment-17300592
]
ASF GitHub Bot commented on AIRFLOW-5270:
-----------------------------------------
alexras commented on pull request #5876:
URL: https://github.com/apache/airflow/pull/5876#issuecomment-797729133
In case someone else finds this: I ran into this issue on 1.10.7, but
bumping to 1.10.14 fixed it. There was a (potentially-unrelated) JSON decoding
issue that prompted a refactoring of the function in which this error occurs
(#8287). I suspect that refactoring fixed this problem as a side-effect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> JSONDecodeError in DockerOperator when force_pull
> -------------------------------------------------
>
> Key: AIRFLOW-5270
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5270
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.3
> Environment: MacOS Mojave
> Reporter: Matthew Petersen
> Assignee: Matthew Petersen
> Priority: Minor
>
> Lines of the Docker API response usually end with "\r\n". We .strip() them to
> correct for this, but sometimes two lines are joined together, creating one
> line that is not valid JSON. The exact error we see is
> ```
> json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 55)
> ```
> And can be reproduced with the following code
> ```
> pip install docker
> from docker import APIClient
> import json
> cli = APIClient(base_url='unix://var/run/docker.sock')
> for l in cli.pull('hello-world', stream=*True*):
> output = json.loads(l.decode('utf-8').strip()
> ```
> which is exactly what this line of `DockerOperator` does which causes it to
> crash
> [https://github.com/apache/airflow/blob/46885ccdbc8618f295b8f986da95c1abbb85bb02/airflow/operators/docker_operator.py#L259]
> The solution is to replace "\r\n" with "\n" before json parsing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
