[
https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097721#comment-17097721
]
ASF GitHub Bot commented on AIRFLOW-4363:
-----------------------------------------
brokenjacobs commented on pull request #8287:
URL: https://github.com/apache/airflow/pull/8287#issuecomment-622597778
The docker docs are definitely wrong. As you can see if you try to run the
operator code as it exists in 10.10.
It doesn’t mean they won’t decide sometime to make the docs be accurate. I’m
supportive of the way this change was implemented.
- Steve Jacobs
> On May 1, 2020, at 4:04 PM, retornam <[email protected]> wrote:
>
>
> @retornam commented on this pull request.
>
> In airflow/providers/docker/operators/docker.py:
>
> > + for output in self.cli.pull(self.image, stream=True,
decode=True):
> + if isinstance(output, str):
> + self.log.info("%s", output)
> + if isinstance(output, dict) and 'status' in output:
> @acroos and then right below it is says (generator or str) for return Type
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or unsubscribe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Encounter JSON Decode Error when using docker operator
> ------------------------------------------------------
>
> Key: AIRFLOW-4363
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4363
> Project: Apache Airflow
> Issue Type: Bug
> Environment: - Mac OS 10.14
> - python 3.6.8
> - airflow 1.10.2
> Reporter: Ben Chen
> Assignee: Ben Chen
> Priority: Blocker
> Fix For: 1.10.10
>
>
> *[Description]* .
> When using the docker_operator, I experienced some issue while using Mac OS
> 10.14.4. The error was json.JSONDecodeError. After my investigation about
> this error, I found that there are several messages for logging aren't well
> separated, for example it contains \n inside one single message which should
> be split into 2 to more different messages.
> *[Update]*
> Confirmed that issue came from the implementation in airflow, issue cannot be
> solved by just passing `decode` to parameter in docker.pull method in docker
> api.
> *[Solution]* .
> For now, I use try-catch to run the original implementation, and in the
> exception part I split the message to list and then parse it. Looking for
> simpler solution to this non critical but still blocking point.
> *[Logs]*
> {docker_operator.py:188}
> INFO - Starting docker container from image hello-world
> {docker_operator.py:202}
> INFO - Pulling docker image hello-world
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Pulling fs layer
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Download complete
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Pull complete
> {docker_operator.py:207}
> INFO - Digest:
> sha256:92695bc579f31df7a63da6922075d0666e565ceccad16b59c3374d2cf4e8e50e
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Digest:
> sha256:1a67c1115b199aa9d964d5da5646917cbac2d5450c71a1deed7b1bfb79c2c82d
> {models.py:1788}
> ERROR - Extra data: line 2 column 1 (char 70)
> Traceback (most recent call last):
> line 1657, in _run_raw_task, result = task_copy.execute(context=context)
> line 205, in execute output = json.loads(line)
> line 354, in loads, return _default_decoder.decode(s)
> line 342, in decode, raise JSONDecodeError("Extra data", s, end)
> json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 70)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
