[
https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067022#comment-17067022
]
ASF subversion and git services commented on AIRFLOW-4363:
----------------------------------------------------------
Commit 5b6325b5796a86a931bb474c12bd008c878471a2 in airflow's branch
refs/heads/v1-10-test from retornam
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=5b6325b ]
[AIRFLOW-4363] Fix JSON encoding error (#7628)
>From the docker-py code comments for APIClient pull,
the decode parameter should be set to True, when the
stream parameter is also set to True. This will allow
decoding JSON data returned from the docker registry
server into dicts
Signed-off-by: Raymond Etornam <[email protected]>
(cherry picked from commit 733d3d3c32e0305691f82102cfc346e8e85478b0)
> Encounter JSON Decode Error when using docker operator
> ------------------------------------------------------
>
> Key: AIRFLOW-4363
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4363
> Project: Apache Airflow
> Issue Type: Bug
> Environment: - Mac OS 10.14
> - python 3.6.8
> - airflow 1.10.2
> Reporter: Ben Chen
> Assignee: Ben Chen
> Priority: Blocker
> Fix For: 1.10.10
>
>
> *[Description]* .
> When using the docker_operator, I experienced some issue while using Mac OS
> 10.14.4. The error was json.JSONDecodeError. After my investigation about
> this error, I found that there are several messages for logging aren't well
> separated, for example it contains \n inside one single message which should
> be split into 2 to more different messages.
> *[Update]*
> Confirmed that issue came from the implementation in airflow, issue cannot be
> solved by just passing `decode` to parameter in docker.pull method in docker
> api.
> *[Solution]* .
> For now, I use try-catch to run the original implementation, and in the
> exception part I split the message to list and then parse it. Looking for
> simpler solution to this non critical but still blocking point.
> *[Logs]*
> {docker_operator.py:188}
> INFO - Starting docker container from image hello-world
> {docker_operator.py:202}
> INFO - Pulling docker image hello-world
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Pulling fs layer
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Download complete
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Pull complete
> {docker_operator.py:207}
> INFO - Digest:
> sha256:92695bc579f31df7a63da6922075d0666e565ceccad16b59c3374d2cf4e8e50e
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Digest:
> sha256:1a67c1115b199aa9d964d5da5646917cbac2d5450c71a1deed7b1bfb79c2c82d
> {models.py:1788}
> ERROR - Extra data: line 2 column 1 (char 70)
> Traceback (most recent call last):
> line 1657, in _run_raw_task, result = task_copy.execute(context=context)
> line 205, in execute output = json.loads(line)
> line 354, in loads, return _default_decoder.decode(s)
> line 342, in decode, raise JSONDecodeError("Extra data", s, end)
> json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 70)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
