[
https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148700#comment-17148700
]
ASF subversion and git services commented on AIRFLOW-4363:
----------------------------------------------------------
Commit 615aa18859e529039a2116204be7a877cc493ee3 in airflow's branch
refs/heads/v1-10-test from retornam
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=615aa18 ]
[AIRFLOW-4363] Fix JSON encoding error (#8287)
(cherry picked from commit 511d98e30ded2bcce9d246b358f806cea45ebcb7)
> Encounter JSON Decode Error when using docker operator
> ------------------------------------------------------
>
> Key: AIRFLOW-4363
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4363
> Project: Apache Airflow
> Issue Type: Bug
> Environment: - Mac OS 10.14
> - python 3.6.8
> - airflow 1.10.2
> Reporter: Ben Chen
> Assignee: Ben Chen
> Priority: Blocker
> Fix For: 1.10.10
>
>
> *[Description]* .
> When using the docker_operator, I experienced some issue while using Mac OS
> 10.14.4. The error was json.JSONDecodeError. After my investigation about
> this error, I found that there are several messages for logging aren't well
> separated, for example it contains \n inside one single message which should
> be split into 2 to more different messages.
> *[Update]*
> Confirmed that issue came from the implementation in airflow, issue cannot be
> solved by just passing `decode` to parameter in docker.pull method in docker
> api.
> *[Solution]* .
> For now, I use try-catch to run the original implementation, and in the
> exception part I split the message to list and then parse it. Looking for
> simpler solution to this non critical but still blocking point.
> *[Logs]*
> {docker_operator.py:188}
> INFO - Starting docker container from image hello-world
> {docker_operator.py:202}
> INFO - Pulling docker image hello-world
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Pulling fs layer
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Download complete
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Pull complete
> {docker_operator.py:207}
> INFO - Digest:
> sha256:92695bc579f31df7a63da6922075d0666e565ceccad16b59c3374d2cf4e8e50e
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Digest:
> sha256:1a67c1115b199aa9d964d5da5646917cbac2d5450c71a1deed7b1bfb79c2c82d
> {models.py:1788}
> ERROR - Extra data: line 2 column 1 (char 70)
> Traceback (most recent call last):
> line 1657, in _run_raw_task, result = task_copy.execute(context=context)
> line 205, in execute output = json.loads(line)
> line 354, in loads, return _default_decoder.decode(s)
> line 342, in decode, raise JSONDecodeError("Extra data", s, end)
> json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 70)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
