MatrixManAtYrService opened a new issue #15952:
URL: https://github.com/apache/airflow/issues/15952
Apache Airflow version
6f8c204
Environment
OS (e.g. from /etc/os-release): Mac OS 11.3
Kernel: Darwin Kernel Version 20.4.0
Install tools: pip install -e .
### The DAG
```python3
@dag
def hello_docker():
DockerOperator(
task_id="say_hi",
image="bash:latest",
command=[
"-c",
"echo Hello World",
],
)
```
### What Happened
Operator tried to push to XCom, but throwse serialize error
```
[2021-05-19 09:13:40,044] {taskinstance.py:1280} INFO - Exporting the
following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=docker_mount
AIRFLOW_CTX_TASK_ID=add_one
AIRFLOW_CTX_EXECUTION_DATE=2021-05-19T00:00:00+00:00
AIRFLOW_CTX_DAG_RUN_ID=backfill__2021-05-19T00:00:00+00:00
[2021-05-19 09:13:40,904] {xcom.py:228} ERROR - Could not serialize the XCom
value into JSON. If you are using pickle instead of JSON for XCom, then you
need to enable pickle support for XCom in your airflow config.
[2021-05-19 09:13:40,904] {taskinstance.py:1481} ERROR - Task failed with
exception
Traceback (most recent call last):
File "/Users/matt/src/airflow/airflow/models/taskinstance.py", line 1137,
in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/Users/matt/src/airflow/airflow/models/taskinstance.py", line 1311,
in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/Users/matt/src/airflow/airflow/models/taskinstance.py", line 1344,
in _execute_task
self.xcom_push(key=XCOM_RETURN_KEY, value=result)
File "/Users/matt/src/airflow/airflow/utils/session.py", line 70, in
wrapper
return func(*args, session=session, **kwargs)
File "/Users/matt/src/airflow/airflow/models/taskinstance.py", line 1919,
in xcom_push
XCom.set(
File "/Users/matt/src/airflow/airflow/utils/session.py", line 67, in
wrapper
return func(*args, **kwargs)
File "/Users/matt/src/airflow/airflow/models/xcom.py", line 79, in set
value = XCom.serialize_value(value)
File "/Users/matt/src/airflow/airflow/models/xcom.py", line 226, in
serialize_value
return json.dumps(value).encode('UTF-8')
File
"/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py",
line 231, in dumps
return _default_encoder.encode(obj)
File
"/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py",
line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File
"/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py",
line 257, in iterencode
return _iterencode(o, 0)
File
"/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/encoder.py",
line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
[2021-05-19 09:13:40,906] {taskinstance.py:1524} INFO - Marking task as
FAILED. dag_id=docker_mount, task_id=add_one, execution_date=20210519T000000,
start_date=20210519T145759, end_date=20210519T151340
[2021-05-19 09:13:40,913] {debug_executor.py:87} ERROR - Failed to execute
task: Object of type bytes is not JSON serializable.
```
### What I Expected to Happen
Something useful ended up in XCom, and no errors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
