[
https://issues.apache.org/jira/browse/AIRFLOW-5039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115798#comment-17115798
]
ASF GitHub Bot commented on AIRFLOW-5039:
-----------------------------------------
rifqifatih edited a comment on pull request #6273:
URL: https://github.com/apache/airflow/pull/6273#issuecomment-633428224
Hi @allrod5, I'm trying to overcome a similar issue and noticed that this PR
is not yet merged. Is there anything left to be done beside the unit test?
Edit: might be able to help on unit test if needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> xcom_push execution_date setting is broken when
> core.enable_xcom_pickling=False
> -------------------------------------------------------------------------------
>
> Key: AIRFLOW-5039
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5039
> Project: Apache Airflow
> Issue Type: Bug
> Components: xcom
> Affects Versions: 1.10.2, 1.10.3
> Reporter: Rodrigo Martins de Oliveira
> Assignee: Rodrigo Martins de Oliveira
> Priority: Critical
>
> Function [xcom_push from airflow.models.taskinstance automatically sets the
> execution_date to
> XCOM|https://github.com/apache/airflow/blob/7f66d0a6dc13a790b4d2d31694cb50737983f82e/airflow/models/taskinstance.py#L1336],
> while this is fine when pickling is enabled it just won't work when it isn't
> as [XCOM will attempt to JSON serialize a `datetime`
> object|https://github.com/apache/airflow/blob/7f66d0a6dc13a790b4d2d31694cb50737983f82e/airflow/models/xcom.py#L229],
> which is unsupported.
>
> This issue is critical as having pickle enabled poses security risks and [in
> environments such as Google Cloud Composer pickling is disabled for
> good|[https://cloud.google.com/composer/docs/concepts/airflow-configurations]].
> Attempting to run DAGs with tasks which pushes content to XCOM when
> `core-enable_xcom_pickling=False` in `airflow.cfg` will result in the
> following stacktrace (tested in version 1.10.2-composer):
> {quote}
> TypeError: Object of type 'datetime' is not JSON serializable
> [2019-07-24 19:14:53,193] \{models.py:1796} ERROR - Object of type
> 'datetime' is not JSON serializable
> Traceback (most recent call last):
> File "/usr/local/lib/airflow/airflow/models.py", line 1668, in _run_raw_task
> self.xcom_push(key=XCOM_RETURN_KEY, value=result)
> File "/usr/local/lib/airflow/airflow/models.py", line 2063, in xcom_push
> execution_date=execution_date or self.execution_date)
> File "/usr/local/lib/airflow/airflow/utils/db.py", line 73, in wrapper
> return func(*args, **kwargs)
> File "/usr/local/lib/airflow/airflow/models.py", line 4785, in set
> value = json.dumps(value).encode('UTF-8')
> File "/opt/python3.6/lib/python3.6/json/__init__.py", line 231, in dumps
> return _default_encoder.encode(obj)
> File "/opt/python3.6/lib/python3.6/json/encoder.py", line 199, in encode
> chunks = self.iterencode(o, _one_shot=True)
> File "/opt/python3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
> return _iterencode(o, 0)
> File "/opt/python3.6/lib/python3.6/json/encoder.py", line 180, in default
> o.__class__.__name__)
> TypeError: Object of type 'datetime' is not JSON serializable
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
