Rémy Frenoy created AIRFLOW-1583:
------------------------------------
Summary: PostgreSQL error when pushing a value bigger than 1GB
Key: AIRFLOW-1583
URL: https://issues.apache.org/jira/browse/AIRFLOW-1583
Project: Apache Airflow
Issue Type: Bug
Components: xcom
Affects Versions: Airflow 1.8, Airflow 1.7.0
Reporter: Rémy Frenoy
Assignee: Rémy Frenoy
Priority: Minor
When a task Operator or a PythonOperator returns a value, this value is
automatically pushed to XCom. This results in a postgresql error when the value
is bigger than 1GB :
ERROR: invalid memory alloc request size 1073741824
A size limit could be added on the value size in the xcom-push() function in
model.py. When value is higher than the limit size, a default value (maybe a
string explaining that the initial value was too high to be saved) could be
pushed to XCom instead of the real one, avoiding the postgresql error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
