Cezary created AIRFLOW-2738:
-------------------------------
Summary: XCOM's don't work with PythonVirtualenvOperator
Key: AIRFLOW-2738
URL: https://issues.apache.org/jira/browse/AIRFLOW-2738
Project: Apache Airflow
Issue Type: Bug
Affects Versions: Airflow 1.8, Airflow 1.9.0
Reporter: Cezary
When you set _provide_context_ to True in PythonVirtualenvOperator, and try to
use xcom_push, Airflow is trying to pickle context - and throws error. __
{code:java}
Traceback (most recent call last):
File
"/home/czarek/.virtualenvs/airflow/lib/python3.6/site-packages/airflow/models.py",
line 1493, in _run_raw_task
result = task_copy.execute(context=context)
File
"/home/czarek/.virtualenvs/airflow/lib/python3.6/site-packages/airflow/operators/python_operator.py",
line 93, in execute
return_value = self.execute_callable()
File
"/home/czarek/.virtualenvs/airflow/lib/python3.6/site-packages/airflow/operators/python_operator.py",
line 249, in execute_callable
self._write_args(input_filename)
File
"/home/czarek/.virtualenvs/airflow/lib/python3.6/site-packages/airflow/operators/python_operator.py",
line 305, in _write_args
pickle.dump(arg_dict, f)
TypeError: can't pickle module objects
{code}
I tried to debug - and it seems that _macros_, _task_instance_, _ti_ and _conf_
in context are guilty in this case.
Using dill doesn't fix this problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
