dsynkov commented on issue #12985: URL: https://github.com/apache/airflow/issues/12985#issuecomment-818158798
Sharing a few notes as I've run into this same issue: * The [docs](https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html#pythonvirtualenvoperator) for `PythonVirtualenvOperator` address this: > You can use the op_args and op_kwargs arguments the same way you use it in the PythonOperator. Unfortunately we currently do not support to serialize var and ti / task_instance **due to incompatibilities with the underlying library**. * I'm not sure what the exact "incompatibilities" are, but searching for open issues mentioning `PythonVirtualenvOperator` there is https://github.com/apache/airflow/issues/7870 which addresses using `dill` vs. `cloudpickle` (but doesn't mention how/if this would enable serialization of `ti`, etc.). Anyone have any insight if https://github.com/apache/airflow/issues/7870 is a pre-req to this current issue? * Also found this Jira story (https://issues.apache.org/jira/browse/AIRFLOW-2738) that's still open that seems to be addressing the same problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
