potiuk commented on code in PR #25780: URL: https://github.com/apache/airflow/pull/25780#discussion_r956377404
########## docs/apache-airflow/howto/operator/python.rst: ########## @@ -89,6 +89,36 @@ If additional parameters for package installation are needed pass them in ``requ All supported options are listed in the `requirements file format <https://pip.pypa.io/en/stable/reference/requirements-file-format/#supported-options>`_. +.. _howto/operator:PythonPreexistingVirtualenvOperator: + +PythonPreexistingVirtualenvOperator +=================================== + +The PythonPreexistingVirtualenvOperator can help you to run some of your tasks with a different set of Python +libraries than other tasks (and than the main Airflow environment). + +Use the :class:`~airflow.operators.python.PythonPreexistingVirtualenvOperator` to execute Python callables inside a +pre-defined virtual environment. The virtualenv should be preinstalled in the environment where +Python is run and in case ``dill`` is used, it has to be preinstalled in the virtualenv (the same +version that is installed in main Airflow environment). + +.. exampleinclude:: /../../airflow/example_dags/example_python_operator.py + :language: python + :dedent: 4 + :start-after: [START howto_operator_preexisting_virtualenv] + :end-before: [END howto_operator_preexisting_virtualenv] + +Passing in arguments +^^^^^^^^^^^^^^^^^^^^ + +You can use the ``op_args`` and ``op_kwargs`` arguments the same way you use it in the PythonOperator. +Unfortunately we currently do not support to serialize ``var`` and ``ti`` / ``task_instance`` due to incompatibilities +with the underlying library. For Airflow context variables make sure that Airflow is also installed as part +of the virtualenv environment in the same version as the Airflow version the task is run on. +Otherwise you won't have access to the most context variables of Airflow in ``op_kwargs``. +If you want the context related to datetime objects like ``data_interval_start`` you can add ``pendulum`` and +``lazy_object_proxy`` to your virtualenv. Review Comment: It is used to get Context resolved properly I believe. This is taken directly from virtualenv documentation and I think it still holds (@uranusjr ?). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
