martinbikandi commented on issue #56295:
URL: https://github.com/apache/airflow/issues/56295#issuecomment-3364727736
Hello, thanks for the answer.
I understand that if I'm passing some airflow-specific classes/functions as
arguments, I need airflow to be installed in the venv or system_site_packages
set to True.
I still have some doubts about why among these two tasks, the first one
fails and the other runs correctly:
```python
# fails
task_run_kwarg = PythonVirtualenvOperator(
task_id="run_kwarg",
python_callable=run_kwarg,
op_kwargs={'context': "{{dag_run.conf}}"},
system_site_packages=True,
# outlets=[dummy_dataset]
)
# works
run_kwarg_no_site_packages = PythonVirtualenvOperator(
task_id="run_kwarg_no-site-packages",
python_callable=run_kwarg,
op_kwargs={'context': "{{dag_run.conf}}"},
system_site_packages=False,
# outlets=[dummy_dataset]
)
```
The arguments that need serializing would be "python_callable" and
"op_kwargs" if I'm not mistaken (outlets is/was commented in the first four
tasks).
In this case the template {'context': "{{dag_run.conf}}"} would be an empty
dictionary as I ran the DAG without parameters, and that is what I see in the
print statement on the second task.
So, the only difference is that `system_site_packages` is true/false. The
`run_kwarg` function does not use any airflow specific class or function. The
first task still fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
