Don't use a wrapper, because only what it does changes the value of the PATH variable. 11:16 $ type python python is /Library/Frameworks/Python.framework/Versions/2.7/bin/python 11:16 $ workon airflow 11:16 $ type python python is /Users/kamilbregula/.virtualenvs/airflow/bin/python
so if you want to use a different environment then you can run the script using the interpreter's direct specification. For example: /Users/kamilbregula/.virtualenvs/airflow/bin/python /path/to/script.py On Wed, Oct 16, 2019 at 2:06 AM Reed Villanueva <rvillanu...@ucera.org> wrote: > > If I have multiple airflow dags with some overlapping python package > dependencies, how can I keep each of these project deps. decoupled? Eg. if I > had project A and B on same server I would run each of them with something > like... > > source /path/to/virtualenv_a/activate > python script_a.py > deactivate > source /path/to/virtualenv_b/activate > python script_b.py > deactivate > > Basically, would like to run dags with the same situation (eg. each dag uses > python scripts that have may have overlapping package deps. that I would like > to develop separately (ie. not have to update all code using a package when > want to update the package just for one project)). Note, I've been using the > BashOperator to run python tasks like... > > do_stuff = BashOperator( > task_id='my_task', > bash_command='python /path/to/script.py'), > execution_timeout=timedelta(minutes=30), > dag=dag) > > Is there a way to get this working? Is there some other best-practice way > that airflow intendeds for people to address (or avoid) these kinds of > problems? > > > This electronic message is intended only for the named > recipient, and may contain information that is confidential or > privileged. If you are not the intended recipient, you are > hereby notified that any disclosure, copying, distribution or > use of the contents of this message is strictly prohibited. If > you have received this message in error or are not the named > recipient, please notify us immediately by contacting the > sender at the electronic mail address noted above, and delete > and destroy all copies of this message. Thank you.
