jens-scheffler-bosch commented on code in PR #33355:
URL: https://github.com/apache/airflow/pull/33355#discussion_r1323595923
##########
airflow/operators/python.py:
##########
@@ -606,7 +614,60 @@ def _prepare_venv(self, venv_path: Path) -> None:
index_urls=self.index_urls,
)
+ def _calculate_cache_hash(self) -> str:
Review Comment:
There is no cache validation implemented and I assume we should not
implement any. If somebody "messes up" the venv then it is anyway hard to track
it down.
Requirements file and pip.conf for the install will be kept inside the
cached venv is somebody want to debug.
For the moment I'd like to keep the complexity on this level, if there is
further demand or issues raised, I think then we can take a look how to harden.
I'd propose if venvs are used, then you should start first with a "normal"
one and if you use it often, then the cached version is a performance tuning.
But it is not a silver bullet for all sorts of problems. Also there is no
pruning and check for sufficient disk space.
The hash is a "simple" approach making the folder from the requirements list
idempotent. Hopefully it is only "write once, use many"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
