potiuk commented on code in PR #35160:
URL: https://github.com/apache/airflow/pull/35160#discussion_r1375559213
##########
tests/operators/test_python.py:
##########
@@ -844,17 +848,30 @@ def f(exit_code):
assert ti.state == expected_state
+venv_cache_path = tempfile.mkdtemp(prefix="venv_cache_path")
Review Comment:
And just tio update this answer 'No it's not possible TODAY". But I can
easily imagine that some time in the future we might run another set of
optimisations to improve performance sof some of our tests - and we remove DB
requirement from them - either via mocking, or maybe even by adding "per test
method" sqlite or in-memory database (I've done both in the past).
However - this is a different CI/DEV/Test optimisation step. This change
which basically aims to not rewrite the tests and change their structure but
merely split out those thests that do not need DB and add mechanism that will
keep it that way.
And yeah. I am happy to work with others on the next steps too :). I believe
that we should continuously optimize and refine our dev tooling - at least
until we grow in terms of community and usage, because that's the only way we
can keep it working :D.
So yeah some day we might want to attempt to convert those
Virtualenv/ExternalPython to non-DB tests :). One more thing though - I think
we really need to make sure it's worth it.. There is a value in hitting real DB
during those tests because they are testing "Real" integration with them. So we
should have quite a good percentage of those. I think we have a little to much
(9000 non-DB vs. 7000 DB sounds like should be 15000 non-DB vs 1500 DB so
having a clear split and numbers laid out our goal shoudl be likely to move
some of the DB tets to become non-DB ones.
I am definitely in that train :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
