[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-589365981 also applied the comments from @efedotova ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-589362536 rebased and hopefully all good now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-587763532 Applied changes from @ashb comments in #7439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-586260298 Sure. I can start a discussion and vote on it - but I think we could merge that one independently (and show it as an example) - for now it's limited only to GCP but it can show others how it can be done in an easy way. I believe AIP-4 is is much more about running the System Tests in automated way rather than separating them out (and it requires Github Actions move to be completed and some serious discussion on how to store credentials). We've already added some separation (skipIfs, _system.py in GCP) - now we have a generic mechanism but it does not mean we introduce it in "global wey" I think it's best if we merge this one first and then I could start a discussion on how to make the AIP-4 automation working. Actually i wanted to start another discussion right after about involving the community to add system tests to all other provider packages - so that we can automate it with AIP-4 as well. So the discussion might be quite big - having a working, generic implementation in GCP that is generic enough to be reused and to show how it works manually is a great way for people to take part in the discussion. WDYT @ashb ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-586177817 Ping. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585978655 Hey @ashb @mik-laj @nuclearpinguin -> all green for the system tests ready for backporting. BTW, I upladed the docs and added "How to write system tests" doc... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585195708 Hello @ashb @mik-laj @nuclearpinguin @kaxil @dimberman -> I updated the implementation now and it works really nicely. Breeze is now turned into "airflow_1.10.*" testing playground. It will be super-useful to automatically run system tests (currently only for google/* operators) using the latests tests and "example_dags" from "providers", but using the installed airflow version. You can run `./breeze --install-airflow-version ` . If you run it with "current" version it works as previously (current version from sources). If you run it with '1.10.2' -> '1.10.9' it will remove the current sources, will uninstall installed airflow master and will install the released version from PIP. The automated installation works out of the box for all versions: 1.10.2 -> 1.10.9. There is an easy way to test installation and run system tests of the released "backported" providers package. If you have binary wheel "providers" package prepared locally or downloaded, you just need to copy the wheel package to "files" dir. Then inside the container with installed airflow version you should be able to run: `pip install /files/.whl` to install the "providers" package. And then the example tests should run out of the box for most operators. The "/opt/airflow/tests" folders and "/providers" folders are mapped into the container (and system test class handles it properly) so that we can still modify the tests locally and run them with airflow 1.10.*. For example, right after installing the 'providers' package you should be able to run: `pytest tests/providers/google/cloud/operators/test_functions_system.py --systems google.cloud` and the system tests It will work for both - 2.0 and 1.10.* series providing that you have proper GCP credentials in "files/airflow-breeze-config/variables.env" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-584021242 @ashb -> one more thing. With few more breeze's modifications, It will actually be possible (bear with me) to also run unit tests with the old version of airflow. I plan to make it super easy to remove "sources" of airflow but leave tests and install any old version of Airflow inside the breeze docker. That will - in theory make it possible to run the "unit" and "integration" tests against old version of Airflow. But I am afraid due to this mocking it will not really work in many cases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-583912240 @ashb > (i.e. why the system bit, Vs unit test is needed to check the backport) The backported packages are already unit-tested. They are copied straight from the master where the unit tests have been run and are green in CI. There is basically no easy way - unless you want to rewrite manually all the tests - to run unit tests with back-ported operators/hooks/sensors from 2.0 in the context of Airlfow 1.10. I at least could not find any way to do it easily - but if you think you can do it easily, it would be great. I just think it is awfully lot of effort. Many of the unit tests use mocking for internal structure of packages and it would not work to convert those. And it would not make a lot of sense because we would just (again) test the same - behaviors of the operators - pretty much all the "operators/hooks/sensors" test are mocking out the communication to the external systems and test internal logic of the operators. We do not need to test it again - we know that this works. What we do not know if the operators are working with airflow-1.10 framework and whether the framework changes did not spoil anything. So we really need to test integration of the operators from 2.0 to 1.10 and the best we can do here is to run those (already unit-tested) operators in the context of 1.10 talking to the real system and see if they are still working there. We have a lot of such tests for GCP and we can fairly easiy test those 2.0 operators to run on top of airflow 1.10.x. Which is really what we want to know. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-583898055 > I've skimmed the changes and they look fine, but I don't see how this is connected to testing backported operators? I am writing devlist message shortly about it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-583890988 @nuclearpinguin @mik-laj (and @ashb / @kaxil ) - this is a change that makes it very easy to run system tests using pytest's markers in the breeze/CI environment. We will use it to test all the backported 2.0 packages on older airlfow versions. For now for Google/GCP for which we have system tests integrated, but it's very generic and we can apply it for other systems as well as we will have a possibility to run the system tests automatically in the future when we move to github actions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
