After testing with Daniell I added fixes for a few "teething" problems. I think the version we have now is pretty solid. If there is anyone willing to test it still (ideally on Linux), that would be great.
Docs are here: https://github.com/PolideaInternal/airflow/blob/ms-travis-ci-tests/CONTRIBUTING.md#integration-test-development-environment You can check-out the branch at: https://github.com/PolideaInternal/airflow/tree/ms-travis-ci-tests Thanks Daniel again for your help! I hope we can get it merged really soon. J. On Mon, Jul 8, 2019 at 8:22 AM Jarek Potiuk <[email protected]> wrote: > OK. The problem is likely solved. It was due to having host-installed > `apache_airflow.egg-info` with Werkzeug dependency still there mounted to > docker container. > > The latest commit in my branch solves it - only mounting what's really > needed. So it should be no problem any more (at least for the new > environment). Separate mail with Werkzeug for local virtualenvs follows. > > J. > > On Mon, Jul 8, 2019 at 7:56 AM Jarek Potiuk <[email protected]> > wrote: > >> First feedback (thanks to Daniel) - due to the werkzeug dependency >> recent removal - there might be a problem with running the test environment >> - I am looking into it - but if you try and have "Werkzeug dependency" >> problem when you try it - I am working on it. But if you try and you do not >> have the problem - also let me know - that might be helpful because I am >> looking for the reason why it works for me and does not work for Daniel. >> >> Thanks Daniel for helping with it ! >> >> J >> >> On Sun, Jul 7, 2019 at 9:21 PM Daniel Imberman <[email protected]> >> wrote: >> >>> Hi Jarek, >>> >>> I will look into this today and tomorrow. Will report back any >>> feedback/findings :). >>> >>> Daniel >>> >>> On Sun, Jul 7, 2019 at 6:47 AM Jarek Potiuk <[email protected]> >>> wrote: >>> >>>> Hello everyone, >>>> >>>> I think after the last round of reviews from Ash, I will be quite ready >>>> this week to merge the final 3/3 step of the new CI Docker image. >>>> >>>> Since it impacts integration-testing workflow of all Airflow >>>> developers, I have a kind request to people who develop Airflow to test it >>>> before we merge it to master. >>>> >>>> The CI builds were extensively tested over the past few months so I am >>>> quite confident it will be ok for Travis, but there are quite a few >>>> improvements for local development. >>>> >>>> Among others you will be able to: >>>> >>>> - Pull the CI images from DockerHub and build your local version >>>> that you will be use for testing via "local_ci_build.sh" (and force pull >>>> with another script) >>>> - Run "local_ci_run_airflow_testing.sh" <test_name> to run a test >>>> - Run "local_ci_enter_environment.sh" to enter the local IT >>>> environment being 1-1 to what you have in Travis CI and run tests more >>>> quickly (also with --skip-db-init flag for even faster runs) >>>> - Run static code analysis (mypy/pylint/flake8/doc build/licence >>>> check) in 1-1 the same way as you run them in Travis using " >>>> ci_pylint.sh/ci_mypy.sh ...." >>>> >>>> I prepared the documentation update in CONTRIBUTING about "Integration >>>> test development environment" >>>> https://github.com/PolideaInternal/airflow/blob/ms-travis-ci-tests/CONTRIBUTING.md#integration-test-development-environment >>>> >>>> You can check the branch out at >>>> https://github.com/PolideaInternal/airflow/tree/ms-travis-ci-tests >>>> >>>> There are more improvements to come/propose soon - pre-commits, >>>> possibly full "simplified development environment" (which follows next as >>>> AIP-7), production-optimised official Airflow image, but it is already >>>> quite a change (improvement I believe) in the development workflow for >>>> Airflow. >>>> >>>> I would love to hear if AIP-10 works as intended for you as well as for >>>> the few people who used it/tested so far. Please let me know your >>>> comments/questions - I am happy to answer them. Especially if you have >>>> Linux desktop - I would love to get it tested. >>>> >>>> If I have enough comments till Monday evening, I would love to merge it >>>> on Tuesday. >>>> >>>> J >>>> >>>> >>>> -- >>>> >>>> Jarek Potiuk >>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>> >>>> M: +48 660 796 129 <+48660796129> >>>> [image: Polidea] <https://www.polidea.com/> >>>> >>>> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
