potiuk commented on issue #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3] URL: https://github.com/apache/airflow/pull/4938#issuecomment-511140442 Hey @ashb - I fixed all comments (or explained) - all are so far in subsequent commits (I split them per fix to easier to review) However I also added one more commit: e6c979b670cc475164d0e85cc7766302b11f1689 as result of testing done by Stephan (when I asked to test it). Stephan noticed (and I fixed) those problems. I think they are important to fix before it gets into hands of developers as this will avoid many first-time issues repeated/reported by many people: * Stephan had some initial struggles with understanding which scripts should be run on host and which in container so I added checks for that (the scripts will print meaningful error if they are run in host but should be in container or the other way round). I also updated docs to emphasise it. I also added more examples with default values for python/backend etc. Plus I added some examples on how to run mypy/pylint/flake when you are inside the environment. * Stephan also had python2.7 set as default so (again I had it implemented in pre-commit hooks) I made python 3.6 as default in local scripts even if python2.7 is default in the path. I am also forcing python3.6 when running pylint/mypy/flake/docs - so that they are always run with the same version of python. * Also he noticed quite some overhead for running pytest/mypy/pylint with checking the docker for re-builds. I have already optimised it in another commit (pre-commit). so I brought it in (uses md5sum to check if files changed) and now the image for pylint/mypy/flake8/docs is only rebuilt when any of the "important" files change (Docker/setup.py etc.). Otherwise it will use already built image - which is almost instanteneuos. I also sped-up pylint along the way (in Mac container it run very long because it was scanning the whole mounted sources - which included 1000s of files in node_modules - and with osxfs being slow it added 20 seconds or so just for 'find' - now it is almost instanteneous. You can also run those static checks for an individual module/file - and I gave some examples how to do it in the docs. * I added VERBOSE environment variable - now the scripts are far less verbose by default but you can enable verbosity by setting VERBOSE="true" - this way if somoene will have a problem, setting VERBOSE might help to send the more detailed output. I hope you find those changes are really needed for first-time users @ashb :).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
