Any chance you can document a flow that ties the CI build artifacts to this repo and how we would use them? Would our build times go down?
-s On Sun, Aug 19, 2018 at 11:43 AM Driesprong, Fokko <fo...@driesprong.frl> wrote: > Hi all, > > Gerardo is doing some awesome work on Dockerizing the CI pipeline. The PR > is still open here: https://github.com/apache/incubator-airflow/pull/3393 > > To make this work, and not store any Docker images in private repositories, > I've created a new repository which stores the Docker image which contains > a lot of the dependencies we use to test against. The repository is here: > https://github.com/apache/incubator-airflow-ci > For most of the tests, we use mocks, but sometimes we also use the actual > service, for example for some Hadoop, Kerberos, Hive functionality. The > images are build in the Dockerhub service: > https://hub.docker.com/r/airflowci/incubator-airflow-ci/builds/ > I've created it for now. I'm open for another name, but Airflow was already > taken, and hyphens are not allowed. > > This change will greatly decouple our dependency on Travis and will make it > much easier to test the code locally. Instead of setting up tox > environments, we can just use Docker instead. I'm really enthusiastic about > the change, and really thankful for Gerardo to put all this effort in > there. > > Recently Holden Karau did a lifestream > <https://www.youtube.com/watch?v=5Il3w2P7IcM>, and did her first PR. I > think the most important learning from it was that testing was not as > trivial as it should be. Therefore I think we should see if we can merge > the PR from Gerardo soon. > > Cheers, Fokko >
