kaxil removed a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#issuecomment-572729963 > @ashb @dimberman @gerardo -> I merged all the smaller PRs separated out from the Kind change and I am working on making the tests work for this one. While trying to fix and tests the kubernetes tests I noticed how difficult it was to iterate and re-test Kubernetes tests, so this one is now optimising even more: > > 1. Only the basic image for CI is built outside and sent to the docker. It is sent via untarred folder rather than as whole file. This makes it much nicer for the slow Mac Host->Docker volume mounting because the untarred image save contains separate .tars for all Docker image layers. I sync the layer dir via rsync based on checksum - this way I make sure than whenever the CI image is rebuilt, only the last rebuilt layers are changed. Then when the image is imported to DinD only the last layers are updated. this way while the environment is up and running and images are loaded to DiND and cluster, only incremental changes will be loaded. > 2. Then I came back to building the kubernetes image inside breeze container. I will speed it up more, but it is actually much better for iterating when you are inside the breeze container. In order to reload last image to the cluster you will just need to run "rebuild_ariflow_image.sh" inside the container and it will (incrementally of course) rebuild and sent the new image layers to DiND and then to cluster. > > I am still working on fixing the tests but I think it is much more developer-friendly than before. Nice work @potiuk
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
