@gerardo thank you for setting this up. I've also been extremely interested in this as well. I've been messing with GCP VM instances in the past few weeks to try to simplify my local build as well. Would definitely be interested in helping with the AIP + implementation.
One thing I believe we should do is set up the ci base-image with all of the pip dependencies pre-loaded. A lot of time is wasted pip installing dependencies. We can auto-generate new images whenever a PR is submitted to this repository and then specify the tag in the .travis.yml when building. On the k8s side, I think we need to move away from minikube for k8s testing. I discussed in a previous email setting travis to work with GKE. I'd be careful about coupling k8s stuff too tightly with a docker infrastructure. That can get pretty dicey. I think as long as we're using a separate k8s cluster the k8s executor tests only need to gather the IP addresses + have access to the kubeconfig. On Tue, Jan 1, 2019 at 8:10 PM Gerardo Curiel <gera...@gerar.do> wrote: > Hi folks, > > I've created an AIP for simplifying Airflow's development workflow: > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-7+Simplified+development+workflow > > The goal of this proposal is to outline the work needed to make local > testing significantly easier and standardise the best practices to > contribute to the Airflow project. > > Any input on it would be greatly appreciated. > > Cheers, > > -- > Gerardo Curiel // https://gerar.do On Tue, Jan 1, 2019 at 8:10 PM Gerardo Curiel <gera...@gerar.do> wrote: > Hi folks, > > I've created an AIP for simplifying Airflow's development workflow: > > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-7+Simplified+development+workflow > > The goal of this proposal is to outline the work needed to make local > testing significantly easier and standardise the best practices to > contribute to the Airflow project. > > Any input on it would be greatly appreciated. > > Cheers, > > -- > Gerardo Curiel // https://gerar.do >
