Hey everyone. I'm working with others on the KubernetesExecutor. As we were 
building out pieces of the executor, we realized it would be nice to have an 
integration test environment to test the bits of code involving the Kubernetes 
integration (specifically around crash safety, but also useful for regression 
purposes). 

I did some testing and found running minikube on travis is possible with 
minikube's `driver=None` mode, which deploys kubernetes locally instead of in a 
VM (see here: 
https://github.com/kubernetes/minikube#linux-continous-integration-with-vm-support).
 I spun up a minikube cluster on travis this way and it's definitely 
technically viable. 

The issue now is it looks like no other contrib executors have integration 
tests, so I'm not sure what pattern I should follow on integration testing (or 
even if airflow wants to accept integration tests for this). There would be 
relatively few tests but they would be heavyweight (ie: spin up minikube, build 
airflow + docker image, deploy airflow+postgres on kubernetes, run tests that 
spin up dags and then kill the scheduler and assert everything succeeds after 
enough time passes for recovery).

I'm open to ideas, just wanting to get a feel from the community before 
starting too much work. 


Reply via email to