Re: K8S deployment operator proposal

2018-07-08 Thread Daniel Imberman
It's a really tough balancing act. On the one hand I agree that ideally we don't want to force people to user k8s's (ridiculously) verbose model. On the other hand the other solution is to hand-craft requests like this

Re: K8S deployment operator proposal

2018-07-05 Thread James Meickle
I think it's going to be an antipattern to write Python configuration in Airflow to configure a Kubernetes deployment since even a "simple" deployment would likely be more classes/objects than the DAG itself. I do like the idea of having a more featured operator than the PodOperator, but if I were

Re: K8S deployment operator proposal

2018-07-03 Thread Daniel Imberman
An example of creating a deployment using the k8s model architecture can be found here: https://github.com/kubernetes-client/python/blob/master/examples/deployment_examples.py def create_deployment_object(): # Configureate Pod template container container = client.V1Container(

K8S deployment operator proposal

2018-07-03 Thread Daniel Imberman
Hi all, Enclosed is a proposal for a kubernetes deployment management operator. I think this would be a good addition to current k8s offerings s.t. users can actually launch persistent applications from airflow DAGs. * What?* Add an operator that monitors a k8s deployment, declaring the task