Thank you for posting this to the wiki Feng Lu :). I'm going to propose an overall "airflow + kubernetes update" meeting in a seperate email to discuss with the community at large. Would love it if you could discuss this further at that meeting!
Daniel On Wed, Aug 30, 2017 at 10:38 PM Feng Lu <[email protected]> wrote: > Hi all, > > *TL;DR* > Airflow doesn't have adequate built-in support for managing per-task > credentials, the concept of connection helps to certain extent but is not > very satisfactory. The current Airflow KubernetesExecutor work opens up the > possibility to handle task credentials at the framework level and separate > workflow business logic from credential/account management by leveraging > the Kubernetes initializer mechanism. At the end of the day, a task/dag > only needs to specify an account name and everything else is taken care by > the Airflow framework in a secure fashion. > > Detailed design: > > https://cwiki.apache.org/confluence/display/AIRFLOW/Managing+Per-task+Credentials+in+KubernetesExecutor > > Critics and comments are welcome :-) > Thank you. > > Feng >
