Hi, The proposal seems rational to me. `BaseOperator.executor_config` seems like a good [new] place to put this. I'd assume that in some environments there would be rules in the policy function <https://airflow.incubator.apache.org/concepts.html#cluster-policy> to force values in certain/all contexts.
Max On Thu, Aug 31, 2017 at 10:17 PM, Feng Lu <[email protected]> wrote: > Sounds great, thanks a lot for setting up the meeting and will be there. > > On Thu, Aug 31, 2017 at 4:10 PM, Daniel Imberman < > [email protected]> > wrote: > > > Thank you for posting this to the wiki Feng Lu :). > > > > I'm going to propose an overall "airflow + kubernetes update" meeting in > a > > seperate email to discuss with the community at large. Would love it if > you > > could discuss this further at that meeting! > > > > Daniel > > > > On Wed, Aug 30, 2017 at 10:38 PM Feng Lu <[email protected]> > > wrote: > > > > > Hi all, > > > > > > *TL;DR* > > > Airflow doesn't have adequate built-in support for managing per-task > > > credentials, the concept of connection helps to certain extent but is > not > > > very satisfactory. The current Airflow KubernetesExecutor work opens up > > the > > > possibility to handle task credentials at the framework level and > > separate > > > workflow business logic from credential/account management by > leveraging > > > the Kubernetes initializer mechanism. At the end of the day, a task/dag > > > only needs to specify an account name and everything else is taken care > > by > > > the Airflow framework in a secure fashion. > > > > > > Detailed design: > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/ > > Managing+Per-task+Credentials+in+KubernetesExecutor > > > > > > Critics and comments are welcome :-) > > > Thank you. > > > > > > Feng > > > > > >
