Update: The first version of task-credentials initializer, which supports GCP service account injection, is available now. Please feel free to check it out here <https://github.com/GoogleCloudPlatform/gke-serviceaccounts-initializer>. I'll add support for AWS_ACCESS_KEY_ID and SSH private key, kindly let me know if you have specific requirements on these credential types.
We are also implementing the corresponding KubernetesExecutor side changes as described in the original design. On Tue, Sep 12, 2017 at 12:15 PM, Feng Lu <[email protected]> wrote: > Thank you Maxime for the confirmation, good suggestion on the use of > policy function! > > On Mon, Sep 11, 2017 at 9:16 AM, Maxime Beauchemin < > [email protected]> wrote: > >> Hi, >> >> The proposal seems rational to me. `BaseOperator.executor_config` seems >> like a good [new] place to put this. I'd assume that in some environments >> there would be rules in the policy function >> <https://airflow.incubator.apache.org/concepts.html#cluster-policy> to >> force values in certain/all contexts. >> >> Max >> >> On Thu, Aug 31, 2017 at 10:17 PM, Feng Lu <[email protected]> >> wrote: >> >> > Sounds great, thanks a lot for setting up the meeting and will be there. >> > >> > On Thu, Aug 31, 2017 at 4:10 PM, Daniel Imberman < >> > [email protected]> >> > wrote: >> > >> > > Thank you for posting this to the wiki Feng Lu :). >> > > >> > > I'm going to propose an overall "airflow + kubernetes update" meeting >> in >> > a >> > > seperate email to discuss with the community at large. Would love it >> if >> > you >> > > could discuss this further at that meeting! >> > > >> > > Daniel >> > > >> > > On Wed, Aug 30, 2017 at 10:38 PM Feng Lu <[email protected]> >> > > wrote: >> > > >> > > > Hi all, >> > > > >> > > > *TL;DR* >> > > > Airflow doesn't have adequate built-in support for managing per-task >> > > > credentials, the concept of connection helps to certain extent but >> is >> > not >> > > > very satisfactory. The current Airflow KubernetesExecutor work >> opens up >> > > the >> > > > possibility to handle task credentials at the framework level and >> > > separate >> > > > workflow business logic from credential/account management by >> > leveraging >> > > > the Kubernetes initializer mechanism. At the end of the day, a >> task/dag >> > > > only needs to specify an account name and everything else is taken >> care >> > > by >> > > > the Airflow framework in a secure fashion. >> > > > >> > > > Detailed design: >> > > > >> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/ >> > > Managing+Per-task+Credentials+in+KubernetesExecutor >> > > > >> > > > Critics and comments are welcome :-) >> > > > Thank you. >> > > > >> > > > Feng >> > > > >> > > >> > >> > >
