Hi Guys, Thanks for putting the thinking in! It is about time that we get this moving.
The design looks pretty sound. One can argue about the different roles that are required, but that will be situation dependent I guess. Implementation wise I would argue together with Max that FAB is a better or best fit. The ER model that is being described is pretty much a copy of a normal security model. So a reimplementation of that is 1) significant duplication of effort and 2) bound to have bugs that have been solved in the other framework. Moreover, FAB does have integration out of the box with some enterprisey systems like IPA, ActiveDirectory, and LDAP. So while you argue that using FAB would increase the scope of the proposal significantly, but I think that is not true. Using FAB would allow you to focus on what kind of out-of-the-box permission sets and roles we would need and maybe address some issues that FAB lacks (maybe how to deal with non web access - ie. in DAGs, maybe Kerberos, probably how to deal with API calls that are not CRUD). Implementation wise it probably simplifies what we need to do. Maybe - using Max’s early POC as an example - we can slowly move over? On a side note: Im planning to hire 2-3 ppl to work on Airflow coming year. Improvement of Security, Enterprise Integration, Revamp UI are on the todo list. However, this is not confirmed yet as business priorities might change. Bolke. > On 15 Jun 2017, at 21:45, kalpesh dharwadkar <kalpeshdharwad...@gmail.com> > wrote: > > @Dan: > > Thanks for your feedback. I will remove the REFRESH_DAG permission. > > @Max: > > Thanks for your response. > > The scope of my proposal was just to add RBAC security feature to Airflow > without replacing any existing frameworks. > > I understand that adopting FAB would serve Airflow better moving forward, > however porting Airflow to using FAB significantly increases the scope of > the proposal and I don't have the time and expertise to carry out the tasks > in the extended scope. > > Hence, I'm curious to know if there's a plan for Airflow to migrate to FAB > this year? > > - Kalpesh > > On Mon, Jun 12, 2017 at 6:16 PM, Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > >> It would be nice to go with a framework for this. I did some >> experimentation using FlaskAppBuilder to go in this direction. It provides >> auth on different authentication backends out of the box (oauth, openid, >> ldap, registration, ...), generates perms for each view that has an >> @has_access decorator, generates at set of perms for each ORM model (show, >> edit, delete, add, ...) and enforces it in the CRUD views as well as in the >> generated REST api that you get for free as a byprdoduct of deriving FAB's >> models (essentially it's SqlAlchemy with a layer on top). >> >> I started a POC on FAB here a while ago: >> https://github.com/mistercrunch/airflow_webserver at the time my main >> motivation was the free/instantaneous REST api. >> >> I think FAB is a decent fit as the porting should be fairly straightforward >> (moving the flask views over and deprecating Flask-Admin in favor of FAB's >> crud) though there was a few blockers. From memory I think FAB didn't like >> the compound PKs we use in some of the Airflow models. We'd have to either >> write a db migration script on the Airflow side, or add support for >> compound keys to FAB (I recently became a maintainer of the project, so I >> could help with that) >> >> The only downside of FAB is that it's not as mature as something like >> Django, but porting to Django would surely be much more work. >> >> Then there's the flask-security suite, but that looks like a bit of a >> patchwork to me, I guess we can pick and choose which we want to use. >> >> Max >> >> On Mon, Jun 12, 2017 at 12:50 PM, Dan Davydov < >> dan.davy...@airbnb.com.invalid> wrote: >> >>> Looks good to me in general, thanks for putting this together! >>> >>> I think the ability to integrate with external RBAC systems like LDAP is >>> important (i.e. the Airflow DB should not be decoupled with the RBAC >>> database wherever possible). >>> >>> I wouldn't be too worried about the permissions about refreshing DAGs, as >>> far as I know this functionality is no longer required with the new >>> webservers which reload state periodically, and will certainly be removed >>> when we have a better DAG consistency story. >>> >>> I think it would also be good to think about this proposal/implementation >>> and how it applied in the API-driven world (e.g. when webserver hits APIs >>> like /clear on behalf of users instead of running commands against the >>> database directly). >>> >>> On Mon, Jun 12, 2017 at 11:12 AM, Bolke de Bruin <bdbr...@gmail.com> >>> wrote: >>> >>>> Will respond but im traveling at the moment. Give me a few days. >>>> >>>> Sent from my iPhone >>>> >>>>> On 12 Jun 2017, at 13:39, Chris Riccomini <criccom...@apache.org> >>> wrote: >>>>> >>>>> Hey all, >>>>> >>>>> Checking in on this. We spent a good chunk of time thinking about >> this, >>>> and >>>>> want to move forward with it, but want to make sure we're all on the >>> same >>>>> page. >>>>> >>>>> Max? Bolke? Dan? Jeremiah? >>>>> >>>>> Cheers, >>>>> Chris >>>>> >>>>> On Thu, Jun 8, 2017 at 1:49 PM, kalpesh dharwadkar < >>>>> kalpeshdharwad...@gmail.com> wrote: >>>>> >>>>>> Hello everyone, >>>>>> >>>>>> As you all know, currently Airflow doesn’t have a built-in Role >> Based >>>>>> Access Control(RBAC) capability. It does provide very limited >>>>>> authorization capability by providing admin, data_profiler, and user >>>> roles. >>>>>> However, associating these roles to authenticated identities is not >> a >>>>>> simple effort. >>>>>> >>>>>> To address this issue, I have created a design proposal for building >>>> RBAC >>>>>> into Airflow and simplifying user access management via the Airflow >>> UI. >>>>>> >>>>>> The design proposal is located at https://cwiki.apache.org/ >>>>>> confluence/display/AIRFLOW/Airflow+RBAC+proposal >>>>>> >>>>>> Any comments/questions/feedback are much appreciated. >>>>>> >>>>>> Thanks >>>>>> Kalpesh >>>>>> >>>> >>> >>
