I think FAB sounds like the right approach. Waiting to hear back with notes on AirBNB H2 discussion to see if they want to take this up.
@Gurer, any idea when this will happen? On Thu, Jun 22, 2017 at 1:00 AM, Bolke de Bruin <[email protected]> wrote: > One downside I see from FAB is that is does not do Business Role mapping > to FAB role. I would prefer to create groups in IPA/LDAP/AD and have those > map to FAB roles instead of needing to manage that in FAB. > > B. > > > On 22 Jun 2017, at 09:36, Bolke de Bruin <[email protected]> wrote: > > > > Hi Guys, > > > > Thanks for putting the thinking in! It is about time that we get this > moving. > > > > The design looks pretty sound. One can argue about the different roles > that are required, but that will be situation dependent I guess. > > > > Implementation wise I would argue together with Max that FAB is a better > or best fit. The ER model that is being described is pretty much a copy of > a normal security model. So a reimplementation of that is 1) significant > duplication of effort and 2) bound to have bugs that have been solved in > the other framework. Moreover, FAB does have integration out of the box > with some enterprisey systems like IPA, ActiveDirectory, and LDAP. > > > > So while you argue that using FAB would increase the scope of the > proposal significantly, but I think that is not true. Using FAB would allow > you to focus on what kind of out-of-the-box permission sets and roles we > would need and maybe address some issues that FAB lacks (maybe how to deal > with non web access - ie. in DAGs, maybe Kerberos, probably how to deal > with API calls that are not CRUD). Implementation wise it probably > simplifies what we need to do. Maybe - using Max’s early POC as an example > - we can slowly move over? > > > > On a side note: Im planning to hire 2-3 ppl to work on Airflow coming > year. Improvement of Security, Enterprise Integration, Revamp UI are on the > todo list. However, this is not confirmed yet as business priorities might > change. > > > > Bolke. > > > > > >> On 15 Jun 2017, at 21:45, kalpesh dharwadkar < > [email protected]> wrote: > >> > >> @Dan: > >> > >> Thanks for your feedback. I will remove the REFRESH_DAG permission. > >> > >> @Max: > >> > >> Thanks for your response. > >> > >> The scope of my proposal was just to add RBAC security feature to > Airflow > >> without replacing any existing frameworks. > >> > >> I understand that adopting FAB would serve Airflow better moving > forward, > >> however porting Airflow to using FAB significantly increases the scope > of > >> the proposal and I don't have the time and expertise to carry out the > tasks > >> in the extended scope. > >> > >> Hence, I'm curious to know if there's a plan for Airflow to migrate to > FAB > >> this year? > >> > >> - Kalpesh > >> > >> On Mon, Jun 12, 2017 at 6:16 PM, Maxime Beauchemin < > >> [email protected]> wrote: > >> > >>> It would be nice to go with a framework for this. I did some > >>> experimentation using FlaskAppBuilder to go in this direction. It > provides > >>> auth on different authentication backends out of the box (oauth, > openid, > >>> ldap, registration, ...), generates perms for each view that has an > >>> @has_access decorator, generates at set of perms for each ORM model > (show, > >>> edit, delete, add, ...) and enforces it in the CRUD views as well as > in the > >>> generated REST api that you get for free as a byprdoduct of deriving > FAB's > >>> models (essentially it's SqlAlchemy with a layer on top). > >>> > >>> I started a POC on FAB here a while ago: > >>> https://github.com/mistercrunch/airflow_webserver at the time my main > >>> motivation was the free/instantaneous REST api. > >>> > >>> I think FAB is a decent fit as the porting should be fairly > straightforward > >>> (moving the flask views over and deprecating Flask-Admin in favor of > FAB's > >>> crud) though there was a few blockers. From memory I think FAB didn't > like > >>> the compound PKs we use in some of the Airflow models. We'd have to > either > >>> write a db migration script on the Airflow side, or add support for > >>> compound keys to FAB (I recently became a maintainer of the project, > so I > >>> could help with that) > >>> > >>> The only downside of FAB is that it's not as mature as something like > >>> Django, but porting to Django would surely be much more work. > >>> > >>> Then there's the flask-security suite, but that looks like a bit of a > >>> patchwork to me, I guess we can pick and choose which we want to use. > >>> > >>> Max > >>> > >>> On Mon, Jun 12, 2017 at 12:50 PM, Dan Davydov < > >>> [email protected]> wrote: > >>> > >>>> Looks good to me in general, thanks for putting this together! > >>>> > >>>> I think the ability to integrate with external RBAC systems like LDAP > is > >>>> important (i.e. the Airflow DB should not be decoupled with the RBAC > >>>> database wherever possible). > >>>> > >>>> I wouldn't be too worried about the permissions about refreshing > DAGs, as > >>>> far as I know this functionality is no longer required with the new > >>>> webservers which reload state periodically, and will certainly be > removed > >>>> when we have a better DAG consistency story. > >>>> > >>>> I think it would also be good to think about this > proposal/implementation > >>>> and how it applied in the API-driven world (e.g. when webserver hits > APIs > >>>> like /clear on behalf of users instead of running commands against the > >>>> database directly). > >>>> > >>>> On Mon, Jun 12, 2017 at 11:12 AM, Bolke de Bruin <[email protected]> > >>>> wrote: > >>>> > >>>>> Will respond but im traveling at the moment. Give me a few days. > >>>>> > >>>>> Sent from my iPhone > >>>>> > >>>>>> On 12 Jun 2017, at 13:39, Chris Riccomini <[email protected]> > >>>> wrote: > >>>>>> > >>>>>> Hey all, > >>>>>> > >>>>>> Checking in on this. We spent a good chunk of time thinking about > >>> this, > >>>>> and > >>>>>> want to move forward with it, but want to make sure we're all on the > >>>> same > >>>>>> page. > >>>>>> > >>>>>> Max? Bolke? Dan? Jeremiah? > >>>>>> > >>>>>> Cheers, > >>>>>> Chris > >>>>>> > >>>>>> On Thu, Jun 8, 2017 at 1:49 PM, kalpesh dharwadkar < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> Hello everyone, > >>>>>>> > >>>>>>> As you all know, currently Airflow doesn’t have a built-in Role > >>> Based > >>>>>>> Access Control(RBAC) capability. It does provide very limited > >>>>>>> authorization capability by providing admin, data_profiler, and > user > >>>>> roles. > >>>>>>> However, associating these roles to authenticated identities is not > >>> a > >>>>>>> simple effort. > >>>>>>> > >>>>>>> To address this issue, I have created a design proposal for > building > >>>>> RBAC > >>>>>>> into Airflow and simplifying user access management via the Airflow > >>>> UI. > >>>>>>> > >>>>>>> The design proposal is located at https://cwiki.apache.org/ > >>>>>>> confluence/display/AIRFLOW/Airflow+RBAC+proposal > >>>>>>> > >>>>>>> Any comments/questions/feedback are much appreciated. > >>>>>>> > >>>>>>> Thanks > >>>>>>> Kalpesh > >>>>>>> > >>>>> > >>>> > >>> > > > >
