We avoid React for the same reasons as the ASF and use Polymer 2 instead. Would that work?
Bolke. > On 20 Jul 2017, at 19:35, Chris Riccomini <[email protected]> wrote: > > Hey Max, > > Want to come down to WePay? We can set up a zoom for those that want to > join online, and record it as well to post for the community. > > Since Joy is just getting started, and it looks like there's going to be a > K8s discussion next week, maybe we can shoot for the week after (the week > of the 31st of July)? Care to float a few times that week? > > Cheers, > Chris > > On Thu, Jul 20, 2017 at 9:31 AM, Maxime Beauchemin < > [email protected]> wrote: > >> Sounds awesome, count me in! >> >> * check out the prototype in my fork, I went far enough to hit some >> hurdles, try different workarounds. I hooked up the Airflow Bootstrap >> template too so that we feel at home in this new UI >> * using a single `id` field is a requirement for FAB that airflow doesn't >> respect (composite pks), either we add the feature to support that in FAB, >> or we align on the Airflow side and modify the models and add a migration >> script. This upgrade would require downtime and might be annoying to the >> Airflow community, but could help with db performance a bit (smaller >> index)... I probably could be convinced either way but I'm leaning on >> improving FAB >> * I'm a maintainer for FAB so I can help get stuff through there >> * React is in limbo at the ASF for licensing reasons, so no React at least >> for now >> * npm/webpack/ES6, javascript only in `.js` files >> * I vote for eslint + eslint-config-airbnb as a set of linting rules for JS >> * Keep out of apache (for now), this new app ships as its own pypi package >> `airflow-webserver`, have a period of overlap (maintaining 2 web apps) >> before ripping out `airflow/www` from the core package >> * You need to get in touch with Marty Kausas, an intern at Airbnb who's >> been working on a Flask blueprint for improved, more personalized views on >> DAGs that we were planning on merging into the main branch eventually. Some >> of Marty's idea and code could be merged into this effort. >> >> These are ideas on how I would proceed personally on this but definitely >> everything here is up for discussion. >> >> Let's meet physically at either WePay or Airbnb. Folks from the community, >> let us know on this thread if you want to be part of this effort, we'll be >> happy to include you. >> >> Thanks, >> >> Max >> >> On Wed, Jul 19, 2017 at 7:33 PM, Joy Gao <[email protected]> wrote: >> >>> Hey everyone, >>> >>> I recently transferred to Data Infra team here at WePay to focus on >>> Airflow-related initiatives. >>> >>> Given the RBAC design is mostly hashed out, I'm happy to get this feature >>> off the ground for Q3, starting with converting Airflow to Fab, if there >>> are no objections. >>> >>> Cheers, >>> Joy >>> >>> On Thu, Jun 29, 2017 at 7:32 AM, Gurer Kiratli < >>> [email protected]> wrote: >>> >>>> Hey all, >>>> >>>> We talked about this internally. We would like to work on this feature >>> but >>>> given the immediate priorities we are not going to be working on it in >>> Q3. >>>> Comes end of Q3 we will reevaluate. Likely scenario is we can work on >> it >>>> late Q4 or Q12018. >>>> >>>> Cheers, >>>> >>>> Gurer >>>> >>>> On Tue, Jun 27, 2017 at 8:08 AM, Chris Riccomini < >> [email protected]> >>>> wrote: >>>> >>>>> I think FAB sounds like the right approach. Waiting to hear back with >>>> notes >>>>> on AirBNB H2 discussion to see if they want to take this up. >>>>> >>>>> @Gurer, any idea when this will happen? >>>>> >>>>> On Thu, Jun 22, 2017 at 1:00 AM, Bolke de Bruin <[email protected]> >>>> wrote: >>>>> >>>>>> One downside I see from FAB is that is does not do Business Role >>>> mapping >>>>>> to FAB role. I would prefer to create groups in IPA/LDAP/AD and >> have >>>>> those >>>>>> map to FAB roles instead of needing to manage that in FAB. >>>>>> >>>>>> B. >>>>>> >>>>>>> On 22 Jun 2017, at 09:36, Bolke de Bruin <[email protected]> >>> wrote: >>>>>>> >>>>>>> Hi Guys, >>>>>>> >>>>>>> Thanks for putting the thinking in! It is about time that we get >>> this >>>>>> moving. >>>>>>> >>>>>>> The design looks pretty sound. One can argue about the different >>>> roles >>>>>> that are required, but that will be situation dependent I guess. >>>>>>> >>>>>>> Implementation wise I would argue together with Max that FAB is a >>>>> better >>>>>> or best fit. The ER model that is being described is pretty much a >>> copy >>>>> of >>>>>> a normal security model. So a reimplementation of that is 1) >>>> significant >>>>>> duplication of effort and 2) bound to have bugs that have been >> solved >>>> in >>>>>> the other framework. Moreover, FAB does have integration out of the >>> box >>>>>> with some enterprisey systems like IPA, ActiveDirectory, and LDAP. >>>>>>> >>>>>>> So while you argue that using FAB would increase the scope of the >>>>>> proposal significantly, but I think that is not true. Using FAB >> would >>>>> allow >>>>>> you to focus on what kind of out-of-the-box permission sets and >> roles >>>> we >>>>>> would need and maybe address some issues that FAB lacks (maybe how >> to >>>>> deal >>>>>> with non web access - ie. in DAGs, maybe Kerberos, probably how to >>> deal >>>>>> with API calls that are not CRUD). Implementation wise it probably >>>>>> simplifies what we need to do. Maybe - using Max’s early POC as an >>>>> example >>>>>> - we can slowly move over? >>>>>>> >>>>>>> On a side note: Im planning to hire 2-3 ppl to work on Airflow >>> coming >>>>>> year. Improvement of Security, Enterprise Integration, Revamp UI >> are >>> on >>>>> the >>>>>> todo list. However, this is not confirmed yet as business >> priorities >>>>> might >>>>>> change. >>>>>>> >>>>>>> Bolke. >>>>>>> >>>>>>> >>>>>>>> On 15 Jun 2017, at 21:45, kalpesh dharwadkar < >>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> @Dan: >>>>>>>> >>>>>>>> Thanks for your feedback. I will remove the REFRESH_DAG >>> permission. >>>>>>>> >>>>>>>> @Max: >>>>>>>> >>>>>>>> Thanks for your response. >>>>>>>> >>>>>>>> The scope of my proposal was just to add RBAC security feature >> to >>>>>> Airflow >>>>>>>> without replacing any existing frameworks. >>>>>>>> >>>>>>>> I understand that adopting FAB would serve Airflow better moving >>>>>> forward, >>>>>>>> however porting Airflow to using FAB significantly increases the >>>> scope >>>>>> of >>>>>>>> the proposal and I don't have the time and expertise to carry >> out >>>> the >>>>>> tasks >>>>>>>> in the extended scope. >>>>>>>> >>>>>>>> Hence, I'm curious to know if there's a plan for Airflow to >>> migrate >>>> to >>>>>> FAB >>>>>>>> this year? >>>>>>>> >>>>>>>> - Kalpesh >>>>>>>> >>>>>>>> On Mon, Jun 12, 2017 at 6:16 PM, Maxime Beauchemin < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> It would be nice to go with a framework for this. I did some >>>>>>>>> experimentation using FlaskAppBuilder to go in this direction. >> It >>>>>> provides >>>>>>>>> auth on different authentication backends out of the box >> (oauth, >>>>>> openid, >>>>>>>>> ldap, registration, ...), generates perms for each view that >> has >>> an >>>>>>>>> @has_access decorator, generates at set of perms for each ORM >>> model >>>>>> (show, >>>>>>>>> edit, delete, add, ...) and enforces it in the CRUD views as >> well >>>> as >>>>>> in the >>>>>>>>> generated REST api that you get for free as a byprdoduct of >>>> deriving >>>>>> FAB's >>>>>>>>> models (essentially it's SqlAlchemy with a layer on top). >>>>>>>>> >>>>>>>>> I started a POC on FAB here a while ago: >>>>>>>>> https://github.com/mistercrunch/airflow_webserver at the time >> my >>>>> main >>>>>>>>> motivation was the free/instantaneous REST api. >>>>>>>>> >>>>>>>>> I think FAB is a decent fit as the porting should be fairly >>>>>> straightforward >>>>>>>>> (moving the flask views over and deprecating Flask-Admin in >> favor >>>> of >>>>>> FAB's >>>>>>>>> crud) though there was a few blockers. From memory I think FAB >>>> didn't >>>>>> like >>>>>>>>> the compound PKs we use in some of the Airflow models. We'd >> have >>> to >>>>>> either >>>>>>>>> write a db migration script on the Airflow side, or add support >>> for >>>>>>>>> compound keys to FAB (I recently became a maintainer of the >>>> project, >>>>>> so I >>>>>>>>> could help with that) >>>>>>>>> >>>>>>>>> The only downside of FAB is that it's not as mature as >> something >>>> like >>>>>>>>> Django, but porting to Django would surely be much more work. >>>>>>>>> >>>>>>>>> Then there's the flask-security suite, but that looks like a >> bit >>>> of a >>>>>>>>> patchwork to me, I guess we can pick and choose which we want >> to >>>> use. >>>>>>>>> >>>>>>>>> Max >>>>>>>>> >>>>>>>>> On Mon, Jun 12, 2017 at 12:50 PM, Dan Davydov < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Looks good to me in general, thanks for putting this together! >>>>>>>>>> >>>>>>>>>> I think the ability to integrate with external RBAC systems >> like >>>>> LDAP >>>>>> is >>>>>>>>>> important (i.e. the Airflow DB should not be decoupled with >> the >>>> RBAC >>>>>>>>>> database wherever possible). >>>>>>>>>> >>>>>>>>>> I wouldn't be too worried about the permissions about >> refreshing >>>>>> DAGs, as >>>>>>>>>> far as I know this functionality is no longer required with >> the >>>> new >>>>>>>>>> webservers which reload state periodically, and will certainly >>> be >>>>>> removed >>>>>>>>>> when we have a better DAG consistency story. >>>>>>>>>> >>>>>>>>>> I think it would also be good to think about this >>>>>> proposal/implementation >>>>>>>>>> and how it applied in the API-driven world (e.g. when >> webserver >>>> hits >>>>>> APIs >>>>>>>>>> like /clear on behalf of users instead of running commands >>> against >>>>> the >>>>>>>>>> database directly). >>>>>>>>>> >>>>>>>>>> On Mon, Jun 12, 2017 at 11:12 AM, Bolke de Bruin < >>>> [email protected] >>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Will respond but im traveling at the moment. Give me a few >>> days. >>>>>>>>>>> >>>>>>>>>>> Sent from my iPhone >>>>>>>>>>> >>>>>>>>>>>> On 12 Jun 2017, at 13:39, Chris Riccomini < >>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hey all, >>>>>>>>>>>> >>>>>>>>>>>> Checking in on this. We spent a good chunk of time thinking >>>> about >>>>>>>>> this, >>>>>>>>>>> and >>>>>>>>>>>> want to move forward with it, but want to make sure we're >> all >>> on >>>>> the >>>>>>>>>> same >>>>>>>>>>>> page. >>>>>>>>>>>> >>>>>>>>>>>> Max? Bolke? Dan? Jeremiah? >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Chris >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jun 8, 2017 at 1:49 PM, kalpesh dharwadkar < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello everyone, >>>>>>>>>>>>> >>>>>>>>>>>>> As you all know, currently Airflow doesn’t have a built-in >>> Role >>>>>>>>> Based >>>>>>>>>>>>> Access Control(RBAC) capability. It does provide very >>> limited >>>>>>>>>>>>> authorization capability by providing admin, data_profiler, >>> and >>>>>> user >>>>>>>>>>> roles. >>>>>>>>>>>>> However, associating these roles to authenticated >> identities >>> is >>>>> not >>>>>>>>> a >>>>>>>>>>>>> simple effort. >>>>>>>>>>>>> >>>>>>>>>>>>> To address this issue, I have created a design proposal for >>>>>> building >>>>>>>>>>> RBAC >>>>>>>>>>>>> into Airflow and simplifying user access management via the >>>>> Airflow >>>>>>>>>> UI. >>>>>>>>>>>>> >>>>>>>>>>>>> The design proposal is located at >> https://cwiki.apache.org/ >>>>>>>>>>>>> confluence/display/AIRFLOW/Airflow+RBAC+proposal >>>>>>>>>>>>> >>>>>>>>>>>>> Any comments/questions/feedback are much appreciated. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> Kalpesh >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> >>> Joy Gao >>> Software Engineer >>> 350 Convention Way, Suite 200 >>> Redwood City, CA 94063 >>> Mobile: 669-224-9305 >>> >>> Payments partner to the platform economy >>> >>
