To my mind, I would expect the MVP of per-DAG RBAC to include three settings: viewing DAG state, executing or modifying DAGs, and viewing tasks within the DAG (logs/code/details). For instance we would love to expose a view of the production dataload state to our engineers, without exposing production logs or allowing them to run/modify anything.
Creating these mappings would be somewhat painful to manage, but that seems fine for an MVP. I would expect some future version to add grouping/tagging to DAGs and to users, to say e.g. "users in group X can see DAGs in group Y and modify DAGs in group Z." On Fri, Mar 30, 2018 at 1:26 PM, Brian Greene < [email protected]> wrote: > I’d think we’d have privilege ‘can_view’ etc, and then a join table (priv) > <-> (dagid) <-> (user/group). Then it’s a simple query to get the perms > for a given dag (as you list In option 2 below). > > It also makes a “secure by default” easy - a lack of entries in that table > for a dag can mean only “admin” access or some such. > > Then any dag can have any combo of permissions for any combo of users. > Adding the groups option raises complexity around nesting, so maybe skip it > for r1? > > My $.02 > > Brian > > Sent from a device with less than stellar autocorrect > > > On Mar 29, 2018, at 10:27 AM, Maxime Beauchemin < > [email protected]> wrote: > > > > Hijacking the thread further here, any thoughts on how to breakdown per > DAG > > access? > > > > Tao & I are talking about introducing per-DAG permissions and one big > > question is whether we'll need to support different operation-types at a > > per-DAG level, which changes the way we need to model the perms. > > > > First [simpler] option is to introduce one perm per DAG. If you have > access > > to 5 DAGs, and you have `can_clear` and `can_run`, you'll have homogenous > > rights on the DAGs you have access to. > > > > Second option is to have a breakdown per DAG. Meaning for each DAG we > > create a set of perms ({dag_id}_can_view, {dag_id}_can_modify, ...). So > one > > user could have modify on some DAGs, view on others, and other DAGs would > > be invisible. This could be broken down further ({dag_id}_can_clear, ...) > > but it gets hard to manage. > > > > Thoughts? > > > > Max > > > >> On Wed, Mar 28, 2018 at 10:02 PM, Tao Feng <[email protected]> wrote: > >> > >> Great work Joy. This is awesome! I am interested in helping out the per > dag > >> level access. Just created a ticket to check(AIRFLOW-2267). Let me > know if > >> you have any suggestions. I will share my proposal once I am ready. > >> > >>> On Fri, Mar 23, 2018 at 6:45 PM, Joy Gao <[email protected]> wrote: > >>> > >>> Hey guys! > >>> > >>> The RBAC UI <https://github.com/apache/incubator-airflow/pull/3015> > has > >>> been merged to master. I'm looking forward to early adopters' feedback > >> and > >>> bug reports. I also hope to have more folks helping out with the RBAC > UI, > >>> especially with introducing DAG-Level access control, which is a > feature > >>> that a lot of people have been asking. If you are interested in helping > >> out > >>> with this effort, let's talk more! > >>> > >>> This commit will be in the 1.10.0 release, and we are going to maintain > >>> both UIs simultaneously for a short period of time. Once RBAC UI is > >> stable > >>> and battle-tested, we will deprecate the old UI and eventually remove > it > >>> from the repo (around Airflow 2.0.0 or 2.1.0 release). This is to > prevent > >>> two UIs from forking into separate paths, as that would become very > >>> difficult to maintain. > >>> > >>> Going forward while both UIs are up, if you are making a change to any > >>> files in airflow/www/ (old UI), where applicable, please also make the > >>> change to the airflow/www_rbac/ (new UI). If you rather not make > changes > >> in > >>> both UIs, it is recommended that you only make the changes to the RBAC > >> UI, > >>> since that is the one we are maintaining in the long term. > >>> > >>> I'm excited that the RBAC UI will be able to bring additional security > to > >>> Airflow, and with FAB framework in place we can look into leveraging it > >> for > >>> a unified set of APIs used by both UI and CLI. > >>> > >>> Joy > >>> > >>> > >>> > >>>> On Thu, Feb 8, 2018 at 11:31 AM, Joy Gao <[email protected]> wrote: > >>>> > >>>> Hi folks, > >>>> > >>>> I have a PR <https://github.com/apache/incubator-airflow/pull/3015> > >> out > >>>> for the new UI. I've included instructions on how to test it out in > the > >>> PR > >>>> description. Looking forward to your feedbacks. > >>>> > >>>> Cheers, > >>>> Joy > >>>> > >>>>> On Fri, Dec 1, 2017 at 6:18 PM, Joy Gao <[email protected]> wrote: > >>>>> > >>>>> Thanks for the background info. Would be really awesome for you to > >> have > >>>>> PyPi access :D I'll make the change to have Airflow Webserver's FAB > >>>>> dependency pointing to my fork for the mean time. > >>>>> > >>>>> For folks who are interested in RBAC, I will be giving a talk/demo at > >>> the Airflow > >>>>> Meet-Up > >>>>> <https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating- > >>> Meetup/events/244525050/> > >>>>> next Monday. Happy to chat afterwards about it as well :) > >>>>> > >>>>> On Thu, Nov 30, 2017 at 8:36 AM, Maxime Beauchemin < > >>>>> [email protected]> wrote: > >>>>> > >>>>>> A bit of related history here: > >>>>>> https://github.com/dpgaspar/Flask-AppBuilder/issues/399 > >>>>>> > >>>>>> On Thu, Nov 30, 2017 at 8:33 AM, Maxime Beauchemin < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> Given I have merge rights on FAB I could probably do another round > >> of > >>>>>>> review and get your PRs through. I would really like to get the > >> main > >>>>>>> maintainer's input on things that touch the core (composite-key > >>>>>> support) as > >>>>>>> he might have concerns/intuitions that we can't know about. > >>>>>>> > >>>>>>> I do not have Pypi access though so I cannot push new releases > >> out. I > >>>>>>> could ask for that. > >>>>>>> > >>>>>>> I've threatened to fork the project before, that's always an > >> option. > >>>>>> I've > >>>>>>> noticed his involvement is sporadic and comes in bursts. > >>>>>>> > >>>>>>> In the meantime, you can have the dependency in Airflow Webserver > >>>>>> pointing > >>>>>>> straight to your fork. > >>>>>>> > >>>>>>> Max > >>>>>>> > >>>>>>>> On Wed, Nov 29, 2017 at 7:02 PM, Joy Gao <[email protected]> wrote: > >>>>>>>> > >>>>>>>> I just created a new webserver instance if you haven't gotten a > >>>>>> chance to > >>>>>>>> fiddle around with the new web UI and the RBAC configurations > >>> (thanks > >>>>>>>> Maxime for getting started with this earlier!): > >>>>>>>> > >>>>>>>> http://104.209.38.171:8080/ > >>>>>>>> > >>>>>>>> Admin Account > >>>>>>>> username: admin > >>>>>>>> password: admin > >>>>>>>> > >>>>>>>> Read-Only Account > >>>>>>>> username: viewer > >>>>>>>> password: password > >>>>>>>> > >>>>>>>> > >>>>>>>>> On Wed, Nov 29, 2017 at 2:58 PM, Joy Gao <[email protected]> wrote: > >>>>>>>>> > >>>>>>>>> Hi folks, > >>>>>>>>> > >>>>>>>>> Thanks for all the feedback regarding to the new Airflow > >> Webserver > >>>>>> UI > >>>>>>>>> <https://github.com/wepay/airflow-webserver/>! I've been > >> actively > >>>>>>>>> addressing all the bugs that were raised on Github. So I want to > >>>>>> take > >>>>>>>> this > >>>>>>>>> opportunity to discuss two issues coming up: > >>>>>>>>> > >>>>>>>>> The first issue is unaddressed PRs in FAB. If these PRs continue > >>> to > >>>>>> stay > >>>>>>>>> unaddressed, RBAC is blocked from making further progress. If > >> this > >>>>>>>> continue > >>>>>>>>> to be an issue, I'm inclined to fork FAB, even though it's not > >>>>>>>> idealistic. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> - PR/631 <https://github.com/dpgaspar/F > >>> lask-AppBuilder/pull/631> > >>>>>>>> Binary > >>>>>>>>> column support (merged, unreleased) > >>>>>>>>> <https://github.com/dpgaspar/Flask-AppBuilder/pull/631> > >>>>>>>>> - PR/639 <https://github.com/dpgaspar/F > >>> lask-AppBuilder/pull/639> > >>>>>>>> Composite > >>>>>>>>> primary key support (unmerged) > >>>>>>>>> - PR/655 <https://github.com/dpgaspar/F > >>> lask-AppBuilder/pull/655> > >>>>>>>> Form > >>>>>>>>> prefill support (unmerged) > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The second issue is an open question about the next step of > >>> Airflow > >>>>>>>>> Webserver itself. Here are the 3 potential directions we could > >>>>>> take, and > >>>>>>>>> I've added my thought on each. > >>>>>>>>> > >>>>>>>>> 1. Permanently keep Airflow Webserver as a separated package > >> from > >>>>>>>> Airflow, > >>>>>>>>> and treat it as another UI option. Keep `www` in Airflow. Allow > >>>>>>>> development > >>>>>>>>> on both UIs. > >>>>>>>>> *I'm not a fan of this. When there is an existing UI in Airflow, > >>>>>> most > >>>>>>>>> contributors would prefer to maintain the official version that > >> is > >>>>>>>>> installed out-of-the-box. **Having a second UI outside of > >> Airflow > >>>>>> will > >>>>>>>>> make maintaining it very difficult, leading to an eventual death > >>> of > >>>>>> the > >>>>>>>> new > >>>>>>>>> UI :(* > >>>>>>>>> > >>>>>>>>> 2. Permanently keep Airflow Webserver as a separated package > >> from > >>>>>>>> Airflow, > >>>>>>>>> but freeze all development on `www` and direct all future UI > >>>>>>>> development > >>>>>>>>> to Airflow Webserver, eventually removing `www` completely when > >>>>>> Airflow > >>>>>>>>> Webserver is stable. > >>>>>>>>> *I'm not a fan of this either. First of all, the views and > >> models > >>>>>> are > >>>>>>>>> tightly coupled in both old and new UI; until we have a > >>> full-fledged > >>>>>>>> REST > >>>>>>>>> API to build the UI (and cli) on top of it, separating them to a > >>>>>>>> separate > >>>>>>>>> package now will potentially cause dependency issues and add > >>>>>>>> complication > >>>>>>>>> to our release cycle. **Secondly, **majority of Airflow users > >> run > >>>>>>>> Airflow > >>>>>>>>> with the UI; it's one of Airflow's best features. Separating UI > >>> out > >>>>>> of > >>>>>>>>> Airflow core will complicate setup and configuration, while > >> making > >>>>>>>> Airflow > >>>>>>>>> core less complete.* > >>>>>>>>> > >>>>>>>>> 3. Merge Airflow Webserver back into Airflow as `www2`, freeze > >> all > >>>>>>>>> development on `www`, eventually removing `www` completely when > >>>>>> `www2` > >>>>>>>> is > >>>>>>>>> stable. > >>>>>>>>> *This makes the most sense to me. Airflow Webserver is developed > >>>>>> with > >>>>>>>> the > >>>>>>>>> goal of feature parity to the current UI, plus additional RBAC > >>>>>>>> capability, > >>>>>>>>> in hope to replace the old UI completely. Yes, this means there > >>>>>> will be > >>>>>>>> a > >>>>>>>>> short period of having to maintain two UIs, but once we freeze > >>>>>>>> development > >>>>>>>>> on www, it shouldn't be a concern for long.* > >>>>>>>>> > >>>>>>>>> I'd love to hear everyone's thoughts on this! I'm excited about > >>>>>> bringing > >>>>>>>>> RBAC to airflow and I hope it's something others will find > >> useful > >>> as > >>>>>>>> well! > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> Joy > >>>>>>>>> > >>>>>>>>> On Mon, Nov 20, 2017 at 11:24 AM, Joy Gao <[email protected]> > >> wrote: > >>>>>>>>> > >>>>>>>>>> Thank you everyone for the active feedback so far, and thanks > >> for > >>>>>>>> setting > >>>>>>>>>> up the demo Maxime! > >>>>>>>>>> > >>>>>>>>>> Going to work on pruning through the issues in the upcoming > >> days. > >>>>>>>>>> > >>>>>>>>>> Fokko/Maxime, do you recall the SQLAlchemy Exception message > >> so I > >>>>>> can > >>>>>>>>>> look into it? Otherwise I'll wait until it's down again =P > >>>>>>>>>> > >>>>>>>>>> Cheers, > >>>>>>>>>> > >>>>>>>>>> Joy > >>>>>>>>>> > >>>>>>>>>> On Mon, Nov 20, 2017 at 9:35 AM, Maxime Beauchemin < > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>>> I just restarted it, not sure how long it will take to get in > >> a > >>>>>> bad > >>>>>>>> state > >>>>>>>>>>> again... > >>>>>>>>>>> > >>>>>>>>>>> Max > >>>>>>>>>>> > >>>>>>>>>>> On Sun, Nov 19, 2017 at 11:55 PM, Driesprong, Fokko > >>>>>>>> <[email protected] > >>>>>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Good morning, > >>>>>>>>>>>> > >>>>>>>>>>>> The demo provided by Max is down, it throws a > >>>>>> SQLAlchemyexception > >>>>>>>> :'( > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers, Fokko > >>>>>>>>>>>> > >>>>>>>>>>>> 2017-11-18 19:14 GMT+01:00 Chris Riccomini < > >>>>>> [email protected]>: > >>>>>>>>>>>> > >>>>>>>>>>>>> @bolke, open issues on the Github repo, please. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Sat, Nov 18, 2017 at 10:13 AM, Bolke de Bruin < > >>>>>>>> [email protected]> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Chris, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Do you want us to report bugs somewhere (I have > >>> encountered > >>>>>> a > >>>>>>>>>>> few)? Or > >>>>>>>>>>>>>> just generic user experiences posted here? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Cheers > >>>>>>>>>>>>>> Bolke > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On 18 Nov 2017, at 00:47, Chris Riccomini < > >>>>>>>> [email protected] > >>>>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Hey all, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I know the weekend is coming up, and for those of us > >> in > >>>>>> the > >>>>>>>> US, > >>>>>>>>>>> next > >>>>>>>>>>>>> week > >>>>>>>>>>>>>>> is a bit of a slow holiday week. Would love to get > >> some > >>>>>>>> feedback > >>>>>>>>>>> from > >>>>>>>>>>>>>>> everyone on this. The goal would ideally to be to > >>>>>> converge on > >>>>>>>>>>> this > >>>>>>>>>>>> and > >>>>>>>>>>>>>>> eventually replace the existing Airflow UI with this > >>> one. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>> Chris > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Fri, Nov 17, 2017 at 1:44 PM, Joy Gao < > >>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi guys. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I've been working on moving airflow from Flask-Admin > >> to > >>>>>>>>>>>>> Flask-AppBuilder > >>>>>>>>>>>>>>>> for RBAC > >>>>>>>>>>>>>>>> <https://cwiki.apache.org/ > >> confluence/display/AIRFLOW/ > >>>>>>>>>>>>>> Airflow+RBAC+proposal > >>>>>>>>>>>>>>>>> , > >>>>>>>>>>>>>>>> check it out at https://github.com/wepay/airfl > >>>>>> ow-webserver. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> It's still a work-in-progress, but most features you > >>> see > >>>>>> in > >>>>>>>> the > >>>>>>>>>>>>>> webserver > >>>>>>>>>>>>>>>> UI today is available there. For those who are > >>>>>> interested in > >>>>>>>>>>> RBAC, > >>>>>>>>>>>> I'd > >>>>>>>>>>>>>> love > >>>>>>>>>>>>>>>> to get some early feedback in terms of the following: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> - New Flask-AppBuilder UI (any bugs/regressions) > >>>>>>>>>>>>>>>> - Setup issues > >>>>>>>>>>>>>>>> - Ease of integration with third party auth (i.e. > >> LDAP, > >>>>>> AD, > >>>>>>>>>>> OAuth, > >>>>>>>>>>>>> etc.) > >>>>>>>>>>>>>>>> - Any other thoughts/concerns > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks a lot! > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>> Joy > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >> >
