Great work Joy. This is awesome! I am interested in helping out the per dag level access. Just created a ticket to check(AIRFLOW-2267). Let me know if you have any suggestions. I will share my proposal once I am ready.
On Fri, Mar 23, 2018 at 6:45 PM, Joy Gao <j...@wepay.com> wrote: > Hey guys! > > The RBAC UI <https://github.com/apache/incubator-airflow/pull/3015> has > been merged to master. I'm looking forward to early adopters' feedback and > bug reports. I also hope to have more folks helping out with the RBAC UI, > especially with introducing DAG-Level access control, which is a feature > that a lot of people have been asking. If you are interested in helping out > with this effort, let's talk more! > > This commit will be in the 1.10.0 release, and we are going to maintain > both UIs simultaneously for a short period of time. Once RBAC UI is stable > and battle-tested, we will deprecate the old UI and eventually remove it > from the repo (around Airflow 2.0.0 or 2.1.0 release). This is to prevent > two UIs from forking into separate paths, as that would become very > difficult to maintain. > > Going forward while both UIs are up, if you are making a change to any > files in airflow/www/ (old UI), where applicable, please also make the > change to the airflow/www_rbac/ (new UI). If you rather not make changes in > both UIs, it is recommended that you only make the changes to the RBAC UI, > since that is the one we are maintaining in the long term. > > I'm excited that the RBAC UI will be able to bring additional security to > Airflow, and with FAB framework in place we can look into leveraging it for > a unified set of APIs used by both UI and CLI. > > Joy > > > > On Thu, Feb 8, 2018 at 11:31 AM, Joy Gao <j...@wepay.com> wrote: > > > Hi folks, > > > > I have a PR <https://github.com/apache/incubator-airflow/pull/3015> out > > for the new UI. I've included instructions on how to test it out in the > PR > > description. Looking forward to your feedbacks. > > > > Cheers, > > Joy > > > > On Fri, Dec 1, 2017 at 6:18 PM, Joy Gao <j...@wepay.com> wrote: > > > >> Thanks for the background info. Would be really awesome for you to have > >> PyPi access :D I'll make the change to have Airflow Webserver's FAB > >> dependency pointing to my fork for the mean time. > >> > >> For folks who are interested in RBAC, I will be giving a talk/demo at > the Airflow > >> Meet-Up > >> <https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating- > Meetup/events/244525050/> > >> next Monday. Happy to chat afterwards about it as well :) > >> > >> On Thu, Nov 30, 2017 at 8:36 AM, Maxime Beauchemin < > >> maximebeauche...@gmail.com> wrote: > >> > >>> A bit of related history here: > >>> https://github.com/dpgaspar/Flask-AppBuilder/issues/399 > >>> > >>> On Thu, Nov 30, 2017 at 8:33 AM, Maxime Beauchemin < > >>> maximebeauche...@gmail.com> wrote: > >>> > >>> > Given I have merge rights on FAB I could probably do another round of > >>> > review and get your PRs through. I would really like to get the main > >>> > maintainer's input on things that touch the core (composite-key > >>> support) as > >>> > he might have concerns/intuitions that we can't know about. > >>> > > >>> > I do not have Pypi access though so I cannot push new releases out. I > >>> > could ask for that. > >>> > > >>> > I've threatened to fork the project before, that's always an option. > >>> I've > >>> > noticed his involvement is sporadic and comes in bursts. > >>> > > >>> > In the meantime, you can have the dependency in Airflow Webserver > >>> pointing > >>> > straight to your fork. > >>> > > >>> > Max > >>> > > >>> > On Wed, Nov 29, 2017 at 7:02 PM, Joy Gao <j...@wepay.com> wrote: > >>> > > >>> >> I just created a new webserver instance if you haven't gotten a > >>> chance to > >>> >> fiddle around with the new web UI and the RBAC configurations > (thanks > >>> >> Maxime for getting started with this earlier!): > >>> >> > >>> >> http://104.209.38.171:8080/ > >>> >> > >>> >> Admin Account > >>> >> username: admin > >>> >> password: admin > >>> >> > >>> >> Read-Only Account > >>> >> username: viewer > >>> >> password: password > >>> >> > >>> >> > >>> >> On Wed, Nov 29, 2017 at 2:58 PM, Joy Gao <j...@wepay.com> wrote: > >>> >> > >>> >> > Hi folks, > >>> >> > > >>> >> > Thanks for all the feedback regarding to the new Airflow Webserver > >>> UI > >>> >> > <https://github.com/wepay/airflow-webserver/>! I've been actively > >>> >> > addressing all the bugs that were raised on Github. So I want to > >>> take > >>> >> this > >>> >> > opportunity to discuss two issues coming up: > >>> >> > > >>> >> > The first issue is unaddressed PRs in FAB. If these PRs continue > to > >>> stay > >>> >> > unaddressed, RBAC is blocked from making further progress. If this > >>> >> continue > >>> >> > to be an issue, I'm inclined to fork FAB, even though it's not > >>> >> idealistic. > >>> >> > > >>> >> > > >>> >> > - PR/631 <https://github.com/dpgaspar/F > lask-AppBuilder/pull/631> > >>> >> Binary > >>> >> > column support (merged, unreleased) > >>> >> > <https://github.com/dpgaspar/Flask-AppBuilder/pull/631> > >>> >> > - PR/639 <https://github.com/dpgaspar/F > lask-AppBuilder/pull/639> > >>> >> Composite > >>> >> > primary key support (unmerged) > >>> >> > - PR/655 <https://github.com/dpgaspar/F > lask-AppBuilder/pull/655> > >>> >> Form > >>> >> > prefill support (unmerged) > >>> >> > > >>> >> > > >>> >> > The second issue is an open question about the next step of > Airflow > >>> >> > Webserver itself. Here are the 3 potential directions we could > >>> take, and > >>> >> > I've added my thought on each. > >>> >> > > >>> >> > 1. Permanently keep Airflow Webserver as a separated package from > >>> >> Airflow, > >>> >> > and treat it as another UI option. Keep `www` in Airflow. Allow > >>> >> development > >>> >> > on both UIs. > >>> >> > *I'm not a fan of this. When there is an existing UI in Airflow, > >>> most > >>> >> > contributors would prefer to maintain the official version that is > >>> >> > installed out-of-the-box. **Having a second UI outside of Airflow > >>> will > >>> >> > make maintaining it very difficult, leading to an eventual death > of > >>> the > >>> >> new > >>> >> > UI :(* > >>> >> > > >>> >> > 2. Permanently keep Airflow Webserver as a separated package from > >>> >> Airflow, > >>> >> > but freeze all development on `www` and direct all future UI > >>> >> development > >>> >> > to Airflow Webserver, eventually removing `www` completely when > >>> Airflow > >>> >> > Webserver is stable. > >>> >> > *I'm not a fan of this either. First of all, the views and models > >>> are > >>> >> > tightly coupled in both old and new UI; until we have a > full-fledged > >>> >> REST > >>> >> > API to build the UI (and cli) on top of it, separating them to a > >>> >> separate > >>> >> > package now will potentially cause dependency issues and add > >>> >> complication > >>> >> > to our release cycle. **Secondly, **majority of Airflow users run > >>> >> Airflow > >>> >> > with the UI; it's one of Airflow's best features. Separating UI > out > >>> of > >>> >> > Airflow core will complicate setup and configuration, while making > >>> >> Airflow > >>> >> > core less complete.* > >>> >> > > >>> >> > 3. Merge Airflow Webserver back into Airflow as `www2`, freeze all > >>> >> > development on `www`, eventually removing `www` completely when > >>> `www2` > >>> >> is > >>> >> > stable. > >>> >> > *This makes the most sense to me. Airflow Webserver is developed > >>> with > >>> >> the > >>> >> > goal of feature parity to the current UI, plus additional RBAC > >>> >> capability, > >>> >> > in hope to replace the old UI completely. Yes, this means there > >>> will be > >>> >> a > >>> >> > short period of having to maintain two UIs, but once we freeze > >>> >> development > >>> >> > on www, it shouldn't be a concern for long.* > >>> >> > > >>> >> > I'd love to hear everyone's thoughts on this! I'm excited about > >>> bringing > >>> >> > RBAC to airflow and I hope it's something others will find useful > as > >>> >> well! > >>> >> > > >>> >> > Cheers, > >>> >> > Joy > >>> >> > > >>> >> > On Mon, Nov 20, 2017 at 11:24 AM, Joy Gao <j...@wepay.com> wrote: > >>> >> > > >>> >> >> Thank you everyone for the active feedback so far, and thanks for > >>> >> setting > >>> >> >> up the demo Maxime! > >>> >> >> > >>> >> >> Going to work on pruning through the issues in the upcoming days. > >>> >> >> > >>> >> >> Fokko/Maxime, do you recall the SQLAlchemy Exception message so I > >>> can > >>> >> >> look into it? Otherwise I'll wait until it's down again =P > >>> >> >> > >>> >> >> Cheers, > >>> >> >> > >>> >> >> Joy > >>> >> >> > >>> >> >> On Mon, Nov 20, 2017 at 9:35 AM, Maxime Beauchemin < > >>> >> >> maximebeauche...@gmail.com> wrote: > >>> >> >> > >>> >> >>> I just restarted it, not sure how long it will take to get in a > >>> bad > >>> >> state > >>> >> >>> again... > >>> >> >>> > >>> >> >>> Max > >>> >> >>> > >>> >> >>> On Sun, Nov 19, 2017 at 11:55 PM, Driesprong, Fokko > >>> >> <fo...@driesprong.frl > >>> >> >>> > > >>> >> >>> wrote: > >>> >> >>> > >>> >> >>> > Good morning, > >>> >> >>> > > >>> >> >>> > The demo provided by Max is down, it throws a > >>> SQLAlchemyexception > >>> >> :'( > >>> >> >>> > > >>> >> >>> > Cheers, Fokko > >>> >> >>> > > >>> >> >>> > 2017-11-18 19:14 GMT+01:00 Chris Riccomini < > >>> criccom...@apache.org>: > >>> >> >>> > > >>> >> >>> > > @bolke, open issues on the Github repo, please. > >>> >> >>> > > > >>> >> >>> > > On Sat, Nov 18, 2017 at 10:13 AM, Bolke de Bruin < > >>> >> bdbr...@gmail.com> > >>> >> >>> > > wrote: > >>> >> >>> > > > >>> >> >>> > > > Chris, > >>> >> >>> > > > > >>> >> >>> > > > Do you want us to report bugs somewhere (I have > encountered > >>> a > >>> >> >>> few)? Or > >>> >> >>> > > > just generic user experiences posted here? > >>> >> >>> > > > > >>> >> >>> > > > Cheers > >>> >> >>> > > > Bolke > >>> >> >>> > > > > >>> >> >>> > > > > On 18 Nov 2017, at 00:47, Chris Riccomini < > >>> >> criccom...@apache.org > >>> >> >>> > > >>> >> >>> > > wrote: > >>> >> >>> > > > > > >>> >> >>> > > > > Hey all, > >>> >> >>> > > > > > >>> >> >>> > > > > I know the weekend is coming up, and for those of us in > >>> the > >>> >> US, > >>> >> >>> next > >>> >> >>> > > week > >>> >> >>> > > > > is a bit of a slow holiday week. Would love to get some > >>> >> feedback > >>> >> >>> from > >>> >> >>> > > > > everyone on this. The goal would ideally to be to > >>> converge on > >>> >> >>> this > >>> >> >>> > and > >>> >> >>> > > > > eventually replace the existing Airflow UI with this > one. > >>> >> >>> > > > > > >>> >> >>> > > > > Cheers, > >>> >> >>> > > > > Chris > >>> >> >>> > > > > > >>> >> >>> > > > > On Fri, Nov 17, 2017 at 1:44 PM, Joy Gao < > j...@wepay.com> > >>> >> wrote: > >>> >> >>> > > > > > >>> >> >>> > > > >> Hi guys. > >>> >> >>> > > > >> > >>> >> >>> > > > >> I've been working on moving airflow from Flask-Admin to > >>> >> >>> > > Flask-AppBuilder > >>> >> >>> > > > >> for RBAC > >>> >> >>> > > > >> <https://cwiki.apache.org/confluence/display/AIRFLOW/ > >>> >> >>> > > > Airflow+RBAC+proposal > >>> >> >>> > > > >>> , > >>> >> >>> > > > >> check it out at https://github.com/wepay/airfl > >>> ow-webserver. > >>> >> >>> > > > >> > >>> >> >>> > > > >> It's still a work-in-progress, but most features you > see > >>> in > >>> >> the > >>> >> >>> > > > webserver > >>> >> >>> > > > >> UI today is available there. For those who are > >>> interested in > >>> >> >>> RBAC, > >>> >> >>> > I'd > >>> >> >>> > > > love > >>> >> >>> > > > >> to get some early feedback in terms of the following: > >>> >> >>> > > > >> > >>> >> >>> > > > >> - New Flask-AppBuilder UI (any bugs/regressions) > >>> >> >>> > > > >> - Setup issues > >>> >> >>> > > > >> - Ease of integration with third party auth (i.e. LDAP, > >>> AD, > >>> >> >>> OAuth, > >>> >> >>> > > etc.) > >>> >> >>> > > > >> - Any other thoughts/concerns > >>> >> >>> > > > >> > >>> >> >>> > > > >> Thanks a lot! > >>> >> >>> > > > >> > >>> >> >>> > > > >> Cheers, > >>> >> >>> > > > >> Joy > >>> >> >>> > > > >> > >>> >> >>> > > > > >>> >> >>> > > > > >>> >> >>> > > > >>> >> >>> > > >>> >> >>> > >>> >> >> > >>> >> >> > >>> >> > > >>> >> > >>> > > >>> > > >>> > >> > >> > > >
