Hey Bolke- To be clear, I'm not suggesting anyone is trying to do anything wrong. Release wasn't mentioned, but a new tar ball with a new version number with a 'beta' tag is published in some way for people to come and test. How is that different than the expected release/RC process (specify a git point, offer a tar ball, add an RCx tag and invite people to test that)? Seems like a parallel process with lots of similarities that could confuse both our end users and the IPMC.
Thanks, Jakob On 1 May 2018 at 13:08, Bolke de Bruin <[email protected]> wrote: > Hi Jakob, > > Understood. But isn’t that in this case not just wording? Ie. this is a > tar-ball that we think is beyond just developer testing (alpha) but more > towards the enthusiasts (beta) but not a version of the tarball that is for > the general public to test (RC) and not a Release (release)? Ie. is the issue > in calling it a ‘release’ which in this case is just meta for a tarball? In > the original email in never mentioned the word release in conjunction with > the beta I think. > > Cheers > Bolke > > >> On 1 May 2018, at 22:01, Jakob Homan <[email protected]> wrote: >> >> Hey all- >> With my Mentor hat on, I need to point out that ASF doesn't really >> have beta releases. This work is awesome, but really needs to go >> through the proper steps. The Release Candidate process is pretty >> well described: >> https://incubator.apache.org/policy/incubation.html#releases. This is >> particularly important since, as was mentioned, graduation should be >> imminent and this process will be heavily scrutinized. >> >> -Jakob >> >> On 1 May 2018 at 12:41, James Meickle <[email protected]> wrote: >>> Thanks for the pointer! I went through and set this up today, using Google >>> OAuth as the RBAC provider. Overall I'm quite enthusiastic about this move, >>> but I thought that it might be helpful to collect feedback as someone who >>> hasn't been following the overall process and is therefore coming at it >>> with fresh eyes. >>> >>> - The Flask appbuilder security documentation is poor quality (e.g., >>> there's some broken sentences); if Airflow is to send people there, it >>> might be worth PRing some of the docs to at least look more professional. >>> >>> - There's not much documentation out there on how to properly set up an >>> OAuth app in Google (in my case, using the G+ API). From an adoption POV, >>> it would be good to screenshot the (current) steps in the process, and >>> point out which values should be used in which fields on Google. For >>> example, I had to grep the code base to find the callback URL. >>> >>> - The initial login UI seems over-complex: you have to click the provider >>> icon, and then click either login or register. The standard for this >>> workflow is that you login by clicking the desired provider's icon, and >>> doing so will register you automatically if you aren't already. In my case >>> I only have one provider, so this menu was even more confusing. >>> >>> - It was not clear to me that the "Public" role has absolutely no >>> permissions. When I set this as the default role and registered, I could no >>> longer access the site until I cleared cookies. I thought it was an OAuth >>> error at first, but it turns out the Public role has fewer effective >>> permissions than an anonymous user; this resulted in a redirect loop >>> because I could not even view the homepage. I had to correct this in the >>> database to be able to log in. >>> >>> - The roles list (at roles/list/ ) is intimidatingly large and hard to >>> parse. For instance, I couldn't tell at a glance what "user" allows >>> relative to "viewer". It would be good to have a narrative description of >>> what each of these roles is intended for, and to present the list of >>> permissions in a more clustered or diffable way. Permissions lists tend to >>> only grow, after all. >>> >>> - A "Viewer" currently lacks enough access to see their own profile. >>> >>> - "User Statistics" (userstatschartview/chart/) uses the internal name, >>> rather than firstname/lastname - which in my case is a `google_idnumber` >>> name. Should probably show both names. >>> >>> Unrelatedly to RBAC (I think), on this branch on my sandbox instance, tasks >>> appear to be failing with the only logs present in the UI as: >>> >>> [{'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True}, >>> {'end_of_log': True}, {'end_of_log': True}, {'end_of_log': True}] >>> >>> >>> Finally, in case anyone else wanted to test run a similar setup, here is >>> the webserver_config.py that I ended up using (note that it has Jinja >>> templating via Ansible): >>> >>> import os >>> from airflow import configuration as conf >>> from flask_appbuilder.security.manager import AUTH_OAUTH >>> basedir = os.path.abspath(os.path.dirname(__file__)) >>> >>> # The SQLAlchemy connection string. >>> SQLALCHEMY_DATABASE_URI = conf.get('core', 'SQL_ALCHEMY_CONN') >>> >>> # Flask-WTF flag for CSRF >>> CSRF_ENABLED = True >>> >>> # The name to display, e.g. "Airflow Staging Sandbox" >>> APP_NAME = "Airflow {{ env }} {{ app_config | capitalize }}" >>> >>> # Use OAuth >>> AUTH_TYPE = AUTH_OAUTH >>> >>> # Will allow user self registration >>> AUTH_USER_REGISTRATION = True >>> >>> # The default user self registration role >>> AUTH_USER_REGISTRATION_ROLE = "{{ airflow_rbac_registration_role | >>> default('Viewer') }}" >>> >>> # Google OAuth: >>> OAUTH_PROVIDERS = [{ >>> # The name of the provider >>> 'name': 'google', >>> # The icon to use >>> 'icon': 'fa-google', >>> # The name of the key that the provider sends >>> 'token_key': 'access_token', >>> # Just in case, whitelist to only @quantopian.com emails >>> 'whitelist': ['@quantopian.com'], >>> # Define the remote app: >>> 'remote_app': { >>> 'base_url': 'https://www.googleapis.com/oauth2/v2/', >>> 'access_token_url': 'https://accounts.google.com/o/oauth2/token', >>> 'authorize_url': 'https://accounts.google.com/o/oauth2/auth', >>> 'request_token_url': None, >>> 'request_token_params': { >>> # Uses the Google+ API, requestingf the 'email' and 'profile' scope >>> 'scope': 'email profile' >>> }, >>> 'consumer_key': '{{ vault_airflow_google_oauth_key }}', >>> 'consumer_secret': '{{ vault_airflow_google_oauth_secret }}' >>> } >>> }] >>> >>> >>> >>> On Mon, Apr 30, 2018 at 12:54 PM, Jørn A Hansen <[email protected]> >>> wrote: >>> >>>> On Mon, 30 Apr 2018 at 15.56, James Meickle <[email protected]> >>>> wrote: >>>> >>>>> Installed this off of the branch, and I do get the Kubernetes executor >>>>> (incl. demo DAG) and some bug fixes - but I don't see any RBAC feature >>>>> anywhere I'd think to look. Do I need to set up some config to get that >>>> to >>>>> show up? >>>> >>>> >>>> See >>>> https://github.com/apache/incubator-airflow/blob/v1-10- >>>> test/UPDATING.md#new-webserver-ui-with-role-based-access-control >>>> >>>> It had me left wondering as well - so I decided to go hunt for it in the >>>> RBAC PR. And there it was :-) >>>> >>>> Cheers, >>>> JornH >>>> >>>>> >>>>> >>>>> On Mon, Apr 23, 2018 at 2:06 PM, Bolke de Bruin <[email protected]> >>>> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I am really happy that Fokko and I have created the v1-10-test branch >>>> and >>>>>> subsequently build the first beta of Apache Airflow 1.10! >>>>>> >>>>>> It is available for testing here: >>>>>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0beta1/ >>>>>> >>>>>> Highlights include: >>>>>> >>>>>> * New RBAC web interface in beta >>>>>> * Timezone support >>>>>> * First class kubernetes operator >>>>>> * Experimental kubernetes executor >>>>>> * Documentation improvements >>>>>> * Performance optimizations for large DAGs >>>>>> * many GCP and S3 integration improvements >>>>>> * many new operators >>>>>> * many many many bug fixes >>>>>> >>>>>> We are aiming for a fully compliant Apache release so we should be able >>>>> to >>>>>> kick off the graduation process after this release. I hope you help us >>>>> out >>>>>> getting there! >>>>>> >>>>>> Kind regards, >>>>>> >>>>>> Bolke & Fokko >>>>> >>>> >
