Hey Robin, Awesome, thanks! I love open source. :) Let me try your patch out and merge it.
Cheers, Chris On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin < [email protected]> wrote: > Hi Chris, > > > I think I ran into this issue when setting up LDAP Auth in our environment > (we're using very close to master as we needed some of the newer > features/bugfixes). The problem turned out to be that the search was > finding no results, so the line: > > > groups_list = [regex.search(i).group(1) for i in user_groups] > > > would fail because it had no matching groups to return. This turned out to > be because Windows Active Directory (the LDAP server we're using) returned > capitals "CN=" where the code expected lowercase: regex = > re.compile("cn=([^,]*).*") > > > I haven't looked it up, but Windows Active Directory is case insensitive > when it comes to usernames and groups, so I wouldn't be surprised if the > protocol itself is case insensitive and both "cn=" and "CN=" should be > considered valid. As such I've a PR open for a simple fix to make this > regex case insensitive: https://github.com/apache/ > incubator-airflow/pull/1945 > > > Hopefully this helps, > > Robin Miller > OLIVER WYMAN > [email protected]<mailto:robin. > [email protected]> > www.oliverwyman.com<http://www.oliverwyman.com/> > > ________________________________ > From: Chris Riccomini <[email protected]> > Sent: 05 January 2017 00:34:27 > To: [email protected] > Subject: Re: Airflow 1.8.0 alpha 2 > > I am now running 1.8.0a2 in our dev environment. It seems to be functioning > well. > > One issue we've hit is that the LDAP auth plugin isn't working for us > anymore: > > Traceback (most recent call last): > File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in > wsgi_app > response = self.full_dispatch_request() > File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in > full_dispatch_request > rv = self.handle_user_exception(e) > File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in > handle_user_exception > reraise(exc_type, exc_value, tb) > File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in > full_dispatch_request > rv = self.dispatch_request() > File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in > dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69, > in inner > return self._run_view(f, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 368, > in _run_view > return fn(self, *args, **kwargs) > File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657, > in login > return airflow.login.login(self, request) > File > "/usr/lib/python2.7/site-packages/airflow/contrib/auth/ > backends/ldap_auth.py", > line 276, in login > flask_login.login_user(LdapUser(user)) > File "<string>", line 4, in __init__ > File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line > 306, in _initialize_instance > manager.dispatch.init_failure(self, args, kwargs) > File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", > line 60, in __exit__ > compat.reraise(exc_type, exc_value, exc_tb) > File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line > 303, in _initialize_instance > return manager.original_init(*mixed[1:], **kwargs) > File > "/usr/lib/python2.7/site-packages/airflow/contrib/auth/ > backends/ldap_auth.py", > line 148, in __init__ > user.username) > File > "/usr/lib/python2.7/site-packages/airflow/contrib/auth/ > backends/ldap_auth.py", > line 106, in groups_user > groups_list = [regex.search(i).group(1) for i in user_groups] > AttributeError: 'NoneType' object has no attribute 'group' > > I believe it's from this patch: > > https://github.com/apache/incubator-airflow/commit/ > d6d3f53673ba3736d7a858531823933cfef2bb4e > > I haven't dug into it yet. We'll need to fix it, though. In the meantime, I > just commented out the call to `groups_user`. More to come. > > On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <[email protected]> wrote: > > > I have another fix that certainly need to be in the final release, but > not > > ready to merge due to failed tests: > > > > https://github.com/apache/incubator-airflow/pull/1961 > > > > > > > > > > On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <[email protected]> > > wrote: > > > > > Great, I will work toward deploying this in our dev cluster today. :D > > > > > > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <[email protected]> > > wrote: > > > > > > > Some issues remain: > > > > > > > > * one_failed not executed as dag run is marked failed seemingly > > > > prematurely (@chris yes you should see this, see below for an example > > > that > > > > is not working properly), confirmed regression > > > > * celery instability Alex > > > > * Wrong DAG state after failure inside branch > > > > > > > > So Alpha 2 is definitely not ready for production, but please do put > in > > > > your canary dags and let them run. I am still quite concerned about > the > > > > scheduler integrity and stability. > > > > > > > > - Bolke > > > > > > > > > > > > > > > > one_failed_not_executed.py > > > > ==== > > > > from airflow import DAG > > > > from airflow.operators.bash_operator import BashOperator > > > > from airflow.operators.dummy_operator import DummyOperator > > > > from datetime import datetime, timedelta > > > > > > > > default_args = { > > > > 'owner': 'airflow', > > > > 'depends_on_past': False, > > > > 'start_date': datetime(2016,10,5,19), > > > > 'email': ['[email protected]'], > > > > 'email_on_failure': False, > > > > 'email_on_retry': False, > > > > 'retries': 1, > > > > 'retry_delay': timedelta(seconds=1), > > > > } > > > > > > > > dag = DAG('tutorial', default_args=default_args, > > > schedule_interval='@once') > > > > > > > > task1 = BashOperator( > > > > task_id='first_one', > > > > bash_command='date', > > > > dag=dag) > > > > > > > > task2 = BashOperator( > > > > task_id='second_one', > > > > bash_command='this_should_not_work', > > > > dag=dag) > > > > > > > > task2.set_upstream(task1) > > > > > > > > > > > > > > > > task3 = BashOperator( > > > > task_id='third_one', > > > > bash_command='random_command_third', > > > > dag=dag) > > > > > > > > task3.set_upstream(task2) > > > > > > > > fail_task = DummyOperator( > > > > task_id='one_failed', > > > > trigger_rule='one_failed', > > > > dag=dag) > > > > > > > > fail_task.set_upstream([task1,task2,task3]) > > > > > > > > > On 4 Jan 2017, at 20:35, Chris Riccomini <[email protected]> > > > wrote: > > > > > > > > > > Bolke, can you describe the current state of the alpha 2 release? I > > saw > > > > > some comments from Alex yesterday about celery instability. If I'm > > > > running > > > > > on LocalExecutor, should I be seeing any issues? > > > > > > > > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <[email protected]> > > > > wrote: > > > > > > > > > >> Hi All, > > > > >> > > > > >> I have put up Airflow 1.8.0 alpha 2 in > https://people.apache.org/~ > > > > bolke/ < > > > > >> https://people.apache.org/~bolke/> . > > > > >> > > > > >> Note: This still cannot be considered an Apache release. Working > on > > > > this. > > > > >> > > > > >> This build is signed (note it is served over https). > > > > >> > > > > >> Changes are in the area of scheduler stability. > > > > >> > > > > >> - Bolke > > > > > > > > > > > > > -- > > _/ > > _/ Alex Van Boxel > > > > ________________________________ > This e-mail and any attachments may be confidential or legally privileged. > If you received this message in error or are not the intended recipient, > you should destroy the e-mail message and any attachments or copies, and > you are prohibited from retaining, distributing, disclosing or using any > information contained herein. Please inform us of the erroneous delivery by > return e-mail. Thank you for your cooperation. >
