[jira] [Updated] (AIRFLOW-3506) Fail to query log from elasticsearch
[ https://issues.apache.org/jira/browse/AIRFLOW-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ping Zhang updated AIRFLOW-3506: Description: We have noticed that the elasticsearch_logging_backend use match to query the log_id from elasticsearch, which ends up getting more results from elasticsearch. It should use match_phrase to query the log_id as a phrase. (was: We have noticed that the elasticsearch_logging_backend use match to query the log_id from elasticsearch, which ended up getting more results from elasticsearch. It should use match_phrase to query the log_id as a phrase.) > Fail to query log from elasticsearch > > > Key: AIRFLOW-3506 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3506 > Project: Apache Airflow > Issue Type: Bug > Components: logging >Reporter: Ping Zhang >Assignee: Ping Zhang >Priority: Major > Labels: easyfix, newbie > > We have noticed that the elasticsearch_logging_backend use match to query the > log_id from elasticsearch, which ends up getting more results from > elasticsearch. It should use match_phrase to query the log_id as a phrase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3506) Fail to query log from elasticsearch
Ping Zhang created AIRFLOW-3506: --- Summary: Fail to query log from elasticsearch Key: AIRFLOW-3506 URL: https://issues.apache.org/jira/browse/AIRFLOW-3506 Project: Apache Airflow Issue Type: Bug Components: logging Reporter: Ping Zhang Assignee: Ping Zhang We have noticed that the elasticsearch_logging_backend use match to query the log_id from elasticsearch, which ended up getting more results from elasticsearch. It should use match_phrase to query the log_id as a phrase. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (AIRFLOW-3453) Failed to run airflow resetdb
[ https://issues.apache.org/jira/browse/AIRFLOW-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ping Zhang closed AIRFLOW-3453. --- Resolution: Invalid > Failed to run airflow resetdb > - > > Key: AIRFLOW-3453 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3453 > Project: Apache Airflow > Issue Type: Bug > Components: database >Affects Versions: 1.10.1 > Environment: macOS Mojave, 10.14.1; >Reporter: Ping Zhang >Assignee: Ping Zhang >Priority: Major > > macOS Mojave, _10.14.1_; > Mysql: > _Your MySQL connection id is 6034_ > _Server version: 8.0.12 Homebrew_ > > Right after I ran `airflow initdb`, > {code:java} > > > _$ airflow resetdb -y > [2018-12-05 16:13:49,427] {settings.py:174} INFO - setting.configure_orm(): > Using pool settings. pool_size=5, pool_recycle=1800 > [2018-12-05 16:13:49,744] {_init_.py:51} INFO - Using executor CeleryExecutor > DB: mysql://root:***@localhost/airflow > [2018-12-05 16:13:49,973] {db.py:358} INFO - Dropping tables that exist > /Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: > Warning: (3719L, "'utf8' is currently an alias for the character set > UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please > consider using UTF8MB4 in order to be unambiguous.") > cursor.execute(statement, parameters) > Traceback (most recent call last): > File "/Users/ping_zhang/anaconda2/envs/apache-airflow/bin/airflow", line 4, > in > _import_('pkg_resources').run_script('apache-airflow==1.10.1', 'airflow') > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", > line 664, in run_script > self.require(requires)[0].run_script(script_name, ns) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", > line 1444, in run_script > exec(code, namespace, namespace) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/EGG-INFO/scripts/airflow", > line 32, in > args.func(args) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/bin/cli.py", > line 1020, in resetdb > db_utils.resetdb(settings.RBAC) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/utils/db.py", > line 360, in resetdb > models.Base.metadata.drop_all(settings.engine) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", > line 3976, in drop_all > tables=tables) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1929, in _run_visitor > conn._run_visitor(visitorcallable, element, **kwargs) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1538, in _run_visitor > **kwargs).traverse_single(element) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", > line 121, in traverse_single > return meth(obj, **kw) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 874, in visit_metadata > table, drop_ok=True, _is_metadata_operation=True) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", > line 121, in traverse_single > return meth(obj, **kw) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 923, in visit_table > self.connection.execute(DropTable(table)) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 945, in execute > return meth(self, multiparams, params) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 68, in _execute_on_connection > return connection._execute_ddl(self, multiparams, params) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1002, in _execute_ddl > compiled > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1189, in _execute_context > context) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1402, in _handle_dbapi_exception > exc_info > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/util/compat.py", > line 203, in raise_from_cause >
[jira] [Assigned] (AIRFLOW-3453) Failed to run airflow resetdb
[ https://issues.apache.org/jira/browse/AIRFLOW-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ping Zhang reassigned AIRFLOW-3453: --- Assignee: Ping Zhang > Failed to run airflow resetdb > - > > Key: AIRFLOW-3453 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3453 > Project: Apache Airflow > Issue Type: Bug > Components: database >Affects Versions: 1.10.1 > Environment: macOS Mojave, 10.14.1; >Reporter: Ping Zhang >Assignee: Ping Zhang >Priority: Major > > macOS Mojave, _10.14.1_; > Mysql: > _Your MySQL connection id is 6034_ > _Server version: 8.0.12 Homebrew_ > > Right after I ran `airflow initdb`, > {code:java} > > > _$ airflow resetdb -y > [2018-12-05 16:13:49,427] {settings.py:174} INFO - setting.configure_orm(): > Using pool settings. pool_size=5, pool_recycle=1800 > [2018-12-05 16:13:49,744] {_init_.py:51} INFO - Using executor CeleryExecutor > DB: mysql://root:***@localhost/airflow > [2018-12-05 16:13:49,973] {db.py:358} INFO - Dropping tables that exist > /Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: > Warning: (3719L, "'utf8' is currently an alias for the character set > UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please > consider using UTF8MB4 in order to be unambiguous.") > cursor.execute(statement, parameters) > Traceback (most recent call last): > File "/Users/ping_zhang/anaconda2/envs/apache-airflow/bin/airflow", line 4, > in > _import_('pkg_resources').run_script('apache-airflow==1.10.1', 'airflow') > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", > line 664, in run_script > self.require(requires)[0].run_script(script_name, ns) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", > line 1444, in run_script > exec(code, namespace, namespace) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/EGG-INFO/scripts/airflow", > line 32, in > args.func(args) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/bin/cli.py", > line 1020, in resetdb > db_utils.resetdb(settings.RBAC) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/utils/db.py", > line 360, in resetdb > models.Base.metadata.drop_all(settings.engine) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", > line 3976, in drop_all > tables=tables) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1929, in _run_visitor > conn._run_visitor(visitorcallable, element, **kwargs) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1538, in _run_visitor > **kwargs).traverse_single(element) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", > line 121, in traverse_single > return meth(obj, **kw) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 874, in visit_metadata > table, drop_ok=True, _is_metadata_operation=True) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", > line 121, in traverse_single > return meth(obj, **kw) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 923, in visit_table > self.connection.execute(DropTable(table)) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 945, in execute > return meth(self, multiparams, params) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", > line 68, in _execute_on_connection > return connection._execute_ddl(self, multiparams, params) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1002, in _execute_ddl > compiled > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1189, in _execute_context > context) > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", > line 1402, in _handle_dbapi_exception > exc_info > File > "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/util/compat.py", > line 203, in
[jira] [Updated] (AIRFLOW-3453) Failed to run airflow resetdb
[ https://issues.apache.org/jira/browse/AIRFLOW-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ping Zhang updated AIRFLOW-3453: Description: macOS Mojave, _10.14.1_; Mysql: _Your MySQL connection id is 6034_ _Server version: 8.0.12 Homebrew_ Right after I ran `airflow initdb`, {code:java} _$ airflow resetdb -y [2018-12-05 16:13:49,427] {settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800 [2018-12-05 16:13:49,744] {_init_.py:51} INFO - Using executor CeleryExecutor DB: mysql://root:***@localhost/airflow [2018-12-05 16:13:49,973] {db.py:358} INFO - Dropping tables that exist /Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: Warning: (3719L, "'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.") cursor.execute(statement, parameters) Traceback (most recent call last): File "/Users/ping_zhang/anaconda2/envs/apache-airflow/bin/airflow", line 4, in _import_('pkg_resources').run_script('apache-airflow==1.10.1', 'airflow') File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", line 664, in run_script self.require(requires)[0].run_script(script_name, ns) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/_init_.py", line 1444, in run_script exec(code, namespace, namespace) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/EGG-INFO/scripts/airflow", line 32, in args.func(args) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/bin/cli.py", line 1020, in resetdb db_utils.resetdb(settings.RBAC) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/utils/db.py", line 360, in resetdb models.Base.metadata.drop_all(settings.engine) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 3976, in drop_all tables=tables) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1929, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1538, in _run_visitor **kwargs).traverse_single(element) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single return meth(obj, **kw) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 874, in visit_metadata table, drop_ok=True, _is_metadata_operation=True) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single return meth(obj, **kw) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 923, in visit_table self.connection.execute(DropTable(table)) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute return meth(self, multiparams, params) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection return connection._execute_ddl(self, multiparams, params) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1002, in _execute_ddl compiled File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception exc_info File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/MySQLdb/cursors.py", line 250, in execute self.errorhandler(self, exc, value) File
[jira] [Created] (AIRFLOW-3453) Failed to run airflow resetdb
Ping Zhang created AIRFLOW-3453: --- Summary: Failed to run airflow resetdb Key: AIRFLOW-3453 URL: https://issues.apache.org/jira/browse/AIRFLOW-3453 Project: Apache Airflow Issue Type: Bug Components: database Affects Versions: 1.10.1 Environment: macOS Mojave, 10.14.1; Reporter: Ping Zhang macOS Mojave, _10.14.1_; Mysql: _Your MySQL connection id is 6034_ _Server version: 8.0.12 Homebrew_ Right after I ran `airflow initdb`, _$ airflow resetdb -y [2018-12-05 16:13:49,427] \{settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800 [2018-12-05 16:13:49,744] \{__init__.py:51} INFO - Using executor CeleryExecutor DB: mysql://root:***@localhost/airflow [2018-12-05 16:13:49,973] \{db.py:358} INFO - Dropping tables that exist /Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py:470: Warning: (3719L, "'utf8' is currently an alias for the character set UTF8MB3, but will be an alias for UTF8MB4 in a future release. Please consider using UTF8MB4 in order to be unambiguous.") cursor.execute(statement, parameters) Traceback (most recent call last): File "/Users/ping_zhang/anaconda2/envs/apache-airflow/bin/airflow", line 4, in __import__('pkg_resources').run_script('apache-airflow==1.10.1', 'airflow') File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/__init__.py", line 664, in run_script self.require(requires)[0].run_script(script_name, ns) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1444, in run_script exec(code, namespace, namespace) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/EGG-INFO/scripts/airflow", line 32, in args.func(args) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/bin/cli.py", line 1020, in resetdb db_utils.resetdb(settings.RBAC) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/apache_airflow-1.10.1-py2.7.egg/airflow/utils/db.py", line 360, in resetdb models.Base.metadata.drop_all(settings.engine) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 3976, in drop_all tables=tables) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1929, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1538, in _run_visitor **kwargs).traverse_single(element) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single return meth(obj, **kw) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 874, in visit_metadata table, drop_ok=True, _is_metadata_operation=True) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single return meth(obj, **kw) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 923, in visit_table self.connection.execute(DropTable(table)) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute return meth(self, multiparams, params) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection return connection._execute_ddl(self, multiparams, params) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1002, in _execute_ddl compiled File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception exc_info File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/Users/ping_zhang/anaconda2/envs/apache-airflow/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) File