[jira] [Commented] (IMPALA-13634) ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
[
https://issues.apache.org/jira/browse/IMPALA-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930227#comment-17930227
]
ASF subversion and git services commented on IMPALA-13634:
--
Commit 841ac3c14b5e92d04b9c065d50a9f0db7485b013 in impala's branch
refs/heads/branch-4.5.0 from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=841ac3c14 ]
IMPALA-13747: Use fresh HS2 client for unique_database fixture
Just like IMPALA-13634 bug, unique_database fixture may taint
ImpalaTestSuite.client by setting 'sync_ddl=True' option and never
cleaning it up for test method usage.
This patch fixes unique_database fixture by always creating fresh HS2
client during CREATE DATABASE and DROP DATABASE.
Testing:
- Caught an assert error at test_set_fallback_db_for_functions
Expected: "default.fn() unknown for database default"
Actual: "functional.fn() unknown for database functional"
In this case, shared ImpalaTestSuite.client may have changed database
via ImpalaTestSuite.run_test_case() or
ImpalaTestSuite.execute_wrapper() in other test method.
Removing unused 'vector' argument somehow fixed the issue.
- For both query_test/test_udfs.py and metadata/test_ddl.py:
- Fixed flake8 issues and remove unused pytest fixture.
- Run and pass the tests in exhaustive exploration.
Change-Id: Ib503e829552d436035c57b489ffda0d0299f8405
Reviewed-on: http://gerrit.cloudera.org:8080/22471
Reviewed-by: Riza Suminto
Tested-by: Impala Public Jenkins
> ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
>
>
> Key: IMPALA-13634
> URL: https://issues.apache.org/jira/browse/IMPALA-13634
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Fix For: Impala 4.5.0
>
>
> ImpalaTestSuite.cleanup_db() drops the database with sync_ddl=1 by default:
> {code:python}
> @classmethod
> def cleanup_db(cls, db_name, sync_ddl=1):
> cls.client.execute("use default")
> cls.client.set_configuration({'sync_ddl': sync_ddl})
> cls.client.execute("drop database if exists `" + db_name + "`
> cascade"){code}
> https://github.com/apache/impala/blob/b1628d8644d21c3b45acd8b2715a284cf5e8379b/tests/common/impala_test_suite.py#L428-L432
> Without setting back the sync_ddl option, the following tests using
> cls.client will keep using the same value. The default sync_ddl=1 slows down
> some tests like TestMetadataQueryStatements.test_describe_db(), and make them
> lose the test coverage on sync_ddl=0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13634) ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
[
https://issues.apache.org/jira/browse/IMPALA-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927843#comment-17927843
]
ASF subversion and git services commented on IMPALA-13634:
--
Commit e83a8e312ab00426db903b6ff51bb2e4d21f07f0 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e83a8e312 ]
IMPALA-13747: Use fresh HS2 client for unique_database fixture
Just like IMPALA-13634 bug, unique_database fixture may taint
ImpalaTestSuite.client by setting 'sync_ddl=True' option and never
cleaning it up for test method usage.
This patch fixes unique_database fixture by always creating fresh HS2
client during CREATE DATABASE and DROP DATABASE.
Testing:
- Caught an assert error at test_set_fallback_db_for_functions
Expected: "default.fn() unknown for database default"
Actual: "functional.fn() unknown for database functional"
In this case, shared ImpalaTestSuite.client may have changed database
via ImpalaTestSuite.run_test_case() or
ImpalaTestSuite.execute_wrapper() in other test method.
Removing unused 'vector' argument somehow fixed the issue.
- For both query_test/test_udfs.py and metadata/test_ddl.py:
- Fixed flake8 issues and remove unused pytest fixture.
- Run and pass the tests in exhaustive exploration.
Change-Id: Ib503e829552d436035c57b489ffda0d0299f8405
Reviewed-on: http://gerrit.cloudera.org:8080/22471
Reviewed-by: Riza Suminto
Tested-by: Impala Public Jenkins
> ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
>
>
> Key: IMPALA-13634
> URL: https://issues.apache.org/jira/browse/IMPALA-13634
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Fix For: Impala 4.5.0
>
>
> ImpalaTestSuite.cleanup_db() drops the database with sync_ddl=1 by default:
> {code:python}
> @classmethod
> def cleanup_db(cls, db_name, sync_ddl=1):
> cls.client.execute("use default")
> cls.client.set_configuration({'sync_ddl': sync_ddl})
> cls.client.execute("drop database if exists `" + db_name + "`
> cascade"){code}
> https://github.com/apache/impala/blob/b1628d8644d21c3b45acd8b2715a284cf5e8379b/tests/common/impala_test_suite.py#L428-L432
> Without setting back the sync_ddl option, the following tests using
> cls.client will keep using the same value. The default sync_ddl=1 slows down
> some tests like TestMetadataQueryStatements.test_describe_db(), and make them
> lose the test coverage on sync_ddl=0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13634) ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
[
https://issues.apache.org/jira/browse/IMPALA-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17927209#comment-17927209
]
ASF subversion and git services commented on IMPALA-13634:
--
Commit e4b5f36003dffb73256c5e614f88ceb5b64e1972 in impala's branch
refs/heads/branch-4.5.0 from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e4b5f3600 ]
IMPALA-13747: Use fresh HS2 client for unique_database fixture
Just like IMPALA-13634 bug, unique_database fixture may taint
ImpalaTestSuite.client by setting 'sync_ddl=True' option and never
cleaning it up for test method usage.
This patch fixes unique_database fixture by always creating fresh HS2
client during CREATE DATABASE and DROP DATABASE.
Testing:
- Caught an assert error at test_set_fallback_db_for_functions
Expected: "default.fn() unknown for database default"
Actual: "functional.fn() unknown for database functional"
In this case, shared ImpalaTestSuite.client may have changed database
via ImpalaTestSuite.run_test_case() or
ImpalaTestSuite.execute_wrapper() in other test method.
Removing unused 'vector' argument somehow fixed the issue.
- For both query_test/test_udfs.py and metadata/test_ddl.py:
- Fixed flake8 issues and remove unused pytest fixture.
- Run and pass the tests in exhaustive exploration.
Change-Id: Ib503e829552d436035c57b489ffda0d0299f8405
Reviewed-on: http://gerrit.cloudera.org:8080/22471
Reviewed-by: Riza Suminto
Tested-by: Impala Public Jenkins
> ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
>
>
> Key: IMPALA-13634
> URL: https://issues.apache.org/jira/browse/IMPALA-13634
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Fix For: Impala 4.5.0
>
>
> ImpalaTestSuite.cleanup_db() drops the database with sync_ddl=1 by default:
> {code:python}
> @classmethod
> def cleanup_db(cls, db_name, sync_ddl=1):
> cls.client.execute("use default")
> cls.client.set_configuration({'sync_ddl': sync_ddl})
> cls.client.execute("drop database if exists `" + db_name + "`
> cascade"){code}
> https://github.com/apache/impala/blob/b1628d8644d21c3b45acd8b2715a284cf5e8379b/tests/common/impala_test_suite.py#L428-L432
> Without setting back the sync_ddl option, the following tests using
> cls.client will keep using the same value. The default sync_ddl=1 slows down
> some tests like TestMetadataQueryStatements.test_describe_db(), and make them
> lose the test coverage on sync_ddl=0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-13634) ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
[
https://issues.apache.org/jira/browse/IMPALA-13634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17913177#comment-17913177
]
ASF subversion and git services commented on IMPALA-13634:
--
Commit fdc43466350db4437b3e917d4ff24dac58af63c3 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fdc434663 ]
IMPALA-13634: ImpalaTestSuite.cleanup_db() should use its own client
ImpalaTestSuite.cleanup_db(cls, db_name, sync_ddl=1) is used to drop the
entire db. Currently it uses cls.client and sets sync_ddl based on the
parameter. This clears all the query options of cls.client and makes it
always run with the same sync_ddl value unless the test code explicitly
set query options again.
This patch changes cleanup_db() to use a dedicated client. Tested with
some tests that uses this method and see performance improvement:
TestName
Before After
metadata/test_explain.py::TestExplainEmptyPartition
52s 9s
query_test/test_insert_behaviour.py::TestInsertBehaviour::test_insert_select_with_empty_resultset
62s 15s
metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::test_describe_db
(exhaustive) 160s 25s
Change-Id: Icb01665bc18d24e2fce4383df87c4607cf4562f1
Reviewed-on: http://gerrit.cloudera.org:8080/22286
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> ImpalaTestSuite.cleanup_db() should keep the original sync_ddl value
>
>
> Key: IMPALA-13634
> URL: https://issues.apache.org/jira/browse/IMPALA-13634
> Project: IMPALA
> Issue Type: Bug
> Components: Test
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
>
> ImpalaTestSuite.cleanup_db() drops the database with sync_ddl=1 by default:
> {code:python}
> @classmethod
> def cleanup_db(cls, db_name, sync_ddl=1):
> cls.client.execute("use default")
> cls.client.set_configuration({'sync_ddl': sync_ddl})
> cls.client.execute("drop database if exists `" + db_name + "`
> cascade"){code}
> https://github.com/apache/impala/blob/b1628d8644d21c3b45acd8b2715a284cf5e8379b/tests/common/impala_test_suite.py#L428-L432
> Without setting back the sync_ddl option, the following tests using
> cls.client will keep using the same value. The default sync_ddl=1 slows down
> some tests like TestMetadataQueryStatements.test_describe_db(), and make them
> lose the test coverage on sync_ddl=0.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
