[
https://issues.apache.org/jira/browse/IMPALA-11747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17910434#comment-17910434
]
Riza Suminto commented on IMPALA-11747:
---------------------------------------
I'm hitting this too here:
[https://jenkins.impala.io/job/ubuntu-20.04-from-scratch/4133/testReport/junit/query_test.test_acid/TestAcid/test_acid_compaction_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_/]
> TestAcid.test_acid_compaction is flaky
> --------------------------------------
>
> Key: IMPALA-11747
> URL: https://issues.apache.org/jira/browse/IMPALA-11747
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Priority: Major
> Attachments:
> catalogd.56643cfb8723.invalid-user.log.INFO.20221122-052015.1.gz,
> hive-metastore.log.gz, hive-server2.log.gz,
> impalad_coord_exec-0_impalad.b0cb221be0c8.invalid-user.log.INFO.20221122-052016.1.bz2,
>
> impalad_coord_exec-1_impalad.f6d753b6d706.invalid-user.log.INFO.20221122-052017.1.gz,
>
> impalad_coord_exec-2_impalad.4d15644b3cc3.invalid-user.log.INFO.20221122-052019.1.gz
>
>
> TestAcid.test_acid_compaction fails on an unrelated patch:
> https://gerrit.cloudera.org/c/19055/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/6664
> {noformat}
> query_test/test_acid.py:62: in test_acid_compaction
> self.run_test_case('QueryTest/acid-compaction', vector,
> use_db=unique_database)
> common/impala_test_suite.py:742: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:578: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:469: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E
> row_regex:'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/base_0000003_v\d+/000000_0','\d+B',''
> !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000001_0000001_0000/000000_0','2B',''
> E None !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000002_0000002_0000/000000_0','2B',''
> E None !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000003_0000003_0000/000000_0','2B',''
> E Number of rows returned (expected vs actual): 1 != 3
> {noformat}
> stderr:
> {noformat}
> SET
> client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> -- connecting to: localhost:21000
> -- connecting to localhost:21050 with impyla
> -- 2022-11-22 06:06:10,804 INFO MainThread: Closing active operation
> -- connecting to localhost:28000 with impyla
> -- 2022-11-22 06:06:10,839 INFO MainThread: Closing active operation
> SET
> client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET sync_ddl=False;
> -- executing against localhost:21000
> DROP DATABASE IF EXISTS `test_acid_compaction_5c682e9f` CASCADE;
> -- 2022-11-22 06:06:10,858 INFO MainThread: Started query
> cc4f4280c721d09e:45d406dc00000000
> SET
> client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET sync_ddl=False;
> -- executing against localhost:21000
> CREATE DATABASE `test_acid_compaction_5c682e9f`;
> -- 2022-11-22 06:06:14,245 INFO MainThread: Started query
> c246e80a313b2897:bc7ad62200000000
> -- 2022-11-22 06:06:14,308 INFO MainThread: Created database
> "test_acid_compaction_5c682e9f" for test ID
> "query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:
> beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]"
> SET
> client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> -- executing against localhost:21000
> use test_acid_compaction_5c682e9f;
> -- 2022-11-22 06:06:14,313 INFO MainThread: Started query
> 944cc29ee418013f:3684f69900000000
> SET
> client_identifier=query_test/test_acid.py::TestAcid::()::test_acid_compaction[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
> SET test_replan=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- 2022-11-22 06:06:14,317 INFO MainThread: Loading query test file:
> /home/ubuntu/Impala/testdata/workloads/functional-query/queries/QueryTest/acid-compaction.test
> -- connecting to localhost:11050 with impyla
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120e50>
> -- 2022-11-22 06:06:16,810 INFO MainThread: Closing active operation
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:16,851 INFO MainThread: Closing active operation
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:38,018 INFO MainThread: Closing active operation
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:39,238 INFO MainThread: Closing active operation
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc11200d0>
> -- 2022-11-22 06:06:40,595 INFO MainThread: Closing active operation
> -- closing connection to: localhost:11050
> -- executing against localhost:21000
> invalidate metadata tt;
> -- 2022-11-22 06:06:40,652 INFO MainThread: Started query
> 524a12a9f7e2a52b:cabf7f6800000000
> -- executing against localhost:21000
> select * from tt;
> -- 2022-11-22 06:06:40,738 INFO MainThread: Started query
> 2c4445f501722473:205c850600000000
> -- connecting to localhost:11050 with impyla
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120550>
> -- 2022-11-22 06:06:40,912 INFO MainThread: Closing active operation
> -- closing query for operation handle:
> <tests.common.impala_connection.OperationHandle object at 0x7f9dc1120650>
> -- 2022-11-22 06:07:35,112 INFO MainThread: Closing active operation
> -- closing connection to: localhost:11050
> -- executing against localhost:21000
> refresh tt;
> -- 2022-11-22 06:07:35,121 INFO MainThread: Started query
> e14cbd422da3bc24:c48a680b00000000
> -- executing against localhost:21000
> select * from tt;
> -- 2022-11-22 06:07:35,194 INFO MainThread: Started query
> 0147f79f0d7cdd37:6cd2423b00000000
> -- executing against localhost:21000
> show files in tt;
> -- 2022-11-22 06:07:35,445 INFO MainThread: Started query
> a6494ba76809ffa6:e0ccc4f000000000
> -- 2022-11-22 06:07:35,474 ERROR MainThread: Comparing QueryTestResults
> (expected vs actual):
> row_regex:'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/base_0000003_v\d+/000000_0','\d+B',''
> !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000001_0000001_0000/000000_0','2B',''
> None !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000002_0000002_0000/000000_0','2B',''
> None !=
> 'hdfs://192.168.124.1:20500/test-warehouse/managed/test_acid_compaction_5c682e9f.db/tt/delta_0000003_0000003_0000/000000_0','2B',''
> Number of rows returned (expected vs actual): 1 != 3
> {noformat}
> It seems the major compaction didn't finish (or fail) so the test still saw
> the original files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
