[
https://issues.apache.org/jira/browse/IMPALA-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17925280#comment-17925280
]
ASF subversion and git services commented on IMPALA-13727:
----------------------------------------------------------
Commit 4273bcb65dfbf09bcce475430d316334e7d894e1 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4273bcb65 ]
IMPALA-13727: Fix wrong regex for hostname in test_multiple_blocks_mt_dop
The test uses r'host=(\S+:[0-9]*)' to match the hosts of all the
fragment instances in query profile, i.e.
host_list = re.findall(r'host=(\S+:[0-9]*)', result.runtime_profile)
This is able to find the instance name in lines like
Instance d843c27e276bfa7a:a86450f600000006 (host=name:27002):
and get "name:27002" from it.
However, the runtime profile we get right after fetching the query
result and before closing the query might have a section of total time
after the fragment instance host, e.g.
Instance d843c27e276bfa7a:a86450f600000006 (host=name:27002):(Total:
19.999ms, non-child: 0.000ns, % non-child: 0.00%)
The regex pattern will match "name:27002):Total:" in the string. If not
all the instances have this "Total" section, the instance names mismatch
and the test will fail. See the query profile attached in the JIRA as an
example. Not sure how this happens. Maybe it's not guaranteed that the
profile is ready immediately when the last result row is fetched. There
might be a delay for the coordinator to process the final update of the
profile from executors. I think it's ok to fix the test first.
This fixes the regex pattern to be r'\(host=([^:]+:[0-9]*)\)' so it can
exactly match the instance name.
Tests
- Verified the fix locally
Change-Id: If70daa344403f2ae8617bf86e7896c2bbfd9e736
Reviewed-on: http://gerrit.cloudera.org:8080/22457
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestParquet.test_multiple_blocks_mt_dop failed by unexpected ranges_per_host
> ----------------------------------------------------------------------------
>
> Key: IMPALA-13727
> URL: https://issues.apache.org/jira/browse/IMPALA-13727
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Attachments: profile.txt
>
>
> The test could fail in any exec vectors, e.g.
> {code}
> query_test.test_scanners.TestParquet.test_multiple_blocks_mt_dop[protocol:
> beeswax | table_format: parquet/none | exec_option: {'test_replan': 1,
> 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': True, 'abort_on_error': 1, 'debug_action':
> '-1:OPEN:[email protected]',
> 'exec_single_node_rows_threshold': 0}] {code}
> Stacktrace
> {code:python}
> query_test/test_scanners.py:903: in test_multiple_blocks_mt_dop
> assert ranges_per_host[host] == 2
> E assert 1 == 2{code}
> Standard Error
> {noformat}
> SET
> client_identifier=query_test/test_scanners.py::TestParquet::()::test_multiple_blocks_mt_dop[protocol:beeswax|table_format:parquet/none|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':True;'abort_on_error':1;'de;
> SET test_replan=1;
> SET mt_dop=2;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=True;
> SET abort_on_error=1;
> SET debug_action=-1:OPEN:[email protected];
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select count(l_orderkey) from functional_parquet.lineitem_sixblocks;
> -- 2025-02-01 20:42:19,750 INFO MainThread: Started query
> 0347a7702c366f22:89a1448a00000000
> SET
> client_identifier=query_test/test_scanners.py::TestParquet::()::test_multiple_blocks_mt_dop[protocol:beeswax|table_format:parquet/none|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':True;'abort_on_error':1;'de;{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
