[
https://issues.apache.org/jira/browse/IMPALA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080917#comment-18080917
]
ASF subversion and git services commented on IMPALA-12679:
----------------------------------------------------------
Commit 06629bf0f0828e0e6c95e9883e4b118d345e0441 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=06629bf0f ]
IMPALA-12679: Deflake rpc_count check in TestFetch.test_rows_sent_counters
The test expects the RPCCount in profile is in range of [5, 9]. However,
in parallel e2e tests the cluster is busy. The query needs more time to
run into the FINISHED state, which leads to the client (test code) sends
more GetOperationStatus RPCs to check the query state. The total number
of RPCs could exceed 9.
To count the RPCs, this patch improves wait_for_impala_state() to also
return the number of state checks that have been performed. Also
saves one RPC from get_runtime_profile() by using the runtime_profile in
fetch result directly. The expected rpc_count is num_state_checks + 5.
The additional 5 RPCs are GetResultSetMetadata, FetchResults * 2,
GetLog, GetRuntimeProfile.
Note that Impyla (0.21a3) uses two FetchResults RPCs to fetch the
results. It doesn't check hasMoreRows=false in the response of the first
FetchResults, so sends an additional FetchResults to get an empty batch
then stop.
Testing
- Ran TestFetch.test_rows_sent_counters 20 times.
- Ran tests in custom_cluster/test_admission_controller.py and
custom_cluster/test_events_custom_configs.py that use
wait_for_any_impala_state()
Change-Id: Icb790895e4ed31eda0b858fb6f1484eca401af73
Reviewed-on: http://gerrit.cloudera.org:8080/24288
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> test_rows_sent_counters failed to match RPCCount
> ------------------------------------------------
>
> Key: IMPALA-12679
> URL: https://issues.apache.org/jira/browse/IMPALA-12679
> Project: IMPALA
> Issue Type: Bug
> Reporter: Michael Smith
> Assignee: Kurt Deschler
> Priority: Major
> Attachments: test_rows_sent_counters.profile.txt
>
>
> {code}
> query_test.test_fetch.TestFetch.test_rows_sent_counters[protocol: beeswax |
> exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none]
> {code}
> failed with
> {code}
> query_test/test_fetch.py:69: in test_rows_sent_counters
> assert re.search("RPCCount: [5-9]", runtime_profile)
> E assert None
> E + where None = <function search at 0x7fbec6fa7578>('RPCCount: [5-9]',
> 'Query (id=c8476e5c065757bf:b436769800000000):\n DEBUG MODE WARNING: Query
> profile created while running a DEBUG buil...: 0.000ns\n -
> WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
> WriteIoWaitTime: 0.000ns\n')
> E + where <function search at 0x7fbec6fa7578> = re.search
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
