[
https://issues.apache.org/jira/browse/IMPALA-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468053#comment-16468053
]
ASF subversion and git services commented on IMPALA-6227:
---------------------------------------------------------
Commit ab2fc5c8b894ef7332d9e4307eacc7842d986aae in impala's branch
refs/heads/master from [[email protected]]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=ab2fc5c ]
IMPALA-6227: reduce window of metric inconsistency
The admission controller test fetches multiple metrics relating to the
admission controller. Before this patch it fetched the whole metrics
list for each metric, meaning there was a substantial window for
the metrics to be inconsistent for a single backend. Now the metrics are
only fetched once. Metric updates are not transactional so there is
still a small window for raciness if an admission decision is made
exactly when the metrics are fetched.
Also try to detect the specific race between updating "dequeued"
and "admitted" that we saw in practice, since the race is still
possible with a smaller window. In that case we retry getting
the metrics.
Change-Id: I2f16edbec53e49446c4c37ef5f926eedb5604319
Reviewed-on: http://gerrit.cloudera.org:8080/10330
Reviewed-by: Tim Armstrong <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> TestAdmissionControllerStress can be flaky
> ------------------------------------------
>
> Key: IMPALA-6227
> URL: https://issues.apache.org/jira/browse/IMPALA-6227
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.11.0
> Reporter: Csaba Ringhofer
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: flaky
> Attachments: TEST-impala-custom-cluster.xml
>
>
> jenkins build https://jenkins.impala.io/job/gerrit-verify-dryrun/1503/console
> failed at the following test:
> {noformat}
> 01:30:11 ] =================================== FAILURES
> ===================================
> 01:30:11 ] TestAdmissionControllerStress.test_mem_limit[num_queries: 30 |
> submission_delay_ms: 0 | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none | round_robin_submission: True]
> 01:30:11 ] custom_cluster/test_admission_controller.py:877: in test_mem_limit
> 01:30:11 ] {'request_pool': self.pool_name, 'mem_limit': query_mem_limit})
> 01:30:11 ] custom_cluster/test_admission_controller.py:760: in
> run_admission_test
> 01:30:11 ] assert metric_deltas['rejected'] ==\
> 01:30:11 ] E assert 5 == ((30 - 15) - 15)
> {noformat}
> This is probably related to the following recent commit:
> https://github.com/apache/incubator-impala/commit/7487c5de04c2c5d97b8a8d5c935d10568f1ed686
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
