[
https://issues.apache.org/jira/browse/IMPALA-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17745484#comment-17745484
]
ASF subversion and git services commented on IMPALA-12300:
----------------------------------------------------------
Commit 0bee563073ef04b04446a2d1acf279ac4bc596e2 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0bee56307 ]
IMPALA-12300: Turn CheckEffectiveInstanceCount to print warning
Scheduler::CheckEffectiveInstanceCount was added to check consistency
between FE planning and BE scheduling if COMPUTE_PROCESSING_COST=true.
This consistency can be broken if there is a cluster membership
change (new executor becomes online) between FE planning and BE
scheduling. Say, in executor group size 10 with 90% health threshold,
admission-controller is allowed to run a query when only 9 executor is
available. If 10th executor is online during the time between FE
planning and BE scheduling, CheckEffectiveInstanceCount can fail and
return error.
This patch turn two error status in CheckEffectiveInstanceCount into
warning, either to query profile as InfoString or WARNING log.
MAX_FRAGMENT_INSTANCES_PER_NODE violation check stays to return error.
Testing:
- Add test_75_percent_availability
- Pass test_executors.py
Change-Id: Ieaf6a46c4f12dbf8b03d1618c2f090ab4f2ac665
Reviewed-on: http://gerrit.cloudera.org:8080/20231
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Turn Scheduler::CheckEffectiveInstanceCount to print warning
> -------------------------------------------------------------
>
> Key: IMPALA-12300
> URL: https://issues.apache.org/jira/browse/IMPALA-12300
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
>
> Scheduler::CheckEffectiveInstanceCount was added to check consistency between
> FE planning and BE scheduling if COMPUTE_PROCESSING_COST=true. This
> consistency can be broken if there is a cluster membership change (new
> executor becomes online) between FE planning and BE scheduling. Say, in
> executor group size 10 with 90% health threshold, admission-controller is
> allowed to run a query when only 9 executor is available. If 10th executor is
> online during BE scheduling, CheckEffectiveInstanceCount can fail and return
> error.
> It should be OK to turn this error form CheckEffectiveInstanceCount into
> warning since query plan for fewer executor should generally OK to run in
> larger executor group size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
