[
https://issues.apache.org/jira/browse/IMPALA-11891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YifanZhang updated IMPALA-11891:
--------------------------------
Description:
Currently, when updating an executor group's min size, we need to restart all
executors in this executor group and all coordinators. Restarting coordinators
will interrupt queries running on all executor groups.
It is necessary to restart coordinators because executor groups will never be
removed from a cluster during the lifetime of an impalad and inconsistent
executor group configurations will be ignored when adding a new executor:
[https://github.com/apache/impala/blob/c3ed44268838f4a00624e7e7fbf6e614f0a54a48/be/src/scheduling/executor-group.cc#L89-L93]
We can improve this by removing empty/unused executor groups from the
membership, and then new configurations of the same group names can be applied.
was:
Currently, when updating an executor group's min size, we need to restart all
executors in this executor group and all coordinators. Restarting coordinators
will interrupt queries running on all executor groups.
It is necessary to restart coordinators because executor groups will never be
removed from a cluster during the lifetime of an impalad and inconsistent
executor group configurations will be ignored when adding a new executor:
[https://github.com/apache/impala/blob/c3ed44268838f4a00624e7e7fbf6e614f0a54a48/be/src/scheduling/executor-group.cc#L90]
We can improve this by removing empty/unused executor groups from the
membership, and then new configurations of the same group names can be applied.
> Update executor group min size without restarting coordinators
> --------------------------------------------------------------
>
> Key: IMPALA-11891
> URL: https://issues.apache.org/jira/browse/IMPALA-11891
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 4.2.0
> Reporter: YifanZhang
> Priority: Minor
>
> Currently, when updating an executor group's min size, we need to restart all
> executors in this executor group and all coordinators. Restarting
> coordinators will interrupt queries running on all executor groups.
> It is necessary to restart coordinators because executor groups will never be
> removed from a cluster during the lifetime of an impalad and inconsistent
> executor group configurations will be ignored when adding a new executor:
> [https://github.com/apache/impala/blob/c3ed44268838f4a00624e7e7fbf6e614f0a54a48/be/src/scheduling/executor-group.cc#L89-L93]
>
> We can improve this by removing empty/unused executor groups from the
> membership, and then new configurations of the same group names can be
> applied.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
