[
https://issues.apache.org/jira/browse/IMPALA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bikramjeet Vig updated IMPALA-3088:
-----------------------------------
Description:
The metrics that expose the per-pool configurations are incorrect (will be 0)
if the node has never received a request placed into this pool, i.e. it only
learns of this pool from a statestore update from another node.
This affects the pool metrics:
admission-controller.pool-max-mem-resources
admission-controller.pool-max-running
admission-controller.pool-max-queued
All of the per-pool metrics are created as soon as the admission controller
either (a) receives a request placed into the new pool or (b) receives a
statestore update for a new pool.
Impala looks up the pool configurations on path (a), which is when the metrics
are set. In case (b) the metrics are created with all other metrics but not set
(or set later if path (a) is later hit). This happens because the pool's config
metrics are only updated based on the local pool config file. Unfortunately,
pool config is only fetched if a query is issued to the coordinator, that is,
if it goes through path (a).
E.g. if we have hosts h1 and h2, and you submit a query to pool1 on h1, you'll
start seeing metrics on h1 for all of the resource pool metrics for pool1.
However, on h2, it learns about pool1 from the statestore and the metrics get
created and reflect the right values on that host _except_ for those listed
above, which are all 0.
was:
The metrics that expose the per-pool configurations are incorrect (will be 0)
if the node has never received a request placed into this pool, i.e. it only
learns of this pool from a statestore update from another node.
This affects the pool metrics:
admission-controller.pool-max-mem-resources
admission-controller.pool-max-running
admission-controller.pool-max-queued
All of the per-pool metrics are created as soon as the admission controller
either (a) receives a request placed into the new pool or (b) receives a
statestore update for a new pool.
Impala looks up the pool configurations on path (a), which is when the metrics
are set. In case (b) the metrics are created with all other metrics but not set
(or set later if path (a) is later hit).
E.g. if we have hosts h1 and h2, and you submit a query to pool1 on h1, you'll
start seeing metrics on h1 for all of the resource pool metrics for pool1.
However, on h2, it learns about pool1 from the statestore and the metrics get
created and reflect the right values on that host _except_ for those listed
above, which are all 0.
> Pool config metrics wrong on non-coordinator nodes
> --------------------------------------------------
>
> Key: IMPALA-3088
> URL: https://issues.apache.org/jira/browse/IMPALA-3088
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.5.0
> Reporter: Matthew Jacobs
> Priority: Minor
> Labels: admission-control
>
> The metrics that expose the per-pool configurations are incorrect (will be 0)
> if the node has never received a request placed into this pool, i.e. it only
> learns of this pool from a statestore update from another node.
> This affects the pool metrics:
> admission-controller.pool-max-mem-resources
> admission-controller.pool-max-running
> admission-controller.pool-max-queued
> All of the per-pool metrics are created as soon as the admission controller
> either (a) receives a request placed into the new pool or (b) receives a
> statestore update for a new pool.
> Impala looks up the pool configurations on path (a), which is when the
> metrics are set. In case (b) the metrics are created with all other metrics
> but not set (or set later if path (a) is later hit). This happens because the
> pool's config metrics are only updated based on the local pool config file.
> Unfortunately, pool config is only fetched if a query is issued to the
> coordinator, that is, if it goes through path (a).
> E.g. if we have hosts h1 and h2, and you submit a query to pool1 on h1,
> you'll start seeing metrics on h1 for all of the resource pool metrics for
> pool1. However, on h2, it learns about pool1 from the statestore and the
> metrics get created and reflect the right values on that host _except_ for
> those listed above, which are all 0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
