[
https://issues.apache.org/jira/browse/IMPALA-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702661#comment-16702661
]
bharath v commented on IMPALA-6843:
-----------------------------------
[~alexmc_bp] Please see IMPALA-7127 for the newer architecture (v2) of the
metadata redesign. It ships with release 3.1.0 and is disabled by default. We
decided to invest more time into redesigning it to suit a multi-tenant cluster
model rather than fixing the current concurrency model which is quite tricky
(inherently by design) and error-prone. The newer architecture makes this jira
idea obsolete.
[~tarmstrong] I'm not super sure but it appears this jira (and IMPALA-6671) is
kept open because the v2 features are disabled by default and any user who runs
with out of the box defaults are still susceptible to these bugs. Once we gain
more confidence in v2 and enable them by default, we can probably mark these as
fixed.
> Responses to prioritizedLoad() requests should be returned directly and not
> via the statestore
> ----------------------------------------------------------------------------------------------
>
> Key: IMPALA-6843
> URL: https://issues.apache.org/jira/browse/IMPALA-6843
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 2.11.0
> Reporter: Dimitris Tsirogiannis
> Priority: Major
> Labels: catalog, frontend, latency, perfomance
>
> Currently, when a statement (e.g. SELECT) needs to access some unloaded
> tables, it issues a prioritizedLoad() request to the catalog. The catalog
> loads the table metadata but does not respond directly to the coordinator
> that issued the request. Instead, the metadata for the newly loaded tables
> are broadcast via the statestore. The problem with this approach is that the
> latency of the response may vary significantly and may depend on the
> latencies of other unrelated metadata operations (e.g. REFRESH) that happen
> to be in the same topic update.
> The response to a prioritizedLoad() request should come directly to the
> issuing coordinator. Other coordinators will receive the metadata of the
> newly loaded table via the statestore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
