Dimitris Tsirogiannis created IMPALA-6843:

             Summary: Responses to prioritizedLoad() requests are returned 
directly and not via the statestore
                 Key: IMPALA-6843
                 URL: https://issues.apache.org/jira/browse/IMPALA-6843
             Project: IMPALA
          Issue Type: Improvement
          Components: Catalog
    Affects Versions: Impala 2.11.0
            Reporter: Dimitris Tsirogiannis
            Assignee: Dimitris Tsirogiannis

Currently, when a statement (e.g. SELECT) needs to access some unloaded tables, 
it issues a prioritizedLoad() request to the catalog. The catalog loads the 
table metadata but does not respond directly to the coordinator that issued the 
request. Instead, the metadata for the newly loaded tables are broadcast via 
the statestore. The problem with this approach is that the latency of the 
response may vary significantly and may depend on the latencies of other 
unrelated metadata operations (e.g. REFRESH) that happen to be in the same 
topic update.

The response to a prioritizedLoad() request should come directly to the issuing 
coordinator. Other coordinators will receive the metadata of the newly loaded 
table via the statestore. 

This message was sent by Atlassian JIRA

Reply via email to