[
https://issues.apache.org/jira/browse/IMPALA-12408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-12408:
-------------------------------------
Description: For queries with lot of partitions the majority of planning
time can be spent decoding compressed partition locations. This can be often
avoided, e.g. by caching FsType instead of always decompressing the path to get
its prefix:
https://github.com/apache/impala/blob/218c4c447eadb14fadb8310db4b46ab8c04cb1ba/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java#L914
(was: For queries with lot of partitions the majority of planning time can be
spent decoding compressed partition locations. This can be often avoid, e.g. by
caching FsType instead of always decompressing the path to get its prefix:
https://github.com/apache/impala/blob/218c4c447eadb14fadb8310db4b46ab8c04cb1ba/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java#L914)
> Planner spends lot of time in HdfsPartition.getLocation()
> ---------------------------------------------------------
>
> Key: IMPALA-12408
> URL: https://issues.apache.org/jira/browse/IMPALA-12408
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
> Labels: performance
>
> For queries with lot of partitions the majority of planning time can be spent
> decoding compressed partition locations. This can be often avoided, e.g. by
> caching FsType instead of always decompressing the path to get its prefix:
> https://github.com/apache/impala/blob/218c4c447eadb14fadb8310db4b46ab8c04cb1ba/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java#L914
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
