[
https://issues.apache.org/jira/browse/HIVE-9573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938990#comment-14938990
]
Ashutosh Chauhan commented on HIVE-9573:
----------------------------------------
This indeed sounds interesting. Will you like to upload a patch?
> Lazy load partitions for SELECT LIMIT type query
> ------------------------------------------------
>
> Key: HIVE-9573
> URL: https://issues.apache.org/jira/browse/HIVE-9573
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Selina Zhang
> Assignee: Selina Zhang
>
> Some tools such as HUE uses
> SELECT * FROM table LIMIT 100;
> to grab the sample content of table. For table with large partitions it
> causes a large amount of partition objects been loaded. Thus slow down the
> HS2 even cause OOM from time to time.
> My solution is lazy load partition objects in FetchOperator for this type of
> query. We can skip retrieving the whole partitions but only partition names
> in PartitionPrunner, and lazy load the partition objects when needed in
> execution time (for local job only).
> I have a patch ready. But want to hear more suggestions. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
