[
https://issues.apache.org/jira/browse/HIVE-439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720989#action_12720989
]
Zheng Shao commented on HIVE-439:
---------------------------------
@hive.439.1.patch
rand_partitionpruner2.q:
-query: select * from tmptable x sort by x.key
+query: select * from tmptable x sort by x.key,value
...
+103 val_103 2008-04-08 12
103 val_103 2008-04-08 11
-103 val_103 2008-04-08 12
I guess we have to sort by all columns to make the result deterministic.
The same thing with input_part2.
Also, shall we rename "Conditional Operator" to "Conditional Task" in explain
plan? In Hive world, Operator usually means one operator in the map-reduce
tasks.
> merge small files whenever possible
> -----------------------------------
>
> Key: HIVE-439
> URL: https://issues.apache.org/jira/browse/HIVE-439
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Namit Jain
> Attachments: hive.439.1.patch
>
>
> There are cases when the input to a Hive job are thousands of small files. In
> this case, there is a mapper for each file. Most of the overhead for spawning
> all these mappers can be avoided if these small files are combined into fewer
> larger files.
> The problem can also be addressed by having a mapper span multiple blocks as
> in:
> https://issues.apache.org/jira/browse/HIVE-74
> Bit, it also makes sense in HIVE to merge files whenever possible.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
