[
https://issues.apache.org/jira/browse/HIVE-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12903655#action_12903655
]
Joydeep Sen Sarma commented on HIVE-1602:
-----------------------------------------
how will this be made transparent from queryability perspective? i think i
still don't understand the details
i agree - if the user does it themselves they have to duplicate the column. but
this doesn't seem like a big deal to me (we compress everything anyway and
partitioning columns are highly compressible since they will be repeated like
crazy).
my worry is that this change might be a very big one (representing multiple
partitions in one storage container). it seems to me a much more fundamental
change than just fixing dynamic partitioning.
> List Partitioning
> -----------------
>
> Key: HIVE-1602
> URL: https://issues.apache.org/jira/browse/HIVE-1602
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.7.0
> Reporter: Ning Zhang
>
> Dynamic partition inserts create partitions bases on the dynamic partition
> column values. Currently it creates one partition for each distinct DP column
> value. This could result in skews in the created dynamic partitions in that
> some partitions are large but there could be large number of small partitions
> as well. This results in burdens in HDFS as well as metastore. A list
> partitioning scheme that aggregate a number of small partitions into one big
> one is more preferable for skewed partitions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
