[
https://issues.apache.org/jira/browse/HIVE-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655646#action_12655646
]
Johan Oskarsson commented on HIVE-91:
-------------------------------------
My approach would be to have a command to add partitions manually, I have
created a jira ticket for it: HIVE-115. There's also already a method for this
in the metastore thrift interface if I'm not mistaken. For us it would be
fairly simple to add another command after loading our data into hdfs.
It would also be a bit tricky to automatically find partitions from HDFS if
they have a custom format. I can't think of a way off the top of my head if you
have directories like so: /dataset/2008/12/10/spain where 2008/12/11 is one
partition and spain is another. Then we'd have to save more information on the
exact directory structure for each partition and it seems to get more complex
then it has to at this stage.
> Allow external tables with different partition directory structure
> ------------------------------------------------------------------
>
> Key: HIVE-91
> URL: https://issues.apache.org/jira/browse/HIVE-91
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Minor
>
> A lot of users have datasets in a directory structures similar to this in
> hdfs: /dataset/yyyy/MM/dd/<one or more files>
> Instead of loading these into Hive the normal way it would be useful to
> create an external table with the /dataset location and then one partition
> per yyyy/mm/dd. This would require the partition "naming to
> directory"-function to be made more flexible.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
