[
https://issues.apache.org/jira/browse/HIVE-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Yang updated HIVE-1377:
----------------------------
Description:
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}
Then calling
{code}
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}
will throw an IOException because /user/warehouse/tableName is not a prefix of
hdfs://host:9000/user/warehouse/tableName/abc. Currently, this is not an issue
but will come up if CombineFileInputFormat is modified so what the scheme and
authority are not stripped out when generating splits (see MAPREDUCE-1806).
The proposed solution is add a case where matching is done by just the path
component of the URI's.
was:
The use case is:
{code}
dir = hdfs://host:9000/user/warehouse/tableName/abc
pathToPartitionInfo = {/user/warehouse/tableName : myPart}
{code}
Then calling
{code}
getPartitionDescFromPath(dir, pathToPartitionInfo)
{code}
will throw an IOException. Currently, this is not an issue but will come up if
CombineFileInputFormat is modified to so what the scheme and authority are not
stripped out in MAPREDUCE-1806.
The proposed solution is add a case where matching is done by just the path
component of the URI's.
> getPartitionDescFromPath() in CombeHiveInputFormat should handle matching by
> path
> ---------------------------------------------------------------------------------
>
> Key: HIVE-1377
> URL: https://issues.apache.org/jira/browse/HIVE-1377
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
>
> The use case is:
> {code}
> dir = hdfs://host:9000/user/warehouse/tableName/abc
> pathToPartitionInfo = {/user/warehouse/tableName : myPart}
> {code}
> Then calling
> {code}
> getPartitionDescFromPath(dir, pathToPartitionInfo)
> {code}
> will throw an IOException because /user/warehouse/tableName is not a prefix
> of hdfs://host:9000/user/warehouse/tableName/abc. Currently, this is not an
> issue but will come up if CombineFileInputFormat is modified so what the
> scheme and authority are not stripped out when generating splits (see
> MAPREDUCE-1806).
> The proposed solution is add a case where matching is done by just the path
> component of the URI's.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
