[
https://issues.apache.org/jira/browse/HIVE-23347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17099072#comment-17099072
]
Adesh Kumar Rao commented on HIVE-23347:
----------------------------------------
[~srahman]
In your example:
{noformat}
partition from metastore: <tablepath>/year=2020/month=3/day=2;
partition from fileSystem: <tablepath>/Year=2020/Month=3/Day=2;
{noformat}
The partition from metastore is wrong. It will contain:
"tablepath/Year=2020/Month=3/Day=2" (the actual hdfs path)
because partition path fetched is being fetched from metastore (the partition
name is "year=2020/month=3/day=2", but the path will be
"tablepath/Year=2020/Month=3/Day=2")
Links for how the partPaths variable is populated:
[Path partPath = getDataLocation(table,
partition);|https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreChecker.java#L316]
and
[getDataLocation.|https://github.com/apache/hive/blob/c34ee9d79bf6931a92b61af98a6c8f09c6b9ad73/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreServerUtils.java#L1374]
> MSCK REPAIR cannot discover partitions with upper case directory names.
> -----------------------------------------------------------------------
>
> Key: HIVE-23347
> URL: https://issues.apache.org/jira/browse/HIVE-23347
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
> Affects Versions: 3.1.0
> Reporter: Sankar Hariappan
> Assignee: Adesh Kumar Rao
> Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23347.01.patch, HIVE-23347.2.patch,
> HIVE-23347.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For the following scenario, we expect MSCK REPAIR to discover partitions but
> it couldn't.
> 1. Have partitioned data path as follows.
> hdfs://mycluster/datapath/t1/Year=2020/Month=03/Day=10
> hdfs://mycluster/datapath/t1/Year=2020/Month=03/Day=11
> 2. create external table t1 (key int, value string) partitioned by (Year int,
> Month int, Day int) stored as orc location hdfs://mycluster/datapath/t1'';
> 3. msck repair table t1;
> 4. show partitions t1; --> Returns zero partitions
> 5. select * from t1; --> Returns empty data.
> When the partition directory names are changed to lower case, this works fine.
> hdfs://mycluster/datapath/t1/year=2020/month=03/day=10
> hdfs://mycluster/datapath/t1/year=2020/month=03/day=11
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
