[
https://issues.apache.org/jira/browse/HIVE-24742?focusedWorklogId=551936&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-551936
]
ASF GitHub Bot logged work on HIVE-24742:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Feb/21 18:09
Start Date: 12/Feb/21 18:09
Worklog Time Spent: 10m
Work Description: aihuaxu opened a new pull request #1973:
URL: https://github.com/apache/hive/pull/1973
### What changes were proposed in this pull request?
Support router path or viewfs path in Hive table location by resolving the
path to the physical cluster path and comparing them to determine if source or
destination are on the same cluster.
### Why are the changes needed?
HDFS router or viewfs is for file system federation and the path is not the
physical path. Currently Hive uses the scheme and authority to check if the
source and destination are on the same cluster and decides to rename the path
or copy the data. That would cause issue for router or viewfs path since the
physical paths could be on different clusters.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
New UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 551936)
Remaining Estimate: 0h
Time Spent: 10m
> Support router path or view fs path in Hive table location
> ----------------------------------------------------------
>
> Key: HIVE-24742
> URL: https://issues.apache.org/jira/browse/HIVE-24742
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Affects Versions: 3.1.2
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Priority: Major
> Attachments: HIVE-24742.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In
> [FileUtils.java|https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/common/FileUtils.java#L747],
> equalsFileSystem function checks the base URL to determine if source and
> destination are on the same cluster and decides copy or move the data. That
> will not work for viewfs or router base file system since
> viewfs://ns-default/a and viewfs://ns-default/b may be on different physical
> clusters.
> FileSystem in HDFS supports resolvePath() function to resolve to the physical
> path. We can support viewfs and router through such function.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
