Syed Shameerur Rahman created HIVE-23751:

             Summary: QTest: Override #mkdirs() method in ProxyFileSystem To 
Align After HADOOP-16582
                 Key: HIVE-23751
             Project: Hive
          Issue Type: Task
            Reporter: Syed Shameerur Rahman
            Assignee: Syed Shameerur Rahman
             Fix For: 4.0.0, 3.2.0

HADOOP-16582 have changed the way how mkdirs() work:

*Before HADOOP-16582:*
All calls to mkdirs(p) were fast-tracked to FileSystem.mkdirs which were then 
re-routed to mkdirs(p, permission) method. For ProxyFileSytem the call would 
look like

FileUtiles.mkdir(p)  ----->  FileSystem.mkdirs(p) ---> 
An implementation of FileSystem have only needed implement mkdirs(p, permission)

*After HADOOP-16582:*

Since FilterFileSystem overrides mkdirs(p) method the new call to 
ProxyFileSystem would look like

FileUtiles.mkdir(p) ---> FilterFileSystem.mkdirs(p) -->

This will make all the qtests fails with the below exception 
Caused by: java.lang.IllegalArgumentException: Wrong FS: 
 expected: file:///
Note: We will hit this issue when we bump up hadoop version in hive.

So as per the discussion in HADOOP-16963 ProxyFileSystem would need to override 
the mkdirs(p) method inorder to solve the above problem. So now the new flow 
would look like

FileUtiles.mkdir(p)  ---->   ProxyFileSytem.mkdirs(p) ---> 
ProxyFileSytem.mkdirs(p, permission) --->

This message was sent by Atlassian Jira

Reply via email to