[ 
https://issues.apache.org/jira/browse/HDFS-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongjun Zhang updated HDFS-12294:
---------------------------------
    Description: 
This is an alternative solution for HDFS-12202, which proposed introducing a 
new set of API, with an additional boolean parameter bypassExtAttrProvider, so 
to let NN bypass external attribute provider when getFileStatus. The goal is to 
avoid distcp from copying attributes from one cluster's external attribute 
provider and save to another cluster's fsimage.

The solution here is, instead of having an additional parameter, encode this 
parameter to the path itself, when calling getFileStatus (and some other 
calls), NN will parse the path, and figure out that whether external attribute 
provider need to be bypassed. The suggested encoding is to have a prefix to the 
path before calling getFileStatus, e.g. /ab/c becomes 
/.reserved/bypassExtAttr/a/b/c. NN will parse the path at the very beginning.

Thanks much to [~andrew.wang] for this suggestion. The scope of change is 
smaller and we don't have to change the FileSystem APIs.






 

  was:
This is an alternative solution for HDFS-12202, which proposed introducing a 
new set of API, with an additional boolean parameter bypassExtAttrProvider, so 
to let NN bypass external attribute provider when getFileStatus. The goal is to 
avoid distcp from copying attributes from one cluster's external attribute 
provider and save to another cluster's fsimage.

The solution here is, instead of having an additional parameter, encode this 
parameter to the path itself, when calling getFileStatus (and some other 
calls), NN will parse the path, and figure out that whether external attribute 
provider need to be bypassed.

Thanks much to [~andrew.wang] for this suggestion. The scope of change is 
smaller and we don't have to change the FileSystem APIs.






 


> Support /.reserved/bypassExtAttr prefix to file path so getFileStatus etc can 
> bypass external attribute provider
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12294
>                 URL: https://issues.apache.org/jira/browse/HDFS-12294
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>
> This is an alternative solution for HDFS-12202, which proposed introducing a 
> new set of API, with an additional boolean parameter bypassExtAttrProvider, 
> so to let NN bypass external attribute provider when getFileStatus. The goal 
> is to avoid distcp from copying attributes from one cluster's external 
> attribute provider and save to another cluster's fsimage.
> The solution here is, instead of having an additional parameter, encode this 
> parameter to the path itself, when calling getFileStatus (and some other 
> calls), NN will parse the path, and figure out that whether external 
> attribute provider need to be bypassed. The suggested encoding is to have a 
> prefix to the path before calling getFileStatus, e.g. /ab/c becomes 
> /.reserved/bypassExtAttr/a/b/c. NN will parse the path at the very beginning.
> Thanks much to [~andrew.wang] for this suggestion. The scope of change is 
> smaller and we don't have to change the FileSystem APIs.
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to