[jira] [Updated] (HADOOP-14211) FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"

2017-03-24 Thread Zhe Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhe Zhang updated HADOOP-14211:
---
Fix Version/s: 2.8.1
   2.7.4

Thanks [~xkrogen] for the work and [~andrew.wang] for the review. +1 on the 
patch as well. I just cherry-picked to branch-2.8 and branch-2.7.

> FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"
> 
>
> Key: HADOOP-14211
> URL: https://issues.apache.org/jira/browse/HADOOP-14211
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: viewfs
>Affects Versions: 2.6.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
> Fix For: 2.9.0, 2.7.4, 2.8.1, 3.0.0-alpha3
>
> Attachments: HADOOP-14211.000.patch, HADOOP-14211.001.patch
>
>
> Right now {{FilterFs}} and {{ChRootedFs}} pass the following up to the 
> {{AbstractFileSystem}} superconstructor:
> {code}
> super(fs.getUri(), fs.getUri().getScheme(),
> fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
> {code}
> This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
> authority, but this isn't necessarily the case - ViewFS is an example of 
> this. You will encounter this issue if you try to filter a ViewFS, or nest 
> one ViewFS within another. The {{authorityNeeded}} check isn't necessary in 
> this case anyway; {{fs}} is already an instantiated {{AbstractFileSystem}} 
> which means it has already used the same constructor with the value of 
> {{authorityNeeded}} (and corresponding validation) that it actually requires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14211) FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"

2017-03-24 Thread Andrew Wang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HADOOP-14211:
-
   Resolution: Fixed
Fix Version/s: 3.0.0-alpha3
   2.9.0
   Status: Resolved  (was: Patch Available)

Thanks Erik, committed to trunk and branch-2.

Have you checked if FilterFileSystem and ViewFileSystem suffer from a similar 
problem? Most apps are more comfortable with the FileSystem APIs. I don't know 
anything that uses FileContext besides YARN and MR2.

> FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"
> 
>
> Key: HADOOP-14211
> URL: https://issues.apache.org/jira/browse/HADOOP-14211
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: viewfs
>Affects Versions: 2.6.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: HADOOP-14211.000.patch, HADOOP-14211.001.patch
>
>
> Right now {{FilterFs}} and {{ChRootedFs}} pass the following up to the 
> {{AbstractFileSystem}} superconstructor:
> {code}
> super(fs.getUri(), fs.getUri().getScheme(),
> fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
> {code}
> This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
> authority, but this isn't necessarily the case - ViewFS is an example of 
> this. You will encounter this issue if you try to filter a ViewFS, or nest 
> one ViewFS within another. The {{authorityNeeded}} check isn't necessary in 
> this case anyway; {{fs}} is already an instantiated {{AbstractFileSystem}} 
> which means it has already used the same constructor with the value of 
> {{authorityNeeded}} (and corresponding validation) that it actually requires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14211) FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"

2017-03-24 Thread Erik Krogen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erik Krogen updated HADOOP-14211:
-
Summary: FilterFs and ChRootedFs are too aggressive about enforcing 
"authorityNeeded"  (was: ChRootedFs is too aggressive about enforcing 
"authorityNeeded")

> FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"
> 
>
> Key: HADOOP-14211
> URL: https://issues.apache.org/jira/browse/HADOOP-14211
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: viewfs
>Affects Versions: 2.6.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
> Attachments: HADOOP-14211.000.patch
>
>
> Right now {{ChRootedFs}} passes the following up to the 
> {{AbstractFileSystem}} superconstructor:
> {code}
> super(fs.getUri(), fs.getUri().getScheme(),
> fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
> {code}
> This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
> authority, but this isn't necessarily the case - ViewFS itself is an example 
> of this. In fact you will encounter this issue if you try to nest one ViewFS 
> within another - I can't think of any reason why you would want to do that 
> but there's no reason why you shouldn't be able to and in general ViewFS is 
> making an assumption that it then proves invalid by its own behavior. The 
> {{authorityNeeded}} check isn't necessary in this case anyway; {{fs}} is 
> already an instantiated {{AbstractFileSystem}} which means it has already 
> used the same constructor with the value of {{authorityNeeded}} (and 
> corresponding validation) that it actually requires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14211) FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"

2017-03-24 Thread Erik Krogen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erik Krogen updated HADOOP-14211:
-
Attachment: HADOOP-14211.001.patch

Thanks for pointing that out [~andrew.wang]! I updated the ticket and added a 
new v001 patch to reflect changes to both {{FilterFs}} and {{ChRootedFs}}. This 
makes for a bit more sensical unit test as well.

> FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"
> 
>
> Key: HADOOP-14211
> URL: https://issues.apache.org/jira/browse/HADOOP-14211
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: viewfs
>Affects Versions: 2.6.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
> Attachments: HADOOP-14211.000.patch, HADOOP-14211.001.patch
>
>
> Right now {{FilterFs}} and {{ChRootedFs}} pass the following up to the 
> {{AbstractFileSystem}} superconstructor:
> {code}
> super(fs.getUri(), fs.getUri().getScheme(),
> fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
> {code}
> This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
> authority, but this isn't necessarily the case - ViewFS is an example of 
> this. You will encounter this issue if you try to filter a ViewFS, or nest 
> one ViewFS within another. The {{authorityNeeded}} check isn't necessary in 
> this case anyway; {{fs}} is already an instantiated {{AbstractFileSystem}} 
> which means it has already used the same constructor with the value of 
> {{authorityNeeded}} (and corresponding validation) that it actually requires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14211) FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"

2017-03-24 Thread Erik Krogen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erik Krogen updated HADOOP-14211:
-
Description: 
Right now {{FilterFs}} and {{ChRootedFs}} pass the following up to the 
{{AbstractFileSystem}} superconstructor:
{code}
super(fs.getUri(), fs.getUri().getScheme(),
fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
{code}
This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
authority, but this isn't necessarily the case - ViewFS is an example of this. 
You will encounter this issue if you try to filter a ViewFS, or nest one ViewFS 
within another. The {{authorityNeeded}} check isn't necessary in this case 
anyway; {{fs}} is already an instantiated {{AbstractFileSystem}} which means it 
has already used the same constructor with the value of {{authorityNeeded}} 
(and corresponding validation) that it actually requires.

  was:
Right now {{ChRootedFs}} passes the following up to the {{AbstractFileSystem}} 
superconstructor:
{code}
super(fs.getUri(), fs.getUri().getScheme(),
fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
{code}
This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
authority, but this isn't necessarily the case - ViewFS itself is an example of 
this. In fact you will encounter this issue if you try to nest one ViewFS 
within another - I can't think of any reason why you would want to do that but 
there's no reason why you shouldn't be able to and in general ViewFS is making 
an assumption that it then proves invalid by its own behavior. The 
{{authorityNeeded}} check isn't necessary in this case anyway; {{fs}} is 
already an instantiated {{AbstractFileSystem}} which means it has already used 
the same constructor with the value of {{authorityNeeded}} (and corresponding 
validation) that it actually requires.


> FilterFs and ChRootedFs are too aggressive about enforcing "authorityNeeded"
> 
>
> Key: HADOOP-14211
> URL: https://issues.apache.org/jira/browse/HADOOP-14211
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: viewfs
>Affects Versions: 2.6.0
>Reporter: Erik Krogen
>Assignee: Erik Krogen
> Attachments: HADOOP-14211.000.patch
>
>
> Right now {{FilterFs}} and {{ChRootedFs}} pass the following up to the 
> {{AbstractFileSystem}} superconstructor:
> {code}
> super(fs.getUri(), fs.getUri().getScheme(),
> fs.getUri().getAuthority() != null, fs.getUriDefaultPort());
> {code}
> This passes a value of {{authorityNeeded==true}} for any {{fs}} which has an 
> authority, but this isn't necessarily the case - ViewFS is an example of 
> this. You will encounter this issue if you try to filter a ViewFS, or nest 
> one ViewFS within another. The {{authorityNeeded}} check isn't necessary in 
> this case anyway; {{fs}} is already an instantiated {{AbstractFileSystem}} 
> which means it has already used the same constructor with the value of 
> {{authorityNeeded}} (and corresponding validation) that it actually requires.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org