[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15872: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15763 > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Fix For: 3.2.1 > > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch, > HADOOP-15872-003.patch, HADOOP-15872-004.patch, HADOOP-15872-005.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15872: Resolution: Fixed Fix Version/s: 3.2.1 Status: Resolved (was: Patch Available) +1 committed to branch-3.2 & trunk one thing which surprised me is that there were some merge issues on the patch; nothing serious but enough to force the merge to go to three-way merge. Whatever branch you are developing on, it's not in sync with the AWS one. Which means that although you've tested the patch on your branch, there may not have been any tests that this patch as applied to the asf branches works. I'm going to have to do this myself now, which complicates my life as maven and object store test runs in one source tree can break those in another (i.e the branch where I'm coding right now) please, do make the final patch & test runs off the ASF codebase, just to round off the due diligence. > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Fix For: 3.2.1 > > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch, > HADOOP-15872-003.patch, HADOOP-15872-004.patch, HADOOP-15872-005.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15872: - Attachment: HADOOP-15872-005.patch > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch, > HADOOP-15872-003.patch, HADOOP-15872-004.patch, HADOOP-15872-005.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15872: - Attachment: HADOOP-15872-004.patch > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch, > HADOOP-15872-003.patch, HADOOP-15872-004.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15872: - Attachment: HADOOP-15872-003.patch > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch, > HADOOP-15872-003.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15872: Status: Patch Available (was: Open) thanks. remember to hit the "Submit patch" button to trigger the jenkins review too > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15872: - Attachment: HADOOP-15872-002.patch > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch, HADOOP-15872-002.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junhua gu updated HADOOP-15872: --- Attachment: HADOOP-15872-001.patch > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > Attachments: HADOOP-15872-001.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junhua gu updated HADOOP-15872: --- Attachment: (was: HADOOP-15872-001.patch) > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: junhua gu >Priority: Major > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15872: -- Description: This update to the latest REST version (2018-11-09) will make the following changes to the ABFS driver: 1) The ABFS implementation of getFileStatus currently requires read permission. According to HDFS permissions guide, it should only require execute on the parent folders (traversal access). A new REST API has been introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. 2) The new "2018-11-09" REST version introduces support to i) automatically translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) optionally translate OIDs to UPNs in the responses when getting the owner, owning group, or ACL. Configuration will be introduced to optionally translate OIDs to UPNs in the responses. Since translation has a performance impact, the default will be to perform no translation and return the OIDs. was: The ABFS implementation of getFileStatus currently requires read permission. According to HDFS permissions guide, it should only require execute on the parent folders (traversal access). getFileStatus should only require execute permission on the parent folders > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15872-001.patch > > > This update to the latest REST version (2018-11-09) will make the following > changes to the ABFS driver: > 1) The ABFS implementation of getFileStatus currently requires read > permission. According to HDFS permissions guide, it should only require > execute on the parent folders (traversal access). A new REST API has been > introduced in REST version "2018-11-09" of ADLS Gen 2 to fix this problem. > 2) The new "2018-11-09" REST version introduces support to i) automatically > translate UPNs to OIDs when setting the owner, owning group, or ACL and ii) > optionally translate OIDs to UPNs in the responses when getting the owner, > owning group, or ACL. Configuration will be introduced to optionally > translate OIDs to UPNs in the responses. Since translation has a performance > impact, the default will be to perform no translation and return the OIDs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15872: -- Summary: ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 (was: ABFS: ) > ABFS: Update to target 2018-11-09 REST version for ADLS Gen 2 > - > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15872-001.patch > > > The ABFS implementation of getFileStatus currently requires read permission. > According to HDFS permissions guide, it should only require execute on the > parent folders (traversal access). > > getFileStatus should only require execute permission on the parent folders -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS:
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15872: -- Summary: ABFS: (was: ABFS: getFileStatus should only require execute permission on the parent folders) > ABFS: > -- > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15872-001.patch > > > The ABFS implementation of getFileStatus currently requires read permission. > According to HDFS permissions guide, it should only require execute on the > parent folders (traversal access). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS:
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Marquardt updated HADOOP-15872: -- Description: The ABFS implementation of getFileStatus currently requires read permission. According to HDFS permissions guide, it should only require execute on the parent folders (traversal access). getFileStatus should only require execute permission on the parent folders was:The ABFS implementation of getFileStatus currently requires read permission. According to HDFS permissions guide, it should only require execute on the parent folders (traversal access). > ABFS: > -- > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15872-001.patch > > > The ABFS implementation of getFileStatus currently requires read permission. > According to HDFS permissions guide, it should only require execute on the > parent folders (traversal access). > > getFileStatus should only require execute permission on the parent folders -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15872) ABFS: getFileStatus should only require execute permission on the parent folders
[ https://issues.apache.org/jira/browse/HADOOP-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] junhua gu updated HADOOP-15872: --- Attachment: HADOOP-15872-001.patch > ABFS: getFileStatus should only require execute permission on the parent > folders > > > Key: HADOOP-15872 > URL: https://issues.apache.org/jira/browse/HADOOP-15872 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure >Affects Versions: 3.2.0 >Reporter: Thomas Marquardt >Assignee: Thomas Marquardt >Priority: Major > Attachments: HADOOP-15872-001.patch > > > The ABFS implementation of getFileStatus currently requires read permission. > According to HDFS permissions guide, it should only require execute on the > parent folders (traversal access). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
