[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087536#comment-14087536 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Yarn-trunk #635 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/635/]) Edit CHANGES.txt files to move HADOOP-10759 and HDFS-6717 from 2.5.0 to 2.6.0 (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1616036) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087628#comment-14087628 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1829 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1829/]) Edit CHANGES.txt files to move HADOOP-10759 and HDFS-6717 from 2.5.0 to 2.6.0 (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1616036) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087652#comment-14087652 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1855 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1855/]) Edit CHANGES.txt files to move HADOOP-10759 and HDFS-6717 from 2.5.0 to 2.6.0 (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1616036) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087900#comment-14087900 ] Karthik Kambatla commented on HDFS-6717: No problem, [~brandonli]. The addendum caught me between RCs. I just reverted the original documentation fix as well from branch-2 and branch-2.5, so that the entire fix goes in one release - 2.6.0. I hope this is okay, given it is a doc improvement. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087939#comment-14087939 ] Brandon Li commented on HDFS-6717: -- I agree that the doc fixes don't have to go in 2.5. Thanks! Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086091#comment-14086091 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Yarn-trunk #634 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/634/]) commit the additional doc change for HDFS-6717 (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1615801) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086196#comment-14086196 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1828 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1828/]) commit the additional doc change for HDFS-6717 (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1615801) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086253#comment-14086253 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1853 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1853/]) commit the additional doc change for HDFS-6717 (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1615801) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086944#comment-14086944 ] Brandon Li commented on HDFS-6717: -- Thank you, [~kasha] and sorry for committing it the 2.5. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087007#comment-14087007 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-trunk-Commit #6019 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6019/]) Edit CHANGES.txt files to move HADOOP-10759 and HDFS-6717 from 2.5.0 to 2.6.0 (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1616036) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.6.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085326#comment-14085326 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-trunk-Commit #6011 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6011/]) commit the additional doc change for HDFS-6717 (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1615801) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085331#comment-14085331 ] Brandon Li commented on HDFS-6717: -- I've committed the additional doc change to trunk/branch2/2.5. Thank you, [~dscheffy]. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HDFS-6717.more-change2.patch, HDFS-6717.more-change3.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079698#comment-14079698 ] Jeff Hansen commented on HDFS-6717: --- Looks good! By the way, when I said I was having trouble concentrating, that had more to do with the state of my mind -- I may have been tired, hungover, in the middle of a beer... cough... One comment I will make -- I found the name of the configuration property hadoop.proxyuser.*.groups to be somewhat misleading. There was a moment when I thought, great, now I have to create a unix group and add my user to it. Then I realized groups had nothing to do with unix groups, it was just a bit of a misnomer and really meant users. Anyway, thanks for all the help -- it's been a learning experience! Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[
https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079763#comment-14079763
]
Brandon Li commented on HDFS-6717:
--
{quote}Then I realized groups had nothing to do with unix groups, it was just
a bit of a misnomer and really meant users. {quote}
They are goups. Maybe the example root is misleading. The user root on
Linux usually belongs to the group root. I updated the doc to make that clear.
I used to think Tech Writer is a easy job. Changing my mind now... :-)
Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
---
Key: HDFS-6717
URL: https://issues.apache.org/jira/browse/HDFS-6717
Project: Hadoop HDFS
Issue Type: Sub-task
Components: nfs
Affects Versions: 2.4.0
Reporter: Jeff Hansen
Assignee: Brandon Li
Priority: Minor
Fix For: 2.5.0
Attachments: HDFS-6717.001.patch, HDFS-6717.more-change.patch,
HDFS-6717.more-change2.patch, HdfsNfsGateway.html
I believe this is just a matter of needing to update documentation. As a
result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and
unsecure code paths appear to have been merged -- this is great because it
means less code to test. However, it means that the default unsecure behavior
requires additional configuration that needs to be documented.
I'm not the first to have trouble following the instructions documented in
http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html
I kept hitting a RemoteException with the message that hdfs user cannot
impersonate root -- apparently under the old code, there was no impersonation
going on, so the nfs3 service could and should be run under the same user id
that runs hadoop (I assumed this meant the user id hdfs). However, with the
new unified code path, that would require hdfs to be able to impersonate root
(because root is always the local user that mounts a drive). The comments in
jira hdfs-5804 seem to indicate nobody has a problem with requiring the
nfsserver user to impersonate root -- if that means it's necessary for the
configuration to include root as a user nfsserver can impersonate, that
should be included in the setup instructions.
More to the point, it appears to be absolutely necessary now to provision a
user named nfsserver in order to be able to give that nfsserver ability to
impersonate other users. Alternatively I think we'd need to configure hdfs to
be able to proxy other users. I'm not really sure what the best practice
should be, but it should be documented since it wasn't needed in the past.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077623#comment-14077623 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Yarn-trunk #627 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/627/]) HDFS-6717. JIRA HDFS-5804 breaks default nfs-gateway behavior for unsecured config. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1614125) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077712#comment-14077712 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1819 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1819/]) HDFS-6717. JIRA HDFS-5804 breaks default nfs-gateway behavior for unsecured config. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1614125) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077784#comment-14077784 ] Hudson commented on HDFS-6717: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1846 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1846/]) HDFS-6717. JIRA HDFS-5804 breaks default nfs-gateway behavior for unsecured config. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1614125) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077793#comment-14077793 ] Jeff Hansen commented on HDFS-6717: --- I would probably recommend adding a comment to line 77 of http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm?view=markuppathrev=1614125 Specifically: The above are the only required configuration for the NFS gateway in non-secure mode. However, note that in most cases of non-secure installations, you will need to include root in the list of users provided under `hadoop.proxyuser.nfsserver.groups` as root will generally be the user that initially executes the mount. Thanks Brandon! By the way, I'd like to concede that I may have made commented (in my stack overflow response) about the lack of certain details in the documentation that were always there – as I recall, I was VERY tired and distracted the first time I went through the instructions and had trouble concentrating =) When I re-read it, I thought, that's funny, many of those things that I complained about not being there were in fact there... Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076524#comment-14076524 ] Jing Zhao commented on HDFS-6717: - +1. Thanks for the fix [~brandonli]! Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076580#comment-14076580 ] Brandon Li commented on HDFS-6717: -- Thank you, [~jingzhao], for the review. I've committed the patch. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076680#comment-14076680 ] Hudson commented on HDFS-6717: -- FAILURE: Integrated in Hadoop-trunk-Commit #5979 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5979/]) HDFS-6717. JIRA HDFS-5804 breaks default nfs-gateway behavior for unsecured config. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1614125) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076902#comment-14076902 ] Jeff Hansen commented on HDFS-6717: --- Sorry I didn't comment on this before -- I guess I still don't see how the updates to the documentation would fix the nfsserver cannot impersonate root remote exception I described above (hdfs was the user I was running nfsserver as at the time when I got the exception). My fix was to allow nfsserver to impersonate anybody -- I set the proxy users to star (*) or wildcard so that nfsserver was allowed to impersonate anybody including root. It kind of seems the documentation needs to say that you should explicitly add root to the proxy users list for the nfsserver person. Is there a better way to get around this? If so, I'm missing it if it's already been spelled out in the documentation. The reason nfsserver needs to impersonate root is because most users can't run the unix mount command -- even if you add it to the fstab file and allow users to mount an nfs mount, the user can run the mount command, but the service still mounts the directory as root. If there's a better way to mount the directory short of being root, it's not clear to me from the documentation. Thanks Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076928#comment-14076928 ] Brandon Li commented on HDFS-6717: -- [~dscheffy], you are right that root group needs to be added to the the value of hadoop.proxyuser.nfsserver.groups. The change in the patch was focusing on answering who is proxy user, who should start NFS gateway and etc. Based on your last comment, I think we should also add some description saying group root should be added to hadoop.proxyuser.nfsserver.groups in most Linux platforms where root privilege is by default required to mount NFS exports. Please let me know if there is any unclear description in the doc and I will fix it. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Assignee: Brandon Li Priority: Minor Fix For: 2.5.0 Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6717) Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
[ https://issues.apache.org/jira/browse/HDFS-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14072422#comment-14072422 ] Brandon Li commented on HDFS-6717: -- With the fix in HDFS-5804, users don't have to use the same use to start NFS gateway and HDFS. One should always specify the following two properties regardless the HDFS cluster is secure or not: hadoop.proxyuser.nfsserver.groups and hadoop.proxyuser.nfsserver.hosts. As pointed out in the user guide, nfsserver should be replace by the user who starts NFS gateway. For secure HDFS cluster, it doesn't matter who starts NFS gateway. It's all about the user account in the keytab. In the above two properties, nfsserver should be replaced by the user in the keytab. Uploaded a patch, which also fixed the rmax and wmax related descriptions. Also uploaded the generated html file for easy review. Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config --- Key: HDFS-6717 URL: https://issues.apache.org/jira/browse/HDFS-6717 Project: Hadoop HDFS Issue Type: Sub-task Components: nfs Affects Versions: 2.4.0 Reporter: Jeff Hansen Priority: Minor Attachments: HDFS-6717.001.patch, HdfsNfsGateway.html I believe this is just a matter of needing to update documentation. As a result of https://issues.apache.org/jira/browse/HDFS-5804, the secure and unsecure code paths appear to have been merged -- this is great because it means less code to test. However, it means that the default unsecure behavior requires additional configuration that needs to be documented. I'm not the first to have trouble following the instructions documented in http://hadoop.apache.org/docs/r2.4.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html I kept hitting a RemoteException with the message that hdfs user cannot impersonate root -- apparently under the old code, there was no impersonation going on, so the nfs3 service could and should be run under the same user id that runs hadoop (I assumed this meant the user id hdfs). However, with the new unified code path, that would require hdfs to be able to impersonate root (because root is always the local user that mounts a drive). The comments in jira hdfs-5804 seem to indicate nobody has a problem with requiring the nfsserver user to impersonate root -- if that means it's necessary for the configuration to include root as a user nfsserver can impersonate, that should be included in the setup instructions. More to the point, it appears to be absolutely necessary now to provision a user named nfsserver in order to be able to give that nfsserver ability to impersonate other users. Alternatively I think we'd need to configure hdfs to be able to proxy other users. I'm not really sure what the best practice should be, but it should be documented since it wasn't needed in the past. -- This message was sent by Atlassian JIRA (v6.2#6252)
