[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952139#comment-16952139 ] Siddharth Wagle commented on HDFS-14890: Hi [~mohansella], the changes in HDFS-2470 do not affect datanode data dir. Also, the clearDirectory() in the patch is called when NN is formatted, are you testing this on a freshly deployed cluster or explicitly calling format? If the set permission is failing can you provide some log details? [~hirik] This Jira does address the NN failure, I am not sure if there is an issue yet with setting permissions, waiting on some logs/exception. > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948693#comment-16948693 ] Mohan Kumar commented on HDFS-14890: [~swagle], on both `namenode/current/` and `datanode/` but not on `namenode/` > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947888#comment-16947888 ] Siddharth Wagle commented on HDFS-14890: [~mohansella] By data folder, you mean namenode current directory or its parent, can you please provide specifics? Also, take a look at the discussion on HDFS-2470. https://issues.apache.org/jira/browse/HDFS-2470?focusedCommentId=16900435=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16900435 > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947407#comment-16947407 ] Mohan Kumar commented on HDFS-14890: [~swagle], in Windows, with the current patch, the data folder is shared to Everyone with Read permission which is not needed leading to security issues. > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944699#comment-16944699 ] Hudson commented on HDFS-14890: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17479 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17479/]) HDFS-14890. Fixed namenode and journalnode startup on Windows. (eyang: rev aa24add8f0e9812d1f787efb3c40155b0fdeed9c) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944688#comment-16944688 ] Eric Yang commented on HDFS-14890: -- Thank you [~swagle] for the patch. Thank you [~elgoiri] [~hirik] for the reviews. I just committed this to trunk and branch-3.2. > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Fix For: 3.3.0, 3.2.2 > > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944685#comment-16944685 ] Siddharth Wagle commented on HDFS-14890: [~eyang] Updated title, let me know if further edits needed. Thanks. > Setting permissions on name directory fails on non posix compliant filesystems > -- > > Key: HDFS-14890 > URL: https://issues.apache.org/jira/browse/HDFS-14890 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.2.1 > Environment: Windows 10. >Reporter: hirik >Assignee: Siddharth Wagle >Priority: Blocker > Attachments: HDFS-14890.01.patch > > > Hi, > HDFS NameNode and JournalNode are not starting in Windows machine. Found > below related exception in logs. > Caused by: java.lang.UnsupportedOperationExceptionCaused by: > java.lang.UnsupportedOperationException > at java.base/java.nio.file.Files.setPosixFilePermissions(Files.java:2155) > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591) > at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613) > at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206) > at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:422) > at > com.slog.dfs.hdfs.nn.NameNodeServiceImpl.delayedStart(NameNodeServiceImpl.java:147) > > Code changes related to this issue: > [https://github.com/apache/hadoop/commit/07e3cf952eac9e47e7bd5e195b0f9fc28c468313#diff-1a56e69d50f21b059637cfcbf1d23f11] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
