[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844712#comment-13844712 ] Chris Nauroth commented on HADOOP-10020: +1 for the branch-2 backport. Thanks for the patch, Colin! > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > HADOOP-10020-b2.001.patch, Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844701#comment-13844701 ] Colin Patrick McCabe commented on HADOOP-10020: --- Thanks for the careful review. bq. Question: are you aware of any downstream projects that are dependent on creating local symlinks using a RawLocalFileSystem? I'm not personally aware of any, but I also didn't do an exhaustive search. I don't think there are any, because HADOOP-9417 (support for symlink resolution in LocalFileSystem / RawLocalFileSystem) has never made it to a stable release. It wasn't in branch-2.3. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > HADOOP-10020-b2.001.patch, Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13844635#comment-13844635 ] Chris Nauroth commented on HADOOP-10020: Hi Colin. I agree that we need to disable HDFS symlinks in branch-2. The branch-2 patch looks good. I assume you've done a test run already. (We don't have the benefit of Jenkins on this.) I see that you've disabled symlinks in {{RawLocalFileSystem}} too. The original patch hadn't done this. Question: are you aware of any downstream projects that are dependent on creating local symlinks using a {{RawLocalFileSystem}}? I'm not personally aware of any, but I also didn't do an exhaustive search. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > HADOOP-10020-b2.001.patch, Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825640#comment-13825640 ] Colin Patrick McCabe commented on HADOOP-10020: --- I posted the patch over in HADOOP-10109. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825618#comment-13825618 ] Suresh Srinivas commented on HADOOP-10020: -- Can this be done as a separate jira instead of addendum to the existing patch? > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825617#comment-13825617 ] Chris Nauroth commented on HADOOP-10020: +1 for the addendum patch. I verified that {{TestOfflineEditsViewer}} passes on branch-2.2 with this patch. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825612#comment-13825612 ] Andrew Wang commented on HADOOP-10020: -- +1 patch looks good to me, but maybe we should addenum over on HADOOP-10052 since that's what introduced this new-style disable logic. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.2.0 >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Fix For: 2.2.0 > > Attachments: > 0001-HADOOP-10020-addendum.-Fix-TestOfflineEditsViewer.patch, > Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787816#comment-13787816 ] Sanjay Radia commented on HADOOP-10020: --- As noted by chris, the TestRetryCacheWithHA is not related to this patch; further TestRetryCacheWithHA passed on my desktop. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4-forBranch2.1beta.patch, Hadoop-10020-4.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787761#comment-13787761 ] Hadoop QA commented on HADOOP-10020: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607091/Hadoop-10020-3.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3183//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3183//console This message is automatically generated. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020-4.patch, Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787756#comment-13787756 ] Hadoop QA commented on HADOOP-10020: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607091/Hadoop-10020-3.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3185//console This message is automatically generated. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787748#comment-13787748 ] Hadoop QA commented on HADOOP-10020: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607091/Hadoop-10020-3.patch against trunk revision . {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3184//console This message is automatically generated. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020-3.patch, > Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787485#comment-13787485 ] Chris Nauroth commented on HADOOP-10020: bq. -1 javac. The applied patch generated 1536 javac compiler warnings (more than the trunk's current 1535 warnings). It looks like {{DistributedFileSystem#createSymlink}} needs a {{@SuppressWarnings("deprecation')}} tag. (I'm still +1 for the patch after adding that.) {quote} -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA {quote} I think this is unrelated. I can't reproduce it locally, and it didn't happen in the prior test run of the patch, which was mostly equivalent. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787480#comment-13787480 ] Hadoop QA commented on HADOOP-10020: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12607045/Hadoop-10020-2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:red}-1 javac{color}. The applied patch generated 1536 javac compiler warnings (more than the trunk's current 1535 warnings). {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3181//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/3181//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3181//console This message is automatically generated. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020-2.patch, Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13787240#comment-13787240 ] Chris Nauroth commented on HADOOP-10020: Hi, [~sanjay.radia]. Thanks for picking up this patch so quickly. I think it looks good. Here are just a couple of small things: # Can we add {{@Deprecated}} tags on {{FileSystem#isSymlinksEnabled}} and {{FileSystem#enableSymlinks}}? That way, clients will get warnings if they try to call them directly. # Shall we add a call to {{FileSystem#isSymlinksEnabled}} in {{FSNamesystem#createSymlink}} too? Right now, the enforcement happens client-side in the {{FileSystem}} subclasses. If someone connects a 2.1.x client to a 2.2.x cluster or recompiles the 2.2.x client code with the check removed, then they could circumvent the check. Adding a check on the server side would prevent this. (Having the client-side check is still good too though.) > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-10020) disable symlinks temporarily
[ https://issues.apache.org/jira/browse/HADOOP-10020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786983#comment-13786983 ] Hadoop QA commented on HADOOP-10020: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12606976/Hadoop-10020.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3179//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3179//console This message is automatically generated. > disable symlinks temporarily > > > Key: HADOOP-10020 > URL: https://issues.apache.org/jira/browse/HADOOP-10020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.1.2-beta >Reporter: Colin Patrick McCabe >Assignee: Sanjay Radia >Priority: Blocker > Attachments: Hadoop-10020.patch > > > disable symlinks temporarily until we can make them production-ready in > Hadoop 2.3 -- This message was sent by Atlassian JIRA (v6.1#6144)