[jira] [Commented] (HDFS-9351) No checkNNStartup() is called when fsck calls FSNamesystem.getSnapshottableDirs()

2015-11-03 Thread Yongjun Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987727#comment-14987727
 ] 

Yongjun Zhang commented on HDFS-9351:
-

Thanks [~xiaochen] for the patch, +1 and will commit soon.


> No checkNNStartup() is called when fsck calls 
> FSNamesystem.getSnapshottableDirs()
> -
>
> Key: HDFS-9351
> URL: https://issues.apache.org/jira/browse/HDFS-9351
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: HDFS
>Reporter: Yongjun Zhang
>Assignee: Xiao Chen
> Attachments: HDFS-9351.patch
>
>
> I looked further at HDFS-9231 change I reviewed earlier, and realize I missed 
> one thing.
> The patch changed
> {code}
>   if (snapshottableDirs != null) {
> SnapshottableDirectoryStatus[] snapshotDirs = namenode.getRpcServer()
> .getSnapshottableDirListing();
> if (snapshotDirs != null) {
>   for (SnapshottableDirectoryStatus dir : snapshotDirs) {
> snapshottableDirs.add(dir.getFullPath().toString());
>   }
> }
>   }
> {code}
> to
> {code}
>   if (snapshottableDirs != null) {
> snapshottableDirs = namenode.getNamesystem().getSnapshottableDirs();
>   }
> {code}
> In old code, namenode.getRpcServer().getSnapshottableDirListing() calls 
> checkNNStartup(), however, this is not done with the new code. This is a hole 
> of the earlier HDFS-9231 patch.
> Create this jira to fix the hole. Suggest to revert this portion of the 
> change, and add code to do what we need at NamenodeFsck.
> Sorry for missing this in my earlier review of HDFS-9231.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9351) No checkNNStartup() is called when fsck calls FSNamesystem.getSnapshottableDirs()

2015-11-03 Thread Xiao Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987757#comment-14987757
 ] 

Xiao Chen commented on HDFS-9351:
-

Thanks Yongjun!

> No checkNNStartup() is called when fsck calls 
> FSNamesystem.getSnapshottableDirs()
> -
>
> Key: HDFS-9351
> URL: https://issues.apache.org/jira/browse/HDFS-9351
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: HDFS
>Reporter: Yongjun Zhang
>Assignee: Xiao Chen
> Attachments: HDFS-9351.patch
>
>
> I looked further at HDFS-9231 change I reviewed earlier, and realize I missed 
> one thing.
> The patch changed
> {code}
>   if (snapshottableDirs != null) {
> SnapshottableDirectoryStatus[] snapshotDirs = namenode.getRpcServer()
> .getSnapshottableDirListing();
> if (snapshotDirs != null) {
>   for (SnapshottableDirectoryStatus dir : snapshotDirs) {
> snapshottableDirs.add(dir.getFullPath().toString());
>   }
> }
>   }
> {code}
> to
> {code}
>   if (snapshottableDirs != null) {
> snapshottableDirs = namenode.getNamesystem().getSnapshottableDirs();
>   }
> {code}
> In old code, namenode.getRpcServer().getSnapshottableDirListing() calls 
> checkNNStartup(), however, this is not done with the new code. This is a hole 
> of the earlier HDFS-9231 patch.
> Create this jira to fix the hole. Suggest to revert this portion of the 
> change, and add code to do what we need at NamenodeFsck.
> Sorry for missing this in my earlier review of HDFS-9231.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9351) No checkNNStartup() is called when fsck calls FSNamesystem.getSnapshottableDirs()

2015-11-02 Thread Xiao Chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986495#comment-14986495
 ] 

Xiao Chen commented on HDFS-9351:
-

The findbugs warning / test failures / asflicense are unrelated.
This patch has no test because the parent HDFS-9231 already added test, which 
passed with this patch.

> No checkNNStartup() is called when fsck calls 
> FSNamesystem.getSnapshottableDirs()
> -
>
> Key: HDFS-9351
> URL: https://issues.apache.org/jira/browse/HDFS-9351
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: HDFS
>Reporter: Yongjun Zhang
>Assignee: Xiao Chen
> Attachments: HDFS-9351.patch
>
>
> I looked further at HDFS-9231 change I reviewed earlier, and realize I missed 
> one thing.
> The patch changed
> {code}
>   if (snapshottableDirs != null) {
> SnapshottableDirectoryStatus[] snapshotDirs = namenode.getRpcServer()
> .getSnapshottableDirListing();
> if (snapshotDirs != null) {
>   for (SnapshottableDirectoryStatus dir : snapshotDirs) {
> snapshottableDirs.add(dir.getFullPath().toString());
>   }
> }
>   }
> {code}
> to
> {code}
>   if (snapshottableDirs != null) {
> snapshottableDirs = namenode.getNamesystem().getSnapshottableDirs();
>   }
> {code}
> In old code, namenode.getRpcServer().getSnapshottableDirListing() calls 
> checkNNStartup(), however, this is not done with the new code. This is a hole 
> of the earlier HDFS-9231 patch.
> Create this jira to fix the hole. Suggest to revert this portion of the 
> change, and add code to do what we need at NamenodeFsck.
> Sorry for missing this in my earlier review of HDFS-9231.
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9351) No checkNNStartup() is called when fsck calls FSNamesystem.getSnapshottableDirs()

2015-11-02 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986369#comment-14986369
 ] 

Hadoop QA commented on HDFS-9351:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s 
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
44s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s 
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s 
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 50s 
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run 
convertXmlToText from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 5s 
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s 
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 31s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s 
{color} | {color:red} Patch generated 1 new checkstyle issues in 
hadoop-hdfs-project/hadoop-hdfs (total was 359, now 359). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s 
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 8s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 58s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 25s 
{color} | {color:red} Patch generated 58 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 124m 43s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_60 Failed junit tests | 
hadoop.hdfs.server.blockmanagement.TestNodeCount |
|   | hadoop.hdfs.security.TestDelegationTokenForProxyUser |
|   | hadoop.hdfs.TestSetrepDecreasing |
| JDK v1.7.0_79 Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
|   | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
|   | hadoop.hdfs.TestRecoverStripedFile |
|   | hadoop.hdfs.server.datanode.TestBlockScanner |
|   | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.7.1 Server=1.7.1 
Image:test-patch-base-hadoop-date2015-11-02 |
| JIRA Patch URL |

[jira] [Commented] (HDFS-9351) No checkNNStartup() is called when fsck calls FSNamesystem.getSnapshottableDirs()

2015-10-30 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983829#comment-14983829
 ] 

Hadoop QA commented on HDFS-9351:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 5s 
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 
10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s 
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s 
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
19s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 4s 
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run 
convertXmlToText from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 21s 
{color} | {color:green} trunk passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 1s 
{color} | {color:green} trunk passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 20s 
{color} | {color:red} Patch generated 1 new checkstyle issues in 
hadoop-hdfs-project/hadoop-hdfs (total was 359, now 359). {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed with JDK v1.8.0_60 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 56s 
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 64m 43s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_60. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 63m 27s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 17s 
{color} | {color:red} Patch generated 56 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 149m 45s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_60 Failed junit tests | 
hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
|   | hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes |
|   | hadoop.hdfs.server.namenode.TestAddStripedBlocks |
|   | hadoop.hdfs.TestSafeModeWithStripedFile |
|   | hadoop.hdfs.server.namenode.TestFileTruncate |
|   | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100 |
|   | hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock |
|   | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
|   | hadoop.hdfs.server.namenode.TestRecoverStripedBlocks |
|   | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS |
|   |