[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357016#comment-14357016
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #129 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/129/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357045#comment-14357045
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2079 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2079/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356764#comment-14356764
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #129 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/129/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356772#comment-14356772
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #863 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/863/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356949#comment-14356949
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2061 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2061/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356969#comment-14356969
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #120 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/120/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357967#comment-14357967
]
Akira AJISAKA commented on HDFS-7830:
-
Committed to branch-2.7 also. Hi [~cmccabe], if you didn't commit this to
branch-2.7 intentionally, please feel free to revert it and bump up the fix
version.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356068#comment-14356068
]
Colin Patrick McCabe commented on HDFS-7830:
+1. Thanks, [~eddyxu].
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356134#comment-14356134
]
Hudson commented on HDFS-7830:
--
FAILURE: Integrated in Hadoop-trunk-Commit #7302 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7302/])
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. McCabe) (cmccabe: rev
5c1036d598051cf6af595740f1ab82092b0b6554)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeHotSwapVolumes.java
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Fix For: 2.7.0
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355330#comment-14355330
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703695/HDFS-7830.003.patch
against trunk revision 20b8ee1.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9819//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355391#comment-14355391
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703695/HDFS-7830.003.patch
against trunk revision a380643.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9820//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14355997#comment-14355997
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703742/HDFS-7830.004.patch
against trunk revision aa92b76.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
org.apache.hadoop.hdfs.TestAppendSnapshotTruncate
org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
org.apache.hadoop.hdfs.server.balancer.TestBalancer
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9823//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9823//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356007#comment-14356007
]
Hadoop QA commented on HDFS-7830:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703742/HDFS-7830.004.patch
against trunk revision aa92b76.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9824//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9824//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch, HDFS-7830.003.patch, HDFS-7830.004.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353555#comment-14353555
]
Lei (Eddy) Xu commented on HDFS-7830:
-
The failure is due to errors on jenkins server.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353396#comment-14353396
]
Colin Patrick McCabe commented on HDFS-7830:
{code}
108 } finally {
109 if (lock != null) {
110 try {
111 lock.release();
112 } catch (IOException e) {
113 FsDatasetImpl.LOG.warn(String.format(I/O error releasing
file lock %s.,
114 lockFile.getAbsolutePath()), e);
115 }
{code}
We shouldn't swallow the exception here in the unit tests. If the lock file
can't be released, the unit test should fail. So we should not catch the
exception (or if we do, we should rethrow it).
{code}
420 if (!exceptions.isEmpty()) {
421 sd.unlock();
422 throw MultipleIOException.createIOException(exceptions);
{code}
In the non-unit-test case, we do need to catch the exception and prevent it
from propagating, since then we won't see any other exceptions.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353523#comment-14353523
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703489/HDFS-7830.001.patch
against trunk revision 3241fc2.
{color:red}-1 patch{color}. Trunk compilation may be broken.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9798//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353769#comment-14353769
]
Colin Patrick McCabe commented on HDFS-7830:
{code}
420 if (!exceptions.isEmpty()) {
421 sd.unlock();
422 throw MultipleIOException.createIOException(exceptions);
423 }
{code}
The point that I was making is that if line 421 throws, we won't see the
exceptions on line 422. We should gather up this exception into our list of
exceptions like the rest.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354124#comment-14354124
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703521/HDFS-7830.002.patch
against trunk revision d6e05c5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1156 javac
compiler warnings (more than the trunk's current 1155 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9804//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/9804//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9804//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14354087#comment-14354087
]
Colin Patrick McCabe commented on HDFS-7830:
Thanks, [~eddyxu]. +1 pending jenkins.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch, HDFS-7830.001.patch,
HDFS-7830.002.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344423#comment-14344423
]
Hadoop QA commented on HDFS-7830:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12702015/HDFS-7830.000.patch
against trunk revision c5eac9c.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1156 javac
compiler warnings (more than the trunk's current 1155 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestReplaceDatanodeOnFailure
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9703//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/9703//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9703//console
This message is automatically generated.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-7830.000.patch
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335216#comment-14335216
]
Lei (Eddy) Xu commented on HDFS-7830:
-
[~cnauroth] Would you mind to file a separate JIRA and assign to me? Thanks!
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335230#comment-14335230
]
Chris Nauroth commented on HDFS-7830:
-
Thank you, Eddy. I filed HDFS-7833.
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.
[
https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335185#comment-14335185
]
Chris Nauroth commented on HDFS-7830:
-
Hi [~eddyxu]. Another potential problem that I've noticed in the DataNode
reconfiguration code is that it never recalculates
{{FsDatasetImpl#validVolsRequired}}. This is a {{final}} variable calculated
as (# volumes configured) - (# volume failures tolerated):
{code}
this.validVolsRequired = volsConfigured - volFailuresTolerated;
{code}
If this variable is not updated for DataNode reconfigurations, then it could
lead to some unexpected situations. For example:
# DataNode starts running with 6 volumes (all healthy) and
{{dfs.datanode.failed.volumes.tolerated}} set to 2.
# {{FsDatasetImpl#validVolsRequired}} is set to 6 - 2 = 4.
# DataNode is reconfigured to run with 8 volumes (all still healthy).
# Now 3 volumes fail. The admin would expect the DataNode to abort, but there
are 8 - 3 = 5 good volumes left, and {{FsDatasetImpl#validVolsRequired}} is
still 4, so {{FsDatasetImpl#hasEnoughResource}} returns {{true}}.
Is this something that makes sense for you to address as part of the patch
you're working on now, or would you prefer I file a separate jira to track
this? Thanks!
DataNode does not release the volume lock when adding a volume fails.
-
Key: HDFS-7830
URL: https://issues.apache.org/jira/browse/HDFS-7830
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
When there is a failure in adding volume process, the {{in_use.lock}} is not
released. Also, doing another {{-reconfig}} to remove the new dir in order to
cleanup doesn't remove the lock. lsof still shows datanode holding on to the
lock file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
