[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534207#comment-14534207
]
Vinayakumar B commented on HDFS-8209:
-
+1 for the latest patch.
bq. I think this assert condition is not working and I am not getting why.
May be assertions disabled when you run the test.
When I run the test with assertion statement in eclipse with -ea VM argument,
assertion failed.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Labels: BB2015-05-RFC
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534251#comment-14534251
]
Hudson commented on HDFS-8209:
--
FAILURE: Integrated in Hadoop-trunk-Commit #7773 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7773/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534502#comment-14534502
]
Hudson commented on HDFS-8209:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #921 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/921/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534538#comment-14534538
]
Hudson commented on HDFS-8209:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #190 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/190/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534696#comment-14534696
]
Hudson commented on HDFS-8209:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk #2119 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2119/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534929#comment-14534929
]
Hudson commented on HDFS-8209:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2137 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2137/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534797#comment-14534797
]
Hudson commented on HDFS-8209:
--
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #179 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/179/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534999#comment-14534999
]
Hudson commented on HDFS-8209:
--
SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #189 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/189/])
HDFS-8209. Support different number of datanode directories in MiniDFSCluster.
(Contributed by surendra singh lilhore) (vinayakumarb: rev
4c6816faf83726c71c9c3981e7280b461f824a4f)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMiniDFSCluster.java
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Fix For: 2.8.0
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532221#comment-14532221
]
Hadoop QA commented on HDFS-8209:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 17s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 2m 20s | The applied patch generated
19 new checkstyle issues (total was 734, now 735). |
| {color:green}+1{color} | whitespace | 0m 3s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 7s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 1m 20s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 82m 37s | Tests failed in hadoop-hdfs. |
| | | 104m 55s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDataTransferKeepalive |
| | hadoop.hdfs.server.datanode.TestDatanodeStartupOptions |
| Timed out tests | org.apache.hadoop.hdfs.TestDatanodeRegistration |
| | org.apache.hadoop.hdfs.server.datanode.TestBlockReplacement |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12731087/HDFS-8209_1.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 449e442 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10841/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10841/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10841/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10841/console |
This message was automatically generated.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Labels: BB2015-05-TBR
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14532507#comment-14532507
]
surendra singh lilhore commented on HDFS-8209:
--
Failed test case and checkstyle is not related to this patch.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Labels: BB2015-05-TBR
Attachments: HDFS-8209.patch, HDFS-8209_1.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513119#comment-14513119
]
surendra singh lilhore commented on HDFS-8209:
--
[~szetszwo] Thanks for reviewing. I think this assert condition is not working
and I am not getting why. When I got {{ArrayIndexOutOfBoundsException}} that
time *storageTypes.length* is 1 and *storagesPerDatanode* is 2 but assert
condition didn't failed. I have submitted the patch, we will see the test
result here.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Attachments: HDFS-8209.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513213#comment-14513213
]
Hadoop QA commented on HDFS-8209:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 6s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. |
| {color:red}-1{color} | javac | 7m 24s | The applied patch generated 125
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 7m 42s | There were no new checkstyle
issues. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 2s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | native | 1m 19s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 165m 43s | Tests failed in hadoop-hdfs. |
| | | 192m 43s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.TestFsck |
| | hadoop.hdfs.TestMiniDFSCluster |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12728171/HDFS-8209.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 47279c3 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/10401/artifact/patchprocess/whitespace.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/10401/artifact/patchprocess/diffJavacWarnings.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10401/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10401/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10401/console |
This message was automatically generated.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Attachments: HDFS-8209.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512713#comment-14512713
]
Tsz Wo Nicholas Sze commented on HDFS-8209:
---
{code}
assert storageTypes == null || storageTypes.length == storagesPerDatanode;
for (int j = 0; j storagesPerDatanode; ++j) {
+ if ((storageTypes != null) (j = storageTypes.length)) {
+break;
+ }
{code}
With the assert storageTypes.length == storagesPerDatanode and j
storagesPerDatanode, the condition j = storageTypes.length is always false.
So I guess the patch won't work.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Attachments: HDFS-8209.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8209) Support different number of datanode directories in MiniDFSCluster.
[
https://issues.apache.org/jira/browse/HDFS-8209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512678#comment-14512678
]
surendra singh lilhore commented on HDFS-8209:
--
Thanks [~szetszwo]... Attached patch, please review.
Support different number of datanode directories in MiniDFSCluster.
---
Key: HDFS-8209
URL: https://issues.apache.org/jira/browse/HDFS-8209
Project: Hadoop HDFS
Issue Type: Improvement
Components: test
Affects Versions: 2.6.0
Reporter: surendra singh lilhore
Assignee: surendra singh lilhore
Priority: Minor
Attachments: HDFS-8209.patch
I want to create MiniDFSCluster with 2 datanode and for each datanode I want
to set different number of StorageTypes, but in this case I am getting
ArrayIndexOutOfBoundsException.
My cluster schema is like this.
{code}
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(2)
.storageTypes(new StorageType[][] {{
StorageType.DISK, StorageType.ARCHIVE },{ StorageType.DISK } })
.build();
{code}
*Exception* :
{code}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.hdfs.MiniDFSCluster.makeDataNodeDirs(MiniDFSCluster.java:1218)
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1402)
at
org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:832)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
