Hi all Nice to meet you all. I am Gao, an engineer working in Japan. And I have got two issues about Bigtop which I want to share.
The first, I want to proposal that we add HDFS Erasure Coding(HDFS-7285 ,
HDFS-8031) test cases in to Bigtop HDFS smoke tests. This could be very helpful
when users want to test HDFS EC(Erasure Coding) related functions before they
use Erasure Coding in production clusters. I wonder if there are engineers also
interested in developing Erasure Coding test cases, so we could gather a team
to work together? I have added a basic demo test case “TestECPolicy.groovy”,
and send it in attached patch.
The second, when I am trying to reuse HDFS test cases by simply copying them
and setting EC policy to TESTDIR, like I copied “TestPut.groovy” and make some
minor changes to make a new EC test case “TestPutEC.groovy”. This test case
failed dur to unpack resource errors. Then I try to run the original
“TestPut.groovy”, some how it failed with the same reason too. Could you give
some tips to resolve these problem? Thank you very much.
Looking forward to your reply.
Gao.
Following is the detailed failure log of the second issue.
TestPut.groovy:
After build with “pom.xml” and “clean package -Pnative,dist -Dtar -DskipTests”
command, “install-hadoop” and “bigtop-tests:smoke-tests:hdfs:test -Psmoke.tests
–info” grandle tasks was ran, then I got following error log in line 53(
JarContent.unpackJarContainer(TestPut.class, ".", null);).
Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.
org.apache.bigtop.itest.hadoop.hdfs.TestPut > classMethod FAILED
java.io.IOException: Class TestPut doesn't belong to any jar file in the
classpath
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
at
org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
at
org.apache.bigtop.itest.JarContent.unpackJarContainer(JarContent.groovy:181)
at org.apache.bigtop.itest.JarContent$unpackJarContainer.call(Unknown
Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at org.apache.bigtop.itest.hadoop.hdfs.TestPut.setUp(TestPut.groovy:53)
1 test completed, 1 failed
TestPutEC.groovy:
After build with “pom.xml” and “clean package -Pnative,dist -Dtar -DskipTests”
command, “install-hadoop” and “bigtop-tests:smoke-tests:hdfs-ec:test
-Psmoke.tests –info” grandle tasks was ran, then I got following error log in
line 53( JarContent.unpackJarContainer(TestPutEC.class, ".", null);). While
“TestECPolicy.groovy” was passed without problem.
Gradle Test Executor 2 started executing tests.
Gradle Test Executor 2 finished executing tests.
org.apache.bigtop.itest.hadoop.hdfs.TestPutEC > classMethod FAILED
java.io.IOException: Class TestPutEC doesn't belong to any jar file in the
classpath
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
at
org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:54)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
at
org.apache.bigtop.itest.JarContent.unpackJarContainer(JarContent.groovy:181)
at org.apache.bigtop.itest.JarContent$unpackJarContainer.call(Unknown
Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at
org.apache.bigtop.itest.hadoop.hdfs.TestPutEC.setUp(TestPutEC.groovy:53)
2 tests completed, 1 failed
Bigtop-add-HDFS-Erasure-Coding-demo.patch
Description: Bigtop-add-HDFS-Erasure-Coding-demo.patch
