[ https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600446#comment-13600446 ]
Hadoop QA commented on HBASE-8031: ---------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12573277/hbase-8031_v1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 2 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 lineLengths{color}. The patch introduces lines longer than 100 {color:red}-1 site{color}. The patch appears to cause mvn site goal to fail. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4782//console This message is automatically generated. > Adopt goraci as an Integration test > ----------------------------------- > > Key: HBASE-8031 > URL: https://issues.apache.org/jira/browse/HBASE-8031 > Project: HBase > Issue Type: Improvement > Components: test > Reporter: Enis Soztutar > Assignee: Enis Soztutar > Fix For: 0.95.0, 0.98.0, 0.94.7 > > Attachments: hbase-8031_v1.patch > > > As you might know, I am a big fan of the goraci test that Keith Turner has > developed, which in turn is inspired by the Accumulo test called Continuous > Ingest. > As much as I hate to say it, having to rely on gora and and external github > library makes using this lib cumbersome. And lately we had to use this for > testing against secure clusters and with Hadoop2, which gora does not support > for now. > So, I am proposing we add this test as an IT in the HBase code base so that > all HBase devs can benefit from it. > The original source code can be found here: > * https://github.com/keith-turner/goraci > * https://github.com/enis/goraci/ > From the javadoc: > {code} > Apache Accumulo [0] has a simple test suite that verifies that data is not > * lost at scale. This test suite is called continuous ingest. This test runs > * many ingest clients that continually create linked lists containing 25 > * million nodes. At some point the clients are stopped and a map reduce job > is > * run to ensure no linked list has a hole. A hole indicates data was lost.·· > * > * The nodes in the linked list are random. This causes each linked list to > * spread across the table. Therefore if one part of a table loses data, then > it > * will be detected by references in another part of the table. > * > Below is rough sketch of how data is written. For specific details look at > * the Generator code. > * > * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that > * reference previous million· 4 If this is the 25th set of 1 million nodes, > * then update 1st set of million to point to last· 5 goto 1 > * > * The key is that nodes only reference flushed nodes. Therefore a node should > * never reference a missing node, even if the ingest client is killed at any > * point in time. > * > * Some ASCII art time: > * [ . . . ] represents one batch of random longs of length WIDTH > * > * _________________________ > * | ______ | > * | | || > * __+_________________+_____ || > * v v v ||| > * first = [ . . . . . . . . . . . ] ||| > * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ||| > * | | | | | | | | | | | ||| > * prev = [ . . . . . . . . . . . ] ||| > * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ||| > * | | | | | | | | | | | ||| > * current = [ . . . . . . . . . . . ] ||| > * ||| > * ... ||| > * ||| > * last = [ . . . . . . . . . . . ] ||| > * | | | | | | | | | | |-----||| > * | |--------|| > * |___________________________| > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira