[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405056#comment-13405056 ] stack commented on HBASE-6203: -- I made HBASE-6302 for doc. after we settle on how IT stuff is run. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404049#comment-13404049 ] Jesse Yates commented on HBASE-6203: Any chance for an addendum to just have a parameter like -Donly-integration-tests so we don't have to have a switch for each module (though having those too would be nice)? Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404067#comment-13404067 ] stack commented on HBASE-6203: -- Nice one Enis. Can you add your 'how-to' above as release note on this issue. What happens if I do 'mvn verify' at the top level? All other modules but hbase-it are noops? There used to be a bit of doc in the manual but it looks like we lost it when I gave up on integration tests. Let me attach patch of old stuff. I updated it a little. What else would you add (I'll take care of commit and add to doc. if you just say what you'd add). Thanks boss. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404103#comment-13404103 ] stack commented on HBASE-6203: -- So, to make this module of use, we need to do the Interface talked of elsewhere. Even w/ the Interface, is 'mvn verify'/failsafe rich enough in the steps/goals offered? Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404115#comment-13404115 ] Hadoop QA commented on HBASE-6203: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12534017/it-doc.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2296//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2296//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2296//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2296//console This message is automatically generated. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404170#comment-13404170 ] Enis Soztutar commented on HBASE-6203: -- @Jesse, Agreed. We now have a switch for every module, but -Donly-integration-tests might not be necessary since you can do {{cd hbase-it;mvn-verify}}. With maven modules, I think the standard way to only execute one module is to cd into that module and execute. I'll add that to the doc. Do you think that would be enough? @Stack Thanks a lot for the docs, I totally missed that. failsafe is basically a fork of surefire, which only adds pre- and post-integration-tests targets. We are not using them right now, but we can make use of those targets, for recovering the cluster after the test for example. I totally share your doubts about this failsafe/surefire thing, but I do not know of any better solution. Suggestions welcome :) mvn verify now executes the classes {{IntegrationTestXXX}} as unit tests. There is also an IntegrationTestsDriver class in the patch at HBASE-6241, which is executed by: {code} bin/hbase --config org.apache.hadoop.hbase.IntegrationTestsDriver {code} I'll open another subtask for HBASE-6201, for making {{mvn verify}} to work with real clusters. I have checked how bigtop does it, and it seems they have: {code} bigtop-tests/test-artifacts/ -- contains actual test code bigtop-tests/test-execution/ -- contains code + configuration for executing the tests {code} Especially, if you look into {{bigtop-tests/test-execution/smokes/hbase/pom.xml}}, it passes HBASE_HOME, HBASE_CONF_DIR, etc from evn to failsafe. It works for bigtop, so I think we can make it work for our cases as well. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404181#comment-13404181 ] Jesse Yates commented on HBASE-6203: @Enis - except you have to build everything first (more specifically mvn instal), which is kind of a pain. Adding the flag means you can do it all in one step. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404331#comment-13404331 ] Enis Soztutar commented on HBASE-6203: -- bq. which is kind of a pain such is maven :) My only concern for the flag is that, for implementing it, every module has to know about and honor only-integration-tests parameter, which seems not clean to me. I am not a maven guy, if you have a suggestion, I'll be more than happy to try it out. Can we instruct reactor to only run test-compile for everything, but test just for hbase-it? Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13404378#comment-13404378 ] Zhihong Ted Yu commented on HBASE-6203: --- In recent Hadoop QA builds (such as https://builds.apache.org/job/PreCommit-HBASE-Build/2298/console), I saw: {code} [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 38:30.129s [INFO] Finished at: Sat Jun 30 02:50:21 UTC 2012 [INFO] Final Memory: 47M/435M [INFO] # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 32776 bytes for Chunk::new # An error report file with more information is saved as: # /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hs_err_pid16801.log /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/dev-support/test-patch.sh: line 594: 16801 Aborted $MVN clean test -P runAllTests -D${PROJECT_NAME}PatchProcess -Dsurefire.secondPartThreadCount=4 {code} I wonder if the above is related this JIRA. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch, it-doc.txt Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403357#comment-13403357 ] Zhihong Ted Yu commented on HBASE-6203: --- Can one integration test be added to the patch ? Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403361#comment-13403361 ] Enis Soztutar commented on HBASE-6203: -- How about waiting for HBASE-6241 and committing this and that consecutively? Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403369#comment-13403369 ] Zhihong Ted Yu commented on HBASE-6203: --- How about bundling patch here into patch for HBASE-6241 ? Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403647#comment-13403647 ] Enis Soztutar commented on HBASE-6203: -- HBASE-6241 is currently a large patch, and I don't want to add more complexity to it. Let's keep the patches separate. For convenience though, I'll upload both merged and unmerged patches at HBASE-6241. Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403666#comment-13403666 ] Hadoop QA commented on HBASE-6203: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531885/HBASE-6203_v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2288//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2288//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2288//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2288//console This message is automatically generated. Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403672#comment-13403672 ] Zhihong Ted Yu commented on HBASE-6203: --- Integrated to trunk. Thanks for the patch, Enis. Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it module
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403685#comment-13403685 ] Hudson commented on HBASE-6203: --- Integrated in HBase-TRUNK #3085 (See [https://builds.apache.org/job/HBase-TRUNK/3085/]) HBASE-6203 Create hbase-it module, pom under hbase-it (Enis) (Revision 1355221) HBASE-6203 Create hbase-it module (Enis) (Revision 1355220) Result = SUCCESS tedyu : Files : * /hbase/trunk/hbase-it * /hbase/trunk/hbase-it/pom.xml tedyu : Files : * /hbase/trunk/pom.xml Create hbase-it module -- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6203) Create hbase-it
[ https://issues.apache.org/jira/browse/HBASE-6203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13293972#comment-13293972 ] Enis Soztutar commented on HBASE-6203: -- Some notes: {code} mvn verify {code} runs the tests under hbase-it, named IntegrationTestXXX. Note that {{mvn test}} does not run these tests. You can run just the integration tests, but cd'ing under hbase-it module, or use {code} mvn verify -Dskip-server-tests -Dskip-common-tests {code} You can also skip integration tests with {{-Dskip-integration-tests}}. Failsafe also honors {{-DskipTests}}. Create hbase-it --- Key: HBASE-6203 URL: https://issues.apache.org/jira/browse/HBASE-6203 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Attachments: HBASE-6203_v1.patch Create hbase-it, as per parent issue, and re-introduce HBASE-4454 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira