[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464332#comment-13464332 ] Hudson commented on HBASE-6702: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #194 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/194/]) HBASE-6702 ResourceChecker refinement (Revision 1390433) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-common/pom.xml * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/LargeTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/MediumTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceCheckerJUnitListener.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/SmallTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestThreads.java * /hbase/trunk/hbase-it/pom.xml * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/LargeTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MediumTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ResourceCheckerJUnitRule.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ServerResourceCheckerJUnitListener.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/SmallTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestCompare.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaMigrationConvertingToPB.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAttributes.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestGet.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hb
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463941#comment-13463941 ] nkeywal commented on HBASE-6702: I wrote something in "HBASE-6884 Update documentation on unit tests". But I mainly suppressed the old stuff, without explaining the listener itself. About it, we could write: A specific listener checks a number of resources before and after each tests and logs them. These resources are the number of threads, the number of file descriptor, ... If the number has increased, it adds a "LEAK?" comment in the logs. As you can have an HBase instance running in the background, some threads can be deleted/created without any specific action in the test. However, if the test does not work as expected, or if the test should not impact these resources, it's worth checking these log lines "hbase.ResourceChecker(157): before" and "hbase.ResourceChecker(157): after" > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463932#comment-13463932 ] stack commented on HBASE-6702: -- Let me know if you want me to update the doc. I can do it no problem. Would remove the old stuff and talk about how we have a listener in surefire that dumps a line at end of each unit test noting thread count, etc. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[
https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463930#comment-13463930
]
stack commented on HBASE-6702:
--
I don't mind us failing on missing license. I heart resource checker.
Confirmed its still doing its stuff in its new listener format:
{code}
2012-09-26 09:22:15,315 INFO [pool-1-thread-1] hbase.ResourceChecker(157):
after: regionserver.TestColumnSeeking#testReseeking Thread=65 (was 65),
OpenFileDescriptor=107 (was 107), MaxFileDescriptor=10240 (was 10240),
ConnectionCount=1 (was 1)
{code}
> ResourceChecker refinement
> --
>
> Key: HBASE-6702
> URL: https://issues.apache.org/jira/browse/HBASE-6702
> Project: HBase
> Issue Type: Improvement
> Components: test
>Affects Versions: 0.96.0
>Reporter: Jesse Yates
>Assignee: nkeywal
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch
>
>
> This was based on some discussion from HBASE-6234.
> The ResourceChecker was added by N. Keywal to help resolve some hadoop qa
> issues, but has since not be widely utilized. Further, with modularization we
> have had to drop the ResourceChecker from the tests that are moved into the
> hbase-common module because bringing the ResourceChecker up to hbase-common
> would involved bringing all its dependencies (which are quite far reaching).
> The question then is, what should we do with it? Get rid of it? Refactor and
> resuse?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463923#comment-13463923 ] stack commented on HBASE-6702: -- This is nice cleanup [~nkeywal] > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463908#comment-13463908 ] nkeywal commented on HBASE-6702: [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase: Too many unapproved licenses: 1 -> [Help 1] btw, May be it could be changed to be a non blocking error? > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463904#comment-13463904 ] Hudson commented on HBASE-6702: --- Integrated in HBase-TRUNK #3380 (See [https://builds.apache.org/job/HBase-TRUNK/3380/]) HBASE-6702 ResourceChecker refinement (Revision 1390433) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-common/pom.xml * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/LargeTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/MediumTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/ResourceCheckerJUnitListener.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/SmallTests.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestThreads.java * /hbase/trunk/hbase-it/pom.xml * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/IntegrationTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/LargeTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/MediumTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ResourceChecker.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ResourceCheckerJUnitRule.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/ServerResourceCheckerJUnitListener.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/SmallTests.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestCompare.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestFullLogReconstruction.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestInfoServers.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestMultiVersions.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerName.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaMigrationConvertingToPB.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAttributes.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFakeKeyInFilter.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestGet.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTablePool.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPaginat
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463887#comment-13463887 ] nkeywal commented on HBASE-6702: Committed revision 1390433. Committed the v5, except the change on docbkx/developer.xml. It includes Jesse's comments. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463728#comment-13463728 ] Hadoop QA commented on HBASE-6702: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546677/6702.v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 858 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 140 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2933//console This message is automatically generated. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch, 6702.v5.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463586#comment-13463586 ] nkeywal commented on HBASE-6702: [~saint@gmail.com] There is everything in this patch, except the documentation update that I will do in a different jira. Instead of migrating to a newer surefire version, the local tests version now uses the same as the parallel tests (so it's our patched version). > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463528#comment-13463528 ] stack commented on HBASE-6702: -- +1 on commit after addressing Jesse comments. The rest of the convertion work would be done in another issue? Good stuff N. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[
https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463063#comment-13463063
]
Jesse Yates commented on HBASE-6702:
Good stuff keywal! Just a couple comments:
{code}
+ hbase-common
+ ${project.version}
+ test-jar
+ test
+
+
{code}
To keep DRY, the aboves should go into hbase/pom.xml's dependencyManagement
section and then the children projects should just use:
{code}
+ hbase-common
+ test-jar
+
+
{code}
Also, any chance for some javadocs on things like:
{code}
+ public ResourceChecker(String tagLine) {
+this.tagLine = tagLine;
+ }
{code}
Otherwise, this is a really sweet add.
> ResourceChecker refinement
> --
>
> Key: HBASE-6702
> URL: https://issues.apache.org/jira/browse/HBASE-6702
> Project: HBase
> Issue Type: Improvement
> Components: test
>Affects Versions: 0.96.0
>Reporter: Jesse Yates
>Assignee: nkeywal
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6702.v1.patch, 6702.v4.patch
>
>
> This was based on some discussion from HBASE-6234.
> The ResourceChecker was added by N. Keywal to help resolve some hadoop qa
> issues, but has since not be widely utilized. Further, with modularization we
> have had to drop the ResourceChecker from the tests that are moved into the
> hbase-common module because bringing the ResourceChecker up to hbase-common
> would involved bringing all its dependencies (which are quite far reaching).
> The question then is, what should we do with it? Get rid of it? Refactor and
> resuse?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462726#comment-13462726 ] nkeywal commented on HBASE-6702: Seems ok... > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462724#comment-13462724 ] Hadoop QA commented on HBASE-6702: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546497/6702.v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 858 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 140 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2930//console This message is automatically generated. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch, 6702.v4.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462526#comment-13462526 ] nkeywal commented on HBASE-6702: bq. What is this change? I've changed the interface of the resource checker, but not yet removed ResourceCheckerJUnitRule, so I've just commented the removed methods. bq. Whats this mean 'migrate the localTests to a newer version of surefire'? The log lines don't show up with surefire 2.10. It works with my patched version. But the localTests profile uses the 2.10. It's historical: I've done it this way because we don't use categories nor parallelization for localTests. The v2 should be "ready for commit' and will include your comments. Thanks for the review! > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[
https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462453#comment-13462453
]
stack commented on HBASE-6702:
--
License missing from ResourceCheckerJUnitListener.java
ResourceCheckerJUnitListener could do w/ a class comment saying what its about
What is this change?
- cu.logInfo("after " + testName);
- cu.check("after "+testName);
ServerResourceCheckerJUnitListener is probably an ok name for this subclass...
adding server resource checking.
It looks promising.
Whats this mean 'migrate the localTests to a newer version of surefire'?
> ResourceChecker refinement
> --
>
> Key: HBASE-6702
> URL: https://issues.apache.org/jira/browse/HBASE-6702
> Project: HBase
> Issue Type: Improvement
> Components: test
>Affects Versions: 0.96.0
>Reporter: Jesse Yates
>Assignee: nkeywal
>Priority: Critical
> Fix For: 0.96.0
>
> Attachments: 6702.v1.patch
>
>
> This was based on some discussion from HBASE-6234.
> The ResourceChecker was added by N. Keywal to help resolve some hadoop qa
> issues, but has since not be widely utilized. Further, with modularization we
> have had to drop the ResourceChecker from the tests that are moved into the
> hbase-common module because bringing the ResourceChecker up to hbase-common
> would involved bringing all its dependencies (which are quite far reaching).
> The question then is, what should we do with it? Get rid of it? Refactor and
> resuse?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461880#comment-13461880 ] nkeywal commented on HBASE-6702: The issue with the official 2.10 release is likely coming SUREFIRE-800. But using the patched version for all types of test seems ok to me. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461869#comment-13461869 ] nkeywal commented on HBASE-6702: v1. Done: - move categories tests to common - add a dependency from hbase-server to common for the tests - put the checkers in common, restructure to allow to add new checker - add a checker specific to HBaseServer Still needed: - migrate the localTests to a newer version of surefire - remove the JUnitRule stuff - check the impact of hbase integration But globally everything is there. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > Attachments: 6702.v1.patch > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461752#comment-13461752 ] nkeywal commented on HBASE-6702: I will be able to make it work with a listener. It has been messy as usual (it seems it doesn't work with surefire 2.10), but it should be ok. I need to do some tests with the last surefire version and the forkMode=once as well. The advantage is that there is nothing to add in the test class. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Assignee: nkeywal >Priority: Critical > Fix For: 0.96.0 > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459841#comment-13459841 ] stack commented on HBASE-6702: -- Thanks [~nkeywal] > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459459#comment-13459459 ] nkeywal commented on HBASE-6702: Yeah. I don't have any opinion in the field vs. common base class approach. Both are ok to me. What I don't remember is if I was not constrained by the JUnit annotation approach. I just remember I had to try multiple options, before finding a suitable one. Something to consider as well is to declare the runlistener in the pom. I know I didn't try it at this time, but it could be simpler actually (it would be nothing to add at all :-) ). I will give it a try. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.96.0 >Reporter: Jesse Yates >Priority: Critical > Fix For: 0.96.0 > > > This was based on some discussion from HBASE-6234. > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The question then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6702) ResourceChecker refinement
[ https://issues.apache.org/jira/browse/HBASE-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446122#comment-13446122 ] Jesse Yates commented on HBASE-6702: One of the most annoying parts about writing tests has been making sure to include the ResourceChecker in all the tests. Its a one liner but needs to be done _everywhere_. I'd propose that as part of this ticket we also go back to a HBaseTest superclass for tests that includes this one-liner so we don't have duplicate code that is common all over the place (one of the best gains from OOP). WRT what to do with the ResourceChecker itself, I'd like to see a split of its responsibilities into two classes. (1) A general resource checker that checks on things like threads, thread pools, files, etc., and (2) an HBaseResourceChecker that extends the above and also checks HBase specific things like HConnections, etc. (1) would then be moved into hbase-common and used for those tests, while (2) would continue as the resource checker for any of the hbase-server (or its dependencies) tests. > ResourceChecker refinement > -- > > Key: HBASE-6702 > URL: https://issues.apache.org/jira/browse/HBASE-6702 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.96.0 >Reporter: Jesse Yates > Fix For: 0.96.0 > > > The ResourceChecker was added by N. Keywal to help resolve some hadoop qa > issues, but has since not be widely utilized. Further, with modularization we > have had to drop the ResourceChecker from the tests that are moved into the > hbase-common module because bringing the ResourceChecker up to hbase-common > would involved bringing all its dependencies (which are quite far reaching). > The quest then is, what should we do with it? Get rid of it? Refactor and > resuse? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
