[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165314#comment-14165314 ] Elliott Clark commented on HBASE-12199: --- Different versions of Hadoop have different syncing behaviors. Some will sync the local filesystem (1.0 and FB's old 0.20) others won't sync (2.0+). The patch speeds up the older hadoops a lot by making everything even. It also speeds up the newer hadoops a little bit by not waiting for the hlog syncer at all. I think this behavior was chosen because it can't change semantics of any test while it can provide a speed up. Since our tests are a single process that never continues after crash syncing is wasted time; linux and osx will always give a read by this process the dirty pag. > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164760#comment-14164760 ] Lars Hofhansl commented on HBASE-12199: --- Why not SKIP_WAL? > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164206#comment-14164206 ] Hudson commented on HBASE-12199: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #554 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/554/]) HBASE-12199 Make TestAtomicOperation and TestEncodedSeekers faster (apurtell: rev be64e4892f1b59da96be2c556200bf974d797b60) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164175#comment-14164175 ] Hudson commented on HBASE-12199: SUCCESS: Integrated in HBase-0.98 #584 (See [https://builds.apache.org/job/HBase-0.98/584/]) HBASE-12199 Make TestAtomicOperation and TestEncodedSeekers faster (apurtell: rev be64e4892f1b59da96be2c556200bf974d797b60) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163971#comment-14163971 ] Hudson commented on HBASE-12199: SUCCESS: Integrated in HBase-1.0 #285 (See [https://builds.apache.org/job/HBase-1.0/285/]) HBASE-12199 Make TestAtomicOperation and TestEncodedSeekers faster (eclark: rev 283ec576413c0c958c95a8c1812b78d7e019218d) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14163806#comment-14163806 ] Hudson commented on HBASE-12199: SUCCESS: Integrated in HBase-TRUNK #5635 (See [https://builds.apache.org/job/HBase-TRUNK/5635/]) HBASE-12199 Make TestAtomicOperation and TestEncodedSeekers faster (eclark: rev e952e736242cd79f9eae523773ab3b9ce788341c) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestEncodedSeekers.java > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 2.0.0, 0.99.1 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[
https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162949#comment-14162949
]
Hadoop QA commented on HBASE-12199:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673489/0001-Fix-TestAtomicOperation.patch
against trunk revision .
ATTACHMENT ID: 12673489
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11256//console
This message is automatically generated.
> Make TestAtomicOperation and TestEncodedSeekers faster
> --
>
> Key: HBASE-12199
> URL: https://issues.apache.org/jira/browse/HBASE-12199
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.0.0
>Reporter: Manukranth Kolloju
>Assignee: Manukranth Kolloju
>Priority: Trivial
> Fix For: 1.0.0
>
> Attachments: 0001-Fix-TestAtomicOperation.patch,
> 0001-INTERNAL-Fix-TestAtomicOperation.patch
>
>
> TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do
> puts, appends, deletes, inc and then does gets to make sure that things
> happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object.
> So, in essence the test doesn't need to sync to disk to make sure the edits
> have persisted. There is no regionserver/master to take care of wal replay or
> any failure for that matter. So, converting all the Durability settings for
> the edits to ASYNC_WAL seems like the right thing to do here to make the test
> run faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12199) Make TestAtomicOperation and TestEncodedSeekers faster
[ https://issues.apache.org/jira/browse/HBASE-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162878#comment-14162878 ] Manukranth Kolloju commented on HBASE-12199: Done. Thanks for the plus one. > Make TestAtomicOperation and TestEncodedSeekers faster > -- > > Key: HBASE-12199 > URL: https://issues.apache.org/jira/browse/HBASE-12199 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.0.0 >Reporter: Manukranth Kolloju >Assignee: Manukranth Kolloju >Priority: Trivial > Fix For: 1.0.0 > > Attachments: 0001-Fix-TestAtomicOperation.patch, > 0001-INTERNAL-Fix-TestAtomicOperation.patch > > > TestAtomicOperation has a bunch of tests which spawn a bunch of tests that do > puts, appends, deletes, inc and then does gets to make sure that things > happened. It uses TEST_UTIL.getLocalHRegion() to obtain an HRegion object. > So, in essence the test doesn't need to sync to disk to make sure the edits > have persisted. There is no regionserver/master to take care of wal replay or > any failure for that matter. So, converting all the Durability settings for > the edits to ASYNC_WAL seems like the right thing to do here to make the test > run faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
