[jira] [Updated] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12112: --- Attachment: HBASE-12112_V2.patch Avoiding ensureKeyValue() from HStore#getRowKeyAtOrBefore() as well in this patch. That is also straight forward. > Avoid KeyValueUtil#ensureKeyValue some more simple cases > > > Key: HBASE-12112 > URL: https://issues.apache.org/jira/browse/HBASE-12112 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12112.patch, HBASE-12112_V2.patch, > HBASE-12112_V2.patch > > > This include fixes with > - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) > - Printing the key portion of a cell (rk+cf+q+ts+type). These are in > Exception messages > - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to > cell#getxxx() which involves bytes copying. This is not a hot area still we > can avoid as much usage of deprecated methods as possible in core code. I > believe these bytes copying methods are used in many other parts and later we > can try fixing those as per area importance > - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[
https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151417#comment-14151417
]
Hadoop QA commented on HBASE-12042:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671741/hbase-12042_v2.patch
against trunk revision .
ATTACHMENT ID: 12671741
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 195
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Table table = TEST_UTIL.createTable(tableName, new byte[][] { FAMILY
}, conf, Integer.MAX_VALUE);
+HRegion region =
TEST_UTIL.getRSForFirstRegionInTable(tableName).getFromOnlineRegions(regionName);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.regionserver.TestHRegion
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//console
This message is automatically generated.
> Replace internal uses of HTable(Configuration, String) with
> HTable(Configuration, TableName)
>
>
> Key: HBASE-12042
> URL: https://issues.apache.org/jira/browse/HBASE-12042
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.99.0, 2.0.0
>Reporter: Solomon Duskis
>Assignee: Solomon Duskis
> Fix For: 2.0.0, 0.99.1
>
> Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch,
> hbase-12042_v2.patch, hbase-12042_v2.patch
>
>
> Replace internal uses of HTable(Configuration, String) with
> HTable(Configuration, TableName)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12112: --- Attachment: HBASE-12112_V2.patch > Avoid KeyValueUtil#ensureKeyValue some more simple cases > > > Key: HBASE-12112 > URL: https://issues.apache.org/jira/browse/HBASE-12112 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12112.patch, HBASE-12112_V2.patch > > > This include fixes with > - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) > - Printing the key portion of a cell (rk+cf+q+ts+type). These are in > Exception messages > - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to > cell#getxxx() which involves bytes copying. This is not a hot area still we > can avoid as much usage of deprecated methods as possible in core code. I > believe these bytes copying methods are used in many other parts and later we > can try fixing those as per area importance > - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12082) Find a way to set timestamp on Cells on the server
[ https://issues.apache.org/jira/browse/HBASE-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12082: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to 0.99+ Thanks Stack & Enis > Find a way to set timestamp on Cells on the server > -- > > Key: HBASE-12082 > URL: https://issues.apache.org/jira/browse/HBASE-12082 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12082.patch > > > On write path, we have to replace the ts on cells when the coming in ts is > HConstants.LATEST_TIMESTAMP. Also on delete version cells we have to adjust > the ts. All these places, now we do Cell to KV convert. > We can provide a similar way as we have given for setting seqId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[
https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151370#comment-14151370
]
stack commented on HBASE-12040:
---
Here is test where I've checked out branch-1 at Nick's commit so 11331 is
included.It perturbs our numbers. FilteredScans are 'faster' because of
it(?) and RandomRead looks a bit slower after this single commit.
{code}
099.at.filterscan.txt:2014-09-28 21:07:47,858 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 223008ms over 100 rows
099.at.filterscan.txt:2014-09-28 21:11:27,928 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 215842ms over 100 rows
099.at.filterscan.txt:2014-09-28 21:15:07,705 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 215566ms over 100 rows
099.at.filterscan.txt:2014-09-28 21:18:47,816 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 215858ms over 100 rows
099.at.filterscan.txt:2014-09-28 21:22:28,202 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 216107ms over 100 rows
099.at.randomread.txt:2014-09-28 21:30:39,885 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 425668ms over 1048576 rows
099.at.randomread.txt:2014-09-28 21:38:43,537 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 479408ms over 1048576 rows
099.at.randomread.txt:2014-09-28 21:46:00,051 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 432500ms over 1048576 rows
099.at.randomread.txt:2014-09-28 21:53:15,114 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 430753ms over 1048576 rows
099.at.randomread.txt:2014-09-28 22:01:23,224 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 483916ms over 1048576 rows
{code}
> Performances issues with FilteredScanTest
> --
>
> Key: HBASE-12040
> URL: https://issues.apache.org/jira/browse/HBASE-12040
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.99.1
>Reporter: Jean-Marc Spaggiari
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.7, 0.99.1
>
> Attachments: at-HBASE-11331.html, pre-HBASE-11331.html
>
>
> While testing 0.99.0RC1 release performances, compared to 0.98.6, figured
> that:
> - FilteredScanTest is 100 times slower;
> - RandomReadTest is 1.5 times slower;
> - RandomSeekScanTest is 3.2 times slower;
> - RandomScanWithRange10Test is 1,2 times slower;
> - RandomScanWithRange100Test is 1,3 times slower;
> - RandomScanWithRange1000Test is 4 times slower;
> - SequentialReadTest is 1,7 times slower;
> - SequentialWriteTest is just a bit faster;
> - RandomWriteTest is just a bit faster;
> - GaussianRandomReadBenchmark is just a beat slower;
> - SequentialReadBenchmark is 1,1 times slower;
> - SequentialWriteBenchmark is 1,1 times slower;
> - UniformRandomReadBenchmark crashed;
> - UniformRandomSmallScan is 1,3 times slower.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12082) Find a way to set timestamp on Cells on the server
[ https://issues.apache.org/jira/browse/HBASE-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151369#comment-14151369 ] Enis Soztutar commented on HBASE-12082: --- LGTM for branch-1 as well. > Find a way to set timestamp on Cells on the server > -- > > Key: HBASE-12082 > URL: https://issues.apache.org/jira/browse/HBASE-12082 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 0.99.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12082.patch > > > On write path, we have to replace the ts on cells when the coming in ts is > HConstants.LATEST_TIMESTAMP. Also on delete version cells we have to adjust > the ts. All these places, now we do Cell to KV convert. > We can provide a similar way as we have given for setting seqId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[ https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12042: -- Attachment: hbase-12042_v2.patch > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) > > > Key: HBASE-12042 > URL: https://issues.apache.org/jira/browse/HBASE-12042 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 2.0.0 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch, > hbase-12042_v2.patch, hbase-12042_v2.patch > > > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[ https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151367#comment-14151367 ] Enis Soztutar commented on HBASE-12042: --- Let me reattach the patch. It seems that hadoopqa is backed up picking up new patches recently. > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) > > > Key: HBASE-12042 > URL: https://issues.apache.org/jira/browse/HBASE-12042 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 2.0.0 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch, > hbase-12042_v2.patch, hbase-12042_v2.patch > > > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12079) Deprecate KeyValueUtil#ensureKeyValue(s)
[ https://issues.apache.org/jira/browse/HBASE-12079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151366#comment-14151366 ] Hudson commented on HBASE-12079: FAILURE: Integrated in HBase-1.0 #240 (See [https://builds.apache.org/job/HBase-1.0/240/]) HBASE-12079 Deprecate KeyValueUtil#ensureKeyValue(s). (anoopsamjohn: rev 4488dad5618aa53cba6c513749a5e77342d54e36) * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java > Deprecate KeyValueUtil#ensureKeyValue(s) > > > Key: HBASE-12079 > URL: https://issues.apache.org/jira/browse/HBASE-12079 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12079.patch > > > We can deprecate this in 2.0 and remove later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12079) Deprecate KeyValueUtil#ensureKeyValue(s)
[ https://issues.apache.org/jira/browse/HBASE-12079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151345#comment-14151345 ] Hudson commented on HBASE-12079: FAILURE: Integrated in HBase-TRUNK #5570 (See [https://builds.apache.org/job/HBase-TRUNK/5570/]) HBASE-12079 Deprecate KeyValueUtil#ensureKeyValue(s). (anoopsamjohn: rev 8895ee694c7b28d93ad6abc4faaeb805bd211a88) * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java > Deprecate KeyValueUtil#ensureKeyValue(s) > > > Key: HBASE-12079 > URL: https://issues.apache.org/jira/browse/HBASE-12079 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12079.patch > > > We can deprecate this in 2.0 and remove later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12111) Remove deprecated APIs from Mutation(s)
[ https://issues.apache.org/jira/browse/HBASE-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151344#comment-14151344 ] Hudson commented on HBASE-12111: FAILURE: Integrated in HBase-TRUNK #5570 (See [https://builds.apache.org/job/HBase-TRUNK/5570/]) HBASE-12111 Remove deprecated APIs from Mutation(s). (anoopsamjohn: rev b48a474988a56c0a4775ec1dfb72473e6eb64140) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Append.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Mutation.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Put.java > Remove deprecated APIs from Mutation(s) > --- > > Key: HBASE-12111 > URL: https://issues.apache.org/jira/browse/HBASE-12111 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-12111.patch, HBASE-12111.patch > > > Mutation setWriteToWAL(boolean) > boolean getWriteToWAL() > Mutation setFamilyMap(NavigableMap>) > NavigableMap> getFamilyMap() > To be removed from Mutation and the setters from Put/Delete/Increment/Append > as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151338#comment-14151338 ] Devaraj Das commented on HBASE-12040: - I am still clueless as to why I see a degradation with HBASE-11331. Maybe, something in my config is not right. I'll work with Nick to see if he can repro my results and go from there. > Performances issues with FilteredScanTest > -- > > Key: HBASE-12040 > URL: https://issues.apache.org/jira/browse/HBASE-12040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Critical > Fix For: 0.98.7, 0.99.1 > > Attachments: at-HBASE-11331.html, pre-HBASE-11331.html > > > While testing 0.99.0RC1 release performances, compared to 0.98.6, figured > that: > - FilteredScanTest is 100 times slower; > - RandomReadTest is 1.5 times slower; > - RandomSeekScanTest is 3.2 times slower; > - RandomScanWithRange10Test is 1,2 times slower; > - RandomScanWithRange100Test is 1,3 times slower; > - RandomScanWithRange1000Test is 4 times slower; > - SequentialReadTest is 1,7 times slower; > - SequentialWriteTest is just a bit faster; > - RandomWriteTest is just a bit faster; > - GaussianRandomReadBenchmark is just a beat slower; > - SequentialReadBenchmark is 1,1 times slower; > - SequentialWriteBenchmark is 1,1 times slower; > - UniformRandomReadBenchmark crashed; > - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151339#comment-14151339 ] stack commented on HBASE-12112: --- ok on the toJSON. Expensive -- espcially when it is subsequently json'ized -- but makes sense. Ok on the rawcells. Maybe comment that no iterator is made in this case. Otherwise all good by me. > Avoid KeyValueUtil#ensureKeyValue some more simple cases > > > Key: HBASE-12112 > URL: https://issues.apache.org/jira/browse/HBASE-12112 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12112.patch > > > This include fixes with > - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) > - Printing the key portion of a cell (rk+cf+q+ts+type). These are in > Exception messages > - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to > cell#getxxx() which involves bytes copying. This is not a hot area still we > can avoid as much usage of deprecated methods as possible in core code. I > believe these bytes copying methods are used in many other parts and later we > can try fixing those as per area importance > - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151337#comment-14151337 ] Anoop Sam John commented on HBASE-12112: bq. Shall we call it getKey? Should we make a new object Key that has the Key stuff from Cell in it? I like this idea. A key object which is having all rk, cf, q, ts and typo info. A getKey(Cell) in CellUtil would be good. > Avoid KeyValueUtil#ensureKeyValue some more simple cases > > > Key: HBASE-12112 > URL: https://issues.apache.org/jira/browse/HBASE-12112 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12112.patch > > > This include fixes with > - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) > - Printing the key portion of a cell (rk+cf+q+ts+type). These are in > Exception messages > - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to > cell#getxxx() which involves bytes copying. This is not a hot area still we > can avoid as much usage of deprecated methods as possible in core code. I > believe these bytes copying methods are used in many other parts and later we > can try fixing those as per area importance > - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[
https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151334#comment-14151334
]
Anoop Sam John commented on HBASE-12112:
{quote}
Can we avoid creating iterator here?
for (Cell cell : rs.rawCells()) {
{quote}
rawCells() return and array of Cells. So no worry.
> Avoid KeyValueUtil#ensureKeyValue some more simple cases
>
>
> Key: HBASE-12112
> URL: https://issues.apache.org/jira/browse/HBASE-12112
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0, 0.99.1
>
> Attachments: HBASE-12112.patch
>
>
> This include fixes with
> - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell)
> - Printing the key portion of a cell (rk+cf+q+ts+type). These are in
> Exception messages
> - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to
> cell#getxxx() which involves bytes copying. This is not a hot area still we
> can avoid as much usage of deprecated methods as possible in core code. I
> believe these bytes copying methods are used in many other parts and later we
> can try fixing those as per area importance
> - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151332#comment-14151332 ] stack commented on HBASE-12040: --- Comparing 0.99.0 and just before HBASE-11331. I checked out at commit 17b2cea0bd3a98eba4f036d90f7920b1e71776a8 Author: Enis Soztutar Date: Tue Sep 9 19:03:26 2014 -0700 Update CHANGES.txt for 0.99.0 > Performances issues with FilteredScanTest > -- > > Key: HBASE-12040 > URL: https://issues.apache.org/jira/browse/HBASE-12040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Critical > Fix For: 0.98.7, 0.99.1 > > Attachments: at-HBASE-11331.html, pre-HBASE-11331.html > > > While testing 0.99.0RC1 release performances, compared to 0.98.6, figured > that: > - FilteredScanTest is 100 times slower; > - RandomReadTest is 1.5 times slower; > - RandomSeekScanTest is 3.2 times slower; > - RandomScanWithRange10Test is 1,2 times slower; > - RandomScanWithRange100Test is 1,3 times slower; > - RandomScanWithRange1000Test is 4 times slower; > - SequentialReadTest is 1,7 times slower; > - SequentialWriteTest is just a bit faster; > - RandomWriteTest is just a bit faster; > - GaussianRandomReadBenchmark is just a beat slower; > - SequentialReadBenchmark is 1,1 times slower; > - SequentialWriteBenchmark is 1,1 times slower; > - UniformRandomReadBenchmark crashed; > - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5622) Improve efficiency of mapred vesion of RowCounter
[ https://issues.apache.org/jira/browse/HBASE-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151331#comment-14151331 ] Ashish Singhi commented on HBASE-5622: -- Thanks Lars for taking a look into the patch. In mapreduce.RowCounter we prepare a Scan object based on user arguments which includes start and end row key before submitting the job. Where as in mapred.RowCounter we do not prepare any such Scan object. So all the changes done in HBASE-4657 are not suitable to this. > Improve efficiency of mapred vesion of RowCounter > - > > Key: HBASE-5622 > URL: https://issues.apache.org/jira/browse/HBASE-5622 > Project: HBase > Issue Type: Sub-task >Reporter: Lars Hofhansl >Assignee: Ashish Singhi >Priority: Minor > Attachments: HBASE-5622.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12079) Deprecate KeyValueUtil#ensureKeyValue(s)
[ https://issues.apache.org/jira/browse/HBASE-12079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12079: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Deprecate KeyValueUtil#ensureKeyValue(s) > > > Key: HBASE-12079 > URL: https://issues.apache.org/jira/browse/HBASE-12079 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12079.patch > > > We can deprecate this in 2.0 and remove later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12079) Deprecate KeyValueUtil#ensureKeyValue(s)
[ https://issues.apache.org/jira/browse/HBASE-12079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12079: --- Hadoop Flags: Reviewed Status: Patch Available (was: Open) Pushed to branch-1 and master. Thanks Stack. > Deprecate KeyValueUtil#ensureKeyValue(s) > > > Key: HBASE-12079 > URL: https://issues.apache.org/jira/browse/HBASE-12079 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12079.patch > > > We can deprecate this in 2.0 and remove later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12111) Remove deprecated APIs from Mutation(s)
[ https://issues.apache.org/jira/browse/HBASE-12111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12111: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks Stack > Remove deprecated APIs from Mutation(s) > --- > > Key: HBASE-12111 > URL: https://issues.apache.org/jira/browse/HBASE-12111 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-12111.patch, HBASE-12111.patch > > > Mutation setWriteToWAL(boolean) > boolean getWriteToWAL() > Mutation setFamilyMap(NavigableMap>) > NavigableMap> getFamilyMap() > To be removed from Mutation and the setters from Put/Delete/Increment/Append > as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12110) Fix .arcconfig
[ https://issues.apache.org/jira/browse/HBASE-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151284#comment-14151284 ] Hudson commented on HBASE-12110: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #521 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/521/]) HBASE-12110 Fix .arcconfig (eclark: rev c220be925ee4b1a730e23b0b5474d1414a806f31) * .arcconfig > Fix .arcconfig > -- > > Key: HBASE-12110 > URL: https://issues.apache.org/jira/browse/HBASE-12110 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 0.98.7, 0.99.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-HBASE-12110-Fix-.arcconfig.patch > > > Not many people are currently using arc but it's a nice tool for the > developers who are used to it. Since it's already there let's make it work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12110) Fix .arcconfig
[ https://issues.apache.org/jira/browse/HBASE-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151267#comment-14151267 ] Hudson commented on HBASE-12110: FAILURE: Integrated in HBase-1.0 #239 (See [https://builds.apache.org/job/HBase-1.0/239/]) HBASE-12110 Fix .arcconfig (eclark: rev 10490bfbde31cd935c15468e5deb0530dbf8d2a0) * .arcconfig > Fix .arcconfig > -- > > Key: HBASE-12110 > URL: https://issues.apache.org/jira/browse/HBASE-12110 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 0.98.7, 0.99.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-HBASE-12110-Fix-.arcconfig.patch > > > Not many people are currently using arc but it's a nice tool for the > developers who are used to it. Since it's already there let's make it work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12110) Fix .arcconfig
[ https://issues.apache.org/jira/browse/HBASE-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151268#comment-14151268 ] Hudson commented on HBASE-12110: FAILURE: Integrated in HBase-TRUNK #5569 (See [https://builds.apache.org/job/HBase-TRUNK/5569/]) HBASE-12110 Fix .arcconfig (eclark: rev 21e31f0c8685b38015e1d24347a554d45a6e7b8d) * .arcconfig > Fix .arcconfig > -- > > Key: HBASE-12110 > URL: https://issues.apache.org/jira/browse/HBASE-12110 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 0.98.7, 0.99.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-HBASE-12110-Fix-.arcconfig.patch > > > Not many people are currently using arc but it's a nice tool for the > developers who are used to it. Since it's already there let's make it work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[ https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151263#comment-14151263 ] Solomon Duskis commented on HBASE-12042: I ran the v2 patch locally and 3 of the tests passed. TestWALPlayer did not pass. I does not pass in master either. Is there a way to re-run the QA process on the v2 file? > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) > > > Key: HBASE-12042 > URL: https://issues.apache.org/jira/browse/HBASE-12042 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.0, 2.0.0 >Reporter: Solomon Duskis >Assignee: Solomon Duskis > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch, > hbase-12042_v2.patch > > > Replace internal uses of HTable(Configuration, String) with > HTable(Configuration, TableName) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12110) Fix .arcconfig
[ https://issues.apache.org/jira/browse/HBASE-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151262#comment-14151262 ] Hudson commented on HBASE-12110: FAILURE: Integrated in HBase-0.98 #548 (See [https://builds.apache.org/job/HBase-0.98/548/]) HBASE-12110 Fix .arcconfig (eclark: rev c220be925ee4b1a730e23b0b5474d1414a806f31) * .arcconfig > Fix .arcconfig > -- > > Key: HBASE-12110 > URL: https://issues.apache.org/jira/browse/HBASE-12110 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 0.98.7, 0.99.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-HBASE-12110-Fix-.arcconfig.patch > > > Not many people are currently using arc but it's a nice tool for the > developers who are used to it. Since it's already there let's make it work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12110) Fix .arcconfig
[ https://issues.apache.org/jira/browse/HBASE-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12110: -- Resolution: Fixed Fix Version/s: 0.99.1 0.98.7 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed. Thanks > Fix .arcconfig > -- > > Key: HBASE-12110 > URL: https://issues.apache.org/jira/browse/HBASE-12110 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 0.98.7, 0.99.1 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 0.98.7, 0.99.1 > > Attachments: 0001-HBASE-12110-Fix-.arcconfig.patch > > > Not many people are currently using arc but it's a nice tool for the > developers who are used to it. Since it's already there let's make it work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[
https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151238#comment-14151238
]
Hadoop QA commented on HBASE-12029:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671717/HBASE-12029_v5.patch
against trunk revision .
ATTACHMENT ID: 12671717
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6//console
This message is automatically generated.
> Use Table and RegionLocator in HTable.getRegionLocations()
> ---
>
> Key: HBASE-12029
> URL: https://issues.apache.org/jira/browse/HBASE-12029
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 0.99.1
>Reporter: Solomon Duskis
>Assignee: Solomon Duskis
> Attachments: HBASE-12029.patch, HBASE-12029_v1.patch,
> HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch,
> HBASE-12029_v4.patch, HBASE-12029_v5.patch
>
>
> HTable.getRegionLocations() is an internal deprecated API. There are plenty
> of internal uses of the method, and it would be ideal to use the Table and
> RegionLocator interfaces instead of the HTable method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151223#comment-14151223 ] Hudson commented on HBASE-12113: SUCCESS: Integrated in HBase-0.94-JDK7 #194 (See [https://builds.apache.org/job/HBase-0.94-JDK7/194/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb * src/main/ruby/hbase/admin.rb > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151222#comment-14151222 ] Hudson commented on HBASE-5525: --- SUCCESS: Integrated in HBase-0.94-JDK7 #194 (See [https://builds.apache.org/job/HBase-0.94-JDK7/194/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb * src/main/ruby/hbase/admin.rb > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.2 >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Fix For: 0.95.1 > > Attachments: HBASE-5525-v1-0.94.patch, HBASE-5525-v1-0.95.patch, > HBASE-5525-v1-trunk-addendum.patch, HBASE-5525-v1-trunk.patch, > HBASE-5525-v1-trunk.patch, HBASE-5525.patch > > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151218#comment-14151218 ] Hudson commented on HBASE-12113: SUCCESS: Integrated in HBase-0.94 #1424 (See [https://builds.apache.org/job/HBase-0.94/1424/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/hbase/admin.rb * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151217#comment-14151217 ] Hudson commented on HBASE-5525: --- SUCCESS: Integrated in HBase-0.94 #1424 (See [https://builds.apache.org/job/HBase-0.94/1424/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/hbase/admin.rb * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.2 >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Fix For: 0.95.1 > > Attachments: HBASE-5525-v1-0.94.patch, HBASE-5525-v1-0.95.patch, > HBASE-5525-v1-trunk-addendum.patch, HBASE-5525-v1-trunk.patch, > HBASE-5525-v1-trunk.patch, HBASE-5525.patch > > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5525) Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151212#comment-14151212 ] Hudson commented on HBASE-5525: --- SUCCESS: Integrated in HBase-0.94-security #540 (See [https://builds.apache.org/job/HBase-0.94-security/540/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb * src/main/ruby/hbase/admin.rb > Truncate and preserve region boundaries option > -- > > Key: HBASE-5525 > URL: https://issues.apache.org/jira/browse/HBASE-5525 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.2 >Reporter: Jean-Daniel Cryans >Assignee: Kevin Odell > Fix For: 0.95.1 > > Attachments: HBASE-5525-v1-0.94.patch, HBASE-5525-v1-0.95.patch, > HBASE-5525-v1-trunk-addendum.patch, HBASE-5525-v1-trunk.patch, > HBASE-5525-v1-trunk.patch, HBASE-5525.patch > > > A tool that would be useful for testing (and maybe in prod too) would be a > truncate option to keep the current region boundaries. Right now what you > have to do is completely kill the table and recreate it with the correct > regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151213#comment-14151213 ] Hudson commented on HBASE-12113: SUCCESS: Integrated in HBase-0.94-security #540 (See [https://builds.apache.org/job/HBase-0.94-security/540/]) HBASE-12113 Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option. (Sean Busbey, original patch by Kevin Odell) (larsh: rev ec6ea58df0361cb3f8bb62ed8a55240032957b8c) * src/main/ruby/shell/commands/truncate_preserve.rb * src/main/ruby/shell.rb * src/main/ruby/hbase/admin.rb > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[
https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Solomon Duskis updated HBASE-12029:
---
Attachment: HBASE-12029_v5.patch
{quote}
... is the new method getAllRegionLocations a replacement for
getRegionLocations? (It makes use of it...Its the preferred way going forward?)
If so, should the @deprecated have a note added that points folks to the new
method?
{quote}
Given the deprecation comments and the amount of time getRegionLocations() is
used, it does look like there's a need for a replacement. I took a shot at a
possible alternative. I added the note. Do you think that
getAllRegionLocations() is appropriate for mapreduce jobs?
metaScanner is now closed.
Removed the odd additions to RegionLocator. I replaced the RegionLocator calls
with calls to Admin in RegionSizeCalculator.
> Use Table and RegionLocator in HTable.getRegionLocations()
> ---
>
> Key: HBASE-12029
> URL: https://issues.apache.org/jira/browse/HBASE-12029
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 0.99.1
>Reporter: Solomon Duskis
>Assignee: Solomon Duskis
> Attachments: HBASE-12029.patch, HBASE-12029_v1.patch,
> HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch,
> HBASE-12029_v4.patch, HBASE-12029_v5.patch
>
>
> HTable.getRegionLocations() is an internal deprecated API. There are plenty
> of internal uses of the method, and it would be ideal to use the Table and
> RegionLocator interfaces instead of the HTable method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12065) Import tool is not restoring multiple DeleteFamily markers of a row
[ https://issues.apache.org/jira/browse/HBASE-12065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12065: -- Fix Version/s: (was: 0.94.24) 0.94.25 > Import tool is not restoring multiple DeleteFamily markers of a row > > > Key: HBASE-12065 > URL: https://issues.apache.org/jira/browse/HBASE-12065 > Project: HBase > Issue Type: Bug > Components: util >Affects Versions: 0.98.2 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.99.1, 0.94.25 > > Attachments: hbase-12065-fix-2.patch, hbase-12065-fix.patch, > hbase-12065-unit-test.patch > > > When a row has more than one DeleteFamily markers, Import tool is not > restoring all DeleteFamily markers. > Scenario: Insert entries into hbase in below order > Put Row1 with Value-A > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Delete Row1 with DeleteFamily Marker > Using Export tool export this data and Import it into another table, you will > see below entries > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Put Row1 with Value-A > One DeleteFamily marker is missing here... In Import tool, > Importer.writeResult() method we are batching all deletes into a single > Delete request and pushing into hbase. Here we are pushing only one delete > family marker into hbase table. > I tried same with normal HTable.delete command also. > If you pass multiple DeleteFamily markers of a row in a single Delete request > to hbase then table is maintaining only one. > If that is the expected behavior of hbase then we should change logic in > Import tool to push DeleteFamily markers individually one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-12113. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to 0.94. Thanks [~busbey] > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12113: -- Status: Open (was: Patch Available) > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12065) Import tool is not restoring multiple DeleteFamily markers of a row
[ https://issues.apache.org/jira/browse/HBASE-12065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151199#comment-14151199 ] Lars Hofhansl commented on HBASE-12065: --- +1 on this one. > Import tool is not restoring multiple DeleteFamily markers of a row > > > Key: HBASE-12065 > URL: https://issues.apache.org/jira/browse/HBASE-12065 > Project: HBase > Issue Type: Bug > Components: util >Affects Versions: 0.98.2 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 > > Attachments: hbase-12065-fix-2.patch, hbase-12065-fix.patch, > hbase-12065-unit-test.patch > > > When a row has more than one DeleteFamily markers, Import tool is not > restoring all DeleteFamily markers. > Scenario: Insert entries into hbase in below order > Put Row1 with Value-A > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Delete Row1 with DeleteFamily Marker > Using Export tool export this data and Import it into another table, you will > see below entries > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Put Row1 with Value-A > One DeleteFamily marker is missing here... In Import tool, > Importer.writeResult() method we are batching all deletes into a single > Delete request and pushing into hbase. Here we are pushing only one delete > family marker into hbase table. > I tried same with normal HTable.delete command also. > If you pass multiple DeleteFamily markers of a row in a single Delete request > to hbase then table is maintaining only one. > If that is the expected behavior of hbase then we should change logic in > Import tool to push DeleteFamily markers individually one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (HBASE-12065) Import tool is not restoring multiple DeleteFamily markers of a row
[ https://issues.apache.org/jira/browse/HBASE-12065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12065: -- Comment: was deleted (was: Hi Lars, Along with ReplicationSink I think copy table tool also has the same logic.I will change that and write unit tests for each. Thanks Sukumar On Saturday, September 27, 2014, Lars Hofhansl (JIRA) Import tool is not restoring multiple DeleteFamily markers of a row > > > Key: HBASE-12065 > URL: https://issues.apache.org/jira/browse/HBASE-12065 > Project: HBase > Issue Type: Bug > Components: util >Affects Versions: 0.98.2 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 > > Attachments: hbase-12065-fix-2.patch, hbase-12065-fix.patch, > hbase-12065-unit-test.patch > > > When a row has more than one DeleteFamily markers, Import tool is not > restoring all DeleteFamily markers. > Scenario: Insert entries into hbase in below order > Put Row1 with Value-A > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Delete Row1 with DeleteFamily Marker > Using Export tool export this data and Import it into another table, you will > see below entries > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Put Row1 with Value-A > One DeleteFamily marker is missing here... In Import tool, > Importer.writeResult() method we are batching all deletes into a single > Delete request and pushing into hbase. Here we are pushing only one delete > family marker into hbase table. > I tried same with normal HTable.delete command also. > If you pass multiple DeleteFamily markers of a row in a single Delete request > to hbase then table is maintaining only one. > If that is the expected behavior of hbase then we should change logic in > Import tool to push DeleteFamily markers individually one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12065) Import tool is not restoring multiple DeleteFamily markers of a row
[ https://issues.apache.org/jira/browse/HBASE-12065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151198#comment-14151198 ] Lars Hofhansl commented on HBASE-12065: --- Let use another jira for the Replication fix. > Import tool is not restoring multiple DeleteFamily markers of a row > > > Key: HBASE-12065 > URL: https://issues.apache.org/jira/browse/HBASE-12065 > Project: HBase > Issue Type: Bug > Components: util >Affects Versions: 0.98.2 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar >Priority: Minor > Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 > > Attachments: hbase-12065-fix-2.patch, hbase-12065-fix.patch, > hbase-12065-unit-test.patch > > > When a row has more than one DeleteFamily markers, Import tool is not > restoring all DeleteFamily markers. > Scenario: Insert entries into hbase in below order > Put Row1 with Value-A > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Delete Row1 with DeleteFamily Marker > Using Export tool export this data and Import it into another table, you will > see below entries > Delete Row1 with DeleteFamily Marker > Put Row1 with Value-B > Put Row1 with Value-A > One DeleteFamily marker is missing here... In Import tool, > Importer.writeResult() method we are batching all deletes into a single > Delete request and pushing into hbase. Here we are pushing only one delete > family marker into hbase table. > I tried same with normal HTable.delete command also. > If you pass multiple DeleteFamily markers of a row in a single Delete request > to hbase then table is maintaining only one. > If that is the expected behavior of hbase then we should change logic in > Import tool to push DeleteFamily markers individually one by one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151197#comment-14151197 ] Lars Hofhansl commented on HBASE-12113: --- +1 (Hadoop QA only work on HBase master) > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12113) Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12113: -- Summary: Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option (was: Backport HBASE-5525 to 0.94) > Backport to 0.94: HBASE-5525 Truncate and preserve region boundaries option > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12113) Backport HBASE-5525 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12113: -- Fix Version/s: 0.94.24 > Backport HBASE-5525 to 0.94 > --- > > Key: HBASE-12113 > URL: https://issues.apache.org/jira/browse/HBASE-12113 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.94.24 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > Fix For: 0.94.24 > > Attachments: HBASE-12113-0.94.1.patch.txt > > > per [mailing > list|http://mail-archives.apache.org/mod_mbox/hbase-user/201409.mbox/%3C1411880636.89071.YahooMailNeo%40web140604.mail.bf1.yahoo.com%3E] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151141#comment-14151141 ] Andrew Purtell commented on HBASE-12040: bq. Here is from the commit just before 11331. It looks like between this point and the 0.99.0, Are you referring to the 11331 commit to branch-1 and the head of branch-1? > Performances issues with FilteredScanTest > -- > > Key: HBASE-12040 > URL: https://issues.apache.org/jira/browse/HBASE-12040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Critical > Fix For: 0.98.7, 0.99.1 > > Attachments: at-HBASE-11331.html, pre-HBASE-11331.html > > > While testing 0.99.0RC1 release performances, compared to 0.98.6, figured > that: > - FilteredScanTest is 100 times slower; > - RandomReadTest is 1.5 times slower; > - RandomSeekScanTest is 3.2 times slower; > - RandomScanWithRange10Test is 1,2 times slower; > - RandomScanWithRange100Test is 1,3 times slower; > - RandomScanWithRange1000Test is 4 times slower; > - SequentialReadTest is 1,7 times slower; > - SequentialWriteTest is just a bit faster; > - RandomWriteTest is just a bit faster; > - GaussianRandomReadBenchmark is just a beat slower; > - SequentialReadBenchmark is 1,1 times slower; > - SequentialWriteBenchmark is 1,1 times slower; > - UniformRandomReadBenchmark crashed; > - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[
https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151121#comment-14151121
]
stack commented on HBASE-12040:
---
No worries [~jmspaggi] Thanks for doing these tests. Lets see how your reruns
do. If close enough, we can close this and instead take on a more extensive
perf compare JIRA. We need to do it before 1.0 release. These tests are a bit
small. 1M and one client. We should probably do larger scale tests.
Here is from the commit just before 11331. It looks like between this point
and the 0.99.0, we lost some randomread perf (thanks against [~jmspaggi] for
getting us to dig in here) while filteredScan was slower back then. Let me try
and get more detail on why.
{code}
099.before.filterscan.txt:2014-09-27 22:08:55,251 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 250721ms over 100 rows
099.before.filterscan.txt:2014-09-27 22:13:07,068 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 247603ms over 100 rows
099.before.filterscan.txt:2014-09-27 22:17:18,743 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 247378ms over 100 rows
099.before.filterscan.txt:2014-09-27 22:21:30,468 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 247487ms over 100 rows
099.before.filterscan.txt:2014-09-27 22:25:42,518 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 247838ms over 100 rows
099.before.randomread.txt:2014-09-27 22:34:07,019 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 412051ms over 1048576 rows
099.before.randomread.txt:2014-09-27 22:41:21,056 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 429792ms over 1048576 rows
099.before.randomread.txt:2014-09-27 22:47:37,499 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 372189ms over 1048576 rows
099.before.randomread.txt:2014-09-27 22:54:33,021 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 411320ms over 1048576 rows
099.before.randomread.txt:2014-09-27 23:01:36,469 INFO [TestClient-0]
hbase.PerformanceEvaluation: Finished in 419260ms over 1048576 rows
{code}
> Performances issues with FilteredScanTest
> --
>
> Key: HBASE-12040
> URL: https://issues.apache.org/jira/browse/HBASE-12040
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.99.1
>Reporter: Jean-Marc Spaggiari
>Assignee: stack
>Priority: Critical
> Fix For: 0.98.7, 0.99.1
>
> Attachments: at-HBASE-11331.html, pre-HBASE-11331.html
>
>
> While testing 0.99.0RC1 release performances, compared to 0.98.6, figured
> that:
> - FilteredScanTest is 100 times slower;
> - RandomReadTest is 1.5 times slower;
> - RandomSeekScanTest is 3.2 times slower;
> - RandomScanWithRange10Test is 1,2 times slower;
> - RandomScanWithRange100Test is 1,3 times slower;
> - RandomScanWithRange1000Test is 4 times slower;
> - SequentialReadTest is 1,7 times slower;
> - SequentialWriteTest is just a bit faster;
> - RandomWriteTest is just a bit faster;
> - GaussianRandomReadBenchmark is just a beat slower;
> - SequentialReadBenchmark is 1,1 times slower;
> - SequentialWriteBenchmark is 1,1 times slower;
> - UniformRandomReadBenchmark crashed;
> - UniformRandomSmallScan is 1,3 times slower.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11920) Add CP hooks for ReplicationEndPoint
[ https://issues.apache.org/jira/browse/HBASE-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151105#comment-14151105 ] Hudson commented on HBASE-11920: FAILURE: Integrated in HBase-1.0 #238 (See [https://builds.apache.org/job/HBase-1.0/238/]) HBASE-11920 Add CP hooks for ReplicationEndPoint (ramkrishna: rev 8a7920ecdbe4b70878a363355818f4803ba88c22) * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionServerObserver.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerCoprocessorHost.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionServerObserver.java > Add CP hooks for ReplicationEndPoint > > > Key: HBASE-11920 > URL: https://issues.apache.org/jira/browse/HBASE-11920 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, Replication >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-11920.patch, HBASE-11920_1.patch, > HBASE-11920_2.patch, HBASE-11920_3.patch, HBASE-11920_4.patch, > HBASE-11920_6.patch > > > If we want to create internal replication endpoints other than the one > created thro configuration we may need new hooks. This is something like an > internal scanner that we create during compaction so that the actual > compaction scanner can be used as a delegator. > [~enis] > If I can give a patch by tomorrow will it be possible to include in the RC? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151101#comment-14151101 ] Jean-Marc Spaggiari commented on HBASE-12040: - Ok. Stop spending to much time on that. I retried few branches last night and figured that at some point, the SequentialWrite because mapred by default and was --nomapred before. However, in mapred mode it failed because it looks for the build jars to be in HDFS but is stacktrace is lost into the gigantic log file where I just search for the "Finished" lines... So sound like some FilteredScanTest are running against an empty table while some other are running against a full table. I will update the SequentialWrite part of the script and update my numbers for the 10 last releases (At least up to where this mapred default value changed) I have tested, update the numbers and report back here. I changed the priority as I have good suspucions that it's coming from here and should not be a blocker anymore... If that's the case, I'm very sorry about that! > Performances issues with FilteredScanTest > -- > > Key: HBASE-12040 > URL: https://issues.apache.org/jira/browse/HBASE-12040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Critical > Fix For: 0.98.7, 0.99.1 > > Attachments: at-HBASE-11331.html, pre-HBASE-11331.html > > > While testing 0.99.0RC1 release performances, compared to 0.98.6, figured > that: > - FilteredScanTest is 100 times slower; > - RandomReadTest is 1.5 times slower; > - RandomSeekScanTest is 3.2 times slower; > - RandomScanWithRange10Test is 1,2 times slower; > - RandomScanWithRange100Test is 1,3 times slower; > - RandomScanWithRange1000Test is 4 times slower; > - SequentialReadTest is 1,7 times slower; > - SequentialWriteTest is just a bit faster; > - RandomWriteTest is just a bit faster; > - GaussianRandomReadBenchmark is just a beat slower; > - SequentialReadBenchmark is 1,1 times slower; > - SequentialWriteBenchmark is 1,1 times slower; > - UniformRandomReadBenchmark crashed; > - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-12040: Priority: Critical (was: Blocker) > Performances issues with FilteredScanTest > -- > > Key: HBASE-12040 > URL: https://issues.apache.org/jira/browse/HBASE-12040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.99.1 >Reporter: Jean-Marc Spaggiari >Assignee: stack >Priority: Critical > Fix For: 0.98.7, 0.99.1 > > Attachments: at-HBASE-11331.html, pre-HBASE-11331.html > > > While testing 0.99.0RC1 release performances, compared to 0.98.6, figured > that: > - FilteredScanTest is 100 times slower; > - RandomReadTest is 1.5 times slower; > - RandomSeekScanTest is 3.2 times slower; > - RandomScanWithRange10Test is 1,2 times slower; > - RandomScanWithRange100Test is 1,3 times slower; > - RandomScanWithRange1000Test is 4 times slower; > - SequentialReadTest is 1,7 times slower; > - SequentialWriteTest is just a bit faster; > - RandomWriteTest is just a bit faster; > - GaussianRandomReadBenchmark is just a beat slower; > - SequentialReadBenchmark is 1,1 times slower; > - SequentialWriteBenchmark is 1,1 times slower; > - UniformRandomReadBenchmark crashed; > - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11920) Add CP hooks for ReplicationEndPoint
[ https://issues.apache.org/jira/browse/HBASE-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11920: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1 also. Thanks Enis. > Add CP hooks for ReplicationEndPoint > > > Key: HBASE-11920 > URL: https://issues.apache.org/jira/browse/HBASE-11920 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, Replication >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 0.99.1 > > Attachments: HBASE-11920.patch, HBASE-11920_1.patch, > HBASE-11920_2.patch, HBASE-11920_3.patch, HBASE-11920_4.patch, > HBASE-11920_6.patch > > > If we want to create internal replication endpoints other than the one > created thro configuration we may need new hooks. This is something like an > internal scanner that we create during compaction so that the actual > compaction scanner can be used as a delegator. > [~enis] > If I can give a patch by tomorrow will it be possible to include in the RC? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12090) Bytes: more Unsafe, more Faster
[ https://issues.apache.org/jira/browse/HBASE-12090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151055#comment-14151055 ] Hudson commented on HBASE-12090: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #520 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/520/]) HBASE-12090 Addendum (larsh: rev b3783d221a83819b4697b6763d6c45bd06620920) * hbase-common/src/main/java/org/apache/hadoop/hbase/util/Bytes.java > Bytes: more Unsafe, more Faster > > > Key: HBASE-12090 > URL: https://issues.apache.org/jira/browse/HBASE-12090 > Project: HBase > Issue Type: Improvement > Components: Performance >Affects Versions: 0.94.23, 0.98.6 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 > > Attachments: 12090-v1.1.txt, 12090-v3.txt, HBASE-12090.2.patch, > HBASE-12090.ADD.patch, HBASE-12090.patch > > > Additional optimizations to *org.apache.hadoop.hbase.util.Bytes*: > * New version of compareTo method. > * New versions for primitive converters : putXXX/toXXX. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11957) Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151018#comment-14151018 ] Lars Hofhansl commented on HBASE-11957: --- [~liushaohui], no problem. You just backported the patch. I wasn't happy because I committed it and did not watch the tests for close to a week :) Still no successful secure build, but at least we got a full JDK6 and JDK7 build. Not sure the secure build is related to this. > Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() > seems incorrect > - > > Key: HBASE-11957 > URL: https://issues.apache.org/jira/browse/HBASE-11957 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Critical > Fix For: 0.94.24 > > Attachments: 11957-addendum-2.txt, 11957-addendum.txt, > HBASE-5974-0.94-v1.diff, verify-test.patch > > > HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, > which cause data missing in hbase scan. > I think we should fix it in 0.94. > [~lhofhansl] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11957) Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151013#comment-14151013 ] Liu Shaohui commented on HBASE-11957: - [~lhofhansl] Sorry for troubling you. I will test the patch more carefully before submitting it. > Backport to 0.94 HBASE-5974 Scanner retry behavior with RPC timeout on next() > seems incorrect > - > > Key: HBASE-11957 > URL: https://issues.apache.org/jira/browse/HBASE-11957 > Project: HBase > Issue Type: Bug >Reporter: Liu Shaohui >Assignee: Liu Shaohui >Priority: Critical > Fix For: 0.94.24 > > Attachments: 11957-addendum-2.txt, 11957-addendum.txt, > HBASE-5974-0.94-v1.diff, verify-test.patch > > > HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, > which cause data missing in hbase scan. > I think we should fix it in 0.94. > [~lhofhansl] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
