[jira] [Updated] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12112: --- Attachment: HBASE-12112_V2.patch Avoid KeyValueUtil#ensureKeyValue some more simple cases Key: HBASE-12112 URL: https://issues.apache.org/jira/browse/HBASE-12112 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12112.patch, HBASE-12112_V2.patch This include fixes with - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) - Printing the key portion of a cell (rk+cf+q+ts+type). These are in Exception messages - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to cell#getxxx() which involves bytes copying. This is not a hot area still we can avoid as much usage of deprecated methods as possible in core code. I believe these bytes copying methods are used in many other parts and later we can try fixing those as per area importance - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[
https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151417#comment-14151417
]
Hadoop QA commented on HBASE-12042:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671741/hbase-12042_v2.patch
against trunk revision .
ATTACHMENT ID: 12671741
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 195
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Table table = TEST_UTIL.createTable(tableName, new byte[][] { FAMILY
}, conf, Integer.MAX_VALUE);
+HRegion region =
TEST_UTIL.getRSForFirstRegionInTable(tableName).getFromOnlineRegions(regionName);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.regionserver.TestHRegion
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7//console
This message is automatically generated.
Replace internal uses of HTable(Configuration, String) with
HTable(Configuration, TableName)
Key: HBASE-12042
URL: https://issues.apache.org/jira/browse/HBASE-12042
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Fix For: 2.0.0, 0.99.1
Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch,
hbase-12042_v2.patch, hbase-12042_v2.patch
Replace internal uses of HTable(Configuration, String) with
HTable(Configuration, TableName)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[ https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-12112: --- Attachment: HBASE-12112_V2.patch Avoiding ensureKeyValue() from HStore#getRowKeyAtOrBefore() as well in this patch. That is also straight forward. Avoid KeyValueUtil#ensureKeyValue some more simple cases Key: HBASE-12112 URL: https://issues.apache.org/jira/browse/HBASE-12112 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12112.patch, HBASE-12112_V2.patch, HBASE-12112_V2.patch This include fixes with - Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell) - Printing the key portion of a cell (rk+cf+q+ts+type). These are in Exception messages - HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to cell#getxxx() which involves bytes copying. This is not a hot area still we can avoid as much usage of deprecated methods as possible in core code. I believe these bytes copying methods are used in many other parts and later we can try fixing those as per area importance - Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12082) Find a way to set timestamp on Cells on the server
[ https://issues.apache.org/jira/browse/HBASE-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151444#comment-14151444 ] Hudson commented on HBASE-12082: FAILURE: Integrated in HBase-TRUNK #5571 (See [https://builds.apache.org/job/HBase-TRUNK/5571/]) HBASE-12082 Find a way to set timestamp on Cells on the server. (anoopsamjohn: rev df2ce0c3896bec9fe128a13490b05f7fea0757c9) * hbase-common/src/main/java/org/apache/hadoop/hbase/SettableTimestamp.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowMutationProcessor.java Find a way to set timestamp on Cells on the server -- Key: HBASE-12082 URL: https://issues.apache.org/jira/browse/HBASE-12082 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.99.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12082.patch On write path, we have to replace the ts on cells when the coming in ts is HConstants.LATEST_TIMESTAMP. Also on delete version cells we have to adjust the ts. All these places, now we do Cell to KV convert. We can provide a similar way as we have given for setting seqId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11871) Avoid usage of KeyValueUtil#ensureKeyValue
[ https://issues.apache.org/jira/browse/HBASE-11871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151448#comment-14151448 ] Anoop Sam John commented on HBASE-11871: We are almost there once HBASE-12112 also goes in :) The only left out will be in WALEdit#write (I don't think this is getting used really) and places in mapreduce package classes mapreduce places actually we get KeyValues but because of API signature we have to do ensureKeyValue(Cell). To change to Cell, we will have to change the mapred job map/reduce o/p value format. As these are public exposed we can not change directly. So I am not planing to work on this immediately. Over all in read and write paths we don't anymore use KeyValueUtil#ensureKeyValue() and all in Cell also now. :) Avoid usage of KeyValueUtil#ensureKeyValue -- Key: HBASE-11871 URL: https://issues.apache.org/jira/browse/HBASE-11871 Project: HBase Issue Type: Umbrella Components: regionserver Reporter: Anoop Sam John Assignee: Anoop Sam John Now we try to use diff impls of Cells (other than KV) in server side for optimizations. (HBASE-11870 is one like this) Having code KeyValueUtil#ensureKeyValue can be dangerous as it recreate the Cell and do buffer copy. This is an umbrella issue. Will raise small sub tasks in specific areas and tackle this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[
https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151462#comment-14151462
]
Hadoop QA commented on HBASE-12112:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671747/HBASE-12112_V2.patch
against trunk revision .
ATTACHMENT ID: 12671747
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.client.TestMultiParallel
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
at
org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed.testTimeRangeMapRed(TestTimeRangeMapRed.java:156)
at
org.apache.camel.test.blueprint.CamelBlueprintHelper.getOsgiService(CamelBlueprintHelper.java:227)
at
org.apache.camel.test.blueprint.CamelBlueprintHelper.getOsgiService(CamelBlueprintHelper.java:202)
at
org.apache.camel.test.blueprint.CamelBlueprintTestSupport.createCamelContext(CamelBlueprintTestSupport.java:352)
at
org.apache.camel.test.junit4.CamelTestSupport.doSetUp(CamelTestSupport.java:247)
at
org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:217)
at
org.apache.camel.test.blueprint.CamelBlueprintTestSupport.setUp(CamelBlueprintTestSupport.java:183)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8//console
This message is automatically generated.
Avoid KeyValueUtil#ensureKeyValue some more simple cases
Key: HBASE-12112
URL: https://issues.apache.org/jira/browse/HBASE-12112
Project: HBase
Issue Type: Sub-task
Components: regionserver
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0, 0.99.1
Attachments: HBASE-12112.patch, HBASE-12112_V2.patch,
HBASE-12112_V2.patch
This include fixes with
- Replace
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[
https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151464#comment-14151464
]
Hadoop QA commented on HBASE-12112:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671752/HBASE-12112_V2.patch
against trunk revision .
ATTACHMENT ID: 12671752
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.camel.test.blueprint.CamelBlueprintHelper.getOsgiService(CamelBlueprintHelper.java:227)
at
org.apache.camel.test.blueprint.CamelBlueprintHelper.getOsgiService(CamelBlueprintHelper.java:202)
at
org.apache.camel.test.blueprint.CamelBlueprintTestSupport.createCamelContext(CamelBlueprintTestSupport.java:352)
at
org.apache.camel.test.junit4.CamelTestSupport.doSetUp(CamelTestSupport.java:247)
at
org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:217)
at
org.apache.camel.test.blueprint.CamelBlueprintTestSupport.setUp(CamelBlueprintTestSupport.java:183)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9//console
This message is automatically generated.
Avoid KeyValueUtil#ensureKeyValue some more simple cases
Key: HBASE-12112
URL: https://issues.apache.org/jira/browse/HBASE-12112
Project: HBase
Issue Type: Sub-task
Components: regionserver
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0, 0.99.1
Attachments: HBASE-12112.patch, HBASE-12112_V2.patch,
HBASE-12112_V2.patch
This include fixes with
- Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell)
- Printing the key portion of a cell (rk+cf+q+ts+type). These are in
Exception messages
- HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to
cell#getxxx() which involves bytes copying. This is not a hot area still we
can avoid as much usage of deprecated methods as possible in core code. I
believe these
[jira] [Commented] (HBASE-12082) Find a way to set timestamp on Cells on the server
[ https://issues.apache.org/jira/browse/HBASE-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151514#comment-14151514 ] Hudson commented on HBASE-12082: FAILURE: Integrated in HBase-1.0 #241 (See [https://builds.apache.org/job/HBase-1.0/241/]) HBASE-12082 Find a way to set timestamp on Cells on the server. (anoopsamjohn: rev 97a458010dff897049ccb9e5884c17172e34f080) * hbase-common/src/main/java/org/apache/hadoop/hbase/SettableTimestamp.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowMutationProcessor.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java Find a way to set timestamp on Cells on the server -- Key: HBASE-12082 URL: https://issues.apache.org/jira/browse/HBASE-12082 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 0.99.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12082.patch On write path, we have to replace the ts on cells when the coming in ts is HConstants.LATEST_TIMESTAMP. Also on delete version cells we have to adjust the ts. All these places, now we do Cell to KV convert. We can provide a similar way as we have given for setting seqId -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12114) meta table cache is broken and cause access the wrong table
He Liangliang created HBASE-12114: - Summary: meta table cache is broken and cause access the wrong table Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-12114 started by He Liangliang. - meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Liangliang updated HBASE-12114: -- Attachment: HBASE-12114.patch meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12096) In ZKSplitLog Coordination and AggregateImplementation replace enhaced for statements with basic for statement to avoid unnecessary object allocation
[
https://issues.apache.org/jira/browse/HBASE-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151585#comment-14151585
]
Ashish Singhi commented on HBASE-12096:
---
Thanks Ted for the review.
{noformat}
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
{noformat}
I ran twice locally passed both the times.
In ZKSplitLog Coordination and AggregateImplementation replace enhaced for
statements with basic for statement to avoid unnecessary object allocation
-
Key: HBASE-12096
URL: https://issues.apache.org/jira/browse/HBASE-12096
Project: HBase
Issue Type: Sub-task
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-12096.patch
In ZKSplitLog Coordination classes and AggregateImplementation replace
enhanced for statements with basic for statement to avoid unnecessary object
allocation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11879) Change TableInputFormatBase to take interface arguments
[
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151642#comment-14151642
]
Solomon Duskis commented on HBASE-11879:
[~ndimiduk]: What's the good word on this patch?
Change TableInputFormatBase to take interface arguments
---
Key: HBASE-11879
URL: https://issues.apache.org/jira/browse/HBASE-11879
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Fix For: 0.99.1
Attachments: HBASE_11879.patch, HBASE_11879_v1.patch
As part of the ongoing interface abstraction work, I'm now investigating
{{TableInputFormatBase}}, which has two methods that break encapsulation:
{code}
protected HTable getHTable();
protected void setHTable(HTable table);
{code}
While these are protected methods, the base @InterfaceAudience.Public is
abstract, meaning that it supports extension by user code.
I propose deprecating these two methods and replacing them with these four,
once the Table interface is merged:
{code}
protected Table getTable();
protected void setTable(Table table);
protected RegionLocator getRegionLocator();
protected void setRegionLocator(RegionLocator regionLocator);
{code}
Since users will frequently call {{setTable}} and {{setRegionLocator}}
together, it probably also makes sense to add the following convenience
method:
{code}
protected void setTableAndRegionLocator(Table table, RegionLocator
regionLocator);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11995) Use Connection and ConnectionFactory where possible
[ https://issues.apache.org/jira/browse/HBASE-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-11995: --- Attachment: HBASE-11995_v2.patch Merged from master. Use Connection and ConnectionFactory where possible --- Key: HBASE-11995 URL: https://issues.apache.org/jira/browse/HBASE-11995 Project: HBase Issue Type: New Feature Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11995_v2.patch, HBASE_11995.patch, HBASE_11995.patch, hbase-11955-0.99.patch HBASE-11825 introduced Connection and ConnectionFactory. Uses of HConnection and HConnectionManager should be migrated to use Connection and ConnectionFactory, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11995) Use Connection and ConnectionFactory where possible
[ https://issues.apache.org/jira/browse/HBASE-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-11995: --- Attachment: (was: HBASE-11995_v2.patch) Use Connection and ConnectionFactory where possible --- Key: HBASE-11995 URL: https://issues.apache.org/jira/browse/HBASE-11995 Project: HBase Issue Type: New Feature Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.1 Attachments: HBASE_11995.patch, HBASE_11995.patch, hbase-11955-0.99.patch HBASE-11825 introduced Connection and ConnectionFactory. Uses of HConnection and HConnectionManager should be migrated to use Connection and ConnectionFactory, if possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151732#comment-14151732 ] Ted Yu commented on HBASE-12114: Can you add a test showing that there is no collision for the above two table names ? meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Help，please！Hbase standalone won't start
I did all http://hbase.apache.org/book/quickstart.html told. when I start Hbase,it gives. I am running it on win7 x64 without hadoop. I really need get this running, thanks. Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method) sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:295 ) sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.ja va:277) sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:158) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.ja va:578) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:564 ) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 45) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 15) java.lang.Thread.run(Thread.java:722) Thread 30 (Thread-5): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.net.dns.ResolverConfigurationImpl.notifyAddrChange0(Native Method) sun.net.dns.ResolverConfigurationImpl$AddressChangeListener.run(ResolverConf igurationImpl.java:142) Thread 28 (Timer for 'HBase' metrics system): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Thread 6 (Attach Listener): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 5 (Signal Dispatcher): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 3 (Finalizer): State: WAITING Blocked count: 14 Waited count: 6 Waiting on java.lang.ref.ReferenceQueue$Lock@1aa75223 Stack: java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177) Thread 2 (Reference Handler): State: WAITING Blocked count: 5 Waited count: 6 Waiting on java.lang.ref.Reference$Lock@5f018580 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) Thread 1 (main): State: RUNNABLE Blocked count: 12 Waited count: 457 Stack: sun.management.ThreadImpl.getThreadInfo1(Native Method) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:174) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:139) org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java: 165) org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:219) org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:425 ) org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommand Line.java:182) org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.jav a:135) org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java :126) org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2785) 2014-09-29 14:48:18,316 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Master not initialized after 20ms seconds at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.ja va:221) at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.j ava:425) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMaster CommandLine.java:182) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandL ine.java:135) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLi ne.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2785) 2014-09-29 14:48:18,319 INFO [Thread-6] regionserver.ShutdownHook: Shutdown hoo k starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSy stem$Cache$ClientFinalizer@54c053e3 2014-09-29 14:48:18,320 INFO [Thread-6] regionserver.HRegionServer: STOPPED: Sh utdown hook 2014-09-29 14:48:18,452 INFO [RS:0;gta-like:54149-SendThread(127.0.0.1:2181)] z ookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:218 1. Will not attempt to authenticate using SASL (unknown error) 2014-09-29 14:48:19,239 INFO [RS:0;gta-like:54149] ipc.RpcServer: Stopping serv er on 54149 2014-09-29 14:48:19,240 FATAL [RS:0;gta-like:54149] regionserver.HRegionServer: ABORTING region server gta-like.gtadata.com,54149,1411973097012: Initialization of RS failed. Hence aborting RS. java.io.IOException: Received the shutdown message while waiting. at org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfSto
[jira] [Updated] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju updated HBASE-12075: --- Attachment: 0001-Implement-Preemptive-Fast-Fail.patch Fixing the javadoc warnings. Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11763: --- Attachment: HBASE-11763-0.98.patch Move TTL handling into ScanQueryMatcher --- Key: HBASE-11763 URL: https://issues.apache.org/jira/browse/HBASE-11763 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch, HBASE-11763.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Status: Open (was: Patch Available) Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Attachment: HBASE-11764-0.98.patch Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11763: --- Status: Patch Available (was: Open) Move TTL handling into ScanQueryMatcher --- Key: HBASE-11763 URL: https://issues.apache.org/jira/browse/HBASE-11763 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch, HBASE-11763.patch, HBASE-11763.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11871) Avoid usage of KeyValueUtil#ensureKeyValue
[ https://issues.apache.org/jira/browse/HBASE-11871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151862#comment-14151862 ] stack commented on HBASE-11871: --- bq. Over all in read and write paths we don't anymore use KeyValueUtil#ensureKeyValue() and all in Cell also now. Wahooo!!! Test would be a Cell whose implementation was other than that of KVs say something from prefix key or something simple that just changes location of one of the fields -- e.g. inlined tags with key. Avoid usage of KeyValueUtil#ensureKeyValue -- Key: HBASE-11871 URL: https://issues.apache.org/jira/browse/HBASE-11871 Project: HBase Issue Type: Umbrella Components: regionserver Reporter: Anoop Sam John Assignee: Anoop Sam John Now we try to use diff impls of Cells (other than KV) in server side for optimizations. (HBASE-11870 is one like this) Having code KeyValueUtil#ensureKeyValue can be dangerous as it recreate the Cell and do buffer copy. This is an umbrella issue. Will raise small sub tasks in specific areas and tackle this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12112) Avoid KeyValueUtil#ensureKeyValue some more simple cases
[
https://issues.apache.org/jira/browse/HBASE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151859#comment-14151859
]
stack commented on HBASE-12112:
---
CellKey would replace places where we make a Cell with a null value. I think
that makes sense.
Add a class comment on CellKey on commit on what it is for.
Change this on commit to be getCellKey:
public static CellKey getKey(Cell cell){
Still need this even though you just added CellKey?
public static Cell createKeyOnlyCell(Cell c, boolean lenAsVal) {
In hfile v2 and v3 where you convert cell to be a cellkey just so you can
toString it, its 'clean' but expensive. I suppose it is fine. Ignore this
remark.
+1 on commit. Nice.
Avoid KeyValueUtil#ensureKeyValue some more simple cases
Key: HBASE-12112
URL: https://issues.apache.org/jira/browse/HBASE-12112
Project: HBase
Issue Type: Sub-task
Components: regionserver
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0, 0.99.1
Attachments: HBASE-12112.patch, HBASE-12112_V2.patch,
HBASE-12112_V2.patch
This include fixes with
- Replace KeyValue#heapSize() with CellUtil#estimatedHeapSizeOf(Cell)
- Printing the key portion of a cell (rk+cf+q+ts+type). These are in
Exception messages
- HFilePrettyPrinter - Avoiding ensureKeyValue() calls and calls to
cell#getxxx() which involves bytes copying. This is not a hot area still we
can avoid as much usage of deprecated methods as possible in core code. I
believe these bytes copying methods are used in many other parts and later we
can try fixing those as per area importance
- Creating CellUtil#createKeyOnlyCell and using that in KeyOnlyFilter
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Attachment: HBASE-11764.patch Sigh. Attached combined patch with HBASE-11763 so Jenkins can apply it. Can I get a review please? Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Status: Open (was: Patch Available) Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Status: Patch Available (was: Open) Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[
https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151863#comment-14151863
]
Hadoop QA commented on HBASE-11764:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671817/HBASE-11764.patch
against trunk revision .
ATTACHMENT ID: 12671817
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11122//console
This message is automatically generated.
Support per cell TTLs
-
Key: HBASE-11764
URL: https://issues.apache.org/jira/browse/HBASE-11764
Project: HBase
Issue Type: Sub-task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 0.98.7, 0.99.1
Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch,
HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch,
HBASE-11764.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[ https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151871#comment-14151871 ] stack commented on HBASE-11853: --- See above comment [~posix4e] ? What else to make it so build on travis? Any doc on what you do to make it run? Provide an alternative to the apache build for developers (like me) who aren't committers - Key: HBASE-11853 URL: https://issues.apache.org/jira/browse/HBASE-11853 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11853-testing-v0.patch, HBASE-11853-testing-v1.patch, HBASE-11853-v3.patch, HBASE-11853-v4.patch Travis CI and Circle-CI now provide free builds for open source projects. I created the capability to run builds this way. Although they are closed source (and thus not a replacement for jenkins IMHO), they are super convenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Attachment: HBASE-11764.patch Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11764) Support per cell TTLs
[ https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11764: --- Status: Patch Available (was: Open) Support per cell TTLs - Key: HBASE-11764 URL: https://issues.apache.org/jira/browse/HBASE-11764 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[ https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151876#comment-14151876 ] Alex Newman commented on HBASE-11853: - Good question. I have everything but yarn working on circle-ci, running in 45 minutes. https://circleci.com/gh/OhmData/hbase-public/329 Every once in a while we get a flakey failure but it works (sans yarn). I am reaching out to the yarn guys, i need to rebase and i need to a bit more for travis. However, it's probably good enough for a start. I'll try to put a patch up for circle today. Provide an alternative to the apache build for developers (like me) who aren't committers - Key: HBASE-11853 URL: https://issues.apache.org/jira/browse/HBASE-11853 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11853-testing-v0.patch, HBASE-11853-testing-v1.patch, HBASE-11853-v3.patch, HBASE-11853-v4.patch Travis CI and Circle-CI now provide free builds for open source projects. I created the capability to run builds this way. Although they are closed source (and thus not a replacement for jenkins IMHO), they are super convenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11763: --- Attachment: HBASE-11763.patch Move TTL handling into ScanQueryMatcher --- Key: HBASE-11763 URL: https://issues.apache.org/jira/browse/HBASE-11763 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch, HBASE-11763.patch, HBASE-11763.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[ https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11853: Attachment: HBASE-11853-v0.patch Here's the patch. MR/YARN tests don't run yet. Provide an alternative to the apache build for developers (like me) who aren't committers - Key: HBASE-11853 URL: https://issues.apache.org/jira/browse/HBASE-11853 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11853-testing-v0.patch, HBASE-11853-testing-v1.patch, HBASE-11853-v0.patch, HBASE-11853-v3.patch, HBASE-11853-v4.patch Travis CI and Circle-CI now provide free builds for open source projects. I created the capability to run builds this way. Although they are closed source (and thus not a replacement for jenkins IMHO), they are super convenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11763) Move TTL handling into ScanQueryMatcher
[
https://issues.apache.org/jira/browse/HBASE-11763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151892#comment-14151892
]
Hadoop QA commented on HBASE-11763:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671814/HBASE-11763.patch
against trunk revision .
ATTACHMENT ID: 12671814
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 8 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11121//console
This message is automatically generated.
Move TTL handling into ScanQueryMatcher
---
Key: HBASE-11763
URL: https://issues.apache.org/jira/browse/HBASE-11763
Project: HBase
Issue Type: Sub-task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 0.98.7, 0.99.1
Attachments: HBASE-11763-0.98.patch, HBASE-11763.patch,
HBASE-11763.patch, HBASE-11763.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151895#comment-14151895 ] Lars Hofhansl commented on HBASE-12114: --- +1 There will be no collissions as the Java HashMap uses a hashCode *and* equals. meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151897#comment-14151897 ] Lars Hofhansl commented on HBASE-12114: --- Still... A unit test would be great, so that we do not break this again. meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151898#comment-14151898 ] Lars Hofhansl commented on HBASE-12114: --- I sunk the current 0.94.24 RC for this. meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151902#comment-14151902 ] Lars Hofhansl commented on HBASE-12114: --- This is not a problem in 0.98+ as we use TableName, which properly performs hash/equals. meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[ https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151911#comment-14151911 ] stack commented on HBASE-11853: --- Is that build on circle-ci skipping tests? When you say except mr/yarn, you mean not running hbase tests that have been annotated as mr? Provide an alternative to the apache build for developers (like me) who aren't committers - Key: HBASE-11853 URL: https://issues.apache.org/jira/browse/HBASE-11853 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11853-testing-v0.patch, HBASE-11853-testing-v1.patch, HBASE-11853-v0.patch, HBASE-11853-v3.patch, HBASE-11853-v4.patch Travis CI and Circle-CI now provide free builds for open source projects. I created the capability to run builds this way. Although they are closed source (and thus not a replacement for jenkins IMHO), they are super convenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11964) Improve spreading replication load from failed regionservers
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151915#comment-14151915 ] stack commented on HBASE-11964: --- Go with new configs that are backed with experience trying to use replication; can only be better than what was there previous. Turn on as default in branch-1 and master. In 0.94 suggest new configs. in doc. In 0.98, ?, enable with big release note? Improve spreading replication load from failed regionservers Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1, 0.94.25 Attachments: HBASE-11964-0.98.patch, HBASE-11964-0.98.patch, HBASE-11964.patch, HBASE-11964.patch Improve replication source thread handling. Improve fanout when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-6290) Add a function a mark a server as dead and start the recovery the process
[ https://issues.apache.org/jira/browse/HBASE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-6290. -- Resolution: Fixed Fix Version/s: 2.0.0 Release Note: Adds a script to mark a server as dead. Usage: considerAsDead.sh --hostname serverName Hadoop Flags: Reviewed Committed to master. Thanks [~talat] Add a function a mark a server as dead and start the recovery the process - Key: HBASE-6290 URL: https://issues.apache.org/jira/browse/HBASE-6290 Project: HBase Issue Type: Improvement Components: monitoring Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Talat UYARER Priority: Minor Labels: beginner Fix For: 2.0.0 Attachments: 6290v2.txt, HBASE-6290.patch ZooKeeper is used a a monitoring tool: we use znode and we start the recovery process when a znode is deleted by ZK because it got a timeout. This timeout is defaulted to 90 seconds, and often set to 30s However, some HW issues could be detected by specialized hw monitoring tools before the ZK timeout. For this reason, it makes sense to offer a very simple function to mark a RS as dead. This should not take in It could be a hbase shell function such as considerAsDead ipAddress|serverName This would delete all the znodes of the server running on this box, starting the recovery process. Such a function would be easily callable (at callers risk) by any fault detection tool... We could have issues to identify the right master region servers around ipv4 vs ipv6 vs and multi networked boxes however. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[ https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151930#comment-14151930 ] Alex Newman commented on HBASE-11853: - Correct. Currently the yarn tests take more than 4G of memory. I'm working with the yarn guys on fixing it now. Provide an alternative to the apache build for developers (like me) who aren't committers - Key: HBASE-11853 URL: https://issues.apache.org/jira/browse/HBASE-11853 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11853-testing-v0.patch, HBASE-11853-testing-v1.patch, HBASE-11853-v0.patch, HBASE-11853-v3.patch, HBASE-11853-v4.patch Travis CI and Circle-CI now provide free builds for open source projects. I created the capability to run builds this way. Although they are closed source (and thus not a replacement for jenkins IMHO), they are super convenient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-6290) Add a function a mark a server as dead and start the recovery the process
[ https://issues.apache.org/jira/browse/HBASE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-6290: - Attachment: 6290.doc.addendum.txt Committed this documentation addendum to master branch. Add a function a mark a server as dead and start the recovery the process - Key: HBASE-6290 URL: https://issues.apache.org/jira/browse/HBASE-6290 Project: HBase Issue Type: Improvement Components: monitoring Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Talat UYARER Priority: Minor Labels: beginner Fix For: 2.0.0 Attachments: 6290.doc.addendum.txt, 6290v2.txt, HBASE-6290.patch ZooKeeper is used a a monitoring tool: we use znode and we start the recovery process when a znode is deleted by ZK because it got a timeout. This timeout is defaulted to 90 seconds, and often set to 30s However, some HW issues could be detected by specialized hw monitoring tools before the ZK timeout. For this reason, it makes sense to offer a very simple function to mark a RS as dead. This should not take in It could be a hbase shell function such as considerAsDead ipAddress|serverName This would delete all the znodes of the server running on this box, starting the recovery process. Such a function would be easily callable (at callers risk) by any fault detection tool... We could have issues to identify the right master region servers around ipv4 vs ipv6 vs and multi networked boxes however. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-11394) Replication can have data loss if peer id contains hyphen -
[ https://issues.apache.org/jira/browse/HBASE-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Talat UYARER reassigned HBASE-11394: Assignee: Talat UYARER Replication can have data loss if peer id contains hyphen - - Key: HBASE-11394 URL: https://issues.apache.org/jira/browse/HBASE-11394 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Talat UYARER Fix For: 2.0.0, 0.99.1 This is an extension to HBASE-8207. It seems that there is no check for the peer id string (which is the short name for the replication peer) format. So in case a peer id containing -, it will cause data loss silently on server failure. I did not verify the claim via testing though, this is just purely from reading the code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12109) user_permission command for namespace does not return correct result
[ https://issues.apache.org/jira/browse/HBASE-12109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-12109: - Attachment: HBASE-12109_1.patch user_permission command for namespace does not return correct result Key: HBASE-12109 URL: https://issues.apache.org/jira/browse/HBASE-12109 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-12109_0.patch, HBASE-12109_1.patch The existing user_permission command does not return permissions related to namespace. The permissions exist in the acl table, but the user_permission.rb does not handle namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) meta table cache is broken and cause access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152001#comment-14152001 ] Lars Hofhansl commented on HBASE-12114: --- A unittest might be JDK specific. Going to commit to 0.94 as is. And run a new RC. meta table cache is broken and cause access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12109) user_permission command for namespace does not return correct result
[ https://issues.apache.org/jira/browse/HBASE-12109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152003#comment-14152003 ] Ted Yu commented on HBASE-12109: lgtm user_permission command for namespace does not return correct result Key: HBASE-12109 URL: https://issues.apache.org/jira/browse/HBASE-12109 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-12109_0.patch, HBASE-12109_1.patch The existing user_permission command does not return permissions related to namespace. The permissions exist in the acl table, but the user_permission.rb does not handle namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12109) user_permission command for namespace does not return correct result
[ https://issues.apache.org/jira/browse/HBASE-12109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152022#comment-14152022 ] Matteo Bertozzi commented on HBASE-12109: - I don't think you can get a null namespace descriptor back, the master seems to throw a NamespaceNotFoundException anyway, +1 for me user_permission command for namespace does not return correct result Key: HBASE-12109 URL: https://issues.apache.org/jira/browse/HBASE-12109 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-12109_0.patch, HBASE-12109_1.patch The existing user_permission command does not return permissions related to namespace. The permissions exist in the acl table, but the user_permission.rb does not handle namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[
https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152023#comment-14152023
]
Jean-Marc Spaggiari commented on HBASE-12040:
-
{quote}
These tests are a bit small. 1M and one client. We should probably do larger
scale tests.
{quote}
I hear you there. I will add some more tests with increased load for the next
releases.
Ran tests again. Tests are average of 10 runs.
||Version||results||
|0.98 branch|0.5354156791|
|d6cc2fb|0.5298372735|
|c682d57|0.5359010151|
|b885130|0.5111323656|
|Branch-1|0.5437815828|
I will re-run 0.98.5 and 0.98.6 to correct my numbers...
Performances issues with FilteredScanTest
--
Key: HBASE-12040
URL: https://issues.apache.org/jira/browse/HBASE-12040
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.1
Reporter: Jean-Marc Spaggiari
Assignee: stack
Priority: Critical
Fix For: 0.98.7, 0.99.1
Attachments: at-HBASE-11331.html, pre-HBASE-11331.html
While testing 0.99.0RC1 release performances, compared to 0.98.6, figured
that:
- FilteredScanTest is 100 times slower;
- RandomReadTest is 1.5 times slower;
- RandomSeekScanTest is 3.2 times slower;
- RandomScanWithRange10Test is 1,2 times slower;
- RandomScanWithRange100Test is 1,3 times slower;
- RandomScanWithRange1000Test is 4 times slower;
- SequentialReadTest is 1,7 times slower;
- SequentialWriteTest is just a bit faster;
- RandomWriteTest is just a bit faster;
- GaussianRandomReadBenchmark is just a beat slower;
- SequentialReadBenchmark is 1,1 times slower;
- SequentialWriteBenchmark is 1,1 times slower;
- UniformRandomReadBenchmark crashed;
- UniformRandomSmallScan is 1,3 times slower.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12038) Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents.
[ https://issues.apache.org/jira/browse/HBASE-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12038: --- Affects Version/s: (was: 0.99.0) 0.99.1 Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents. Key: HBASE-12038 URL: https://issues.apache.org/jira/browse/HBASE-12038 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis Signatures with byte[] and String table name variables were deprecated in favor of a TableName input. Replace all byte[] and String table name variables with TableName types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12114) Meta table cache hashing may access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12114: -- Summary: Meta table cache hashing may access the wrong table (was: meta table cache is broken and cause access the wrong table) Meta table cache hashing may access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12114) Meta table cache hashing may access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-12114. --- Resolution: Fixed Committed to 0.94 Thanks for the patch [~heliangliang]. If you get time to think of a test, we can add one in another jira. Meta table cache hashing may access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11964) Improve spreading replication load from failed regionservers
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11964: --- Status: Open (was: Patch Available) Improve spreading replication load from failed regionservers Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1, 0.94.25 Attachments: HBASE-11964-0.98.patch, HBASE-11964-0.98.patch, HBASE-11964.patch, HBASE-11964.patch Improve replication source thread handling. Improve fanout when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11964) Improve spreading replication load from failed regionservers
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11964: --- Status: Patch Available (was: Open) Let me fire off another precommit build. Looks like maybe one consequence of making this change default somewhere is some replication unit tests will need fixing. Improve spreading replication load from failed regionservers Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.99.1, 0.94.25 Attachments: HBASE-11964-0.98.patch, HBASE-11964-0.98.patch, HBASE-11964.patch, HBASE-11964.patch Improve replication source thread handling. Improve fanout when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11853) Provide an alternative to the apache build for developers (like me) who aren't committers
[
https://issues.apache.org/jira/browse/HBASE-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152046#comment-14152046
]
Hadoop QA commented on HBASE-11853:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671822/HBASE-11853-v0.patch
against trunk revision .
ATTACHMENT ID: 12671822
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestMultiParallel
org.apache.hadoop.hbase.TestZooKeeper
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11124//console
This message is automatically generated.
Provide an alternative to the apache build for developers (like me) who
aren't committers
-
Key: HBASE-11853
URL: https://issues.apache.org/jira/browse/HBASE-11853
Project: HBase
Issue Type: Sub-task
Reporter: Alex Newman
Assignee: Alex Newman
Attachments: HBASE-11853-testing-v0.patch,
HBASE-11853-testing-v1.patch, HBASE-11853-v0.patch, HBASE-11853-v3.patch,
HBASE-11853-v4.patch
Travis CI and Circle-CI now provide free builds for open source projects. I
created the capability to run builds this way. Although they are closed
source (and thus not a replacement for jenkins IMHO), they are super
convenient.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[
https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152053#comment-14152053
]
Hadoop QA commented on HBASE-11764:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12671818/HBASE-11764.patch
against trunk revision .
ATTACHMENT ID: 12671818
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ IteratorTag tagIterator =
CellUtil.tagsIterator(cell.getTagsArray(), cell.getTagsOffset(),
+updatedCells.add(new KeyValue(cell.getRowArray(),
cell.getRowOffset(), cell.getRowLength(),
+ IteratorTag i =
CellUtil.tagsIterator(cell.getTagsArray(), cell.getTagsOffset(),
+if (t.getType() == TagType.TTL_TAG_TYPE
increment.getTTL() != Long.MAX_VALUE) {
+ long readPointToUse, long earliestPutTs, long oldestUnexpiredTS, long
now, byte[] dropDeletesFromRow,
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
org.apache.hadoop.hbase.client.TestReplicaWithCluster
{color:red}-1 core zombie tests{color}. There are 4 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339)
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsOnRSRestart(TestVisibilityLabels.java:406)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService.testAddVisibilityLabelsOnRSRestart(TestVisibilityLabelsWithDefaultVisLabelService.java:115)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsOnKillingOfRSContainingLabelsTable(TestVisibilityLabels.java:359)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11123//console
This message is automatically generated.
Support per cell TTLs
-
Key: HBASE-11764
URL:
[jira] [Updated] (HBASE-12109) user_permission command for namespace does not return correct result
[ https://issues.apache.org/jira/browse/HBASE-12109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-12109: - Attachment: HBASE-12109_2.patch Addressed Matteo's comment. user_permission command for namespace does not return correct result Key: HBASE-12109 URL: https://issues.apache.org/jira/browse/HBASE-12109 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-12109_0.patch, HBASE-12109_1.patch, HBASE-12109_2.patch The existing user_permission command does not return permissions related to namespace. The permissions exist in the acl table, but the user_permission.rb does not handle namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152072#comment-14152072 ] Manukranth Kolloju commented on HBASE-12075: Created a phabricator diff to facilitate the review : https://reviews.facebook.net/D24177 Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11804) Raise default heap size if unspecified
[ https://issues.apache.org/jira/browse/HBASE-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152106#comment-14152106 ] Elliott Clark commented on HBASE-11804: --- Working on the release note now. Raise default heap size if unspecified -- Key: HBASE-11804 URL: https://issues.apache.org/jira/browse/HBASE-11804 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 2.0.0 Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 0001-HBASE-11804-Raise-default-heap-size-if-unspecified.patch On master running pe randomWrite with ten threads and the default heap size crashes the system. This works on 0.98 and before. It's been a long time that 1000mb has been our default. Maybe we should start looking at raising that limit on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6290) Add a function a mark a server as dead and start the recovery the process
[ https://issues.apache.org/jira/browse/HBASE-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152118#comment-14152118 ] Hudson commented on HBASE-6290: --- FAILURE: Integrated in HBase-TRUNK #5572 (See [https://builds.apache.org/job/HBase-TRUNK/5572/]) HBASE-6290 Add a function a mark a server as dead and start the recovery the process (Talat Uyarer) (stack: rev 4be3cf8fec8390c4129fa037b63718e84db01656) * bin/considerAsDead.sh HBASE-6290 Add a function a mark a server as dead and start the recovery the process (Talat Uyarer) (stack: rev 8e5aad43a2fbde2aac6aff6dd01582933bd8db9b) * bin/considerAsDead.sh HBASE-6290 Add a function a mark a server as dead and start the recovery the process (Talat Uyarer) DOC ADDENDUM (stack: rev 82bb37070379bb8e135ee566589e8e348e0bdd78) * src/main/docbkx/ops_mgt.xml Add a function a mark a server as dead and start the recovery the process - Key: HBASE-6290 URL: https://issues.apache.org/jira/browse/HBASE-6290 Project: HBase Issue Type: Improvement Components: monitoring Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Talat UYARER Priority: Minor Labels: beginner Fix For: 2.0.0 Attachments: 6290.doc.addendum.txt, 6290v2.txt, HBASE-6290.patch ZooKeeper is used a a monitoring tool: we use znode and we start the recovery process when a znode is deleted by ZK because it got a timeout. This timeout is defaulted to 90 seconds, and often set to 30s However, some HW issues could be detected by specialized hw monitoring tools before the ZK timeout. For this reason, it makes sense to offer a very simple function to mark a RS as dead. This should not take in It could be a hbase shell function such as considerAsDead ipAddress|serverName This would delete all the znodes of the server running on this box, starting the recovery process. Such a function would be easily callable (at callers risk) by any fault detection tool... We could have issues to identify the right master region servers around ipv4 vs ipv6 vs and multi networked boxes however. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-12038) Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents.
[ https://issues.apache.org/jira/browse/HBASE-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-12038 started by Solomon Duskis. -- Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents. Key: HBASE-12038 URL: https://issues.apache.org/jira/browse/HBASE-12038 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis Signatures with byte[] and String table name variables were deprecated in favor of a TableName input. Replace all byte[] and String table name variables with TableName types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12038) Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents.
[ https://issues.apache.org/jira/browse/HBASE-12038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12038: --- Attachment: HBASE-12038.patch Replaced lots of HTable constructors from the byte[] version to the TableName version Replace internal uses of signatures with byte[] and String tableNames to use the TableName equivalents. Key: HBASE-12038 URL: https://issues.apache.org/jira/browse/HBASE-12038 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12038.patch Signatures with byte[] and String table name variables were deprecated in favor of a TableName input. Replace all byte[] and String table name variables with TableName types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) Meta table cache hashing may access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152202#comment-14152202 ] Hudson commented on HBASE-12114: SUCCESS: Integrated in HBase-0.94 #1426 (See [https://builds.apache.org/job/HBase-0.94/1426/]) HBASE-12114 Meta table cache hashing may access the wrong table. (He Liangliang) (larsh: rev ffd3c383abb863c29b00b2e016d8116d7f9c2c1a) * src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java Meta table cache hashing may access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kiran Kumar M R updated HBASE-10200:
Affects Version/s: 2.0.0
Status: Patch Available (was: Open)
This patch adds check to verify if {{hbase.master.info.bindAddress}} and
{{hbase.regionserver.info.bindAddress}} belong to host machine or not.
If address does not belong to the machine, then error message is logged and
exception is thrown during HTTP Server startup/
hbase.util.Addressing#isLocalAddress method is added to check if an address is
local. This method is from Hadoop netutils but not reused(Its marked as
Interface Private in Hadoop.)
Please review and let me know any suggestions.
This patch is for master (2.0.0) branch.
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Priority: Minor
Labels: beginner
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kiran Kumar M R updated HBASE-10200:
Attachment: HBASE-10020.patch
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Priority: Minor
Labels: beginner
Attachments: HBASE-10020.patch
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12003) Fix SecureBulkLoadEndpoint class javadoc formatting
[ https://issues.apache.org/jira/browse/HBASE-12003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-12003: - Resolution: Fixed Status: Resolved (was: Patch Available) Pushed. Fix SecureBulkLoadEndpoint class javadoc formatting --- Key: HBASE-12003 URL: https://issues.apache.org/jira/browse/HBASE-12003 Project: HBase Issue Type: Improvement Components: documentation, security Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Trivial Fix For: 2.0.0 Attachments: HBASE-12003.00.patch, HBASE-12003.00.patch Noticed while reviewing security docs. We link off to the javadoc on said class from the book. Trivial fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) Meta table cache hashing may access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152210#comment-14152210 ] Hudson commented on HBASE-12114: SUCCESS: Integrated in HBase-0.94-JDK7 #195 (See [https://builds.apache.org/job/HBase-0.94-JDK7/195/]) HBASE-12114 Meta table cache hashing may access the wrong table. (He Liangliang) (larsh: rev ffd3c383abb863c29b00b2e016d8116d7f9c2c1a) * src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java Meta table cache hashing may access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11764) Support per cell TTLs
[
https://issues.apache.org/jira/browse/HBASE-11764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152229#comment-14152229
]
Andrew Purtell commented on HBASE-11764:
The reported test failures are unrelated to this patch. I checked out the tip
of master branch and ran the indicated tests and they fail for me locally. For
example TestReplicaWithCluster#testReplicaAndReplication will time out after 30
seconds.
The 0.98 unit test suite can complete cleanly. Here is a run using the 0.98
version of this patch. The changes are identical to the master patch modulo
substitutions of KeyValue for Cell.
{noformat}
[INFO] Building HBase - Common 0.98.7-SNAPSHOT
Tests run: 164, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] Building HBase - Client 0.98.7-SNAPSHOT
Tests run: 51, Failures: 0, Errors: 0, Skipped: 1
...
[INFO] Building HBase - Hadoop Compatibility 0.98.7-SNAPSHOT
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] Building HBase - Hadoop Two Compatibility 0.98.7-SNAPSHOT
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] Building HBase - Prefix Tree 0.98.7-SNAPSHOT
Tests run: 248, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] Building HBase - Server 0.98.7-SNAPSHOT
Tests run: 831, Failures: 0, Errors: 0, Skipped: 3
Tests run: 1256, Failures: 0, Errors: 0, Skipped: 8
...
[INFO] Building HBase - Thrift 0.98.7-SNAPSHOT
Tests run: 77, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 55:05 min
[INFO] Finished at: 2014-09-29T13:15:18-08:00
[INFO] Final Memory: 48M/408M
{noformat}
Support per cell TTLs
-
Key: HBASE-11764
URL: https://issues.apache.org/jira/browse/HBASE-11764
Project: HBase
Issue Type: Sub-task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 0.98.7, 0.99.1
Attachments: HBASE-11764-0.98.patch, HBASE-11764.patch,
HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch, HBASE-11764.patch,
HBASE-11764.patch, HBASE-11764.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152241#comment-14152241
]
Ted Yu commented on HBASE-10200:
+1 if test suite passes.
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Priority: Minor
Labels: beginner
Attachments: HBASE-10020.patch
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11980) Change sync to hsync, remove unused InfoServer, and reference our httpserver instead of hadoops
[ https://issues.apache.org/jira/browse/HBASE-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152247#comment-14152247 ] stack commented on HBASE-11980: --- [~echarles] Thanks for the review. Where you see the java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView ? You have the log still otherwise I'll just try replicate... you were running jdk7 or jdk6 with hbase master branch? If jdk6, yeah, not supported on the master branch. Yeah, voodoo necessary getting things running on different versions. I like your idea of hadoop-current vs hadoop-2. Change sync to hsync, remove unused InfoServer, and reference our httpserver instead of hadoops --- Key: HBASE-11980 URL: https://issues.apache.org/jira/browse/HBASE-11980 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0, 0.99.1 Attachments: 0001-HBASE-11980-Change-sync-to-hsync-remove-unused-InfoS.patch Subtask that fixes a few issues with our building against hadoop3. Get it in. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-12008: - Attachment: HBASE-12008.01.patch Removed a bunch of unused imports as well. Remove IntegrationTestImportTsv#testRunFromOutputCommitter -- Key: HBASE-12008 URL: https://issues.apache.org/jira/browse/HBASE-12008 Project: HBase Issue Type: Test Components: mapreduce, test Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch This test was introduced to cover a dodgy code pathway exercised by the hbase storage handler in HCatalog. Said dodgy code path involves launching a MR job from the output committer of another MR job. As of Hive 0.14, said storage handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-12008: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch-1 and master. Remove IntegrationTestImportTsv#testRunFromOutputCommitter -- Key: HBASE-12008 URL: https://issues.apache.org/jira/browse/HBASE-12008 Project: HBase Issue Type: Test Components: mapreduce, test Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch This test was introduced to cover a dodgy code pathway exercised by the hbase storage handler in HCatalog. Said dodgy code path involves launching a MR job from the output committer of another MR job. As of Hive 0.14, said storage handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152310#comment-14152310 ] Ted Yu commented on HBASE-12075: QA run took over 300 minutes: https://builds.apache.org/job/PreCommit-HBASE-Build/11120/console Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12114) Meta table cache hashing may access the wrong table
[ https://issues.apache.org/jira/browse/HBASE-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152325#comment-14152325 ] Hudson commented on HBASE-12114: SUCCESS: Integrated in HBase-0.94-security #541 (See [https://builds.apache.org/job/HBase-0.94-security/541/]) HBASE-12114 Meta table cache hashing may access the wrong table. (He Liangliang) (larsh: rev ffd3c383abb863c29b00b2e016d8116d7f9c2c1a) * src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java Meta table cache hashing may access the wrong table --- Key: HBASE-12114 URL: https://issues.apache.org/jira/browse/HBASE-12114 Project: HBase Issue Type: Bug Components: Client Reporter: He Liangliang Assignee: He Liangliang Fix For: 0.94.24 Attachments: HBASE-12114.patch The cache key used a hash code from a weak hash algorithm Bytes.mapKey(tableName), which can cause collisions in a quite high probability then make the client accessing a wrong region from another table. Example: 2fx0 and 2fvn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12008) Remove IntegrationTestImportTsv#testRunFromOutputCommitter
[ https://issues.apache.org/jira/browse/HBASE-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152328#comment-14152328 ] Enis Soztutar commented on HBASE-12008: --- belated +1. Remove IntegrationTestImportTsv#testRunFromOutputCommitter -- Key: HBASE-12008 URL: https://issues.apache.org/jira/browse/HBASE-12008 Project: HBase Issue Type: Test Components: mapreduce, test Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12008.00.patch, HBASE-12008.01.patch This test was introduced to cover a dodgy code pathway exercised by the hbase storage handler in HCatalog. Said dodgy code path involves launching a MR job from the output committer of another MR job. As of Hive 0.14, said storage handler has been removed. Let's drop test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12104) Some optimization and bugfix for HTableMultiplexer
[
https://issues.apache.org/jira/browse/HBASE-12104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152338#comment-14152338
]
Ted Yu commented on HBASE-12104:
{code}
+ public static final String TABLE_MULTIPLEXER_MAX_RETRY_IN_QUEUE =
+ hbase.client.max.retried.in.queue;
{code}
Should the config be hbase.client.max.retries.in.queue ?
{code}
+ } catch (RuntimeException e) {
+// To make findbugs happy
+// Log all the exceptions and move on
{code}
Is the above desirable for IOExceptions ?
Some optimization and bugfix for HTableMultiplexer
--
Key: HBASE-12104
URL: https://issues.apache.org/jira/browse/HBASE-12104
Project: HBase
Issue Type: Sub-task
Components: Client
Affects Versions: 2.0.0
Reporter: Yi Deng
Assignee: Yi Deng
Labels: multiplexer
Fix For: 2.0.0
Attachments: 0001-Make-HTableMultiplexerStatus-public.patch
Make HTableMultiplexerStatus public
Delay before resubmit.
Fix some missing counting on total failure.
Use ScheduledExecutorService to simplify the code.
Other refactoring.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11178) Remove deprecation annotations from mapred namespace
[ https://issues.apache.org/jira/browse/HBASE-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11178: - Attachment: HBASE-11178.00.patch Rebased to master. Remove deprecation annotations from mapred namespace Key: HBASE-11178 URL: https://issues.apache.org/jira/browse/HBASE-11178 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11178.00.patch, HBASE-11178.00.patch I guess mapred API is here to stay. We should remove these annotations and clean it up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11178) Remove deprecation annotations from mapred namespace
[ https://issues.apache.org/jira/browse/HBASE-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11178: - Status: Open (was: Patch Available) Remove deprecation annotations from mapred namespace Key: HBASE-11178 URL: https://issues.apache.org/jira/browse/HBASE-11178 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11178.00.patch, HBASE-11178.00.patch I guess mapred API is here to stay. We should remove these annotations and clean it up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12042) Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName)
[ https://issues.apache.org/jira/browse/HBASE-12042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12042: --- Affects Version/s: (was: 0.99.0) 0.99.1 Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName) Key: HBASE-12042 URL: https://issues.apache.org/jira/browse/HBASE-12042 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.1 Attachments: HBASE-12042.patch, HBASE-12042.patch, HBASE-12042.patch, hbase-12042_v2.patch, hbase-12042_v2.patch Replace internal uses of HTable(Configuration, String) with HTable(Configuration, TableName) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11178) Remove deprecation annotations from mapred namespace
[ https://issues.apache.org/jira/browse/HBASE-11178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11178: - Status: Patch Available (was: Open) Remove deprecation annotations from mapred namespace Key: HBASE-11178 URL: https://issues.apache.org/jira/browse/HBASE-11178 Project: HBase Issue Type: Sub-task Components: mapreduce Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11178.00.patch, HBASE-11178.00.patch I guess mapred API is here to stay. We should remove these annotations and clean it up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152394#comment-14152394 ] Manukranth Kolloju commented on HBASE-12075: I added only 2 tests. They take less than a minute to run in total. Not sure why its not running on Hudson. Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju updated HBASE-12075: --- Attachment: 0001-HBASE-12075-Implement-Preemptive-Fast-Fail.patch Adding the latest patch addressing the review comments. Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-HBASE-12075-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12075) Preemptive Fast Fail
[ https://issues.apache.org/jira/browse/HBASE-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152419#comment-14152419 ] Elliott Clark commented on HBASE-12075: --- It didn't even get to tests. Just compiling too 300 mins. Probably something weird is going on with jenkins... BLAH Preemptive Fast Fail Key: HBASE-12075 URL: https://issues.apache.org/jira/browse/HBASE-12075 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 1.0.0 Attachments: 0001-Add-a-test-case-for-Preemptive-Fast-Fail.patch, 0001-HBASE-12075-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch, 0001-Implement-Preemptive-Fast-Fail.patch In multi threaded clients, we use a feature developed on 0.89-fb branch called Preemptive Fast Fail. This allows the client threads which would potentially fail, fail fast. The idea behind this feature is that we allow, among the hundreds of client threads, one thread to try and establish connection with the regionserver and if that succeeds, we mark it as a live node again. Meanwhile, other threads which are trying to establish connection to the same server would ideally go into the timeouts which is effectively unfruitful. We can in those cases return appropriate exceptions to those clients instead of letting them retry. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12115) Fix NumberFormat Exception in TableInputFormatBase.
Manukranth Kolloju created HBASE-12115: -- Summary: Fix NumberFormat Exception in TableInputFormatBase. Key: HBASE-12115 URL: https://issues.apache.org/jira/browse/HBASE-12115 Project: HBase Issue Type: Bug Components: test Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Priority: Minor Fix For: 1.0.0 DNS.reverseDns doesn't work well with IPv6 addresses. This patch is to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12115) Fix NumberFormat Exception in TableInputFormatBase.
[ https://issues.apache.org/jira/browse/HBASE-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju updated HBASE-12115: --- Issue Type: Sub-task (was: Bug) Parent: HBASE-12037 Fix NumberFormat Exception in TableInputFormatBase. --- Key: HBASE-12115 URL: https://issues.apache.org/jira/browse/HBASE-12115 Project: HBase Issue Type: Sub-task Components: test Affects Versions: 1.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Priority: Minor Fix For: 1.0.0 Original Estimate: 24h Remaining Estimate: 24h DNS.reverseDns doesn't work well with IPv6 addresses. This patch is to fix that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12029) Use Table and RegionLocator in HTable.getRegionLocations()
[ https://issues.apache.org/jira/browse/HBASE-12029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152451#comment-14152451 ] Solomon Duskis commented on HBASE-12029: Both of the following pass locally: org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery.testSplitWhileBulkLoadPhase(TestLoadIncrementalHFilesSplitRecovery.java:339) at org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testSimpleLoad(TestLoadIncrementalHFiles.java:100) Are these known zombie tests? Use Table and RegionLocator in HTable.getRegionLocations() --- Key: HBASE-12029 URL: https://issues.apache.org/jira/browse/HBASE-12029 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12029.patch, HBASE-12029_v1.patch, HBASE-12029_v2.patch, HBASE-12029_v3.patch, HBASE-12029_v4.patch, HBASE-12029_v4.patch, HBASE-12029_v5.patch HTable.getRegionLocations() is an internal deprecated API. There are plenty of internal uses of the method, and it would be ideal to use the Table and RegionLocator interfaces instead of the HTable method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables
[
https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl reopened HBASE-12019:
---
[~stack], looks like you misapplied the 0.94 patch. (replaced DRFA\{S} with
RFA\{S})
hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER
variables
---
Key: HBASE-12019
URL: https://issues.apache.org/jira/browse/HBASE-12019
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.99.0, 0.94.23, 0.98.6
Environment: Linux
Reporter: Sebastien Barrier
Assignee: Sebastien Barrier
Priority: Minor
Labels: patch
Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1
Attachments: HBASE-12019.patch
hbase-env.sh is supposed to be used to set environment variables like
HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the
export of thoses, the benefit of hbase-env.sh is lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables
[ https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-12019: -- Attachment: 12019-0.94-addendum.txt Addendum that fixes it. Going to apply. 0.94 only, 0.98+ are good. hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables --- Key: HBASE-12019 URL: https://issues.apache.org/jira/browse/HBASE-12019 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.99.0, 0.94.23, 0.98.6 Environment: Linux Reporter: Sebastien Barrier Assignee: Sebastien Barrier Priority: Minor Labels: patch Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: 12019-0.94-addendum.txt, HBASE-12019.patch hbase-env.sh is supposed to be used to set environment variables like HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the export of thoses, the benefit of hbase-env.sh is lost. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables
[
https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152474#comment-14152474
]
Lars Hofhansl commented on HBASE-12019:
---
Looking at the patch now... It does not make much sense anyway:
{code}
+if [ -z ${HBASE_ROOT_LOGGER} ]; then
export HBASE_ROOT_LOGGER=${HBASE_ROOT_LOGGER:-INFO,RFA}
+fi
{code}
The existing code (at least in 0.98+) {{export
HBASE_ROOT_LOGGER=$\{HBASE_ROOT_LOGGER:\-INFO,RFA}}} seems to do the right
thing already anyway - notice the {{$\{HBASE_ROOT_LOGGER:-}} part, which
evaluates to 2nd part only if the variable is null.
In any case, going to commit the addendum to 0.94 now.
hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER
variables
---
Key: HBASE-12019
URL: https://issues.apache.org/jira/browse/HBASE-12019
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.99.0, 0.94.23, 0.98.6
Environment: Linux
Reporter: Sebastien Barrier
Assignee: Sebastien Barrier
Priority: Minor
Labels: patch
Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1
Attachments: 12019-0.94-addendum.txt, HBASE-12019.patch
hbase-env.sh is supposed to be used to set environment variables like
HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the
export of thoses, the benefit of hbase-env.sh is lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152476#comment-14152476
]
Ted Yu commented on HBASE-10200:
Looks like QA bot consistently encounters the following:
{code}
Caused by: hudson.plugins.git.GitException: Failed to connect to
https://git-wip-us.apache.org/repos/asf/hbase.git (exception:
java.net.UnknownHostException: git-wip-us.apache.org: Name or service not known)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkCredentials(CliGitAPIImpl.java:1960)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1137)
{code}
Running test suite now.
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Priority: Minor
Labels: beginner
Attachments: HBASE-10020.patch
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-10200) Better error message when HttpServer fails to start due to java.net.BindException
[
https://issues.apache.org/jira/browse/HBASE-10200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152476#comment-14152476
]
Ted Yu edited comment on HBASE-10200 at 9/29/14 11:04 PM:
--
Looks like QA bot consistently encounters the following:
{code}
Caused by: hudson.plugins.git.GitException: Failed to connect to
https://git-wip-us.apache.org/repos/asf/hbase.git (exception:
java.net.UnknownHostException: git-wip-us.apache.org: Name or service not known)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkCredentials(CliGitAPIImpl.java:1960)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1137)
{code}
Running test suite locally now.
was (Author: yuzhih...@gmail.com):
Looks like QA bot consistently encounters the following:
{code}
Caused by: hudson.plugins.git.GitException: Failed to connect to
https://git-wip-us.apache.org/repos/asf/hbase.git (exception:
java.net.UnknownHostException: git-wip-us.apache.org: Name or service not known)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkCredentials(CliGitAPIImpl.java:1960)
at
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1137)
{code}
Running test suite now.
Better error message when HttpServer fails to start due to
java.net.BindException
-
Key: HBASE-10200
URL: https://issues.apache.org/jira/browse/HBASE-10200
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: Ted Yu
Priority: Minor
Labels: beginner
Attachments: HBASE-10020.patch
Starting HBase using Hoya, I saw the following in log:
{code}
2013-12-17 21:49:06,758 INFO [master:hor12n19:42587] http.HttpServer:
HttpServer.start() threw a non Bind IOException
java.net.BindException: Port in use: hor12n14.gq1.ygridcore.net:12432
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:742)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:686)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:586)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.BindException: Cannot assign requested address
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:344)
at sun.nio.ch.Net.bind(Net.java:336)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:199)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.apache.hadoop.http.HttpServer.openListener(HttpServer.java:738)
{code}
This was due to hbase.master.info.bindAddress giving static address but Hoya
allocates master dynamically.
Better error message should be provided: when bindAddress points another host
than local host, message should remind user to remove / adjust
hbase.master.info.bindAddress config param from hbase-site.xml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables
[ https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-12019. --- Resolution: Fixed Done. Rolling new RC. hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables --- Key: HBASE-12019 URL: https://issues.apache.org/jira/browse/HBASE-12019 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.99.0, 0.94.23, 0.98.6 Environment: Linux Reporter: Sebastien Barrier Assignee: Sebastien Barrier Priority: Minor Labels: patch Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: 12019-0.94-addendum.txt, HBASE-12019.patch hbase-env.sh is supposed to be used to set environment variables like HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the export of thoses, the benefit of hbase-env.sh is lost. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12040: --- Attachment: ycsb-0.98.6.1-vs-0.98.7.pdf Switching test tools, I did a comparison with YCSB between 0.98.6.1 and 0.98.7-SNAPSHOT (c220be9) on a single host cluster, and found an across the board improvement. Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152489#comment-14152489 ] Andrew Purtell commented on HBASE-12040: Given the results in the OP have been declared invalid in comments above, and we have various divergent reports, I would like to close this issue as Invalid. What do you think? I think we should continue looking closely at perf differences over time and when we think we've found a repeatable regression let's file a JIRA for that, it will be actionable. Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152493#comment-14152493 ] Devaraj Das commented on HBASE-12040: - I am fine with that plan, [~apurtell]. Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152495#comment-14152495 ] Nick Dimiduk commented on HBASE-12040: -- FYI, I'm running the scanRange100 test over a much larger dataset (--size=100) vs various versions and configurations. Will report any meaningful findings. Will also look into the state of Devaraj's cluster. Meanwhile, I think it's alright to proceed with [~apurtell]'s suggestion. Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152496#comment-14152496 ] Andrew Purtell commented on HBASE-12040: I'm also going to scale up. Will resolve this in a few hours unless objection Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12019) hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables
[ https://issues.apache.org/jira/browse/HBASE-12019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152498#comment-14152498 ] stack commented on HBASE-12019: --- Thanks for fixup [~lhofhansl]. hbase-daemon.sh overwrite HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER variables --- Key: HBASE-12019 URL: https://issues.apache.org/jira/browse/HBASE-12019 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.99.0, 0.94.23, 0.98.6 Environment: Linux Reporter: Sebastien Barrier Assignee: Sebastien Barrier Priority: Minor Labels: patch Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: 12019-0.94-addendum.txt, HBASE-12019.patch hbase-env.sh is supposed to be used to set environment variables like HBASE_ROOT_LOGGER and HBASE_SECURITY_LOGGER but hbase-daemon.sh overwrite the export of thoses, the benefit of hbase-env.sh is lost. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12040) Performances issues with FilteredScanTest
[ https://issues.apache.org/jira/browse/HBASE-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152500#comment-14152500 ] Jean-Marc Spaggiari commented on HBASE-12040: - +1 with closing this too. We can continue on the ML. I have updated my scripts to use 10 clients instead of 1, and made side that --nomapred is specified everywhere. I will re-run it for all the 0.98 and 0.99 releases and updated my report. Sorry about that. Performances issues with FilteredScanTest -- Key: HBASE-12040 URL: https://issues.apache.org/jira/browse/HBASE-12040 Project: HBase Issue Type: Bug Affects Versions: 0.99.1 Reporter: Jean-Marc Spaggiari Assignee: stack Priority: Critical Fix For: 0.98.7, 0.99.1 Attachments: at-HBASE-11331.html, pre-HBASE-11331.html, ycsb-0.98.6.1-vs-0.98.7.pdf While testing 0.99.0RC1 release performances, compared to 0.98.6, figured that: - FilteredScanTest is 100 times slower; - RandomReadTest is 1.5 times slower; - RandomSeekScanTest is 3.2 times slower; - RandomScanWithRange10Test is 1,2 times slower; - RandomScanWithRange100Test is 1,3 times slower; - RandomScanWithRange1000Test is 4 times slower; - SequentialReadTest is 1,7 times slower; - SequentialWriteTest is just a bit faster; - RandomWriteTest is just a bit faster; - GaussianRandomReadBenchmark is just a beat slower; - SequentialReadBenchmark is 1,1 times slower; - SequentialWriteBenchmark is 1,1 times slower; - UniformRandomReadBenchmark crashed; - UniformRandomSmallScan is 1,3 times slower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
