[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357194#comment-14357194
]
Srikanth Srungarapu commented on HBASE-13206:
-
+1 (non-binding). Good catch!
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: HBASE-13193-v4.patch
Re-attaching, the previous precommit build tried to apply the patch to branch
0.94 instead of master for some reason.
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13205) [branch-1] Backport HBASE-11598 Add simple rpc throttling
[ https://issues.apache.org/jira/browse/HBASE-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13205: -- Status: Patch Available (was: Open) [branch-1] Backport HBASE-11598 Add simple rpc throttling - Key: HBASE-13205 URL: https://issues.apache.org/jira/browse/HBASE-13205 Project: HBase Issue Type: Task Components: security Reporter: Ashish Singhi Assignee: Ashish Singhi Labels: multitenancy, quota Fix For: 1.1.0 Attachments: HBASE-13205-branch-1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12931) The existing KeyValues in memstore are not removed completely after inserting cell into memStore
[
https://issues.apache.org/jira/browse/HBASE-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357062#comment-14357062
]
Hudson commented on HBASE-12931:
FAILURE: Integrated in HBase-0.98 #891 (See
[https://builds.apache.org/job/HBase-0.98/891/])
HBASE-12931 The existing KeyValues in memstore are not removed completely after
inserting cell into memStore (ChiaPing Tsai) (tedyu: rev
d4e82006940bb2df94810350217a7a2df55eca0e)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
The existing KeyValues in memstore are not removed completely after inserting
cell into memStore
-
Key: HBASE-12931
URL: https://issues.apache.org/jira/browse/HBASE-12931
Project: HBase
Issue Type: Bug
Reporter: ChiaPing Tsai
Assignee: ChiaPing Tsai
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: 12931-0.98.txt, HBASE-12931.patch
If I'm not wrong, the UPSERT method of memStore should remove all existing
KeyValues except the newer version.
In memStore,
{code:title=DefaultMemStore.java|borderStyle=solid}
int versIionsVisible = 0;
...
if (cur.getTypeByte() == KeyValue.Type.Put.getCode()
cur.getSequenceId() = readpoint) {
if (versionsVisible 1) {
// if we get here we have seen at least one version visible to
the oldest scanner,
// which means we can prove that no scanner will see this version
// false means there was a change, so give us the size.
long delta = heapSizeChange(cur, true);
addedSize -= delta;
this.size.addAndGet(-delta);
it.remove();
setOldestEditTimeToNow();
} else {
versionsVisible++;
}
{code}
Does versionsVisible 1 should be changed to versionsVisible = 1 ?
thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357070#comment-14357070
]
Hadoop QA commented on HBASE-13071:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703903/HBASE-13071_trunk_8.patch
against master branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703903
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1926 checkstyle errors (more than the master's current 1924 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13185//console
This message is automatically generated.
Hbase Streaming Scan Feature
Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Reporter: Eshcar Hillel
Attachments: 99.eshcar.png, HBASE-13071_98_1.patch,
HBASE-13071_trunk_1.patch, HBASE-13071_trunk_2.patch,
HBASE-13071_trunk_3.patch, HBASE-13071_trunk_4.patch,
HBASE-13071_trunk_5.patch, HBASE-13071_trunk_6.patch,
HBASE-13071_trunk_7.patch, HBASE-13071_trunk_8.patch,
HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf,
gc.eshcar.png, hits.eshcar.png, network.png
A scan operation iterates over all rows of a table or a subrange of the
table. The synchronous nature in which the data is served at the client side
hinders the speed the application traverses the data: it increases the
overall processing time, and may cause a great variance in the times the
application waits for the next piece of data.
The scanner next() method at the client side invokes an RPC to the
regionserver and then stores the results in a cache. The application can
specify how many rows will be transmitted per RPC; by default this is set to
[jira] [Commented] (HBASE-12931) The existing KeyValues in memstore are not removed completely after inserting cell into memStore
[
https://issues.apache.org/jira/browse/HBASE-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357083#comment-14357083
]
Hudson commented on HBASE-12931:
FAILURE: Integrated in HBase-1.0 #797 (See
[https://builds.apache.org/job/HBase-1.0/797/])
HBASE-12931 The existing KeyValues in memstore are not removed completely after
inserting cell into memStore (ChiaPing Tsai) (tedyu: rev
0bbb606eae19b45d6905b2f453a76e7a45b3bdc2)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultMemStore.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultMemStore.java
The existing KeyValues in memstore are not removed completely after inserting
cell into memStore
-
Key: HBASE-12931
URL: https://issues.apache.org/jira/browse/HBASE-12931
Project: HBase
Issue Type: Bug
Reporter: ChiaPing Tsai
Assignee: ChiaPing Tsai
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: 12931-0.98.txt, HBASE-12931.patch
If I'm not wrong, the UPSERT method of memStore should remove all existing
KeyValues except the newer version.
In memStore,
{code:title=DefaultMemStore.java|borderStyle=solid}
int versIionsVisible = 0;
...
if (cur.getTypeByte() == KeyValue.Type.Put.getCode()
cur.getSequenceId() = readpoint) {
if (versionsVisible 1) {
// if we get here we have seen at least one version visible to
the oldest scanner,
// which means we can prove that no scanner will see this version
// false means there was a change, so give us the size.
long delta = heapSizeChange(cur, true);
addedSize -= delta;
this.size.addAndGet(-delta);
it.remove();
setOldestEditTimeToNow();
} else {
versionsVisible++;
}
{code}
Does versionsVisible 1 should be changed to versionsVisible = 1 ?
thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13201) Remove HTablePool from thrift-server
[
https://issues.apache.org/jira/browse/HBASE-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357110#comment-14357110
]
Hadoop QA commented on HBASE-13201:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703896/HBASE-13201.patch
against master branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703896
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1927 checkstyle errors (more than the master's current 1924 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13187//console
This message is automatically generated.
Remove HTablePool from thrift-server
Key: HBASE-13201
URL: https://issues.apache.org/jira/browse/HBASE-13201
Project: HBase
Issue Type: Sub-task
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Fix For: 2.0.0, 1.0.1
Attachments: HBASE-13201.patch
HTablePool is an old concept that was superseded by Connection.
thrift-server doesn't need HTablePool anymore and can use the ConnectionCache
directly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13202) Procedure v2 - core
[
https://issues.apache.org/jira/browse/HBASE-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357173#comment-14357173
]
Hadoop QA commented on HBASE-13202:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703940/HBASE-13202-v0.patch
against 0.94 branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703940
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13190//console
This message is automatically generated.
Procedure v2 - core
---
Key: HBASE-13202
URL: https://issues.apache.org/jira/browse/HBASE-13202
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Attachments: HBASE-13202-v0.patch
core package, part of HBASE-12439
this is just the proc-v2 submodule. it depends only on hbase-common.
https://reviews.apache.org/r/27703/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357200#comment-14357200
]
Hadoop QA commented on HBASE-13193:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703942/HBASE-13193-v4.patch
against 0.94 branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703942
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13191//console
This message is automatically generated.
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13205) [branch-1] Backport HBASE-11598 Add simple rpc throttling
[ https://issues.apache.org/jira/browse/HBASE-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi updated HBASE-13205: -- Attachment: HBASE-13205-branch-1.patch [branch-1] Backport HBASE-11598 Add simple rpc throttling - Key: HBASE-13205 URL: https://issues.apache.org/jira/browse/HBASE-13205 Project: HBase Issue Type: Task Components: security Reporter: Ashish Singhi Assignee: Ashish Singhi Labels: multitenancy, quota Fix For: 1.1.0 Attachments: HBASE-13205-branch-1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: HBASE-13193-v4.patch
Fix the findbug warning
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13202) Procedure v2 - core
[ https://issues.apache.org/jira/browse/HBASE-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-13202: Attachment: HBASE-13202-v0.patch Procedure v2 - core --- Key: HBASE-13202 URL: https://issues.apache.org/jira/browse/HBASE-13202 Project: HBase Issue Type: Sub-task Components: master Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Attachments: HBASE-13202-v0.patch core package, part of HBASE-12439 this is just the proc-v2 submodule. it depends only on hbase-common. https://reviews.apache.org/r/27703/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357228#comment-14357228
]
Hadoop QA commented on HBASE-13193:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703947/HBASE-13193-v4.patch
against 0.94 branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703947
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13192//console
This message is automatically generated.
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13206:
Attachment: HBASE-13206-v0.patch
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13206) Fix TableLock tableName log format
Matteo Bertozzi created HBASE-13206:
---
Summary: Fix TableLock tableName log format
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13206:
Status: Patch Available (was: Open)
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13207) Backport to 0.98 [PERF] Reuse the IPC buffers...
[
https://issues.apache.org/jira/browse/HBASE-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357542#comment-14357542
]
Hadoop QA commented on HBASE-13207:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703963/13207.0.98.txt
against 0.98 branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703963
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
25 warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3838 checkstyle errors (more than the master's current 3837 errors).
{color:red}-1 findbugs{color}. The patch appears to introduce 7 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13193//console
This message is automatically generated.
Backport to 0.98 [PERF] Reuse the IPC buffers...
--
Key: HBASE-13207
URL: https://issues.apache.org/jira/browse/HBASE-13207
Project: HBase
Issue Type: Sub-task
Components: Performance
Reporter: stack
Assignee: stack
Fix For: 0.98.12
Attachments: 13207.0.98.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12586) Task 6 7 from HBASE-9117, delete all public HTable constructors and delete ConnectionManager#{delete,get}Connection
[
https://issues.apache.org/jira/browse/HBASE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-12586:
Status: Patch Available (was: Open)
Task 6 7 from HBASE-9117, delete all public HTable constructors and delete
ConnectionManager#{delete,get}Connection
--
Key: HBASE-12586
URL: https://issues.apache.org/jira/browse/HBASE-12586
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: beginner, beginners
Attachments: HBASE-12586.patch, HBASE-12586.patch, HBASE-12586.patch,
HBASE-12586.patch, HBASE-12586.patch
Finish cleanup from HBASE-9117 removing old API. This issue covers tasks 6
and 7 from the list here:
https://issues.apache.org/jira/browse/HBASE-9117?focusedCommentId=13919716page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13919716
To be done in master branch only.
Marked as beginner task. The idea is straight-forward. It is just a lot of
work going through all tests converting to use new API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime is buggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-13167:
Status: Patch Available (was: Open)
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13192) IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures
[ https://issues.apache.org/jira/browse/HBASE-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-13192: Resolution: Fixed Fix Version/s: 1.0.1 Status: Resolved (was: Patch Available) Committed. IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures --- Key: HBASE-13192 URL: https://issues.apache.org/jira/browse/HBASE-13192 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.0.1 Attachments: 13192-1.txt, 13192-2.txt Noticed this issue in our internal test runs. It shows up when the test is run with replicas turned on. After the bulk load is complete, the test tries to modify the table with an intention of having the replicas load the new region files. But due to a bug in the way the test was waiting for the modifyTable operation to complete, it would not actually wait. Sometimes a replica wouldn't have loaded the new files yet and it might happen that the 'check' step of the test would go to that replica asking for data. It would give empty results back, and the test would fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12439) Procedure V2
[ https://issues.apache.org/jira/browse/HBASE-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-12439: --- Labels: reliability (was: ) Procedure V2 Key: HBASE-12439 URL: https://issues.apache.org/jira/browse/HBASE-12439 Project: HBase Issue Type: New Feature Components: master Affects Versions: 2.0.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Labels: reliability Attachments: ProcedureV2.pdf, Procedurev2Notification-Bus.pdf Procedure v2 (aka Notification Bus) aims to provide a unified way to build: * multi-steps procedure with a rollback/rollforward ability in case of failure (e.g. create/delete table) ** HBASE-12070 * notifications across multiple machines (e.g. ACLs/Labels/Quotas cache updates) ** Make sure that every machine has the grant/revoke/label ** Enforce space limit quota across the namespace ** HBASE-10295 eliminate permanent replication zk node * procedures across multiple machines (e.g. Snapshots) * coordinated long-running procedures (e.g. compactions, splits, ...) * Synchronous calls, with the ability to see the state/result in case of failure. ** HBASE-11608 sync split still work in progress/initial prototype: https://reviews.apache.org/r/27703/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13210) Procedure V2 - master Modify table
[ https://issues.apache.org/jira/browse/HBASE-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-13210: --- Labels: reliablity (was: ) Procedure V2 - master Modify table -- Key: HBASE-13210 URL: https://issues.apache.org/jira/browse/HBASE-13210 Project: HBase Issue Type: Sub-task Components: master Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang Labels: reliablity Original Estimate: 72h Remaining Estimate: 72h master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the modify table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12931) The existing KeyValues in memstore are not removed completely after inserting cell into memStore
[
https://issues.apache.org/jira/browse/HBASE-12931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357516#comment-14357516
]
Hudson commented on HBASE-12931:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #848 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/848/])
HBASE-12931 The existing KeyValues in memstore are not removed completely after
inserting cell into memStore (ChiaPing Tsai) (tedyu: rev
d4e82006940bb2df94810350217a7a2df55eca0e)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
The existing KeyValues in memstore are not removed completely after inserting
cell into memStore
-
Key: HBASE-12931
URL: https://issues.apache.org/jira/browse/HBASE-12931
Project: HBase
Issue Type: Bug
Reporter: ChiaPing Tsai
Assignee: ChiaPing Tsai
Priority: Minor
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: 12931-0.98.txt, HBASE-12931.patch
If I'm not wrong, the UPSERT method of memStore should remove all existing
KeyValues except the newer version.
In memStore,
{code:title=DefaultMemStore.java|borderStyle=solid}
int versIionsVisible = 0;
...
if (cur.getTypeByte() == KeyValue.Type.Put.getCode()
cur.getSequenceId() = readpoint) {
if (versionsVisible 1) {
// if we get here we have seen at least one version visible to
the oldest scanner,
// which means we can prove that no scanner will see this version
// false means there was a change, so give us the size.
long delta = heapSizeChange(cur, true);
addedSize -= delta;
this.size.addAndGet(-delta);
it.remove();
setOldestEditTimeToNow();
} else {
versionsVisible++;
}
{code}
Does versionsVisible 1 should be changed to versionsVisible = 1 ?
thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357566#comment-14357566
]
Hudson commented on HBASE-13206:
FAILURE: Integrated in HBase-TRUNK #6242 (See
[https://builds.apache.org/job/HBase-TRUNK/6242/])
HBASE-13206 Fix TableLock tableName log format (matteo.bertozzi: rev
b5bfbc8a9f4e9e25ba622c8a6f35f258d72af8ed)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/TableLockChecker.java
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13191) mvn site fails on jenkins due to permgen
[ https://issues.apache.org/jira/browse/HBASE-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357568#comment-14357568 ] Hudson commented on HBASE-13191: FAILURE: Integrated in HBase-TRUNK #6242 (See [https://builds.apache.org/job/HBase-TRUNK/6242/]) HBASE-13191. Addendum missed one spot we hard-code MAVEN_OPTS for jenkins. (busbey: rev 9087febd21fe2f8970fc79cacbb2730057479cff) * dev-support/test-patch.properties * pom.xml mvn site fails on jenkins due to permgen Key: HBASE-13191 URL: https://issues.apache.org/jira/browse/HBASE-13191 Project: HBase Issue Type: Bug Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 2.0.0 Attachments: HBASE-13191.1.patch.txt, HBASE-13191.addendum.1.patch.txt, HBASE-13191.addendum.2.txt, HBASE-13191.addendum.3.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt patch QA is failing the mvn site goal due to permgen. looks like we have ~3 places we try to set it. consolidate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357567#comment-14357567 ] Hudson commented on HBASE-13208: FAILURE: Integrated in HBase-TRUNK #6242 (See [https://builds.apache.org/job/HBase-TRUNK/6242/]) HBASE-13208 Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName (busbey: rev 5bf98d0827a8b53b44394d5477fb2e73aa1b84fb) * dev-support/test-patch.sh Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Fix For: 2.0.0 Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13210) Procedure V2 - master Modify table
Stephen Yuan Jiang created HBASE-13210: -- Summary: Procedure V2 - master Modify table Key: HBASE-13210 URL: https://issues.apache.org/jira/browse/HBASE-13210 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the modify table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12439) Procedure V2
[ https://issues.apache.org/jira/browse/HBASE-12439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-12439: --- Priority: Major (was: Minor) Procedure V2 Key: HBASE-12439 URL: https://issues.apache.org/jira/browse/HBASE-12439 Project: HBase Issue Type: New Feature Components: master Affects Versions: 2.0.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Labels: reliability Attachments: ProcedureV2.pdf, Procedurev2Notification-Bus.pdf Procedure v2 (aka Notification Bus) aims to provide a unified way to build: * multi-steps procedure with a rollback/rollforward ability in case of failure (e.g. create/delete table) ** HBASE-12070 * notifications across multiple machines (e.g. ACLs/Labels/Quotas cache updates) ** Make sure that every machine has the grant/revoke/label ** Enforce space limit quota across the namespace ** HBASE-10295 eliminate permanent replication zk node * procedures across multiple machines (e.g. Snapshots) * coordinated long-running procedures (e.g. compactions, splits, ...) * Synchronous calls, with the ability to see the state/result in case of failure. ** HBASE-11608 sync split still work in progress/initial prototype: https://reviews.apache.org/r/27703/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13212) Procedure V2 - master Create/Delete namespace
Stephen Yuan Jiang created HBASE-13212: -- Summary: Procedure V2 - master Create/Delete namespace Key: HBASE-13212 URL: https://issues.apache.org/jira/browse/HBASE-13212 Project: HBase Issue Type: Sub-task Components: master Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the create/delete namespace handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357587#comment-14357587
]
Hudson commented on HBASE-13206:
FAILURE: Integrated in HBase-1.1 #279 (See
[https://builds.apache.org/job/HBase-1.1/279/])
HBASE-13206 Fix TableLock tableName log format (matteo.bertozzi: rev
f8dced1e6dd63225bf3e43740bc9a227b0c514e7)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/TableLockChecker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Status: Open (was: Patch Available)
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work logged] (HBASE-13209) Procedure V2 - master Add/Modify/Delete Column Family
[ https://issues.apache.org/jira/browse/HBASE-13209?focusedWorklogId=19448page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-19448 ] Stephen Yuan Jiang logged work on HBASE-13209: -- Author: Stephen Yuan Jiang Created on: 11/Mar/15 21:37 Start Date: 11/Mar/15 21:37 Worklog Time Spent: 72h Issue Time Tracking --- Worklog Id: (was: 19448) Time Spent: 72h Remaining Estimate: 96h (was: 168h) Procedure V2 - master Add/Modify/Delete Column Family - Key: HBASE-13209 URL: https://issues.apache.org/jira/browse/HBASE-13209 Project: HBase Issue Type: Sub-task Components: master Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang Labels: reliability Attachments: AlterColumnFamilyPV2-draft.v0-master.patch Original Estimate: 168h Time Spent: 72h Remaining Estimate: 96h master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the add/modify/delete handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13209) Procedure V2 - master Add/Modify/Delete Column Family
[ https://issues.apache.org/jira/browse/HBASE-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang updated HBASE-13209: --- Attachment: AlterColumnFamilyPV2-draft.v0-master.patch Procedure V2 - master Add/Modify/Delete Column Family - Key: HBASE-13209 URL: https://issues.apache.org/jira/browse/HBASE-13209 Project: HBase Issue Type: Sub-task Components: master Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang Labels: reliability Attachments: AlterColumnFamilyPV2-draft.v0-master.patch Original Estimate: 168h Remaining Estimate: 168h master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the add/modify/delete handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: HBASE-13193-v5.patch
New patch rebased for master
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch, HBASE-13193-v5.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13063) Allow to turn off memstore replication for region replicas
[
https://issues.apache.org/jira/browse/HBASE-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-13063:
--
Attachment: hbase-13063-addendum.patch
Thanks [~mbertozzi]. I have committed the addendum with a simple change (use
Boolean.toString(memstoreReplication) instead of hard coding false as the
value. Attaching for ref.
Allow to turn off memstore replication for region replicas
--
Key: HBASE-13063
URL: https://issues.apache.org/jira/browse/HBASE-13063
Project: HBase
Issue Type: New Feature
Components: regionserver, Replication
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-13063-v0.patch, HBASE-13063-v1-branch-1.patch,
HBASE-13063-v1.patch, HBASE-13063-v1.patch, HBASE-13063-v2.patch,
HBASE-13063-v2.patch, hbase-13063-addendum.patch, hbase-13063-addendum.patch
HBASE-11568 allows to use replication to send wal edits from the primary to
the replicas.
sometimes the memstore replication is not required, so it will be nice have a
flag to disable it.
the result will be more or less the same to what we have in phase-1, but with
the row-level consistency provided by the flush edit transfered via
replication to refresh the hfiles.
create 't1', 'f', {REGION_REPLICATION = 2, REGION_MEMSTORE_REPLICATION =
false}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13207) Backport to 0.98 [PERF] Reuse the IPC buffers...
stack created HBASE-13207: - Summary: Backport to 0.98 [PERF] Reuse the IPC buffers... Key: HBASE-13207 URL: https://issues.apache.org/jira/browse/HBASE-13207 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-13208) Patch build should match the patch file name and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez reassigned HBASE-13208: - Assignee: Esteban Gutierrez Patch build should match the patch file name and not the whole relative URL in findBranchNameFromPatchName -- Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-13208: -- Attachment: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357365#comment-14357365
]
Hadoop QA commented on HBASE-13206:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703933/HBASE-13206-v0.patch
against master branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703933
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13188//console
This message is automatically generated.
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-13208. - Resolution: Fixed Fix Version/s: 2.0.0 Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Fix For: 2.0.0 Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: (was: HBASE-13193-master-v4.patch)
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: HBASE-13193-v4.patch
The issue with the patch applying to branch 0.94 has been called out in
HBASE-13208. Re-attaching to trigger a precommit build (last attachment didn't
start one)
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13191) mvn site fails on jenkins due to permgen
[ https://issues.apache.org/jira/browse/HBASE-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13191: Resolution: Fixed Status: Resolved (was: Patch Available) yay! I'll go update the jenkins status thread with what's different. mvn site fails on jenkins due to permgen Key: HBASE-13191 URL: https://issues.apache.org/jira/browse/HBASE-13191 Project: HBase Issue Type: Bug Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 2.0.0 Attachments: HBASE-13191.1.patch.txt, HBASE-13191.addendum.1.patch.txt, HBASE-13191.addendum.2.txt, HBASE-13191.addendum.3.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt patch QA is failing the mvn site goal due to permgen. looks like we have ~3 places we try to set it. consolidate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime is buggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-13167:
Status: Open (was: Patch Available)
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12586) Task 6 7 from HBASE-9117, delete all public HTable constructors and delete ConnectionManager#{delete,get}Connection
[
https://issues.apache.org/jira/browse/HBASE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-12586:
Status: Open (was: Patch Available)
Task 6 7 from HBASE-9117, delete all public HTable constructors and delete
ConnectionManager#{delete,get}Connection
--
Key: HBASE-12586
URL: https://issues.apache.org/jira/browse/HBASE-12586
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: beginner, beginners
Attachments: HBASE-12586.patch, HBASE-12586.patch, HBASE-12586.patch,
HBASE-12586.patch, HBASE-12586.patch
Finish cleanup from HBASE-9117 removing old API. This issue covers tasks 6
and 7 from the list here:
https://issues.apache.org/jira/browse/HBASE-9117?focusedCommentId=13919716page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13919716
To be done in master branch only.
Marked as beginner task. The idea is straight-forward. It is just a lot of
work going through all tests converting to use new API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13165) Fix docs and scripts for default max heaps size after HBASE-11804
[ https://issues.apache.org/jira/browse/HBASE-13165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357604#comment-14357604 ] Misty Stanley-Jones commented on HBASE-13165: - Yes, thanks! Fix docs and scripts for default max heaps size after HBASE-11804 - Key: HBASE-13165 URL: https://issues.apache.org/jira/browse/HBASE-13165 Project: HBase Issue Type: Bug Components: documentation, scripts Affects Versions: 1.0.0 Reporter: Lars George Assignee: Lars George Priority: Minor Fix For: 2.0.0, 1.0.1, 1.1.0 Attachments: HBASE-13165-code.patch, HBASE-13165-docs.patch, HBASE-13165.patch We changed the behavior of JAVA_HEAP_MAX in hbase-env.sh to be unset and default to the JVM default value. But in quite a few places we still say 1GB is default. Fix the scripts and docs accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Status: Patch Available (was: Open)
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13209) Procedure V2 - master Add/Modify/Delete Column Family
Stephen Yuan Jiang created HBASE-13209: -- Summary: Procedure V2 - master Add/Modify/Delete Column Family Key: HBASE-13209 URL: https://issues.apache.org/jira/browse/HBASE-13209 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the add/modify/delete handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13211) Procedure V2 - master Enable/Disable table
Stephen Yuan Jiang created HBASE-13211: -- Summary: Procedure V2 - master Enable/Disable table Key: HBASE-13211 URL: https://issues.apache.org/jira/browse/HBASE-13211 Project: HBase Issue Type: Sub-task Components: master Affects Versions: 2.0.0, 1.1.0 Reporter: Stephen Yuan Jiang Assignee: Stephen Yuan Jiang master side, part of HBASE-12439 starts up the procedure executor on the master and replaces the enable/disable table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13205) [branch-1] Backport HBASE-11598 Add simple rpc throttling
[
https://issues.apache.org/jira/browse/HBASE-13205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357387#comment-14357387
]
Hadoop QA commented on HBASE-13205:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703936/HBASE-13205-branch-1.patch
against branch-1 branch at commit e66dca6cd1fd91bfa65a7cd4c68acb7a7f6a6c4e.
ATTACHMENT ID: 12703936
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 38 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3827 checkstyle errors (more than the master's current 3806 errors).
{color:red}-1 findbugs{color}. The patch appears to introduce 8 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ new java.lang.String[] { UserName, UserGroup,
Namespace, TableName, RemoveAll, BypassGlobals, Throttle, });
+result = result (Float.floatToIntBits(getShare())==
Float.floatToIntBits(other.getShare()));
+ imit\030\002 \001(\004\022\r\n\005share\030\003
\001(\002\022#\n\005scope\030\004 \001(\016 +
+ new java.lang.String[] { ReqNum, ReqSize, WriteNum,
WriteSize, ReadNum, ReadSize, });
+ public void postSetNamespaceQuota(final String namespace, final Quotas
quotas) throws IOException {
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.giraph.examples.BrachaTouegDeadlockComputationTest.testEmptyGraph(BrachaTouegDeadlockComputationTest.java:53)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13189//console
This message is automatically generated.
[branch-1] Backport HBASE-11598 Add simple rpc throttling
-
Key: HBASE-13205
URL: https://issues.apache.org/jira/browse/HBASE-13205
Project: HBase
Issue Type: Task
Components: security
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Labels: multitenancy, quota
Fix For: 1.1.0
Attachments:
[jira] [Commented] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357404#comment-14357404 ] Sean Busbey commented on HBASE-13208: - my bad, looks like I can stop checking on matching the beginning as well. the lack of a start/end anchor should have been an indicator. +1 will push shortly unless someone wants a change. Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Lawlor updated HBASE-13193:
Attachment: HBASE-13193-master-v4.patch
Add master to patch name
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-master-v4.patch, HBASE-13193-v1.patch,
HBASE-13193-v2.patch, HBASE-13193-v3.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-13206:
Resolution: Fixed
Fix Version/s: 0.98.12
1.1.0
1.0.1
2.0.0
Status: Resolved (was: Patch Available)
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357490#comment-14357490
]
Hudson commented on HBASE-13206:
SUCCESS: Integrated in HBase-1.0 #798 (See
[https://builds.apache.org/job/HBase-1.0/798/])
HBASE-13206 Fix TableLock tableName log format (matteo.bertozzi: rev
1b2ea696f5ce94813027ac9aef074c1ee050b9d0)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/TableLockChecker.java
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[
https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357369#comment-14357369
]
Sean Busbey commented on HBASE-13208:
-
{code}
-if [[ $patchName =~ .*$LOCAL_BRANCH_NAME.* ]]; then
+if [[ $patchName =~ /jira/secure/attachment/[0-9]*/.*$LOCAL_BRANCH_NAME
]]; then
{code}
you need a wildcard on the end, other wise we'll skip all hte patches that
include e.g. .1.patch or .1.patch.txt
Patch build should match the patch filename and not the whole relative URL in
findBranchNameFromPatchName
-
Key: HBASE-13208
URL: https://issues.apache.org/jira/browse/HBASE-13208
Project: HBase
Issue Type: Bug
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Trivial
Attachments:
0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch
In HBASE-1319 we saw that the patch got applied to the wrong branch, the
problem is findBranchNameFromPatchName matching a regex that contains
wildcard symbols against the whole URL, in this case the regex is 0.94 and
the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch
where 0394 is a match.
Thanks to [~jonathan.lawlor] for reporting this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13207) Backport to 0.98 [PERF] Reuse the IPC buffers...
[ https://issues.apache.org/jira/browse/HBASE-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13207: -- Attachment: 13207.0.98.txt This patch is HBASE-13142 and HBASE-13188 and the addendum on HBASE-13142 plus a little fix for 0.98 on tail of processResponse, return 'done' rather than decide ourselves dependent on any content remaining as is done in master branch. Backport to 0.98 [PERF] Reuse the IPC buffers... -- Key: HBASE-13207 URL: https://issues.apache.org/jira/browse/HBASE-13207 Project: HBase Issue Type: Sub-task Components: Performance Reporter: stack Assignee: stack Fix For: 0.98.12 Attachments: 13207.0.98.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13142) [PERF] Reuse the IPCUtil#buildCellBlock buffer
[ https://issues.apache.org/jira/browse/HBASE-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357269#comment-14357269 ] stack commented on HBASE-13142: --- Made a subtask for 0.98 backport. [PERF] Reuse the IPCUtil#buildCellBlock buffer -- Key: HBASE-13142 URL: https://issues.apache.org/jira/browse/HBASE-13142 Project: HBase Issue Type: Improvement Components: Performance Reporter: stack Assignee: Liang Xie Labels: beginner Fix For: 2.0.0, 1.1.0 Attachments: 13142.txt, 13142v2.txt, 13142v3.txt, 13142v5.0.98.txt, 13142v5.addendum.txt, 13142v5.txt, 13142v5.txt, buffers.svg, clean.svg, gc.png, gc_time_spent.png, hits.png, net.png, traces.2.svg, traces.svg Running some scan profiling, flight recorder was mildly fingering resize of the buffer allocated in IPCUtil#buildCellBlock as a point of contention. It was half-hearted blaming it for a few hundreds of ms over a five minute sampling with a few tens of instances showing. I tried then w/ flamegraph/lightweight profiler and this reported the buffer allocations taking 22% of our total CPU. See attachment trace.svg. I enabled TRACE-level logging on org.apache.hadoop.hbase.ipc.IPCUtil and indeed every allocation was doing a resize from initial allocation of 16k -- the default up to 220k (this test returns ten randomly sized rows zipfian sized between 0 and 8k). Upping the allocation to 220k meant we now avoided the resize but the initial allocation was now blamed for 10% of allocations (see trace.2.svg attached). Lets do buffer reuse. Will save a bunch of allocation and CPU. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13191) mvn site fails on jenkins due to permgen
[ https://issues.apache.org/jira/browse/HBASE-13191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357347#comment-14357347 ] Mikhail Antonov commented on HBASE-13191: - Oh, I see. thanks for explanation! mvn site fails on jenkins due to permgen Key: HBASE-13191 URL: https://issues.apache.org/jira/browse/HBASE-13191 Project: HBase Issue Type: Bug Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 2.0.0 Attachments: HBASE-13191.1.patch.txt, HBASE-13191.addendum.1.patch.txt, HBASE-13191.addendum.2.txt, HBASE-13191.addendum.3.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt, HBASE-13191.not-really-a.patch.txt patch QA is failing the mvn site goal due to permgen. looks like we have ~3 places we try to set it. consolidate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Esteban Gutierrez updated HBASE-13208: -- Summary: Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName (was: Patch build should match the patch file name and not the whole relative URL in findBranchNameFromPatchName) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357391#comment-14357391 ] Esteban Gutierrez commented on HBASE-13208: --- not really, there is always the misconception that a regex needs to have the ending .* to match the rest of the string (many times is related to the implementation, but for extended POSIX should be fine) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13192) IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures
[ https://issues.apache.org/jira/browse/HBASE-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357741#comment-14357741 ] Hudson commented on HBASE-13192: SUCCESS: Integrated in HBase-TRUNK #6243 (See [https://builds.apache.org/job/HBase-TRUNK/6243/]) HBASE-13192. IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures. (ddas: rev 2adec36b2f57fea7006036e5a1f4d5979737ab34) * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures --- Key: HBASE-13192 URL: https://issues.apache.org/jira/browse/HBASE-13192 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.0.1 Attachments: 13192-1.txt, 13192-2.txt Noticed this issue in our internal test runs. It shows up when the test is run with replicas turned on. After the bulk load is complete, the test tries to modify the table with an intention of having the replicas load the new region files. But due to a bug in the way the test was waiting for the modifyTable operation to complete, it would not actually wait. Sometimes a replica wouldn't have loaded the new files yet and it might happen that the 'check' step of the test would go to that replica asking for data. It would give empty results back, and the test would fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-5289) NullPointerException in resetZooKeeperTrackers in HConnectionManager / HConnectionImplementation
[
https://issues.apache.org/jira/browse/HBASE-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-5289.
--
Resolution: Cannot Reproduce
Resolve as cannot reproduce
NullPointerException in resetZooKeeperTrackers in HConnectionManager /
HConnectionImplementation
Key: HBASE-5289
URL: https://issues.apache.org/jira/browse/HBASE-5289
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.90.5
Reporter: Krystian Nowak
This might happen on heavy load in case of lagging HBase when sharing one
HConnection by multiple threads:
{noformat}
2012-01-26 13:59:38,396 ERROR [http://*:8080-251-EventThread]
zookeeper.ClientCnxn$EventThread(532): Error while calling watcher
java.lang.NullPointerException
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.resetZooKeeperTrackers(HConnectionManager.java:533)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.abort(HConnectionManager.java:1536)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:344)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:262)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
{noformat}
The following code is not protected against NPE:
{code}
private synchronized void resetZooKeeperTrackers()
throws ZooKeeperConnectionException {
LOG.info(Trying to reconnect to zookeeper);
masterAddressTracker.stop();
masterAddressTracker = null;
rootRegionTracker.stop();
rootRegionTracker = null;
clusterId = null;
this.zooKeeper = null;
setupZookeeperTrackers();
}
{code}
In some cases as proven by the log snippet above it might happen that either
masterAddressTracker or rootRegionTracker might be null.
Because of the NPE the code can't reach setupZookeeperTrackers() call.
This should be fixed at least the way as shown in one of the patches in
HBASE-5153
{code}
LOG.info(Trying to reconnect to zookeeper.);
if (this.masterAddressTracker != null) {
this.masterAddressTracker.stop();
this.masterAddressTracker = null;
}
if (this.rootRegionTracker != null) {
this.rootRegionTracker.stop();
this.rootRegionTracker = null;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12586) Task 6 7 from HBASE-9117, delete all public HTable constructors and delete ConnectionManager#{delete,get}Connection
[
https://issues.apache.org/jira/browse/HBASE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357790#comment-14357790
]
Andrey Stepachev commented on HBASE-12586:
--
[~mantonov] great cleanup.
HTable has getRegionLocations, may be we need new jira for cleaning up those
methods too?
(they marked deprecated and seems most tests and code that use HTable actually
does that
only for this method).
thanks.
Task 6 7 from HBASE-9117, delete all public HTable constructors and delete
ConnectionManager#{delete,get}Connection
--
Key: HBASE-12586
URL: https://issues.apache.org/jira/browse/HBASE-12586
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: beginner, beginners
Attachments: HBASE-12586.patch, HBASE-12586.patch, HBASE-12586.patch,
HBASE-12586.patch, HBASE-12586.patch
Finish cleanup from HBASE-9117 removing old API. This issue covers tasks 6
and 7 from the list here:
https://issues.apache.org/jira/browse/HBASE-9117?focusedCommentId=13919716page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13919716
To be done in master branch only.
Marked as beginner task. The idea is straight-forward. It is just a lot of
work going through all tests converting to use new API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-5080) Make HConnectionManager's prefetchRegionCache error trace more helpful
[ https://issues.apache.org/jira/browse/HBASE-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-5080. -- Resolution: Cannot Reproduce Resolve as can't reproduce... a lot has changed here since. Reopen if we have it wrong [~shrijeet] Make HConnectionManager's prefetchRegionCache error trace more helpful -- Key: HBASE-5080 URL: https://issues.apache.org/jira/browse/HBASE-5080 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.90.3 Reporter: Shrijeet Paliwal Priority: Minor We catch RuntimeException in this HConnectionManager's prefetchRegionCache method. The trace is not very helpful since it eats the information about the line where RuntimeException actually happened. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10776) Separate HConnectionManager into several parts
[ https://issues.apache.org/jira/browse/HBASE-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357835#comment-14357835 ] Mikhail Antonov commented on HBASE-10776: - Is this issue still relevant? Trunk seems to be moving towards large changes in this area.. this particular class might be removed. Separate HConnectionManager into several parts -- Key: HBASE-10776 URL: https://issues.apache.org/jira/browse/HBASE-10776 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.89-fb Reporter: @deprecated Yi Deng Priority: Minor Fix For: 0.89-fb HConnectionManager is too large to effectively maintain. This Jira records some refactoring jobs: 1. Move TableServers out as a standalone class 2. Move region-locating code as a class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357833#comment-14357833
]
Hadoop QA commented on HBASE-13193:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704001/HBASE-13193-v4.patch
against master branch at commit 2adec36b2f57fea7006036e5a1f4d5979737ab34.
ATTACHMENT ID: 12704001
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13198//console
This message is automatically generated.
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch, HBASE-13193-v5.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13090) Progress heartbeats for long running scanners
[ https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor updated HBASE-13090: Status: Patch Available (was: Open) Submit patch to see if anything is broken by this Progress heartbeats for long running scanners - Key: HBASE-13090 URL: https://issues.apache.org/jira/browse/HBASE-13090 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Attachments: HBASE-13090-v1.patch It can be necessary to set very long timeouts for clients that issue scans over large regions when all data in the region might be filtered out depending on scan criteria. This is a usability concern because it can be hard to identify what worst case timeout to use until scans are occasionally/intermittently failing in production, depending on variable scan criteria. It would be better if the client-server scan protocol can send back periodic progress heartbeats to clients as long as server scanners are alive and making progress. This is related but orthogonal to streaming scan (HBASE-13071). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13090) Progress heartbeats for long running scanners
[ https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor updated HBASE-13090: Attachment: HBASE-13090-v1.patch Attached is a rough work in progress patch. The patch does provide a working implementation of heartbeats but I believe it could be refined so I am looking to get some feedback. The implementation points that I wanted to highlight for discussion are below: * We wanted to move all time tracking into RegionScanner and StoreScanner and leave RSRpcServices unscathed. I started off with that intention but it was slowly revealed that it may be better to simply have a timeLimit field in the call to nextRaw from RSRpcServices. Logic outlined below: ** While it is certainly possible to add a reset() or newSession() method to the RegionScanner interface that would allow us to reset time tracking, the issue becomes how do we communicate that size limit down from the RegionScanner into the StoreScanner (the scanner that is looping through the cells for a particular column family). ** The StoreScanners are stored in a KeyValueHeap in the RegionScanner... So it would be possible to loop through them all and call a similar reset/newSession method on all of them but that seems dirty and wasteful. It seems more appropriate to communicate the timeLimit down to only the relevant storeScanner via a timeLimit field in the InternalScanner#next(ListCell results, ..., timeLimit) call. ** Since the RegionScanner also implements the InternalScanner interface, that same next method would need to be implemented in RegionScannerImpl. Because of this, I think it makes the most sense to simply have a nextRaw(ListCell, ..., timeLimit) method to specify the timeLimit from RSRpcServices rather than an update/newSession call * To avoid polluting the returned Result array with state information about heartbeats, a new heartbeat flag has been added to the ScanResponse. Since only the ScannerCallable ever sees the ScanResponse returned from the server, I have exposed the method ScannerCallable#isHeartbeatMessage() to allow the ClientScanner to check if the most recent server response was a heartbeat/keep-alive message. * The method postHeapNext(ListCells) was added to RegionScannerImpl to allow me to insert delays in between fetches of column family cells for testing. It didn't feel clean, so I was wondering if anyone had any ideas about alternative approaches to emulate long running scans on the server side * Since heartbeat messages have the potential to create partial results (in the event that the timeout occurs in the middle of a row) we only allow heartbeat messages if the client has specified that heartbeats are supported AND partial results are also supported. Ideas for improvement: * As earlier discussion indicated, the tracking of limits in RSRpcServices is somewhat messy. When a new limit needs to be added, the RegionScanner and InternalScanner interfaces must both be changed. The limit logic may be simplified by defining something along the lines of a ScannerLimit object. The object would have a field per limit and would have an associated Builder that would allow us to specify only the limits we care about (if a limit is not set, then it doesn't get enforced). Then, in the future, if a new limit was needed it would only amount to adding a new field in ScannerLimit and adding the appropriate enforcement logic (no changes to interfaces necessary). What do you guys think? I thought this would clean things up a bit but wanted to see if any objections first Of course the finer implementation points can be seen in the patch itself and any feedback would be appreciated. Will post to reviewboard Thanks Progress heartbeats for long running scanners - Key: HBASE-13090 URL: https://issues.apache.org/jira/browse/HBASE-13090 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Attachments: HBASE-13090-v1.patch It can be necessary to set very long timeouts for clients that issue scans over large regions when all data in the region might be filtered out depending on scan criteria. This is a usability concern because it can be hard to identify what worst case timeout to use until scans are occasionally/intermittently failing in production, depending on variable scan criteria. It would be better if the client-server scan protocol can send back periodic progress heartbeats to clients as long as server scanners are alive and making progress. This is related but orthogonal to streaming scan (HBASE-13071). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13194) TableNamespaceManager not ready cause MasterQuotaManager initialization fail
[
https://issues.apache.org/jira/browse/HBASE-13194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357906#comment-14357906
]
Jimmy Xiang commented on HBASE-13194:
-
The code doesn't match the comment any more, right? If failed to init namespace
table, we throw an exception now. It used to not throw an exception when the
namespace table is just introduced.
TableNamespaceManager not ready cause MasterQuotaManager initialization fail
-
Key: HBASE-13194
URL: https://issues.apache.org/jira/browse/HBASE-13194
Project: HBase
Issue Type: Bug
Components: master
Reporter: zhangduo
This cause TestNamespaceAuditor to fail.
https://builds.apache.org/job/HBase-TRUNK/6237/testReport/junit/org.apache.hadoop.hbase.namespace/TestNamespaceAuditor/testRegionOperations/
{noformat}
2015-03-10 22:42:01,372 ERROR [hemera:48616.activeMasterManager]
namespace.NamespaceStateManager(204): Error while update namespace state.
java.io.IOException: Table Namespace Manager not ready yet, try again later
at
org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:1912)
at
org.apache.hadoop.hbase.master.HMaster.listNamespaceDescriptors(HMaster.java:2131)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.initialize(NamespaceStateManager.java:188)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.start(NamespaceStateManager.java:63)
at
org.apache.hadoop.hbase.namespace.NamespaceAuditor.start(NamespaceAuditor.java:57)
at
org.apache.hadoop.hbase.quotas.MasterQuotaManager.start(MasterQuotaManager.java:88)
at
org.apache.hadoop.hbase.master.HMaster.initQuotaManager(HMaster.java:902)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:756)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:161)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1455)
at java.lang.Thread.run(Thread.java:744)
{noformat}
The direct reason is that we do not have a retry here, if init fails then it
always fails. But I skimmed the code, seems there is no async init operations
when calling finishActiveMasterInitialization, so it is very strange. Need to
dig more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13090) Progress heartbeats for long running scanners
[
https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357903#comment-14357903
]
Jonathan Lawlor commented on HBASE-13090:
-
Thanks for the comments [~stack]
bq. If only timeout, then maybe premature for ScanLimit unless anything in
current Scan structure that might sit better in ScanLimit?
I was thinking that we could combine the batch limit, size limit, and now the
time limit into ScannerLimit object. With this patch, the InternalScanner and
RegionScanner interfaces now have a large cascading call structure that looks
like this:
{code}
NextState next(ListCell result) throws IOException;
...
NextState next(ListCell result, int batchLimit) throws IOException;
...
NextState next(ListCell result, int batchLimit, long sizeLimit) throws
IOException;
...
NextState next(ListCell result, int batchLimit, long sizeLimit, long
timeLimit) throws IOException;
{code}
As more limits are added, it gets uglier and uglier. The idea with ScannerLimit
would be to change it to this:
{code}
NextState next(ListCell result) throws IOException;
...
NextState next(ListCell result, ScannerLimit limit) throws IOException;
{code}
Where the ScannerLimit object can have as many limits specified as it wants
(may only contain a time limit, or may contain a time limit, batch limit and
size limit).
bq. What would be the downsides if default was to allow return of partials to
clients?
So right now partial result support is on by default but in the case that the
scan is specified to be a small scan we disable partial results server side.
This means that in the case of small scans we wouldn't allow heartbeat messages
either since they could potentially create partials. Outside of small scans
heartbeats would be supported.
bq. since you can't specify your own Scanner implementation serverside (you
can't right?)
As far as I can tell there is no nice way to specify your own StoreScanner
implementation but upon further investigation it looks like I can specify my
own KeyValueHeap implementation inside the RegionScanners. This would allow me
to take this method out. Going to investigate further and see if this ugly
postHeapNext method can be taken out.
bq. When do I call isHeartbeatMessage? At want point in the processing?
Currently it is used inside ClientScanner.java after the Result array comes
back from the server. By checking it here, we can see if the most recent
response from the server (the one that returned the Results array) was a
heartbeat message.
Progress heartbeats for long running scanners
-
Key: HBASE-13090
URL: https://issues.apache.org/jira/browse/HBASE-13090
Project: HBase
Issue Type: New Feature
Reporter: Andrew Purtell
Assignee: Jonathan Lawlor
Attachments: HBASE-13090-v1.patch
It can be necessary to set very long timeouts for clients that issue scans
over large regions when all data in the region might be filtered out
depending on scan criteria. This is a usability concern because it can be
hard to identify what worst case timeout to use until scans are
occasionally/intermittently failing in production, depending on variable scan
criteria. It would be better if the client-server scan protocol can send back
periodic progress heartbeats to clients as long as server scanners are alive
and making progress.
This is related but orthogonal to streaming scan (HBASE-13071).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13213) Split out locality metrics among primary and secondary region
[ https://issues.apache.org/jira/browse/HBASE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13213: --- Status: Patch Available (was: Open) Split out locality metrics among primary and secondary region - Key: HBASE-13213 URL: https://issues.apache.org/jira/browse/HBASE-13213 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Attachments: 13213-v1.patch This task provides the ability to track locality of primary and secondary region replicas. We already have percentFilesLocal metric. There should be two sets of metrics - one for primaries and another for secondary / tertiary regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13213) Split out locality metrics among primary and secondary region
[ https://issues.apache.org/jira/browse/HBASE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13213: --- Attachment: 13213-v1.patch Split out locality metrics among primary and secondary region - Key: HBASE-13213 URL: https://issues.apache.org/jira/browse/HBASE-13213 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu Attachments: 13213-v1.patch This task provides the ability to track locality of primary and secondary region replicas. We already have percentFilesLocal metric. There should be two sets of metrics - one for primaries and another for secondary / tertiary regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13213) Split out locality metrics among primary and secondary region
[
https://issues.apache.org/jira/browse/HBASE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357712#comment-14357712
]
Ted Yu commented on HBASE-13213:
Here is result of preliminary testing:
Verified that some regions of
IntegrationTestTimeBoundedRequestsWithRegionReplicas table have replica Id of 1
or 2 using http://xx:16030/rs-status
{code}
IntegrationTestTimeBoundedRequestsWithRegionReplicas,b98e,1425341957962_0002.c0be8d17b9183aa1f6a1c305a5a527cd.
b98ebff42
IntegrationTestTimeBoundedRequestsWithRegionReplicas,c65a,1425341957962_0002.5ebffccd8800b48fe62939250b6bb8db.
c65accc02
{code}
Using http://xx:16030/jmx, verified that locality for secondary regions has
some value:
{code}
percentFilesLocal : 25,
percentFilesLocalSecondaryRegions : 45,
{code}
Split out locality metrics among primary and secondary region
-
Key: HBASE-13213
URL: https://issues.apache.org/jira/browse/HBASE-13213
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 13213-v1.patch
This task provides the ability to track locality of primary and secondary
region replicas.
We already have percentFilesLocal metric.
There should be two sets of metrics - one for primaries and another for
secondary / tertiary regions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12586) Task 6 7 from HBASE-9117, delete all public HTable constructors and delete ConnectionManager#{delete,get}Connection
[
https://issues.apache.org/jira/browse/HBASE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-12586:
--
Hadoop Flags: Incompatible change
+1 from me. Marked as incompatible change. You good w/ this [~ndimiduk]?
A release note when you get a chance [~mantonov] please.
Task 6 7 from HBASE-9117, delete all public HTable constructors and delete
ConnectionManager#{delete,get}Connection
--
Key: HBASE-12586
URL: https://issues.apache.org/jira/browse/HBASE-12586
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: beginner, beginners
Attachments: HBASE-12586.patch, HBASE-12586.patch, HBASE-12586.patch,
HBASE-12586.patch, HBASE-12586.patch
Finish cleanup from HBASE-9117 removing old API. This issue covers tasks 6
and 7 from the list here:
https://issues.apache.org/jira/browse/HBASE-9117?focusedCommentId=13919716page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13919716
To be done in master branch only.
Marked as beginner task. The idea is straight-forward. It is just a lot of
work going through all tests converting to use new API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13208) Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName
[ https://issues.apache.org/jira/browse/HBASE-13208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357759#comment-14357759 ] stack commented on HBASE-13208: --- +1 Patch build should match the patch filename and not the whole relative URL in findBranchNameFromPatchName - Key: HBASE-13208 URL: https://issues.apache.org/jira/browse/HBASE-13208 Project: HBase Issue Type: Bug Reporter: Esteban Gutierrez Assignee: Esteban Gutierrez Priority: Trivial Fix For: 2.0.0 Attachments: 0001-HBASE-13208-Patch-build-should-match-the-patch-filen.patch In HBASE-1319 we saw that the patch got applied to the wrong branch, the problem is findBranchNameFromPatchName matching a regex that contains wildcard symbols against the whole URL, in this case the regex is 0.94 and the relativePatchURL is /jira/secure/attachment/12703942/HBASE-13193-v4.patch where 0394 is a match. Thanks to [~jonathan.lawlor] for reporting this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357781#comment-14357781
]
Hudson commented on HBASE-13206:
SUCCESS: Integrated in HBase-0.98 #893 (See
[https://builds.apache.org/job/HBase-0.98/893/])
HBASE-13206 Fix TableLock tableName log format (matteo.bertozzi: rev
b58e954efcc4481495232de0f7d299f0db9a3e1b)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/TableLockChecker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13193) RegionScannerImpl filters should not be reset if a partial Result is returned
[
https://issues.apache.org/jira/browse/HBASE-13193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357841#comment-14357841
]
stack commented on HBASE-13193:
---
Unless objection, will commit this evening.
RegionScannerImpl filters should not be reset if a partial Result is returned
-
Key: HBASE-13193
URL: https://issues.apache.org/jira/browse/HBASE-13193
Project: HBase
Issue Type: Bug
Reporter: Jonathan Lawlor
Assignee: Jonathan Lawlor
Attachments: HBASE-13193-v1.patch, HBASE-13193-v2.patch,
HBASE-13193-v3.patch, HBASE-13193-v4.patch, HBASE-13193-v4.patch,
HBASE-13193-v4.patch, HBASE-13193-v5.patch
In RegionScannerImpl the filters get reset during calls to nextRaw:
{code}
public NextState nextRaw(ListCell outResults, int batchLimit, long
remainingResultSize)
throws IOException {
...
resetFilters();
if (isFilterDoneInternal()) {
...
}
{code}
This creates a problem when returning partial Results because filters should
only be reset in between rows (i.e. after all of the cells for a particular
row have been returned). If a partial Result is returned, there are still
cells in the row that have not been evaluated yet and we should avoid
resetting the filter since it will wipe away all state information for that
filter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13201) Remove HTablePool from thrift-server
[ https://issues.apache.org/jira/browse/HBASE-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-13201: --- Attachment: HBASE-13201-1.patch Remove HTablePool from thrift-server Key: HBASE-13201 URL: https://issues.apache.org/jira/browse/HBASE-13201 Project: HBase Issue Type: Sub-task Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1 Attachments: HBASE-13201-1.patch, HBASE-13201.patch HTablePool is an old concept that was superseded by Connection. thrift-server doesn't need HTablePool anymore and can use the ConnectionCache directly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13201) Remove HTablePool from thrift-server
[ https://issues.apache.org/jira/browse/HBASE-13201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357842#comment-14357842 ] Solomon Duskis commented on HBASE-13201: HTablePool is in hbase-client in the 1.0 branch. I'll create a separate patch for it. Remove HTablePool from thrift-server Key: HBASE-13201 URL: https://issues.apache.org/jira/browse/HBASE-13201 Project: HBase Issue Type: Sub-task Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 1.0.1 Attachments: HBASE-13201-1.patch, HBASE-13201.patch HTablePool is an old concept that was superseded by Connection. thrift-server doesn't need HTablePool anymore and can use the ConnectionCache directly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
??????[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime isbuggy
iPhone
-- --
??: stack (JIRA) j...@apache.org
: 2015??03??12?? 06:53
??: issues issues@hbase.apache.org
: [jira] [Updated] (HBASE-13167) The check for balancerCutoffTime isbuggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13167:
--
Assignee: Mikhail Antonov
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Assignee: Mikhail Antonov
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13194) TableNamespaceManager not ready cause MasterQuotaManager initialization fail
[
https://issues.apache.org/jira/browse/HBASE-13194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357895#comment-14357895
]
zhangduo commented on HBASE-13194:
--
I see that for 'hbase:meta', we will try to verify whether the target region
server is still alive, otherwise we will force a assign in
finishActiveMasterInitialization.
But for namespace table, we do not have this check, and the code is here
{code:title=TableNamespaceManager.java}
public void start() throws IOException {
if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
TableName.NAMESPACE_TABLE_NAME)) {
LOG.info(Namespace table not found. Creating...);
createNamespaceTable(masterServices);
}
try {
// Wait for the namespace table to be assigned.
// If timed out, we will move ahead without initializing it.
// So that it should be initialized later on lazily.
long startTime = EnvironmentEdgeManager.currentTime();
int timeout = conf.getInt(NS_INIT_TIMEOUT, DEFAULT_NS_INIT_TIMEOUT);
while (!(isTableAssigned() isTableEnabled())) {
if (EnvironmentEdgeManager.currentTime() - startTime + 100 timeout) {
// We can't do anything if ns is not online.
throw new IOException(Timedout + timeout + ms waiting for
namespace table to +
be assigned and enabled: + getTableState());
}
Thread.sleep(100);
}
} catch (InterruptedException e) {
throw (InterruptedIOException)new InterruptedIOException().initCause(e);
}
// initialize namespace table
isTableAvailableAndInitialized();
}
{code}
The comments say that we could do a lazy initialization. It is true that
'isTableAvailableAndInitialized' has initialization code in it, the problem is
NamespaceStateManager. NamespaceStateManager only call its initialize once in
its start method, if it is failed this time, then there is no chance to do a
'lazy initialization'.
To fix this test, I think use 'isTableAvailableAndInitialized' as the while
loop's condition is enough, but it is still not 100% safe. Maybe we should also
introduce lazy initialization in NamespaceStateManager?
Any suggestions? I do not know who are the experts, maybe [~jxiang] [~octo47] ?
Thanks
TableNamespaceManager not ready cause MasterQuotaManager initialization fail
-
Key: HBASE-13194
URL: https://issues.apache.org/jira/browse/HBASE-13194
Project: HBase
Issue Type: Bug
Components: master
Reporter: zhangduo
This cause TestNamespaceAuditor to fail.
https://builds.apache.org/job/HBase-TRUNK/6237/testReport/junit/org.apache.hadoop.hbase.namespace/TestNamespaceAuditor/testRegionOperations/
{noformat}
2015-03-10 22:42:01,372 ERROR [hemera:48616.activeMasterManager]
namespace.NamespaceStateManager(204): Error while update namespace state.
java.io.IOException: Table Namespace Manager not ready yet, try again later
at
org.apache.hadoop.hbase.master.HMaster.checkNamespaceManagerReady(HMaster.java:1912)
at
org.apache.hadoop.hbase.master.HMaster.listNamespaceDescriptors(HMaster.java:2131)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.initialize(NamespaceStateManager.java:188)
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.start(NamespaceStateManager.java:63)
at
org.apache.hadoop.hbase.namespace.NamespaceAuditor.start(NamespaceAuditor.java:57)
at
org.apache.hadoop.hbase.quotas.MasterQuotaManager.start(MasterQuotaManager.java:88)
at
org.apache.hadoop.hbase.master.HMaster.initQuotaManager(HMaster.java:902)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:756)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:161)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1455)
at java.lang.Thread.run(Thread.java:744)
{noformat}
The direct reason is that we do not have a retry here, if init fails then it
always fails. But I skimmed the code, seems there is no async init operations
when calling finishActiveMasterInitialization, so it is very strange. Need to
dig more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13206) Fix TableLock tableName log format
[
https://issues.apache.org/jira/browse/HBASE-13206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357909#comment-14357909
]
Hudson commented on HBASE-13206:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #849 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/849/])
HBASE-13206 Fix TableLock tableName log format (matteo.bertozzi: rev
b58e954efcc4481495232de0f7d299f0db9a3e1b)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/TableLockChecker.java
Fix TableLock tableName log format
--
Key: HBASE-13206
URL: https://issues.apache.org/jira/browse/HBASE-13206
Project: HBase
Issue Type: Bug
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Attachments: HBASE-13206-v0.patch
The TableName log from Table Lock result in something like:
{noformat}
[tableName=
^Gdefault^R^Cfoo, lockOwner=localhost,6
{noformat}
because it uses tableNameProto.toByteArray()
the fix will result in the proper view
{noformat}
[tableName=default:testMissingLastRegion, lockOwner=localh
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime is buggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13167:
--
Priority: Trivial (was: Major)
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Assignee: Mikhail Antonov
Priority: Trivial
Fix For: 2.0.0
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime is buggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13167:
--
Resolution: Fixed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Assignee: Mikhail Antonov
Fix For: 2.0.0
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13213) Split out locality metrics among primary and secondary region
[
https://issues.apache.org/jira/browse/HBASE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357761#comment-14357761
]
Hadoop QA commented on HBASE-13213:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704034/13213-v1.patch
against master branch at commit c938267db3bfe95908f97397f64ffc50933e4343.
ATTACHMENT ID: 12704034
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1925 checkstyle errors (more than the master's current 1924 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13200//console
This message is automatically generated.
Split out locality metrics among primary and secondary region
-
Key: HBASE-13213
URL: https://issues.apache.org/jira/browse/HBASE-13213
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 13213-v1.patch
This task provides the ability to track locality of primary and secondary
region replicas.
We already have percentFilesLocal metric.
There should be two sets of metrics - one for primaries and another for
secondary / tertiary regions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
??????[jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature
feature iPhone -- -- ??: Eshcar Hillel (JIRA) j...@apache.org : 2015??03??11?? 16:27 ??: issues issues@hbase.apache.org : [jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature [ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-13071: -- Attachment: HBASE-13071_trunk_7.patch Hbase Streaming Scan Feature Key: HBASE-13071 URL: https://issues.apache.org/jira/browse/HBASE-13071 Project: HBase Issue Type: New Feature Reporter: Eshcar Hillel Attachments: 99.eshcar.png, HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch, HBASE-13071_trunk_2.patch, HBASE-13071_trunk_3.patch, HBASE-13071_trunk_4.patch, HBASE-13071_trunk_5.patch, HBASE-13071_trunk_6.patch, HBASE-13071_trunk_7.patch, HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf, gc.eshcar.png, hits.eshcar.png, network.png A scan operation iterates over all rows of a table or a subrange of the table. The synchronous nature in which the data is served at the client side hinders the speed the application traverses the data: it increases the overall processing time, and may cause a great variance in the times the application waits for the next piece of data. The scanner next() method at the client side invokes an RPC to the regionserver and then stores the results in a cache. The application can specify how many rows will be transmitted per RPC; by default this is set to 100 rows. The cache can be considered as a producer-consumer queue, where the hbase client pushes the data to the queue and the application consumes it. Currently this queue is synchronous, i.e., blocking. More specifically, when the application consumed all the data from the cache --- so the cache is empty --- the hbase client retrieves additional data from the server and re-fills the cache with new data. During this time the application is blocked. Under the assumption that the application processing time can be balanced by the time it takes to retrieve the data, an asynchronous approach can reduce the time the application is waiting for data. We attach a design document. We also have a patch that is based on a private branch, and some evaluation results of this code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13167) The check for balancerCutoffTime is buggy
[
https://issues.apache.org/jira/browse/HBASE-13167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13167:
--
Assignee: Mikhail Antonov
The check for balancerCutoffTime is buggy
-
Key: HBASE-13167
URL: https://issues.apache.org/jira/browse/HBASE-13167
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.98.8
Reporter: Tianyin Xu
Assignee: Mikhail Antonov
Attachments: HBASE-13167.patch
In HMaster, the checking logic for balancerCutoffTime in buggy.
See the following code:
{code:title=HMaster.java|borderStyle=solid}
1419 private int getBalancerCutoffTime() {
1420 int balancerCutoffTime =
1421 getConfiguration().getInt(hbase.balancer.max.balancing, -1);
1422 if (balancerCutoffTime == -1) {
1423 // No time period set so create one
1424 int balancerPeriod =
1425 getConfiguration().getInt(hbase.balancer.period, 30);
1426 balancerCutoffTime = balancerPeriod;
1427 // If nonsense period, set it to balancerPeriod
1428 if (balancerCutoffTime = 0) balancerCutoffTime = balancerPeriod;
1429 }
1430 return balancerCutoffTime;
1431 }
{code}
Basically, the check for nonsense period is no effect, because
balancerCutoffTime is always assigned to be balancerPeriod whatever the value
is.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13192) IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures
[ https://issues.apache.org/jira/browse/HBASE-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357751#comment-14357751 ] Hudson commented on HBASE-13192: FAILURE: Integrated in HBase-1.1 #280 (See [https://builds.apache.org/job/HBase-1.1/280/]) HBASE-13192. IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures. (ddas: rev e864dd0520dd62e95a98b0e9c92891c546aff999) * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures --- Key: HBASE-13192 URL: https://issues.apache.org/jira/browse/HBASE-13192 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.0.1 Attachments: 13192-1.txt, 13192-2.txt Noticed this issue in our internal test runs. It shows up when the test is run with replicas turned on. After the bulk load is complete, the test tries to modify the table with an intention of having the replicas load the new region files. But due to a bug in the way the test was waiting for the modifyTable operation to complete, it would not actually wait. Sometimes a replica wouldn't have loaded the new files yet and it might happen that the 'check' step of the test would go to that replica asking for data. It would give empty results back, and the test would fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12586) Task 6 7 from HBASE-9117, delete all public HTable constructors and delete ConnectionManager#{delete,get}Connection
[
https://issues.apache.org/jira/browse/HBASE-12586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-12586:
Release Note: HTable class has been marked as private API before, and now
it's no longer directly instantiable from client code (all public constructors
have been removed). All clients should use Connection#getTable() and
Connection#getRegionLocator() when appropriate to obtain Table and
RegionLocator implementations to work with.
Task 6 7 from HBASE-9117, delete all public HTable constructors and delete
ConnectionManager#{delete,get}Connection
--
Key: HBASE-12586
URL: https://issues.apache.org/jira/browse/HBASE-12586
Project: HBase
Issue Type: Task
Affects Versions: 2.0.0
Reporter: stack
Assignee: Mikhail Antonov
Labels: beginner, beginners
Attachments: HBASE-12586.patch, HBASE-12586.patch, HBASE-12586.patch,
HBASE-12586.patch, HBASE-12586.patch
Finish cleanup from HBASE-9117 removing old API. This issue covers tasks 6
and 7 from the list here:
https://issues.apache.org/jira/browse/HBASE-9117?focusedCommentId=13919716page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13919716
To be done in master branch only.
Marked as beginner task. The idea is straight-forward. It is just a lot of
work going through all tests converting to use new API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13192) IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures
[ https://issues.apache.org/jira/browse/HBASE-13192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357764#comment-14357764 ] Hudson commented on HBASE-13192: SUCCESS: Integrated in HBase-1.0 #799 (See [https://builds.apache.org/job/HBase-1.0/799/]) HBASE-13192. IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures. (ddas: rev f4794bd2cb4996989603f3054e20c6cb86a1f2d6) * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures --- Key: HBASE-13192 URL: https://issues.apache.org/jira/browse/HBASE-13192 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.0.1 Attachments: 13192-1.txt, 13192-2.txt Noticed this issue in our internal test runs. It shows up when the test is run with replicas turned on. After the bulk load is complete, the test tries to modify the table with an intention of having the replicas load the new region files. But due to a bug in the way the test was waiting for the modifyTable operation to complete, it would not actually wait. Sometimes a replica wouldn't have loaded the new files yet and it might happen that the 'check' step of the test would go to that replica asking for data. It would give empty results back, and the test would fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13214) Remove deprecated and unused methods from HTable class
Mikhail Antonov created HBASE-13214: --- Summary: Remove deprecated and unused methods from HTable class Key: HBASE-13214 URL: https://issues.apache.org/jira/browse/HBASE-13214 Project: HBase Issue Type: Sub-task Components: API Affects Versions: 2.0.0 Reporter: Mikhail Antonov Methods like #getRegionLocation(), #isTableEnabled() etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-13090) Progress heartbeats for long running scanners
[ https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor reassigned HBASE-13090: --- Assignee: Jonathan Lawlor Progress heartbeats for long running scanners - Key: HBASE-13090 URL: https://issues.apache.org/jira/browse/HBASE-13090 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Jonathan Lawlor Attachments: HBASE-13090-v1.patch It can be necessary to set very long timeouts for clients that issue scans over large regions when all data in the region might be filtered out depending on scan criteria. This is a usability concern because it can be hard to identify what worst case timeout to use until scans are occasionally/intermittently failing in production, depending on variable scan criteria. It would be better if the client-server scan protocol can send back periodic progress heartbeats to clients as long as server scanners are alive and making progress. This is related but orthogonal to streaming scan (HBASE-13071). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13063) Allow to turn off memstore replication for region replicas
[
https://issues.apache.org/jira/browse/HBASE-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357860#comment-14357860
]
Hudson commented on HBASE-13063:
SUCCESS: Integrated in HBase-TRUNK #6244 (See
[https://builds.apache.org/job/HBase-TRUNK/6244/])
HBASE-13063 Allow to turn off memstore replication for region replicas -
ADDENDUM for turning of primary flush tigger when this is enabled (enis: rev
c938267db3bfe95908f97397f64ffc50933e4343)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionReplicaUtil.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
Allow to turn off memstore replication for region replicas
--
Key: HBASE-13063
URL: https://issues.apache.org/jira/browse/HBASE-13063
Project: HBase
Issue Type: New Feature
Components: regionserver, Replication
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-13063-v0.patch, HBASE-13063-v1-branch-1.patch,
HBASE-13063-v1.patch, HBASE-13063-v1.patch, HBASE-13063-v2.patch,
HBASE-13063-v2.patch, hbase-13063-addendum.patch, hbase-13063-addendum.patch
HBASE-11568 allows to use replication to send wal edits from the primary to
the replicas.
sometimes the memstore replication is not required, so it will be nice have a
flag to disable it.
the result will be more or less the same to what we have in phase-1, but with
the row-level consistency provided by the flush edit transfered via
replication to refresh the hfiles.
create 't1', 'f', {REGION_REPLICATION = 2, REGION_MEMSTORE_REPLICATION =
false}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13169) ModifyTable increasing the region replica count should also auto-setup RRRE
[
https://issues.apache.org/jira/browse/HBASE-13169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357868#comment-14357868
]
Hadoop QA commented on HBASE-13169:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704030/hbase-13169_v2-master.patch
against master branch at commit c938267db3bfe95908f97397f64ffc50933e4343.
ATTACHMENT ID: 12704030
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.util.TestProcessBasedCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13201//console
This message is automatically generated.
ModifyTable increasing the region replica count should also auto-setup RRRE
---
Key: HBASE-13169
URL: https://issues.apache.org/jira/browse/HBASE-13169
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 2.0.0, 1.1.0
Attachments: hbase-13169_v1.patch, hbase-13169_v2-master.patch
This is a follow up to Async WAL replication jira (HBASE-11568). In
HBASE-11568 we setup replication automatically in create table if the
configuration is enabled.
We should do the same in case a table with region replication = 1 is
modified, and region replication is set to 1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13090) Progress heartbeats for long running scanners
[ https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357867#comment-14357867 ] stack commented on HBASE-13090: --- Nice writeup [~jonathan.lawlor] If only timeout, then maybe premature for ScanLimit unless anything in current Scan structure that might sit better in ScanLimit? bq. if the client has specified that heartbeats are supported AND partial results are also supported This might be ok for 1.1 but partials should be on all the time in 2.0.. This feature should be on all the time in 2.0. What would be the downsides if default was to allow return of partials to clients? On postHeapNext, yeah, ugly, but since you can't specify your own Scanner implementation serverside (you can't right?), ugly injection is all you have ... so yeah, ugly but we need it (can you make the scan latched rather than slowed) When do I call isHeartbeatMessage? At want point in the processing? Your reasoning that new session or reset doesn't work makes sense to me. Will give review on patch later. Good stuff [~jonathan.lawlor] Progress heartbeats for long running scanners - Key: HBASE-13090 URL: https://issues.apache.org/jira/browse/HBASE-13090 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Jonathan Lawlor Attachments: HBASE-13090-v1.patch It can be necessary to set very long timeouts for clients that issue scans over large regions when all data in the region might be filtered out depending on scan criteria. This is a usability concern because it can be hard to identify what worst case timeout to use until scans are occasionally/intermittently failing in production, depending on variable scan criteria. It would be better if the client-server scan protocol can send back periodic progress heartbeats to clients as long as server scanners are alive and making progress. This is related but orthogonal to streaming scan (HBASE-13071). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13063) Allow to turn off memstore replication for region replicas
[
https://issues.apache.org/jira/browse/HBASE-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357869#comment-14357869
]
Hudson commented on HBASE-13063:
SUCCESS: Integrated in HBase-1.1 #281 (See
[https://builds.apache.org/job/HBase-1.1/281/])
HBASE-13063 Allow to turn off memstore replication for region replicas -
ADDENDUM for turning of primary flush tigger when this is enabled (enis: rev
0f892c23ecb7520d218fdddaa49ff69e6eb09152)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/util/ServerRegionReplicaUtil.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegionReplicaUtil.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Allow to turn off memstore replication for region replicas
--
Key: HBASE-13063
URL: https://issues.apache.org/jira/browse/HBASE-13063
Project: HBase
Issue Type: New Feature
Components: regionserver, Replication
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-13063-v0.patch, HBASE-13063-v1-branch-1.patch,
HBASE-13063-v1.patch, HBASE-13063-v1.patch, HBASE-13063-v2.patch,
HBASE-13063-v2.patch, hbase-13063-addendum.patch, hbase-13063-addendum.patch
HBASE-11568 allows to use replication to send wal edits from the primary to
the replicas.
sometimes the memstore replication is not required, so it will be nice have a
flag to disable it.
the result will be more or less the same to what we have in phase-1, but with
the row-level consistency provided by the flush edit transfered via
replication to refresh the hfiles.
create 't1', 'f', {REGION_REPLICATION = 2, REGION_MEMSTORE_REPLICATION =
false}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning
[ https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-13109: Make better SEEK vs SKIP decisions during scanning -- Key: HBASE-13109 URL: https://issues.apache.org/jira/browse/HBASE-13109 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Attachments: 13109-0.98-v4.txt, 13109-trunk-v2.txt, 13109-trunk-v3.txt, 13109-trunk-v4.txt, 13109-trunk-v5.txt, 13109-trunk.txt, nextIndexKVChange_new.patch I'm re-purposing this issue to add a heuristic as to when to SEEK and when to SKIP Cells. This has come up in various issues, and I think I have a way to finally fix this now. HBASE-9778, HBASE-12311, and friends are related. --- Old description --- This is a continuation of HBASE-9778. We've seen a scenario of a very slow scan over a region using a timerange that happens to fall after the ts of any Cell in the region. Turns out we spend a lot of time seeking. Tested with a 5 column table, and the scan is 5x faster when the timerange falls before all Cells' ts. We can use the lookahead hint introduced in HBASE-9778 to do opportunistic SKIPing before we actually seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13213) Split out locality metrics among primary and secondary region
Ted Yu created HBASE-13213: -- Summary: Split out locality metrics among primary and secondary region Key: HBASE-13213 URL: https://issues.apache.org/jira/browse/HBASE-13213 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu This task provides the ability to track locality of primary and secondary region replicas. We already have percentFilesLocal metric. There should be two sets of metrics - one for primaries and another for secondary / tertiary regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning
[ https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357707#comment-14357707 ] Andrew Purtell commented on HBASE-13109: [~jesse_yates] Make better SEEK vs SKIP decisions during scanning -- Key: HBASE-13109 URL: https://issues.apache.org/jira/browse/HBASE-13109 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Attachments: 13109-0.98-v4.txt, 13109-trunk-v2.txt, 13109-trunk-v3.txt, 13109-trunk-v4.txt, 13109-trunk-v5.txt, 13109-trunk.txt, nextIndexKVChange_new.patch I'm re-purposing this issue to add a heuristic as to when to SEEK and when to SKIP Cells. This has come up in various issues, and I think I have a way to finally fix this now. HBASE-9778, HBASE-12311, and friends are related. --- Old description --- This is a continuation of HBASE-9778. We've seen a scenario of a very slow scan over a region using a timerange that happens to fall after the ts of any Cell in the region. Turns out we spend a lot of time seeking. Tested with a 5 column table, and the scan is 5x faster when the timerange falls before all Cells' ts. We can use the lookahead hint introduced in HBASE-9778 to do opportunistic SKIPing before we actually seek. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13213) Split out locality metrics among primary and secondary region
[ https://issues.apache.org/jira/browse/HBASE-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13213: --- Attachment: (was: 13213-v1.patch) Split out locality metrics among primary and secondary region - Key: HBASE-13213 URL: https://issues.apache.org/jira/browse/HBASE-13213 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Ted Yu This task provides the ability to track locality of primary and secondary region replicas. We already have percentFilesLocal metric. There should be two sets of metrics - one for primaries and another for secondary / tertiary regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
