[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
[ https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261475#comment-13261475 ] Hudson commented on HBASE-5849: --- Integrated in HBase-0.92 #390 (See [https://builds.apache.org/job/HBase-0.92/390/]) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330119) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330118) Result = FAILURE stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java On first cluster startup, RS aborts if root znode is not available -- Key: HBASE-5849 URL: https://issues.apache.org/jira/browse/HBASE-5849 Project: HBase Issue Type: Bug Components: master, regionserver, zookeeper Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.2, 0.94.0 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch, HBASE-5849_v4-0.92.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch When launching a fresh new cluster, the master has to be started first, which might create race conditions for starting master and rs at the same time. Master startup code is smt like this: - establish zk connection - create root znodes in zk (/hbase) - create ephemeral node for master /hbase/master, Region server start up code is smt like this: - establish zk connection - check whether the root znode (/hbase) is there. If not, shutdown. - wait for the master to create znodes /hbase/master So, the problem is on the very first launch of the cluster, RS aborts to start since /hbase znode might not have been created yet (only the master creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on subsequent cluster starts, it does not matter which order the servers are started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261796#comment-13261796 ] Hudson commented on HBASE-5848: --- Integrated in HBase-TRUNK #2812 (See [https://builds.apache.org/job/HBase-TRUNK/2812/]) HBASE-5848 Addendum, try 2 (Revision 1330349) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261895#comment-13261895
]
Hudson commented on HBASE-5871:
---
Integrated in HBase-0.94-security #21 (See
[https://builds.apache.org/job/HBase-0.94-security/21/])
HBASE-5871 Usability regression, we don't parse compression algos anymore
(Revision 1330124)
Result = FAILURE
larsh :
Files :
* /hbase/branches/0.94/src/main/ruby/hbase/admin.rb
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
[ https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261896#comment-13261896 ] Hudson commented on HBASE-5849: --- Integrated in HBase-0.94-security #21 (See [https://builds.apache.org/job/HBase-0.94-security/21/]) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330117) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java On first cluster startup, RS aborts if root znode is not available -- Key: HBASE-5849 URL: https://issues.apache.org/jira/browse/HBASE-5849 Project: HBase Issue Type: Bug Components: master, regionserver, zookeeper Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.2, 0.94.0 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch, HBASE-5849_v4-0.92.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch When launching a fresh new cluster, the master has to be started first, which might create race conditions for starting master and rs at the same time. Master startup code is smt like this: - establish zk connection - create root znodes in zk (/hbase) - create ephemeral node for master /hbase/master, Region server start up code is smt like this: - establish zk connection - check whether the root znode (/hbase) is there. If not, shutdown. - wait for the master to create znodes /hbase/master So, the problem is on the very first launch of the cluster, RS aborts to start since /hbase znode might not have been created yet (only the master creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on subsequent cluster starts, it does not matter which order the servers are started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261897#comment-13261897 ] Hudson commented on HBASE-5848: --- Integrated in HBase-0.94-security #21 (See [https://builds.apache.org/job/HBase-0.94-security/21/]) HBASE-5848 Addendum (Revision 1330106) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262159#comment-13262159
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-TRUNK #2814 (See
[https://builds.apache.org/job/HBase-TRUNK/2814/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330551)
Result = FAILURE
larsh :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262164#comment-13262164
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-0.94 #149 (See
[https://builds.apache.org/job/HBase-0.94/149/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330549)
Result = SUCCESS
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262227#comment-13262227
]
Hudson commented on HBASE-5870:
---
Integrated in HBase-TRUNK #2815 (See
[https://builds.apache.org/job/HBase-TRUNK/2815/])
HBASE-5870 Hadoop 23 compilation broken because
JobTrackerRunner#getJobTracker() method is not found (Revision 1330563)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Zhihong Yu
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870-v2.txt, 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262236#comment-13262236
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-0.92 #391 (See
[https://builds.apache.org/job/HBase-0.92/391/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330558)
Result = SUCCESS
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262274#comment-13262274
]
Hudson commented on HBASE-5870:
---
Integrated in HBase-TRUNK-security #184 (See
[https://builds.apache.org/job/HBase-TRUNK-security/184/])
HBASE-5870 Hadoop 23 compilation broken because
JobTrackerRunner#getJobTracker() method is not found (Revision 1330563)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Zhihong Yu
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870-v2.txt, 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262275#comment-13262275
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-TRUNK-security #184 (See
[https://builds.apache.org/job/HBase-TRUNK-security/184/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330551)
Result = FAILURE
larsh :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262277#comment-13262277 ] Hudson commented on HBASE-5848: --- Integrated in HBase-TRUNK-security #184 (See [https://builds.apache.org/job/HBase-TRUNK-security/184/]) HBASE-5848 Addendum, try 2 (Revision 1330349) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262273#comment-13262273
]
Hudson commented on HBASE-5871:
---
Integrated in HBase-TRUNK-security #184 (See
[https://builds.apache.org/job/HBase-TRUNK-security/184/])
HBASE-5871 Usability regression, we don't parse compression algos anymore
(Revision 1330123)
Result = FAILURE
larsh :
Files :
* /hbase/trunk/src/main/ruby/hbase/admin.rb
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
[ https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262276#comment-13262276 ] Hudson commented on HBASE-5849: --- Integrated in HBase-TRUNK-security #184 (See [https://builds.apache.org/job/HBase-TRUNK-security/184/]) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330116) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java On first cluster startup, RS aborts if root znode is not available -- Key: HBASE-5849 URL: https://issues.apache.org/jira/browse/HBASE-5849 Project: HBase Issue Type: Bug Components: master, regionserver, zookeeper Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.2, 0.94.0 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch, HBASE-5849_v4-0.92.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch When launching a fresh new cluster, the master has to be started first, which might create race conditions for starting master and rs at the same time. Master startup code is smt like this: - establish zk connection - create root znodes in zk (/hbase) - create ephemeral node for master /hbase/master, Region server start up code is smt like this: - establish zk connection - check whether the root znode (/hbase) is there. If not, shutdown. - wait for the master to create znodes /hbase/master So, the problem is on the very first launch of the cluster, RS aborts to start since /hbase znode might not have been created yet (only the master creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on subsequent cluster starts, it does not matter which order the servers are started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262394#comment-13262394 ] Hudson commented on HBASE-5652: --- Integrated in HBase-TRUNK-security #185 (See [https://builds.apache.org/job/HBase-TRUNK-security/185/]) HBASE-5652 [findbugs] Fix lock release on all paths (Gregory Channan) (Revision 1330628) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/dev-support/findbugs-exclude.xml * /hbase/trunk/dev-support/test-patch.properties * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Fix For: 0.96.0 Attachments: HBASE-5652-v0.patch, HBASE-5652-v1.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5672) TestLruBlockCache#testBackgroundEvictionThread fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262789#comment-13262789
]
Hudson commented on HBASE-5672:
---
Integrated in HBase-TRUNK #2817 (See
[https://builds.apache.org/job/HBase-TRUNK/2817/])
HBASE-5672 TestLruBlockCache#testBackgroundEvictionThread fails
occasionally (Revision 1330971)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
TestLruBlockCache#testBackgroundEvictionThread fails occasionally
-
Key: HBASE-5672
URL: https://issues.apache.org/jira/browse/HBASE-5672
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0
Attachments: HBASE-5672.patch, HBASE-5672v2.patch, HBASE-5672v3.patch
We find TestLruBlockCache#testBackgroundEvictionThread fails occasionally.
I think it's a problem of the test case.
Because runEviction() only do evictionThread.evict():
{code}
public void evict() {
synchronized(this) {
this.notify(); // FindBugs NN_NAKED_NOTIFY
}
}
{code}
However when we call evictionThread.evict(), the evictionThread may haven't
been in run() in the TestLruBlockCache#testBackgroundEvictionThread.
If we run the test many times, we could find failture easily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263100#comment-13263100
]
Hudson commented on HBASE-5864:
---
Integrated in HBase-0.94 #151 (See
[https://builds.apache.org/job/HBase-0.94/151/])
HBASE-5864 Error while reading from hfile in 0.94 (Ram) (Revision 1331057)
Result = ABORTED
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5862) After Region Close remove the Operation Metrics.
[ https://issues.apache.org/jira/browse/HBASE-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263099#comment-13263099 ] Hudson commented on HBASE-5862: --- Integrated in HBase-0.94 #151 (See [https://builds.apache.org/job/HBase-0.94/151/]) HBASE-5862 After Region Close remove the Operation Metrics; ADDENDUM -- missing import (Revision 1331040) Result = ABORTED stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java After Region Close remove the Operation Metrics. Key: HBASE-5862 URL: https://issues.apache.org/jira/browse/HBASE-5862 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5862-0.patch, HBASE-5862-1.patch, HBASE-5862-2.patch, HBASE-5862-3.patch, HBASE-5862-4.patch, HBASE-5862-94-3.patch, TSD.png If a region is closed then Hadoop metrics shouldn't still be reporting about that region. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263136#comment-13263136
]
Hudson commented on HBASE-5864:
---
Integrated in HBase-0.94-security #22 (See
[https://builds.apache.org/job/HBase-0.94-security/22/])
HBASE-5864 Error while reading from hfile in 0.94 (Ram) (Revision 1331057)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263135#comment-13263135
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-0.94-security #22 (See
[https://builds.apache.org/job/HBase-0.94-security/22/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330549)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5862) After Region Close remove the Operation Metrics.
[ https://issues.apache.org/jira/browse/HBASE-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263134#comment-13263134 ] Hudson commented on HBASE-5862: --- Integrated in HBase-0.94-security #22 (See [https://builds.apache.org/job/HBase-0.94-security/22/]) HBASE-5862 After Region Close remove the Operation Metrics; ADDENDUM -- missing import (Revision 1331040) HBASE-5862 After Region Close remove the Operation Metrics (Revision 1330998) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionMetricsStorage.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java After Region Close remove the Operation Metrics. Key: HBASE-5862 URL: https://issues.apache.org/jira/browse/HBASE-5862 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5862-0.patch, HBASE-5862-1.patch, HBASE-5862-2.patch, HBASE-5862-3.patch, HBASE-5862-4.patch, HBASE-5862-94-3.patch, TSD.png If a region is closed then Hadoop metrics shouldn't still be reporting about that region. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263185#comment-13263185
]
Hudson commented on HBASE-5864:
---
Integrated in HBase-TRUNK #2819 (See
[https://builds.apache.org/job/HBase-TRUNK/2819/])
HBASE-5864 Error while reading from hfile in 0.94 (Ram) (Revision 1331058)
Result = SUCCESS
larsh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5872) Improve hadoopqa script to include checks for hadoop 0.23 build
[ https://issues.apache.org/jira/browse/HBASE-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263266#comment-13263266 ] Hudson commented on HBASE-5872: --- Integrated in HBase-TRUNK #2820 (See [https://builds.apache.org/job/HBase-TRUNK/2820/]) HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/dev-support/test-patch.sh Improve hadoopqa script to include checks for hadoop 0.23 build --- Key: HBASE-5872 URL: https://issues.apache.org/jira/browse/HBASE-5872 Project: HBase Issue Type: New Feature Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.96.0 Attachments: hbase-5872.patch There have been a few patches that have made it into hbase trunk that have broken the compile of hbase against hadoop 0.23.x, without being known for a few days. We could have the bot do a few things: 1) verify that patch compiles against hadoop 23 2) verify that unit tests pass against hadoop 23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5862) After Region Close remove the Operation Metrics.
[ https://issues.apache.org/jira/browse/HBASE-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263370#comment-13263370 ] Hudson commented on HBASE-5862: --- Integrated in HBase-TRUNK-security #186 (See [https://builds.apache.org/job/HBase-TRUNK-security/186/]) HBASE-5862 After Region Close remove the Operation Metrics (Revision 1330997) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/OperationMetrics.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionMetricsStorage.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerDynamicMetrics.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java After Region Close remove the Operation Metrics. Key: HBASE-5862 URL: https://issues.apache.org/jira/browse/HBASE-5862 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5862-0.patch, HBASE-5862-1.patch, HBASE-5862-2.patch, HBASE-5862-3.patch, HBASE-5862-4.patch, HBASE-5862-94-3.patch, TSD.png If a region is closed then Hadoop metrics shouldn't still be reporting about that region. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263369#comment-13263369 ] Hudson commented on HBASE-5844: --- Integrated in HBase-TRUNK-security #186 (See [https://builds.apache.org/job/HBase-TRUNK-security/186/]) HBASE-5844 Delete the region servers znode after a regions server crash; REVERT (Revision 1330983) Result = SUCCESS stack : Files : * /hbase/trunk/bin/hbase-daemon.sh * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Fix For: 0.96.0 Attachments: 5844.v1.patch, 5844.v2.patch, 5844.v3.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5872) Improve hadoopqa script to include checks for hadoop 0.23 build
[ https://issues.apache.org/jira/browse/HBASE-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263371#comment-13263371 ] Hudson commented on HBASE-5872: --- Integrated in HBase-TRUNK-security #186 (See [https://builds.apache.org/job/HBase-TRUNK-security/186/]) HBASE-5872 Improve hadoopqa script to include checks for hadoop 0.23 build (Revision 1331143) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/dev-support/test-patch.sh Improve hadoopqa script to include checks for hadoop 0.23 build --- Key: HBASE-5872 URL: https://issues.apache.org/jira/browse/HBASE-5872 Project: HBase Issue Type: New Feature Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.96.0 Attachments: hbase-5872.patch There have been a few patches that have made it into hbase trunk that have broken the compile of hbase against hadoop 0.23.x, without being known for a few days. We could have the bot do a few things: 1) verify that patch compiles against hadoop 23 2) verify that unit tests pass against hadoop 23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263373#comment-13263373
]
Hudson commented on HBASE-5864:
---
Integrated in HBase-TRUNK-security #186 (See
[https://builds.apache.org/job/HBase-TRUNK-security/186/])
HBASE-5864 Error while reading from hfile in 0.94 (Ram) (Revision 1331058)
Result = SUCCESS
larsh :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5829) Inconsistency between the regions map and the servers map in AssignmentManager
[
https://issues.apache.org/jira/browse/HBASE-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263372#comment-13263372
]
Hudson commented on HBASE-5829:
---
Integrated in HBase-TRUNK-security #186 (See
[https://builds.apache.org/job/HBase-TRUNK-security/186/])
HBASE-5829 Inconsistency between the regions map and the servers map in
AssignmentManager (Revision 1330993)
Result = SUCCESS
stack :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
Inconsistency between the regions map and the servers map in
AssignmentManager
--
Key: HBASE-5829
URL: https://issues.apache.org/jira/browse/HBASE-5829
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.6, 0.92.1
Reporter: Maryann Xue
Assignee: Maryann Xue
Fix For: 0.96.0
Attachments: HBASE-5829-0.90.patch, HBASE-5829-trunk.patch
There are occurrences in AM where this.servers is not kept consistent with
this.regions. This might cause balancer to offline a region from the RS that
already returned NotServingRegionException at a previous offline attempt.
In AssignmentManager.unassign(HRegionInfo, boolean)
try {
// TODO: We should consider making this look more like it does for the
// region open where we catch all throwables and never abort
if (serverManager.sendRegionClose(server, state.getRegion(),
versionOfClosingNode)) {
LOG.debug(Sent CLOSE to + server + for region +
region.getRegionNameAsString());
return;
}
// This never happens. Currently regionserver close always return true.
LOG.warn(Server + server + region CLOSE RPC returned false for +
region.getRegionNameAsString());
} catch (NotServingRegionException nsre) {
LOG.info(Server + server + returned + nsre + for +
region.getRegionNameAsString());
// Presume that master has stale data. Presume remote side just split.
// Presume that the split message when it comes in will fix up the
master's
// in memory cluster state.
} catch (Throwable t) {
if (t instanceof RemoteException) {
t = ((RemoteException)t).unwrapRemoteException();
if (t instanceof NotServingRegionException) {
if (checkIfRegionBelongsToDisabling(region)) {
// Remove from the regionsinTransition map
LOG.info(While trying to recover the table
+ region.getTableNameAsString()
+ to DISABLED state the region + region
+ was offlined but the table was in DISABLING state);
synchronized (this.regionsInTransition) {
this.regionsInTransition.remove(region.getEncodedName());
}
// Remove from the regionsMap
synchronized (this.regions) {
this.regions.remove(region);
}
deleteClosingOrClosedNode(region);
}
}
// RS is already processing this region, only need to update the
timestamp
if (t instanceof RegionAlreadyInTransitionException) {
LOG.debug(update + state + the timestamp.);
state.update(state.getState());
}
}
In AssignmentManager.assign(HRegionInfo, RegionState, boolean, boolean,
boolean)
synchronized (this.regions) {
this.regions.put(plan.getRegionInfo(), plan.getDestination());
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5672) TestLruBlockCache#testBackgroundEvictionThread fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263374#comment-13263374
]
Hudson commented on HBASE-5672:
---
Integrated in HBase-TRUNK-security #186 (See
[https://builds.apache.org/job/HBase-TRUNK-security/186/])
HBASE-5672 TestLruBlockCache#testBackgroundEvictionThread fails
occasionally (Revision 1330971)
Result = SUCCESS
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
TestLruBlockCache#testBackgroundEvictionThread fails occasionally
-
Key: HBASE-5672
URL: https://issues.apache.org/jira/browse/HBASE-5672
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0
Attachments: HBASE-5672.patch, HBASE-5672v2.patch, HBASE-5672v3.patch
We find TestLruBlockCache#testBackgroundEvictionThread fails occasionally.
I think it's a problem of the test case.
Because runEviction() only do evictionThread.evict():
{code}
public void evict() {
synchronized(this) {
this.notify(); // FindBugs NN_NAKED_NOTIFY
}
}
{code}
However when we call evictionThread.evict(), the evictionThread may haven't
been in run() in the TestLruBlockCache#testBackgroundEvictionThread.
If we run the test many times, we could find failture easily.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5872) Improve hadoopqa script to include checks for hadoop 0.23 build
[ https://issues.apache.org/jira/browse/HBASE-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263483#comment-13263483 ] Hudson commented on HBASE-5872: --- Integrated in HBase-TRUNK #2821 (See [https://builds.apache.org/job/HBase-TRUNK/2821/]) HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248) Result = FAILURE jmhsieh : Files : * /hbase/trunk/dev-support/test-patch.sh Improve hadoopqa script to include checks for hadoop 0.23 build --- Key: HBASE-5872 URL: https://issues.apache.org/jira/browse/HBASE-5872 Project: HBase Issue Type: New Feature Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.96.0 Attachments: hbase-5872-part2.patch, hbase-5872.patch There have been a few patches that have made it into hbase trunk that have broken the compile of hbase against hadoop 0.23.x, without being known for a few days. We could have the bot do a few things: 1) verify that patch compiles against hadoop 23 2) verify that unit tests pass against hadoop 23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5885) Invalid HFile block magic on Local file System
[ https://issues.apache.org/jira/browse/HBASE-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264226#comment-13264226 ] Hudson commented on HBASE-5885: --- Integrated in HBase-0.94 #153 (See [https://builds.apache.org/job/HBase-0.94/153/]) HBASE-5885 Invalid HFile block magic on Local file System (Revision 1331676) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java Invalid HFile block magic on Local file System -- Key: HBASE-5885 URL: https://issues.apache.org/jira/browse/HBASE-5885 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 0.94.0, 0.96.0 Attachments: 5885-trunk-v2.txt, HBASE-5885-94-0.patch, HBASE-5885-94-1.patch, HBASE-5885-trunk-0.patch, HBASE-5885-trunk-1.patch ERROR: java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Thu Apr 26 11:19:18 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader reader=file:/tmp/hbase-eclark/hbase/TestTable/e2d1c846363c75262cbfd85ea278b342/info/bae2681d63734066957b58fe791a0268, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false], firstKey=01/info:data/1335463981520/Put, lastKey=0002588100/info:data/1335463902296/Put, avgKeyLen=30, avgValueLen=1000, entries=1215085, length=1264354417, cur=000248/info:data/1335463994457/Put/vlen=1000/ts=0] at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:135) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:95) at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:368) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:127) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3323) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3279) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3296) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2393) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) Caused by: java.io.IOException: Invalid HFile block magic: \xEC\xD5\x9D\xB4\xC2bfo at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:153) at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:164) at org.apache.hadoop.hbase.io.hfile.HFileBlock.init(HFileBlock.java:254) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1779) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1637) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:327) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:555) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:651) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:130) ... 12 more Thu Apr 26 11:19:19 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1132) at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1121) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2420) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264227#comment-13264227
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-0.94 #153 (See
[https://builds.apache.org/job/HBase-0.94/153/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331639)
Result = FAILURE
stack :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
Allow spaces in coprocessor conf (aka trim() className)
---
Key: HBASE-5893
URL: https://issues.apache.org/jira/browse/HBASE-5893
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: conf, coprocessors
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-5893-v0.patch
This is annoying especially for coprocessors where you've long class name.
but maybe is a bug of Configuration.getStrings() that doesn't trim each
string.
When you've comma separated values like in the coprocessors case, you've to
pack together your values without spaces (v1,v2,v3,...) otherwise the
coprocessor is not loaded because the class name with spaces is not found.
{code}
property
namehbase.coprocessor.master.classes/name
value
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController
/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264225#comment-13264225
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94 #153 (See
[https://builds.apache.org/job/HBase-0.94/153/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331683)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5611-92.patch, HBASE-5611-94-minorchange.patch,
HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264228#comment-13264228
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-TRUNK #2823 (See
[https://builds.apache.org/job/HBase-TRUNK/2823/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331681)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5611-92.patch, HBASE-5611-94-minorchange.patch,
HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264230#comment-13264230
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-TRUNK #2823 (See
[https://builds.apache.org/job/HBase-TRUNK/2823/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331638)
Result = FAILURE
stack :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
Allow spaces in coprocessor conf (aka trim() className)
---
Key: HBASE-5893
URL: https://issues.apache.org/jira/browse/HBASE-5893
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: conf, coprocessors
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-5893-v0.patch
This is annoying especially for coprocessors where you've long class name.
but maybe is a bug of Configuration.getStrings() that doesn't trim each
string.
When you've comma separated values like in the coprocessors case, you've to
pack together your values without spaces (v1,v2,v3,...) otherwise the
coprocessor is not loaded because the class name with spaces is not found.
{code}
property
namehbase.coprocessor.master.classes/name
value
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController
/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5885) Invalid HFile block magic on Local file System
[ https://issues.apache.org/jira/browse/HBASE-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264229#comment-13264229 ] Hudson commented on HBASE-5885: --- Integrated in HBase-TRUNK #2823 (See [https://builds.apache.org/job/HBase-TRUNK/2823/]) HBASE-5885 Invalid HFile block magic on Local file System (Revision 1331675) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java Invalid HFile block magic on Local file System -- Key: HBASE-5885 URL: https://issues.apache.org/jira/browse/HBASE-5885 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 0.94.0, 0.96.0 Attachments: 5885-trunk-v2.txt, HBASE-5885-94-0.patch, HBASE-5885-94-1.patch, HBASE-5885-trunk-0.patch, HBASE-5885-trunk-1.patch ERROR: java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Thu Apr 26 11:19:18 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader reader=file:/tmp/hbase-eclark/hbase/TestTable/e2d1c846363c75262cbfd85ea278b342/info/bae2681d63734066957b58fe791a0268, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false], firstKey=01/info:data/1335463981520/Put, lastKey=0002588100/info:data/1335463902296/Put, avgKeyLen=30, avgValueLen=1000, entries=1215085, length=1264354417, cur=000248/info:data/1335463994457/Put/vlen=1000/ts=0] at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:135) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:95) at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:368) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:127) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3323) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3279) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3296) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2393) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) Caused by: java.io.IOException: Invalid HFile block magic: \xEC\xD5\x9D\xB4\xC2bfo at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:153) at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:164) at org.apache.hadoop.hbase.io.hfile.HFileBlock.init(HFileBlock.java:254) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1779) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1637) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:327) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:555) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:651) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:130) ... 12 more Thu Apr 26 11:19:19 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1132) at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1121) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2420) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264238#comment-13264238
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-TRUNK-security #187 (See
[https://builds.apache.org/job/HBase-TRUNK-security/187/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331681)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5611-92.patch, HBASE-5611-94-minorchange.patch,
HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5872) Improve hadoopqa script to include checks for hadoop 0.23 build
[ https://issues.apache.org/jira/browse/HBASE-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264239#comment-13264239 ] Hudson commented on HBASE-5872: --- Integrated in HBase-TRUNK-security #187 (See [https://builds.apache.org/job/HBase-TRUNK-security/187/]) HBASE-5872 ADDENDUM - improve hadoop 23 compile sucess/fail message (Revision 1331248) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/dev-support/test-patch.sh Improve hadoopqa script to include checks for hadoop 0.23 build --- Key: HBASE-5872 URL: https://issues.apache.org/jira/browse/HBASE-5872 Project: HBase Issue Type: New Feature Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.96.0 Attachments: hbase-5872-part2.patch, hbase-5872.patch There have been a few patches that have made it into hbase trunk that have broken the compile of hbase against hadoop 0.23.x, without being known for a few days. We could have the bot do a few things: 1) verify that patch compiles against hadoop 23 2) verify that unit tests pass against hadoop 23 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264241#comment-13264241
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-TRUNK-security #187 (See
[https://builds.apache.org/job/HBase-TRUNK-security/187/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331638)
Result = SUCCESS
stack :
Files :
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
Allow spaces in coprocessor conf (aka trim() className)
---
Key: HBASE-5893
URL: https://issues.apache.org/jira/browse/HBASE-5893
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: conf, coprocessors
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-5893-v0.patch
This is annoying especially for coprocessors where you've long class name.
but maybe is a bug of Configuration.getStrings() that doesn't trim each
string.
When you've comma separated values like in the coprocessors case, you've to
pack together your values without spaces (v1,v2,v3,...) otherwise the
coprocessor is not loaded because the class name with spaces is not found.
{code}
property
namehbase.coprocessor.master.classes/name
value
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController
/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5885) Invalid HFile block magic on Local file System
[ https://issues.apache.org/jira/browse/HBASE-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264240#comment-13264240 ] Hudson commented on HBASE-5885: --- Integrated in HBase-TRUNK-security #187 (See [https://builds.apache.org/job/HBase-TRUNK-security/187/]) HBASE-5885 Invalid HFile block magic on Local file System (Revision 1331675) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java Invalid HFile block magic on Local file System -- Key: HBASE-5885 URL: https://issues.apache.org/jira/browse/HBASE-5885 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 0.94.0, 0.96.0 Attachments: 5885-trunk-v2.txt, HBASE-5885-94-0.patch, HBASE-5885-94-1.patch, HBASE-5885-trunk-0.patch, HBASE-5885-trunk-1.patch ERROR: java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Thu Apr 26 11:19:18 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader reader=file:/tmp/hbase-eclark/hbase/TestTable/e2d1c846363c75262cbfd85ea278b342/info/bae2681d63734066957b58fe791a0268, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false], firstKey=01/info:data/1335463981520/Put, lastKey=0002588100/info:data/1335463902296/Put, avgKeyLen=30, avgValueLen=1000, entries=1215085, length=1264354417, cur=000248/info:data/1335463994457/Put/vlen=1000/ts=0] at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:135) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:95) at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:368) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:127) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3323) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3279) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3296) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2393) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) Caused by: java.io.IOException: Invalid HFile block magic: \xEC\xD5\x9D\xB4\xC2bfo at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:153) at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:164) at org.apache.hadoop.hbase.io.hfile.HFileBlock.init(HFileBlock.java:254) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1779) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1637) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:327) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:555) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:651) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:130) ... 12 more Thu Apr 26 11:19:19 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1132) at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1121) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2420) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264242#comment-13264242
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.92 #392 (See
[https://builds.apache.org/job/HBase-0.92/392/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331684)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5611-92.patch, HBASE-5611-94-minorchange.patch,
HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264243#comment-13264243
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-0.92 #392 (See
[https://builds.apache.org/job/HBase-0.92/392/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331641)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
Allow spaces in coprocessor conf (aka trim() className)
---
Key: HBASE-5893
URL: https://issues.apache.org/jira/browse/HBASE-5893
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: conf, coprocessors
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-5893-v0.patch
This is annoying especially for coprocessors where you've long class name.
but maybe is a bug of Configuration.getStrings() that doesn't trim each
string.
When you've comma separated values like in the coprocessors case, you've to
pack together your values without spaces (v1,v2,v3,...) otherwise the
coprocessor is not loaded because the class name with spaces is not found.
{code}
property
namehbase.coprocessor.master.classes/name
value
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController
/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264340#comment-13264340
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94 #155 (See
[https://builds.apache.org/job/HBase-0.94/155/])
HBASE-5611 Addendum fixes test failure in TestHeapSize#testSizes (Revision
1331769)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5611-94.addendum, HBASE-5611-92.patch,
HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264414#comment-13264414
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94 #158 (See
[https://builds.apache.org/job/HBase-0.94/158/])
HBASE-5611 Revert due to consistent TestChangingEncoding failure in 0.94
branch (Revision 1331826)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5611-94.addendum, HBASE-5611-92.patch,
HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264422#comment-13264422
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-0.94-security #24 (See
[https://builds.apache.org/job/HBase-0.94-security/24/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331639)
Result = FAILURE
stack :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
Allow spaces in coprocessor conf (aka trim() className)
---
Key: HBASE-5893
URL: https://issues.apache.org/jira/browse/HBASE-5893
Project: HBase
Issue Type: Bug
Components: coprocessors
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: conf, coprocessors
Fix For: 0.92.2, 0.94.0
Attachments: HBASE-5893-v0.patch
This is annoying especially for coprocessors where you've long class name.
but maybe is a bug of Configuration.getStrings() that doesn't trim each
string.
When you've comma separated values like in the coprocessors case, you've to
pack together your values without spaces (v1,v2,v3,...) otherwise the
coprocessor is not loaded because the class name with spaces is not found.
{code}
property
namehbase.coprocessor.master.classes/name
value
org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController
/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5885) Invalid HFile block magic on Local file System
[ https://issues.apache.org/jira/browse/HBASE-5885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264421#comment-13264421 ] Hudson commented on HBASE-5885: --- Integrated in HBase-0.94-security #24 (See [https://builds.apache.org/job/HBase-0.94-security/24/]) HBASE-5885 Invalid HFile block magic on Local file System (Revision 1331676) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestChecksum.java Invalid HFile block magic on Local file System -- Key: HBASE-5885 URL: https://issues.apache.org/jira/browse/HBASE-5885 Project: HBase Issue Type: Bug Affects Versions: 0.94.0, 0.96.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Blocker Fix For: 0.94.0, 0.96.0 Attachments: 5885-trunk-v2.txt, HBASE-5885-94-0.patch, HBASE-5885-94-1.patch, HBASE-5885-trunk-0.patch, HBASE-5885-trunk-1.patch ERROR: java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions: Thu Apr 26 11:19:18 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader reader=file:/tmp/hbase-eclark/hbase/TestTable/e2d1c846363c75262cbfd85ea278b342/info/bae2681d63734066957b58fe791a0268, compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true] [cacheDataOnWrite=false] [cacheIndexesOnWrite=false] [cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false], firstKey=01/info:data/1335463981520/Put, lastKey=0002588100/info:data/1335463902296/Put, avgKeyLen=30, avgValueLen=1000, entries=1215085, length=1264354417, cur=000248/info:data/1335463994457/Put/vlen=1000/ts=0] at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:135) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:95) at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:368) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:127) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3323) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3279) at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3296) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2393) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) Caused by: java.io.IOException: Invalid HFile block magic: \xEC\xD5\x9D\xB4\xC2bfo at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:153) at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:164) at org.apache.hadoop.hbase.io.hfile.HFileBlock.init(HFileBlock.java:254) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1779) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1637) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:327) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:555) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:651) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:130) ... 12 more Thu Apr 26 11:19:19 PDT 2012, org.apache.hadoop.hbase.client.ScannerCallable@190a621a, java.io.IOException: java.io.IOException: java.lang.IllegalArgumentException at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1132) at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:1121) at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2420) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264420#comment-13264420
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94-security #24 (See
[https://builds.apache.org/job/HBase-0.94-security/24/])
HBASE-5611 Revert due to consistent TestChangingEncoding failure in 0.94
branch (Revision 1331826)
HBASE-5611 Addendum fixes test failure in TestHeapSize#testSizes (Revision
1331769)
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331683)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5611-94.addendum, HBASE-5611-92.patch,
HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264714#comment-13264714 ] Hudson commented on HBASE-5712: --- Integrated in HBase-TRUNK #2825 (See [https://builds.apache.org/job/HBase-TRUNK/2825/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332072) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch, hbase-5712.patch On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264717#comment-13264717 ] Hudson commented on HBASE-5712: --- Integrated in HBase-0.94 #160 (See [https://builds.apache.org/job/HBase-0.94/160/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332071) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch, hbase-5712.patch On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264746#comment-13264746 ] Hudson commented on HBASE-5712: --- Integrated in HBase-0.92 #393 (See [https://builds.apache.org/job/HBase-0.92/393/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332070) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch, hbase-5712.patch On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5903) Detect the test classes without categories
[ https://issues.apache.org/jira/browse/HBASE-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265023#comment-13265023 ] Hudson commented on HBASE-5903: --- Integrated in HBase-TRUNK #2826 (See [https://builds.apache.org/job/HBase-TRUNK/2826/]) HBASE-5903 Detect the test classes without categories (Revision 1332260) Result = SUCCESS stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestCheckTestClasses.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java Detect the test classes without categories -- Key: HBASE-5903 URL: https://issues.apache.org/jira/browse/HBASE-5903 Project: HBase Issue Type: Improvement Components: build, test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5903.v3.patch, 5903v4.txt The tests are executed by category. When a test does not have a category, it's not run on prebuild nor central build. This new test checks the test classess and list the ones without category. It fails if it finds one. As it's a small test it will be executed on the developper machine and will fail immediately on the central builds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5906) TestChangingEncoding failing sporadically in 0.94 build
[ https://issues.apache.org/jira/browse/HBASE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265135#comment-13265135 ] Hudson commented on HBASE-5906: --- Integrated in HBase-TRUNK #2827 (See [https://builds.apache.org/job/HBase-TRUNK/2827/]) HBASE-5906 TestChangingEncoding failing sporadically in 0.94 build (Revision 1332320) Result = SUCCESS stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java TestChangingEncoding failing sporadically in 0.94 build --- Key: HBASE-5906 URL: https://issues.apache.org/jira/browse/HBASE-5906 Project: HBase Issue Type: Bug Reporter: stack Attachments: 5906.txt The test passes locally for me and Elliott but takes a long time to run. Timeout is only two minutes for the test though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5906) TestChangingEncoding failing sporadically in 0.94 build
[ https://issues.apache.org/jira/browse/HBASE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265384#comment-13265384 ] Hudson commented on HBASE-5906: --- Integrated in HBase-0.94 #161 (See [https://builds.apache.org/job/HBase-0.94/161/]) HBASE-5906 TestChangingEncoding failing sporadically in 0.94 build (Revision 1332319) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java TestChangingEncoding failing sporadically in 0.94 build --- Key: HBASE-5906 URL: https://issues.apache.org/jira/browse/HBASE-5906 Project: HBase Issue Type: Bug Reporter: stack Attachments: 5906.txt The test passes locally for me and Elliott but takes a long time to run. Timeout is only two minutes for the test though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265383#comment-13265383
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94 #161 (See
[https://builds.apache.org/job/HBase-0.94/161/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size - v2 (Jieshan) (Revision 1332344)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5611-94-v2.txt, 5611-94.addendum, HBASE-5611-92.patch,
HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5884) MapReduce package info has broken link to bulk-loads
[ https://issues.apache.org/jira/browse/HBASE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265476#comment-13265476 ] Hudson commented on HBASE-5884: --- Integrated in HBase-TRUNK #2828 (See [https://builds.apache.org/job/HBase-TRUNK/2828/]) HBASE-5884 MapReduce package info has broken link to bulk-loads (Revision 1332440) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/package-info.java MapReduce package info has broken link to bulk-loads Key: HBASE-5884 URL: https://issues.apache.org/jira/browse/HBASE-5884 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Trivial Fix For: 0.94.0, 0.96.0 Attachments: doc_HBASE-5884.patch Bulk Loads link goes to an old link, which we have dropped recently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265475#comment-13265475 ] Hudson commented on HBASE-5548: --- Integrated in HBase-TRUNK #2828 (See [https://builds.apache.org/job/HBase-TRUNK/2828/]) HBASE-5548 Add ability to get a table in the shell (Revision 1332419) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/ruby/hbase/hbase.rb * /hbase/trunk/src/main/ruby/hbase/table.rb * /hbase/trunk/src/main/ruby/shell.rb * /hbase/trunk/src/main/ruby/shell/commands.rb * /hbase/trunk/src/main/ruby/shell/commands/count.rb * /hbase/trunk/src/main/ruby/shell/commands/create.rb * /hbase/trunk/src/main/ruby/shell/commands/delete.rb * /hbase/trunk/src/main/ruby/shell/commands/deleteall.rb * /hbase/trunk/src/main/ruby/shell/commands/get.rb * /hbase/trunk/src/main/ruby/shell/commands/get_counter.rb * /hbase/trunk/src/main/ruby/shell/commands/get_table.rb * /hbase/trunk/src/main/ruby/shell/commands/incr.rb * /hbase/trunk/src/main/ruby/shell/commands/put.rb * /hbase/trunk/src/main/ruby/shell/commands/scan.rb * /hbase/trunk/src/main/ruby/shell/commands/table_help.rb * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/src/test/ruby/hbase/admin_test.rb Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch, ruby_HBASE-5548-v1.patch, ruby_HBASE-5548-v2.patch, ruby_HBASE-5548-v3.patch, ruby_HBASE-5548-v5.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5884) MapReduce package info has broken link to bulk-loads
[ https://issues.apache.org/jira/browse/HBASE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265493#comment-13265493 ] Hudson commented on HBASE-5884: --- Integrated in HBase-0.94 #163 (See [https://builds.apache.org/job/HBase-0.94/163/]) HBASE-5884 MapReduce package info has broken link to bulk-loads (Revision 1332441) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/package-info.java MapReduce package info has broken link to bulk-loads Key: HBASE-5884 URL: https://issues.apache.org/jira/browse/HBASE-5884 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Trivial Fix For: 0.94.0, 0.96.0 Attachments: doc_HBASE-5884.patch Bulk Loads link goes to an old link, which we have dropped recently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5879) Enable JMX metrics collection for the Thrift proxy
[ https://issues.apache.org/jira/browse/HBASE-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265515#comment-13265515 ] Hudson commented on HBASE-5879: --- Integrated in HBase-TRUNK #2829 (See [https://builds.apache.org/job/HBase-TRUNK/2829/]) HBASE-5879 Enable JMX metrics collection for the Thrift proxy (Revision 1332450) Result = FAILURE stack : Files : * /hbase/trunk/bin/hbase-config.sh Enable JMX metrics collection for the Thrift proxy -- Key: HBASE-5879 URL: https://issues.apache.org/jira/browse/HBASE-5879 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Fix For: 0.96.0 Attachments: 5879_trunk.txt, D2955.1.patch We need to enable JMX on the Thrift proxy on a separate port different from the JMX port used by regionserver. This is necessary for metrics collection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265612#comment-13265612 ] Hudson commented on HBASE-5908: --- Integrated in HBase-TRUNK #2830 (See [https://builds.apache.org/job/HBase-TRUNK/2830/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332495) Result = FAILURE todd : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog Key: HBASE-5908 URL: https://issues.apache.org/jira/browse/HBASE-5908 Project: HBase Issue Type: Bug Components: test, wal Affects Versions: 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5908-trunk.patch TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 The failure: java.io.IOException: Append is not supported. Please see the dfs.support.append configuration parameter. Instead of using append, we can probably just: - copy over the contents to a new file - append the garbage to the new file - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265618#comment-13265618 ] Hudson commented on HBASE-5908: --- Integrated in HBase-0.94 #164 (See [https://builds.apache.org/job/HBase-0.94/164/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332494) Result = FAILURE todd : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog Key: HBASE-5908 URL: https://issues.apache.org/jira/browse/HBASE-5908 Project: HBase Issue Type: Bug Components: test, wal Affects Versions: 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5908-trunk.patch TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 The failure: java.io.IOException: Append is not supported. Please see the dfs.support.append configuration parameter. Instead of using append, we can probably just: - copy over the contents to a new file - append the garbage to the new file - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265637#comment-13265637 ] Hudson commented on HBASE-5908: --- Integrated in HBase-0.92 #394 (See [https://builds.apache.org/job/HBase-0.92/394/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332493) Result = SUCCESS todd : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog Key: HBASE-5908 URL: https://issues.apache.org/jira/browse/HBASE-5908 Project: HBase Issue Type: Bug Components: test, wal Affects Versions: 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5908-trunk.patch TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 The failure: java.io.IOException: Append is not supported. Please see the dfs.support.append configuration parameter. Instead of using append, we can probably just: - copy over the contents to a new file - append the garbage to the new file - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265668#comment-13265668 ] Hudson commented on HBASE-5548: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5548 Add ability to get a table in the shell (Revision 1332419) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/ruby/hbase/hbase.rb * /hbase/trunk/src/main/ruby/hbase/table.rb * /hbase/trunk/src/main/ruby/shell.rb * /hbase/trunk/src/main/ruby/shell/commands.rb * /hbase/trunk/src/main/ruby/shell/commands/count.rb * /hbase/trunk/src/main/ruby/shell/commands/create.rb * /hbase/trunk/src/main/ruby/shell/commands/delete.rb * /hbase/trunk/src/main/ruby/shell/commands/deleteall.rb * /hbase/trunk/src/main/ruby/shell/commands/get.rb * /hbase/trunk/src/main/ruby/shell/commands/get_counter.rb * /hbase/trunk/src/main/ruby/shell/commands/get_table.rb * /hbase/trunk/src/main/ruby/shell/commands/incr.rb * /hbase/trunk/src/main/ruby/shell/commands/put.rb * /hbase/trunk/src/main/ruby/shell/commands/scan.rb * /hbase/trunk/src/main/ruby/shell/commands/table_help.rb * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * /hbase/trunk/src/test/ruby/hbase/admin_test.rb Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch, ruby_HBASE-5548-v1.patch, ruby_HBASE-5548-v2.patch, ruby_HBASE-5548-v3.patch, ruby_HBASE-5548-v5.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5903) Detect the test classes without categories
[ https://issues.apache.org/jira/browse/HBASE-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265670#comment-13265670 ] Hudson commented on HBASE-5903: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5903 Detect the test classes without categories (Revision 1332260) Result = FAILURE stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestCheckTestClasses.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java Detect the test classes without categories -- Key: HBASE-5903 URL: https://issues.apache.org/jira/browse/HBASE-5903 Project: HBase Issue Type: Improvement Components: build, test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.96.0 Attachments: 5903.v3.patch, 5903v4.txt The tests are executed by category. When a test does not have a category, it's not run on prebuild nor central build. This new test checks the test classess and list the ones without category. It fails if it finds one. As it's a small test it will be executed on the developper machine and will fail immediately on the central builds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265666#comment-13265666 ] Hudson commented on HBASE-5908: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332495) Result = FAILURE todd : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog Key: HBASE-5908 URL: https://issues.apache.org/jira/browse/HBASE-5908 Project: HBase Issue Type: Bug Components: test, wal Affects Versions: 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5908-trunk.patch TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 The failure: java.io.IOException: Append is not supported. Please see the dfs.support.append configuration parameter. Instead of using append, we can probably just: - copy over the contents to a new file - append the garbage to the new file - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5879) Enable JMX metrics collection for the Thrift proxy
[ https://issues.apache.org/jira/browse/HBASE-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265669#comment-13265669 ] Hudson commented on HBASE-5879: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5879 Enable JMX metrics collection for the Thrift proxy (Revision 1332450) Result = FAILURE stack : Files : * /hbase/trunk/bin/hbase-config.sh Enable JMX metrics collection for the Thrift proxy -- Key: HBASE-5879 URL: https://issues.apache.org/jira/browse/HBASE-5879 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Priority: Minor Fix For: 0.96.0 Attachments: 5879_trunk.txt, D2955.1.patch We need to enable JMX on the Thrift proxy on a separate port different from the JMX port used by regionserver. This is necessary for metrics collection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265667#comment-13265667 ] Hudson commented on HBASE-5712: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332072) Result = FAILURE jmhsieh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch, hbase-5712.patch On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5884) MapReduce package info has broken link to bulk-loads
[ https://issues.apache.org/jira/browse/HBASE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265672#comment-13265672 ] Hudson commented on HBASE-5884: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5884 MapReduce package info has broken link to bulk-loads (Revision 1332440) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/package-info.java MapReduce package info has broken link to bulk-loads Key: HBASE-5884 URL: https://issues.apache.org/jira/browse/HBASE-5884 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Trivial Fix For: 0.94.0, 0.96.0 Attachments: doc_HBASE-5884.patch Bulk Loads link goes to an old link, which we have dropped recently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5906) TestChangingEncoding failing sporadically in 0.94 build
[ https://issues.apache.org/jira/browse/HBASE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265671#comment-13265671 ] Hudson commented on HBASE-5906: --- Integrated in HBase-TRUNK-security #188 (See [https://builds.apache.org/job/HBase-TRUNK-security/188/]) HBASE-5906 TestChangingEncoding failing sporadically in 0.94 build (Revision 1332320) Result = FAILURE stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java TestChangingEncoding failing sporadically in 0.94 build --- Key: HBASE-5906 URL: https://issues.apache.org/jira/browse/HBASE-5906 Project: HBase Issue Type: Bug Reporter: stack Attachments: 5906.txt The test passes locally for me and Elliott but takes a long time to run. Timeout is only two minutes for the test though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265961#comment-13265961 ] Hudson commented on HBASE-5548: --- Integrated in HBase-TRUNK #2832 (See [https://builds.apache.org/job/HBase-TRUNK/2832/]) HBASE-5548 Add ability to get a table in the shell; ADDENDUM (Revision 1332766) Result = FAILURE stack : Files : * /hbase/trunk/src/main/ruby/hbase/table.rb * /hbase/trunk/src/main/ruby/shell/commands/count.rb * /hbase/trunk/src/main/ruby/shell/commands/deleteall.rb * /hbase/trunk/src/main/ruby/shell/commands/get_counter.rb * /hbase/trunk/src/test/ruby/hbase/admin_test.rb * /hbase/trunk/src/test/ruby/hbase/table_test.rb * /hbase/trunk/src/test/ruby/shell/commands_test.rb * /hbase/trunk/src/test/ruby/test_helper.rb Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch, ruby_HBASE-5548-addendum.patch, ruby_HBASE-5548-v1.patch, ruby_HBASE-5548-v2.patch, ruby_HBASE-5548-v3.patch, ruby_HBASE-5548-v5.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266127#comment-13266127 ] Hudson commented on HBASE-5908: --- Integrated in HBase-0.94-security #25 (See [https://builds.apache.org/job/HBase-0.94-security/25/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332494) Result = SUCCESS todd : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog Key: HBASE-5908 URL: https://issues.apache.org/jira/browse/HBASE-5908 Project: HBase Issue Type: Bug Components: test, wal Affects Versions: 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5908-trunk.patch TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 The failure: java.io.IOException: Append is not supported. Please see the dfs.support.append configuration parameter. Instead of using append, we can probably just: - copy over the contents to a new file - append the garbage to the new file - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266129#comment-13266129 ] Hudson commented on HBASE-5897: --- Integrated in HBase-0.94-security #25 (See [https://builds.apache.org/job/HBase-0.94-security/25/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332810) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prePut coprocessor hook causing substantial CPU usage - Key: HBASE-5897 URL: https://issues.apache.org/jira/browse/HBASE-5897 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, testRegionServerCoprocessorExceptionWithRemove.stack I was running an insert workload against trunk under oprofile and saw that a significant portion of CPU usage was going to calling the prePut coprocessor hook inside doMiniBatchPut, even though I don't have any coprocessors installed. I ran a million-row insert and collected CPU time spent in the RS after commenting out the preput hook, and found CPU usage reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5906) TestChangingEncoding failing sporadically in 0.94 build
[ https://issues.apache.org/jira/browse/HBASE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266130#comment-13266130 ] Hudson commented on HBASE-5906: --- Integrated in HBase-0.94-security #25 (See [https://builds.apache.org/job/HBase-0.94-security/25/]) HBASE-5906 TestChangingEncoding failing sporadically in 0.94 build (Revision 1332319) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java TestChangingEncoding failing sporadically in 0.94 build --- Key: HBASE-5906 URL: https://issues.apache.org/jira/browse/HBASE-5906 Project: HBase Issue Type: Bug Reporter: stack Attachments: 5906.txt The test passes locally for me and Elliott but takes a long time to run. Timeout is only two minutes for the test though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266126#comment-13266126
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.94-security #25 (See
[https://builds.apache.org/job/HBase-0.94-security/25/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size - v2 (Jieshan) (Revision 1332344)
Result = SUCCESS
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5611-94-v2.txt, 5611-94.addendum, HBASE-5611-92.patch,
HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266128#comment-13266128 ] Hudson commented on HBASE-5712: --- Integrated in HBase-0.94-security #25 (See [https://builds.apache.org/job/HBase-0.94-security/25/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332071) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, hbase-5712-v2.patch, hbase-5712.patch On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5884) MapReduce package info has broken link to bulk-loads
[ https://issues.apache.org/jira/browse/HBASE-5884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266131#comment-13266131 ] Hudson commented on HBASE-5884: --- Integrated in HBase-0.94-security #25 (See [https://builds.apache.org/job/HBase-0.94-security/25/]) HBASE-5884 MapReduce package info has broken link to bulk-loads (Revision 1332441) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/package-info.java MapReduce package info has broken link to bulk-loads Key: HBASE-5884 URL: https://issues.apache.org/jira/browse/HBASE-5884 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Priority: Trivial Fix For: 0.94.0, 0.96.0 Attachments: doc_HBASE-5884.patch Bulk Loads link goes to an old link, which we have dropped recently. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266147#comment-13266147 ] Hudson commented on HBASE-5897: --- Integrated in HBase-0.94 #165 (See [https://builds.apache.org/job/HBase-0.94/165/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332810) Result = FAILURE larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prePut coprocessor hook causing substantial CPU usage - Key: HBASE-5897 URL: https://issues.apache.org/jira/browse/HBASE-5897 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, testRegionServerCoprocessorExceptionWithRemove.stack I was running an insert workload against trunk under oprofile and saw that a significant portion of CPU usage was going to calling the prePut coprocessor hook inside doMiniBatchPut, even though I don't have any coprocessors installed. I ran a million-row insert and collected CPU time spent in the RS after commenting out the preput hook, and found CPU usage reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5888) Clover profile in build
[ https://issues.apache.org/jira/browse/HBASE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266165#comment-13266165 ] Hudson commented on HBASE-5888: --- Integrated in HBase-TRUNK #2834 (See [https://builds.apache.org/job/HBase-TRUNK/2834/]) HBASE-5888 Clover profile in build (Revision 1332829) Result = FAILURE stack : Files : * /hbase/trunk/pom.xml Clover profile in build --- Key: HBASE-5888 URL: https://issues.apache.org/jira/browse/HBASE-5888 Project: HBase Issue Type: Improvement Components: build, test Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: hbase-clover_v1.patch, hbase-clover_v2.patch Clover is disabled right now. I would like to add a profile that enables clover reports. We can also backport this to 0.92, and 0.94, since we are also interested in test coverage for those branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266164#comment-13266164 ] Hudson commented on HBASE-5897: --- Integrated in HBase-TRUNK #2834 (See [https://builds.apache.org/job/HBase-TRUNK/2834/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332811) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prePut coprocessor hook causing substantial CPU usage - Key: HBASE-5897 URL: https://issues.apache.org/jira/browse/HBASE-5897 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, testRegionServerCoprocessorExceptionWithRemove.stack I was running an insert workload against trunk under oprofile and saw that a significant portion of CPU usage was going to calling the prePut coprocessor hook inside doMiniBatchPut, even though I don't have any coprocessors installed. I ran a million-row insert and collected CPU time spent in the RS after commenting out the preput hook, and found CPU usage reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5785) Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion
[ https://issues.apache.org/jira/browse/HBASE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266163#comment-13266163 ] Hudson commented on HBASE-5785: --- Integrated in HBase-TRUNK #2834 (See [https://builds.apache.org/job/HBase-TRUNK/2834/]) HBASE-5785 Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion (Revision 1332824) Result = FAILURE stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/protobuf * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/protobuf/TestProtobufUtil.java Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion - Key: HBASE-5785 URL: https://issues.apache.org/jira/browse/HBASE-5785 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Critical Labels: noob Fix For: 0.96.0 Attachments: hbase-5785.patch We need to add some unit tests for the probuf utilities to catch issues earlier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266244#comment-13266244 ] Hudson commented on HBASE-5897: --- Integrated in HBase-0.92 #395 (See [https://builds.apache.org/job/HBase-0.92/395/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332816) Result = SUCCESS larsh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prePut coprocessor hook causing substantial CPU usage - Key: HBASE-5897 URL: https://issues.apache.org/jira/browse/HBASE-5897 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, testRegionServerCoprocessorExceptionWithRemove.stack I was running an insert workload against trunk under oprofile and saw that a significant portion of CPU usage was going to calling the prePut coprocessor hook inside doMiniBatchPut, even though I don't have any coprocessors installed. I ran a million-row insert and collected CPU time spent in the RS after commenting out the preput hook, and found CPU usage reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5901) Use union type protobufs instead of class/byte pairs for multi requests
[ https://issues.apache.org/jira/browse/HBASE-5901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266283#comment-13266283 ] Hudson commented on HBASE-5901: --- Integrated in HBase-TRUNK #2835 (See [https://builds.apache.org/job/HBase-TRUNK/2835/]) HBASE-5901. Use union type protobufs instead of class/byte pairs for multi requests. (Revision 1332882) Result = SUCCESS todd : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Client.proto Use union type protobufs instead of class/byte pairs for multi requests --- Key: HBASE-5901 URL: https://issues.apache.org/jira/browse/HBASE-5901 Project: HBase Issue Type: Improvement Components: ipc, performance Affects Versions: 0.96.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.96.0 Attachments: hbase-5901.txt, hbase-5901.txt, hbase-5901.txt The current implementation of multi actions uses repeated NameBytesPairs for the contents of multi actions. Instead, we should introduce a union type protobuf for the valid actions. This makes the RPCs smaller since they don't need to carry class names, and makes deserialization faster since it can avoid some copying and reflection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5785) Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion
[ https://issues.apache.org/jira/browse/HBASE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266371#comment-13266371 ] Hudson commented on HBASE-5785: --- Integrated in HBase-TRUNK-security #189 (See [https://builds.apache.org/job/HBase-TRUNK-security/189/]) HBASE-5785 Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion (Revision 1332824) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/protobuf * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/protobuf/TestProtobufUtil.java Adding unit tests for protbuf utils introduced for HRegionInterface pb conversion - Key: HBASE-5785 URL: https://issues.apache.org/jira/browse/HBASE-5785 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Affects Versions: 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Critical Labels: noob Fix For: 0.96.0 Attachments: hbase-5785.patch We need to add some unit tests for the probuf utilities to catch issues earlier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266372#comment-13266372 ] Hudson commented on HBASE-5548: --- Integrated in HBase-TRUNK-security #189 (See [https://builds.apache.org/job/HBase-TRUNK-security/189/]) HBASE-5548 Add ability to get a table in the shell; ADDENDUM (Revision 1332766) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/ruby/hbase/table.rb * /hbase/trunk/src/main/ruby/shell/commands/count.rb * /hbase/trunk/src/main/ruby/shell/commands/deleteall.rb * /hbase/trunk/src/main/ruby/shell/commands/get_counter.rb * /hbase/trunk/src/test/ruby/hbase/admin_test.rb * /hbase/trunk/src/test/ruby/hbase/table_test.rb * /hbase/trunk/src/test/ruby/shell/commands_test.rb * /hbase/trunk/src/test/ruby/test_helper.rb Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch, ruby_HBASE-5548-addendum.patch, ruby_HBASE-5548-v1.patch, ruby_HBASE-5548-v2.patch, ruby_HBASE-5548-v3.patch, ruby_HBASE-5548-v5.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266373#comment-13266373 ] Hudson commented on HBASE-5897: --- Integrated in HBase-TRUNK-security #189 (See [https://builds.apache.org/job/HBase-TRUNK-security/189/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332811) Result = SUCCESS larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java prePut coprocessor hook causing substantial CPU usage - Key: HBASE-5897 URL: https://issues.apache.org/jira/browse/HBASE-5897 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, testRegionServerCoprocessorExceptionWithRemove.stack I was running an insert workload against trunk under oprofile and saw that a significant portion of CPU usage was going to calling the prePut coprocessor hook inside doMiniBatchPut, even though I don't have any coprocessors installed. I ran a million-row insert and collected CPU time spent in the RS after commenting out the preput hook, and found CPU usage reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5901) Use union type protobufs instead of class/byte pairs for multi requests
[ https://issues.apache.org/jira/browse/HBASE-5901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266375#comment-13266375 ] Hudson commented on HBASE-5901: --- Integrated in HBase-TRUNK-security #189 (See [https://builds.apache.org/job/HBase-TRUNK-security/189/]) HBASE-5901. Use union type protobufs instead of class/byte pairs for multi requests. (Revision 1332882) Result = SUCCESS todd : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Client.proto Use union type protobufs instead of class/byte pairs for multi requests --- Key: HBASE-5901 URL: https://issues.apache.org/jira/browse/HBASE-5901 Project: HBase Issue Type: Improvement Components: ipc, performance Affects Versions: 0.96.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Critical Fix For: 0.96.0 Attachments: hbase-5901.txt, hbase-5901.txt, hbase-5901.txt The current implementation of multi actions uses repeated NameBytesPairs for the contents of multi actions. Instead, we should introduce a union type protobuf for the valid actions. This makes the RPCs smaller since they don't need to carry class names, and makes deserialization faster since it can avoid some copying and reflection. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5888) Clover profile in build
[ https://issues.apache.org/jira/browse/HBASE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266374#comment-13266374 ] Hudson commented on HBASE-5888: --- Integrated in HBase-TRUNK-security #189 (See [https://builds.apache.org/job/HBase-TRUNK-security/189/]) HBASE-5888 Clover profile in build (Revision 1332829) Result = SUCCESS stack : Files : * /hbase/trunk/pom.xml Clover profile in build --- Key: HBASE-5888 URL: https://issues.apache.org/jira/browse/HBASE-5888 Project: HBase Issue Type: Improvement Components: build, test Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.96.0 Attachments: hbase-clover_v1.patch, hbase-clover_v2.patch Clover is disabled right now. I would like to add a profile that enables clover reports. We can also backport this to 0.92, and 0.94, since we are also interested in test coverage for those branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5869) Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb
[ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266765#comment-13266765 ] Hudson commented on HBASE-5869: --- Integrated in HBase-TRUNK #2836 (See [https://builds.apache.org/job/HBase-TRUNK/2836/]) HBASE-5869 Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb (Revision 1333099) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/DeserializationException.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HBaseException.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/RegionTransition.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ServerName.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/SplitLogTask.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/trunk/src/main/protobuf/ZooKeeper.proto * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestSerialization.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestWALObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/Mocking.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb - Key: HBASE-5869 URL: https://issues.apache.org/jira/browse/HBASE-5869 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.96.0 Attachments: 5869v7.txt, 5869v8.txt, 5869v9.txt, firstcut.txt, secondcut.txt, v10.txt, v11.txt, v12.txt, v13.txt, v13.txt, v4.txt, v5.txt, v6.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators:
[jira] [Commented] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
[ https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266816#comment-13266816 ] Hudson commented on HBASE-5840: --- Integrated in HBase-TRUNK #2837 (See [https://builds.apache.org/job/HBase-TRUNK/2837/]) HBASE-5840 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status (RajeshBabu) (Revision 1333124) HBASE-5548 Add ability to get a table in the shell; BACKING OUT MISTAKEN CO-COMMIT OF HBASE-5840 (Revision 1333123) Result = SUCCESS ramkrishna : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status -- Key: HBASE-5840 URL: https://issues.apache.org/jira/browse/HBASE-5840 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.0 Reporter: Gopinathan A Assignee: rajeshbabu Fix For: 0.96.0, 0.94.1 Attachments: HBASE-5840.patch, HBASE-5840_trunk.patch, HBASE-5840_v2.patch TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will keeps showing old status. This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266818#comment-13266818 ] Hudson commented on HBASE-2214: --- Integrated in HBase-TRUNK #2837 (See [https://builds.apache.org/job/HBase-TRUNK/2837/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333122) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Client.proto * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Ferdy Galema Fix For: 0.96.0, 0.94.1 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5548) Add ability to get a table in the shell
[ https://issues.apache.org/jira/browse/HBASE-5548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266817#comment-13266817 ] Hudson commented on HBASE-5548: --- Integrated in HBase-TRUNK #2837 (See [https://builds.apache.org/job/HBase-TRUNK/2837/]) HBASE-5548 Add ability to get a table in the shell; BACKING OUT MISTAKEN CO-COMMIT OF HBASE-5840 (Revision 1333123) Result = SUCCESS stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java Add ability to get a table in the shell --- Key: HBASE-5548 URL: https://issues.apache.org/jira/browse/HBASE-5548 Project: HBase Issue Type: Improvement Components: shell Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: ruby_HBASE-5528-v0.patch, ruby_HBASE-5548-addendum.patch, ruby_HBASE-5548-v1.patch, ruby_HBASE-5548-v2.patch, ruby_HBASE-5548-v3.patch, ruby_HBASE-5548-v5.patch Currently, all the commands that operate on a table in the shell first have to take the table as name as input. There are two main considerations: * It is annoying to have to write the table name every time, when you should just be able to get a reference to a table * the current implementation is very wasteful - it creates a new HTable for each call (but reuses the connection since it uses the same configuration) We should be able to get a handle to a single HTable and then operate on that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1996) Configure scanner buffer in bytes instead of number of rows
[ https://issues.apache.org/jira/browse/HBASE-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266815#comment-13266815 ] Hudson commented on HBASE-1996: --- Integrated in HBase-TRUNK #2837 (See [https://builds.apache.org/job/HBase-TRUNK/2837/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333122) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Client.proto * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java Configure scanner buffer in bytes instead of number of rows --- Key: HBASE-1996 URL: https://issues.apache.org/jira/browse/HBASE-1996 Project: HBase Issue Type: Improvement Reporter: Dave Latham Assignee: Dave Latham Fix For: 0.90.0 Attachments: 1966.patch, 1996-0.20.3-v2.patch, 1996-0.20.3-v3.patch, 1996-0.20.3.patch Currently, the default scanner fetches a single row at a time. This makes for very slow scans on tables where the rows are not large. You can change the setting for an HTable instance or for each Scan. It would be better to have a default that performs reasonably well so that people stop running into slow scans because they are evaluating HBase, aren't familiar with the setting, or simply forgot. Unfortunately, if we increase the value of the current setting, then we run the risk of running OOM for tables with large rows. Let's change the setting so that it works with a size in bytes, rather than in rows. This will allow us to set a reasonable default so that tables with small rows will scan performantly and tables with large rows will not run OOM. Note that the case is very similar to table writes as well. When disabling auto flush, we buffer a list of Put's to commit at once. That buffer is measured in bytes, so that a small number of large Puts or a lot of small Puts can each fit in a single flush. If that buffer were measured in number of Put's it would have the same problem that we have for the scan buffer, and we wouldn't be able to set a good default value for tables with different size rows. Changing the scan buffer to be configured like the write buffer will make it more consistent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1996) Configure scanner buffer in bytes instead of number of rows
[ https://issues.apache.org/jira/browse/HBASE-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266870#comment-13266870 ] Hudson commented on HBASE-1996: --- Integrated in HBase-0.94 #170 (See [https://builds.apache.org/job/HBase-0.94/170/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333157) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java Configure scanner buffer in bytes instead of number of rows --- Key: HBASE-1996 URL: https://issues.apache.org/jira/browse/HBASE-1996 Project: HBase Issue Type: Improvement Reporter: Dave Latham Assignee: Dave Latham Fix For: 0.90.0 Attachments: 1966.patch, 1996-0.20.3-v2.patch, 1996-0.20.3-v3.patch, 1996-0.20.3.patch Currently, the default scanner fetches a single row at a time. This makes for very slow scans on tables where the rows are not large. You can change the setting for an HTable instance or for each Scan. It would be better to have a default that performs reasonably well so that people stop running into slow scans because they are evaluating HBase, aren't familiar with the setting, or simply forgot. Unfortunately, if we increase the value of the current setting, then we run the risk of running OOM for tables with large rows. Let's change the setting so that it works with a size in bytes, rather than in rows. This will allow us to set a reasonable default so that tables with small rows will scan performantly and tables with large rows will not run OOM. Note that the case is very similar to table writes as well. When disabling auto flush, we buffer a list of Put's to commit at once. That buffer is measured in bytes, so that a small number of large Puts or a lot of small Puts can each fit in a single flush. If that buffer were measured in number of Put's it would have the same problem that we have for the scan buffer, and we wouldn't be able to set a good default value for tables with different size rows. Changing the scan buffer to be configured like the write buffer will make it more consistent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266871#comment-13266871 ] Hudson commented on HBASE-2214: --- Integrated in HBase-0.94 #170 (See [https://builds.apache.org/job/HBase-0.94/170/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333157) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly - Key: HBASE-2214 URL: https://issues.apache.org/jira/browse/HBASE-2214 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Ferdy Galema Fix For: 0.96.0, 0.94.1 Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt The notion that you set size rather than row count specifying how many rows a scanner should return in each cycle was raised over in hbase-1966. Its a good one making hbase regular though the data under it may vary. HBase-1966 was committed but the patch was constrained by the fact that it needed to not change RPC interface. This issue is about doing hbase-1966 for 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2625) Make testDynamicBloom()'s randomness deterministic
[ https://issues.apache.org/jira/browse/HBASE-2625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266912#comment-13266912 ] Hudson commented on HBASE-2625: --- Integrated in HBase-TRUNK #2838 (See [https://builds.apache.org/job/HBase-TRUNK/2838/]) HBASE-2625 Avoid byte buffer allocations when reading a value from a Result object (Tudor Scurtu) (Revision 1333159) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestResult.java Make testDynamicBloom()'s randomness deterministic Key: HBASE-2625 URL: https://issues.apache.org/jira/browse/HBASE-2625 Project: HBase Issue Type: Test Components: test Affects Versions: 0.90.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.90.0 Attachments: hbase-2625.patch Had a failure with testDynamicBloom on Hudson today. Will investigate, however it would be nice to reproduce the problem to make sure it's not the fault of my test assumptions. I plan to seed the Random number generator with the current time and print that out for post-mortem analysis. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4990) Document secure HBase setup
[ https://issues.apache.org/jira/browse/HBASE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266991#comment-13266991 ] Hudson commented on HBASE-4990: --- Integrated in HBase-TRUNK #2839 (See [https://builds.apache.org/job/HBase-TRUNK/2839/]) HBASE-4990 Document secure HBase setup (Revision 1333212) Result = SUCCESS stack : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/security.xml Document secure HBase setup --- Key: HBASE-4990 URL: https://issues.apache.org/jira/browse/HBASE-4990 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.96.0 Attachments: 4990.txt, 4990v2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5913) Speed up the full scan of META
[ https://issues.apache.org/jira/browse/HBASE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267179#comment-13267179 ] Hudson commented on HBASE-5913: --- Integrated in HBase-TRUNK #2840 (See [https://builds.apache.org/job/HBase-TRUNK/2840/]) HBASE-5913 Speed up the full scan of META (Chunhui) (Revision 1333283) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java Speed up the full scan of META -- Key: HBASE-5913 URL: https://issues.apache.org/jira/browse/HBASE-5913 Project: HBase Issue Type: Improvement Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.96.0, 0.94.1 Attachments: 5913-v2.txt, HBASE-5913.patch In the master, we will do the full scan of META in some situations for example, 1.master start up 2.CatalogJanitor do the full scan per 5 mins 3.ServerShutdownHandler, getServerUserRegions for dead server. For the online applications, we should try the best to reduce the process time of ServerShutdownHandler in the situation 3. However, we found MetaReader#getServerUserRegions take 14mins for 10w regions in our production environment. And it is caused by two reasons: The first, we don't use cache and get one row per next() when fully scan .META. The second, hbase.ipc.client.tcpnodelay is false as default, and in our environment it take 40ms for per next() (It is related to the length of row in the .META. , if someone also found, could try to set it true) For this issue, I think we could set the caching when do the full scan of META -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5919) Add fixes for Ted's review comments from HBASE-5869
[ https://issues.apache.org/jira/browse/HBASE-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267212#comment-13267212 ] Hudson commented on HBASE-5919: --- Integrated in HBase-TRUNK #2841 (See [https://builds.apache.org/job/HBase-TRUNK/2841/]) HBASE-5919 Add fixes for Ted's review comments from HBASE-5869 (Revision 104) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Bytes.java Add fixes for Ted's review comments from HBASE-5869 --- Key: HBASE-5919 URL: https://issues.apache.org/jira/browse/HBASE-5919 Project: HBase Issue Type: Bug Reporter: stack Assignee: Ted Yu Priority: Blocker Attachments: 5919-v2.txt, 5919-v4.txt, 5919.txt I missed addressing a few of Ted's comments on the end of my navigating HBASE-5869 commit. Fix here. Make it a blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5869) Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb
[ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267213#comment-13267213 ] Hudson commented on HBASE-5869: --- Integrated in HBase-TRUNK #2841 (See [https://builds.apache.org/job/HBase-TRUNK/2841/]) HBASE-5919 Add fixes for Ted's review comments from HBASE-5869 (Revision 104) Result = SUCCESS tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Bytes.java Move SplitLogManager splitlog taskstate and AssignmentManager RegionTransitionData znode datas to pb - Key: HBASE-5869 URL: https://issues.apache.org/jira/browse/HBASE-5869 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Fix For: 0.96.0 Attachments: 5869v7.txt, 5869v8.txt, 5869v9.txt, firstcut.txt, secondcut.txt, v10.txt, v11.txt, v12.txt, v13.txt, v13.txt, v4.txt, v5.txt, v6.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4990) Document secure HBase setup
[ https://issues.apache.org/jira/browse/HBASE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267219#comment-13267219 ] Hudson commented on HBASE-4990: --- Integrated in HBase-TRUNK-security #190 (See [https://builds.apache.org/job/HBase-TRUNK-security/190/]) HBASE-4990 Document secure HBase setup (Revision 1333212) Result = SUCCESS stack : Files : * /hbase/trunk/src/docbkx/book.xml * /hbase/trunk/src/docbkx/security.xml Document secure HBase setup --- Key: HBASE-4990 URL: https://issues.apache.org/jira/browse/HBASE-4990 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.96.0 Attachments: 4990.txt, 4990v2.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
[ https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267221#comment-13267221 ] Hudson commented on HBASE-5840: --- Integrated in HBase-TRUNK-security #190 (See [https://builds.apache.org/job/HBase-TRUNK-security/190/]) HBASE-5840 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status (RajeshBabu) (Revision 1333124) HBASE-5548 Add ability to get a table in the shell; BACKING OUT MISTAKEN CO-COMMIT OF HBASE-5840 (Revision 1333123) Result = SUCCESS ramkrishna : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status -- Key: HBASE-5840 URL: https://issues.apache.org/jira/browse/HBASE-5840 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.0 Reporter: Gopinathan A Assignee: rajeshbabu Fix For: 0.96.0, 0.94.1 Attachments: HBASE-5840.patch, HBASE-5840_trunk.patch, HBASE-5840_v2.patch TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will keeps showing old status. This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
