[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260253#comment-13260253
 ] 

Hudson commented on HBASE-5849:
---

Integrated in HBase-0.92 #387 (See 
[https://builds.apache.org/job/HBase-0.92/387/])
HBASE-5849 On first cluster startup, RS aborts if root znode is not 
available (Revision 1329548)

 Result = ABORTED
stack : 
Files : 
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java


 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260254#comment-13260254
 ] 

Lars Hofhansl commented on HBASE-5848:
--

@Ram: What I meant was doing this:
{code}
createTable(new HTableDescriptor(x), new byte[][] {null});
{code}
splitKeys is not null, but one of the contained keys is.

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260255#comment-13260255
 ] 

stack commented on HBASE-5849:
--

I killed all running builds in case they'd run into this hang.

 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260257#comment-13260257
 ] 

ramkrishna.s.vasudevan commented on HBASE-5848:
---

@Lars
I tried that.  It fails here throwing NPE in HBaseAdmin.createTableAsync
{code}
Arrays.sort(splitKeys, Bytes.BYTES_COMPARATOR);
{code}
Then there should be a prevalidator to check if the splitkeys has a null 
element in it.
Ok i will add that validation also. 

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260258#comment-13260258
 ] 

Hudson commented on HBASE-5844:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5844 Delete the region servers znode after a regions server crash 
(Revision 1329430)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/bin/hbase-daemon.sh
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java


 Delete the region servers znode after a regions server crash
 

 Key: HBASE-5844
 URL: https://issues.apache.org/jira/browse/HBASE-5844
 Project: HBase
  Issue Type: Improvement
  Components: regionserver, scripts
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
 Fix For: 0.96.0

 Attachments: 5844.v1.patch, 5844.v2.patch, 5844.v3.patch


 today, if the regions server crashes, its znode is not deleted in ZooKeeper. 
 So the recovery process will stop only after a timeout, usually 30s.
 By deleting the znode in start script, we remove this delay and the recovery 
 starts immediately.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5635) If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260259#comment-13260259
 ] 

Hudson commented on HBASE-5635:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5635 If getTaskList() returns null, splitlogWorker would go down and 
it won't serve any requests (Ram) (Revision 1329322)

 Result = SUCCESS
ramkrishna : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java


 If getTaskList() returns null, splitlogWorker would go down and it won't 
 serve any requests
 ---

 Key: HBASE-5635
 URL: https://issues.apache.org/jira/browse/HBASE-5635
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Assignee: Chinna Rao Lalam
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch, 
 HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch


 During the hlog split operation if all the zookeepers are down ,then the 
 paths will be returned as null and the splitworker thread wil be exited
 Now this regionserver wil not be able to acquire any other tasks since the 
 splitworker thread is exited
 Please find the attached code for more details
 {code}
 private ListString getTaskList() {
 for (int i = 0; i  zkretries; i++) {
   try {
 return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
 this.watcher.splitLogZNode));
   } catch (KeeperException e) {
 LOG.warn(Could not get children of znode  +
 this.watcher.splitLogZNode, e);
 try {
   Thread.sleep(1000);
 } catch (InterruptedException e1) {
   LOG.warn(Interrupted while trying to get task list ..., e1);
   Thread.currentThread().interrupt();
   return null;
 }
   }
 }
 {code}
 in the org.apache.hadoop.hbase.regionserver.SplitLogWorker 
  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260256#comment-13260256
 ] 

stack commented on HBASE-5849:
--

Enis, might taking a look at this?

 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5851) TestProcessBasedCluster sometimes fails; currently disabled -- needs to be fixed and reenabled

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260261#comment-13260261
 ] 

Hudson commented on HBASE-5851:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5851 TestProcessBasedCluster sometimes fails; DISABLE THE FAILING 
TEST FOR NOW (Revision 1329497)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestProcessBasedCluster.java


 TestProcessBasedCluster sometimes fails; currently disabled -- needs to be 
 fixed and reenabled
 --

 Key: HBASE-5851
 URL: https://issues.apache.org/jira/browse/HBASE-5851
 Project: HBase
  Issue Type: Test
Reporter: Zhihong Yu
Assignee: Jimmy Xiang
 Attachments: disable.txt, hbase-5851.patch, hbase-5851_v2.patch, 
 metahang.txt, zkfail.txt


 TestProcessBasedCluster failed in 
 https://builds.apache.org/job/HBase-TRUNK-security/178
 Looks like cluster failed to start:
 {code}
 2012-04-21 14:22:32,666 INFO  [Thread-1] 
 util.ProcessBasedLocalHBaseCluster(176): Waiting for HBase to startup. 
 Retries left: 2
 java.io.IOException: Giving up trying to location region in meta: thread is 
 interrupted.
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1173)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:956)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:917)
   at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:252)
   at org.apache.hadoop.hbase.client.HTable.init(HTable.java:192)
   at 
 org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:174)
   at 
 org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
 java.lang.InterruptedException: sleep interrupted at 
 java.lang.Thread.sleep(Native Method)
   at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:134)
   at 
 org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:178)
   at 
 org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5830) Cleanup SequenceFileLogWriter to use syncFs api from SequenceFile#Writer directly in trunk.

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260263#comment-13260263
 ] 

Hudson commented on HBASE-5830:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5830 Cleanup SequenceFileLogWriter to use syncFs api from 
SequenceFile#Writer directly in trunk. (Revision 1329443)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java


 Cleanup SequenceFileLogWriter to use syncFs api from SequenceFile#Writer 
 directly in trunk.
 ---

 Key: HBASE-5830
 URL: https://issues.apache.org/jira/browse/HBASE-5830
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.96.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
 Fix For: 0.96.0

 Attachments: HBASE-5830.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5857) RIT map in RS not getting cleared while region opening

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260264#comment-13260264
 ] 

Hudson commented on HBASE-5857:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5857 RIT map in RS not getting cleared while region opening (Revision 
1329470)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java


 RIT map in RS not getting cleared while region opening
 --

 Key: HBASE-5857
 URL: https://issues.apache.org/jira/browse/HBASE-5857
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
 Fix For: 0.92.2, 0.94.0, 0.96.0

 Attachments: HBASE-5857_0.92.1.patch, HBASE-5857_0.92.patch, 
 HBASE-5857_0.94.1.patch, HBASE-5857_94.patch, HBASE-5857_trunk.1.patch, 
 HBASE-5857_trunk.1.patch, HBASE-5857_trunk.2.patch, HBASE-5857_trunk.patch


 While opening the region in RS after adding the region to 
 regionsInTransitionInRS if tableDescriptors.get() throws exception the region 
 wont be cleared from regionsInTransitionInRS. So next time if it tries to 
 open the region in the same RS it will throw the 
 RegionAlreadyInTransitionException.
 if swap the below statement this issue wont come.
 {code}
 this.regionsInTransitionInRS.putIfAbsent(region.getEncodedNameAsBytes(),true);
 HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260262#comment-13260262
 ] 

Hudson commented on HBASE-5849:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5849 On first cluster startup, RS aborts if root znode is not 
available (Revision 1329527)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java


 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5443) Add PB-based calls to HRegionInterface

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260260#comment-13260260
 ] 

Hudson commented on HBASE-5443:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5443 Convert admin protocol of HRegionInterface to PB (Revision 
1329358)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureServer.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/AdminProtocol.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ClientProtocol.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnection.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SortedCopyOnWriteSet.java
* /hbase/trunk/src/main/protobuf/Admin.proto
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHTableUtil.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 Add PB-based calls to HRegionInterface
 --

 Key: HBASE-5443
 URL: https://issues.apache.org/jira/browse/HBASE-5443
 Project: HBase
  Issue Type: Task
  Components: ipc, master, migration, regionserver
Reporter: Todd Lipcon
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: region_java-proto-mapping.pdf




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260265#comment-13260265
 ] 

Hudson commented on HBASE-5833:
---

Integrated in HBase-TRUNK-security #182 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/182/])
HBASE-5833 0.92 build has been failing pretty consistently on 
TestMasterFailover; ADDENDUM; missed import (Revision 1329411)
HBASE-5833 0.92 build has been failing pretty consistently on 
TestMasterFailover (Revision 1329400)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionOnCluster.java

stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/filter/TestColumnPrefixFilter.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestOpenedRegionHandler.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksRead.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestColumnSeeking.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionOnCluster.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestResettingCounters.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransaction.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java


 0.92 build has been failing pretty consistently on TestMasterFailover
 -

 Key: HBASE-5833
 URL: https://issues.apache.org/jira/browse/HBASE-5833
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.90.7, 0.92.2, 0.94.0

 Attachments: 5833-trunk.txt, 5833-v2.092.txt, 5833.txt, 
 5833v3092.txt, 5833v4090.txt, 5833v4092.txt, 5833v4trunk.txt, 5833v5094.txt, 
 5833v5trunk.txt, closehregions.txt


 Trunk seems fine but 0.92 fails on this test pretty regularly.  Running it 
 local it seems to hang for me.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5842) Passing shell commands as an argument

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260267#comment-13260267
 ] 

stack commented on HBASE-5842:
--

We used to have it so you could pass a formatter class that the shell would 
use.  Default is console formatting.  We used to have an html output one which 
was useful when you could type in shell commands on ui and get results as an 
html page.  Another formatter would emit results per line so greppable.

 Passing shell commands as an argument
 -

 Key: HBASE-5842
 URL: https://issues.apache.org/jira/browse/HBASE-5842
 Project: HBase
  Issue Type: Improvement
  Components: shell
Affects Versions: 0.94.0
Reporter: Harsh J
Priority: Minor

 Many times we've required scans of .META. to analyze issues with the cluster 
 we work on, and to have the result in a file we can pass around we usually 
 end up doing something like:
 {{echo scan '.META.'| hbase shell  meta-scan.txt}}
 This can rather be simplified as something like the following instead, with 
 support for a commands reading argument:
 {{hbase shell -c scan '.META.'}}
 [Note though: File reading is possible already, i.e. {{hbase shell file.hs}}, 
 but then thats two steps and we usually don't keep a file around for just a 
 meta table scan.]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260269#comment-13260269
 ] 

Lars Hofhansl commented on HBASE-5848:
--

@Ram: That is probably overkill, means another loop over the splitKeys before 
they are sorted. As long as it fails on the client (even with an NPE) and does 
not cause the master to abort it's fine I think.

If you like to add the extra check now, that fine too.

+1 in either case. Going to bed now, feel free to commit :)


 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5847) Support createTable(splitKeys) in Thrift

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260270#comment-13260270
 ] 

stack commented on HBASE-5847:
--

Does the thrift2 package support this Nicolas?

 Support createTable(splitKeys) in Thrift
 

 Key: HBASE-5847
 URL: https://issues.apache.org/jira/browse/HBASE-5847
 Project: HBase
  Issue Type: Improvement
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Trivial

 The Thrift API does not allow a user to create a table with multiple split 
 keys.  This is needed for a handful of new internal projects that are written 
 in PHP/C++.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260271#comment-13260271
 ] 

Lars Hofhansl commented on HBASE-5861:
--

I am generally a bit unclear at what point we require a recompile of HBase 
against a specific version of Hadoop and when we make it just work via 
reflection.


 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5677) The master never does balance because duplicate openhandled the one region

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5677:
-

Resolution: Won't Fix
Status: Resolved  (was: Patch Available)

Will be fixed over in HBASE-5850.

 The master never does balance because duplicate openhandled the one region
 --

 Key: HBASE-5677
 URL: https://issues.apache.org/jira/browse/HBASE-5677
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6
 Environment: 0.90
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2

 Attachments: 5677-proposal.txt, 5677-proposal.txt, 
 Backport-HBASE-5454-to-90.patch, Backport-HBASE-5454-to-92.patch, 
 HBASE-5677-90-v1.patch, surefire-report_no_patched_v1.html, 
 surefire-report_patched_v1.html


 If region be assigned When the master is doing initialization(before do 
 processFailover),the region will be duplicate openhandled.
 because the unassigned node in zookeeper will be handled again in 
 AssignmentManager#processFailover()
 it cause the region in RIT,thus the master never does balance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5342) Grant/Revoke global permissions

2012-04-24 Thread Matteo Bertozzi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260273#comment-13260273
 ] 

Matteo Bertozzi commented on HBASE-5342:


@Enis are you planning to release something soon? 
otherwise I can work on that. Do you have already a draft/notes or can I start 
from scratch?

 Grant/Revoke global permissions
 ---

 Key: HBASE-5342
 URL: https://issues.apache.org/jira/browse/HBASE-5342
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar

 HBASE-3025 introduced simple ACLs based on coprocessors. It defines 
 global/table/cf/cq level permissions. However, there is no way to 
 grant/revoke global level permissions, other than the hbase.superuser conf 
 setting. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5850) Refuse operations from Admin before master is initialized - fix for all branches.

2012-04-24 Thread xufeng (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260278#comment-13260278
 ] 

xufeng commented on HBASE-5850:
---

@stack @Ted @Lars
Thanks for your help,Thanks. 

 Refuse operations from Admin before master is initialized - fix for all 
 branches.
 -

 Key: HBASE-5850
 URL: https://issues.apache.org/jira/browse/HBASE-5850
 Project: HBase
  Issue Type: Bug
Reporter: xufeng
Assignee: xufeng
 Fix For: 0.90.7, 0.92.2, 0.94.0

 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, 
 backport-5454(createTable)-to-94.patch, 
 backport-5454(createTable)-to-94_surefire-report.html, 
 backport-5454(createTable)-to-trunk.patch, 
 backport-5454(createTable)-to-trunk_surefire-report.html, 
 backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, 
 backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html


 This issue is needed in 0.90 0.92 also.
 And update the hbase-5454 patch that add the checkInitialized() into 
 HMaster#createTable().

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5564) Bulkload is discarding duplicate records

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260282#comment-13260282
 ] 

stack commented on HBASE-5564:
--

@Laxman Any luck?

 Bulkload is discarding duplicate records
 

 Key: HBASE-5564
 URL: https://issues.apache.org/jira/browse/HBASE-5564
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.96.0
 Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
  Labels: bulkloader
 Fix For: 0.96.0

 Attachments: 5564.lint, 5564v5.txt, HBASE-5564_trunk.1.patch, 
 HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch, 
 HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch


 Duplicate records are getting discarded when duplicate records exists in same 
 input file and more specifically if they exists in same split.
 Duplicate records are considered if the records are from diffrent different 
 splits.
 Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260280#comment-13260280
 ] 

stack commented on HBASE-5732:
--

@Devaraj If this patch only works against 1.0.x hadoop, what do we do when we 
want to run hbase 0.96 on hadoop 2.0.x?

Here is some more feedback on posted patch:

Its big!  Its mostly deletes and generated code though thankfully.

So, you are going to move to UGI over User?

Are you going to get rid of the security dir that is at top level in hbase?  Is 
there anything left in it after this patch?

What kind of hadoop will be required?  One that supports security: i.e. apache 
hadoop 1.0.x, 2.0.x?  And then for the others?  The will need to have an answer 
for the security methods?

Just remove rather than do this commenting out:

{code}
-builder.setError(error != null);
+//builder.setStatus(
{code}

This is going to make a copy of the response?

{code}
+  token = connection.saslServer.wrap(buf.array(),
+  buf.arrayOffset(), buf.remaining());
{code}

Do we have to?  Can't we feed it out on the output stream, first the wrapping, 
then the response?

Any tests?

Good stuff.

 Remove the SecureRPCEngine and merge the security-related logic in the core 
 engine
 --

 Key: HBASE-5732
 URL: https://issues.apache.org/jira/browse/HBASE-5732
 Project: HBase
  Issue Type: Improvement
Reporter: Devaraj Das
Assignee: Devaraj Das
 Attachments: rpcengine-merge.patch


 Remove the SecureRPCEngine and merge the security-related logic in the core 
 engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus

2012-04-24 Thread jiafeng.zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260283#comment-13260283
 ] 

jiafeng.zhang commented on HBASE-5853:
--

Recompiled 
mvn clean site install assembly:single -Dmaven.test.skip –Prelease  
-Dhadoop.profile=23

 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
 --

 Key: HBASE-5853
 URL: https://issues.apache.org/jira/browse/HBASE-5853
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.1
 Environment: hadoop-0.23.1 hbase-0.92.1 
Reporter: jiafeng.zhang
 Fix For: 0.92.1, 0.94.0


 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error 
 reading responses on connection Thread[IPC Client (1260987126) connection to 
 server121/172.16.40.121:9000 from smp,5,main]
 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372)
   at 
 org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223)
   at 
 org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75)
   at 
 org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832)
   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756)
 Caused by: java.lang.ClassNotFoundException: Class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found
   at 
 org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151)
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368)
   ... 4 more
 2012-04-23 12:51:07,797 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 server124,60020,1335152900476: Replay of HLog required. Forcing server 
 shutdown
 org.apache.hadoop.hbase.DroppedSnapshotException: region: 
 hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a.
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: java.io.IOException: Failed on local exception: 
 java.io.IOException: Error reading responses; Host Details : local host is: 
 server124/172.16.40.124; destination host is: server121:9000; 
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724)
   at org.apache.hadoop.ipc.Client.call(Client.java:1094)
   at 
 org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193)
   at $Proxy10.getFileInfo(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65)
   at $Proxy10.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548)
   at 
 org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595)
   at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506)
   at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89)
   at 
 org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254)
   ... 6 more
 Caused by: java.io.IOException: Error reading responses
   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763)
 Caused by: java.lang.RuntimeException: 

[jira] [Updated] (HBASE-5672) TestLruBlockCache#testBackgroundEvictionThread fails occasionally

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5672:
-

Status: Patch Available  (was: Open)

+1 on patch.  Submitting to hadoopqa.

 TestLruBlockCache#testBackgroundEvictionThread fails occasionally
 -

 Key: HBASE-5672
 URL: https://issues.apache.org/jira/browse/HBASE-5672
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5672.patch, HBASE-5672v2.patch


 We find TestLruBlockCache#testBackgroundEvictionThread fails occasionally.
 I think it's a problem of the test case.
 Because runEviction() only do evictionThread.evict():
 {code}
 public void evict() {
   synchronized(this) {
 this.notify(); // FindBugs NN_NAKED_NOTIFY
   }
 }
 {code}
 However when we call evictionThread.evict(), the evictionThread may haven't 
 been in run() in the TestLruBlockCache#testBackgroundEvictionThread.
 If we run the test many times, we could find failture easily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5672) TestLruBlockCache#testBackgroundEvictionThread fails occasionally

2012-04-24 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260288#comment-13260288
 ] 

Hadoop QA commented on HBASE-5672:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12521633/HBASE-5672v2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1624//console

This message is automatically generated.

 TestLruBlockCache#testBackgroundEvictionThread fails occasionally
 -

 Key: HBASE-5672
 URL: https://issues.apache.org/jira/browse/HBASE-5672
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5672.patch, HBASE-5672v2.patch


 We find TestLruBlockCache#testBackgroundEvictionThread fails occasionally.
 I think it's a problem of the test case.
 Because runEviction() only do evictionThread.evict():
 {code}
 public void evict() {
   synchronized(this) {
 this.notify(); // FindBugs NN_NAKED_NOTIFY
   }
 }
 {code}
 However when we call evictionThread.evict(), the evictionThread may haven't 
 been in run() in the TestLruBlockCache#testBackgroundEvictionThread.
 If we run the test many times, we could find failture easily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260295#comment-13260295
 ] 

Hudson commented on HBASE-5849:
---

Integrated in HBase-TRUNK #2804 (See 
[https://builds.apache.org/job/HBase-TRUNK/2804/])
HBASE-5849 On first cluster startup, RS aborts if root znode is not 
available; REVERT (Revision 1329560)

 Result = SUCCESS
stack : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java


 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5672) TestLruBlockCache#testBackgroundEvictionThread fails occasionally

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260307#comment-13260307
 ] 

stack commented on HBASE-5672:
--

Sorry Chunhui, we let the patch rot.  Mind updating it?  Thanks.

 TestLruBlockCache#testBackgroundEvictionThread fails occasionally
 -

 Key: HBASE-5672
 URL: https://issues.apache.org/jira/browse/HBASE-5672
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5672.patch, HBASE-5672v2.patch


 We find TestLruBlockCache#testBackgroundEvictionThread fails occasionally.
 I think it's a problem of the test case.
 Because runEviction() only do evictionThread.evict():
 {code}
 public void evict() {
   synchronized(this) {
 this.notify(); // FindBugs NN_NAKED_NOTIFY
   }
 }
 {code}
 However when we call evictionThread.evict(), the evictionThread may haven't 
 been in run() in the TestLruBlockCache#testBackgroundEvictionThread.
 If we run the test many times, we could find failture easily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Harsh J (JIRA)
Harsh J created HBASE-5863:
--

 Summary: Improve the graceful_stop.sh CLI help (especially about 
reloads)
 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Attachments: HBASE-5863.patch

Right now, graceful_stop.sh prints:

{code}
Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] [--thrift] 
[--rest] hostname
 thrift  If we should stop/start thrift before/after the hbase stop/start
 restIf we should stop/start rest before/after the hbase stop/start
 restart If we should restart after graceful stop
 reload  Move offloaded regions back on to the stopped server
 debug   Move offloaded regions back on to the stopped server
 hostnameHostname of server we are to stop
{code}

This does not help us specify that reload is actually a sub/additive-option 
to restart.

Also, the debug line seems to still have an old copy/paste mistake.

I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Harsh J (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HBASE-5863:
---

Attachment: HBASE-5863.patch

 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260308#comment-13260308
 ] 

Hudson commented on HBASE-5849:
---

Integrated in HBase-0.94 #141 (See 
[https://builds.apache.org/job/HBase-0.94/141/])
HBASE-5849 On first cluster startup, RS aborts if root znode is not 
available; REVERT (Revision 1329561)

 Result = FAILURE
stack : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java


 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Harsh J (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HBASE-5863:
---

Status: Patch Available  (was: Open)

 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4393:
-

   Resolution: Fixed
Fix Version/s: 0.96.0
 Release Note: Tool to check cluster.  See $ ./bin/hbase 
org.apache.hadoop.hbase.tool.Canary -help for how to use.
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk.  Thanks for the patch Matteo. I tried it out.  Does the 
basics.  Nice. Thanks.

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4393:
-

Fix Version/s: 0.94.0

Committed to 0.94 (thought you might like this Lars).

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260345#comment-13260345
 ] 

Hadoop QA commented on HBASE-5863:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523940/HBASE-5863.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 5 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1625//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1625//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1625//console

This message is automatically generated.

 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260352#comment-13260352
 ] 

Hudson commented on HBASE-4393:
---

Integrated in HBase-0.94 #142 (See 
[https://builds.apache.org/job/HBase-0.94/142/])
HBASE-4393 Implement a canary monitoring program (Revision 1329575)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/tools
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/tools/Canary.java


 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Sho Shimauchi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260356#comment-13260356
 ] 

Sho Shimauchi commented on HBASE-5863:
--

This patch changes only help messages in a shell script.
New tests aren't needed for this patch.

 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260358#comment-13260358
 ] 

Hudson commented on HBASE-5849:
---

Integrated in HBase-0.92 #388 (See 
[https://builds.apache.org/job/HBase-0.92/388/])
HBASE-5849 On first cluster startup, RS aborts if root znode is not 
available; REVERT (Revision 1329562)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java


 On first cluster startup, RS aborts if root znode is not available
 --

 Key: HBASE-5849
 URL: https://issues.apache.org/jira/browse/HBASE-5849
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver, zookeeper
Affects Versions: 0.92.2, 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.2, 0.94.0

 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch


 When launching a fresh new cluster, the master has to be started first, which 
 might create race conditions for starting master and rs at the same time. 
 Master startup code is smt like this: 
  - establish zk connection
  - create root znodes in zk (/hbase)
  - create ephemeral node for master /hbase/master, 
  Region server start up code is smt like this: 
  - establish zk connection
  - check whether the root znode (/hbase) is there. If not, shutdown. 
  - wait for the master to create znodes /hbase/master
 So, the problem is on the very first launch of the cluster, RS aborts to 
 start since /hbase znode might not have been created yet (only the master 
 creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on 
 subsequent cluster starts, it does not matter which order the servers are 
 started. So this affects only first launchs. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260370#comment-13260370
 ] 

Hudson commented on HBASE-4393:
---

Integrated in HBase-TRUNK #2805 (See 
[https://builds.apache.org/job/HBase-TRUNK/2805/])
HBASE-4393 Implement a canary monitoring program (Revision 1329574)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/tools
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/tools/Canary.java


 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5856) byte - String is not consistent between HBaseAdmin and HRegionInfo

2012-04-24 Thread binlijin (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

binlijin updated HBASE-5856:


Attachment: HBASE-5856-0.92.patch

 byte - String is not consistent between HBaseAdmin and HRegionInfo
 

 Key: HBASE-5856
 URL: https://issues.apache.org/jira/browse/HBASE-5856
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1
Reporter: binlijin
 Attachments: HBASE-5856-0.92.patch


 In HBaseAdmin 
   public void split(final String tableNameOrRegionName)
   throws IOException, InterruptedException {
 split(Bytes.toBytes(tableNameOrRegionName));  // string - byte 
   }
 In HRegionInfo
   this.regionNameStr = Bytes.toStringBinary(this.regionName);  // byte - 
 string
 Should we use Bytes.toBytesBinary in HBaseAdmin ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Gopinathan A (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gopinathan A updated HBASE-5864:


  Component/s: regionserver
Affects Version/s: 0.94.0
Fix Version/s: 0.94.0

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Priority: Critical
 Fix For: 0.94.0


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Gopinathan A (JIRA)
Gopinathan A created HBASE-5864:
---

 Summary: Error while reading from hfile in 0.94
 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
Reporter: Gopinathan A
Priority: Critical


Got the following stacktrace during region split.

{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: Failed 
getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404, 
lastDataBlockOffset: 84764558
at 
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at 
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at 
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at 
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at 
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at 
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at 
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260456#comment-13260456
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

Please correct me if am wrong.  
Am working on a patch, which i can upload for verification.  Maynot be the 
final one.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-5864:
-

Assignee: ramkrishna.s.vasudevan

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260455#comment-13260455
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

The problem is because of using checksums.  This problem will be coming based 
on the block size and the kv size that are added in HFile.
When we try to read a block, Consider the following code 
In BlockIterator.blockRange
{code}
@Override
public HFileBlock nextBlock() throws IOException {
  if (offset = endOffset)
return null;
  HFileBlock b = readBlockData(offset, -1, -1, false);
  offset += b.getOnDiskSizeWithHeader();
  return b;
}
{code}
In case of noncompressed algo in 
{code}
  private HFileBlock readBlockDataInternal(FSDataInputStream is, long offset, 
long onDiskSizeWithHeaderL,
int uncompressedSize, boolean pread, boolean verifyChecksum) 
{code}
{code}
 b = new HFileBlock(headerBuf, getMinorVersion());

// This will also allocate enough room for the next block's header.
b.allocateBuffer(true);

{code}
Inside allocateBuffer
{code}
int cksumBytes = totalChecksumBytes();
int capacityNeeded = headerSize() + uncompressedSizeWithoutHeader +
cksumBytes +
(extraBytes ? headerSize() : 0);

ByteBuffer newBuf = ByteBuffer.allocate(capacityNeeded);
{code}
This byte allocated for this block is having the checksumbytes also added to it.
After fetching the block and its corresponding stream, we try to get the root 
block.
{code}
// Data index. We also read statistics about the block index written after
// the root level.
dataBlockIndexReader.readMultiLevelIndexRoot(
blockIter.nextBlockAsStream(BlockType.ROOT_INDEX),
trailer.getDataIndexCount());

{code}
Inside readMultiLevelIndexRoot
{code}
readRootIndex(in, numEntries);
  if (in.available()  MID_KEY_METADATA_SIZE) {
// No mid-key metadata available.
return;
  }
{code}
Here the InputStream 'in' is formed from the blk.getByteStream().
Here the inputstream actually will have its available length including the 
checksum bytes.
While doing readRootIndex
{code}
public void readRootIndex(DataInput in, final int numEntries)
throws IOException {
{code}
We read only the data and the remaining checksum bytse allocated are still 
available with the input stream.
We know that for every 16k data size we have 4 byte check sum.
Now if my data size is ~66k i will have 20bytes check sum.  So the 
{code}
   if (in.available()  MID_KEY_METADATA_SIZE) {
// No mid-key metadata available.
return;
  }
{code}
will now be greater than MID_KEY_METADATA_SIZE(16 bytes) and thus giving us 
some invalid 'midLeafBlockOffset'.
So when we try to read the file we get some abnormal values.  The same can be 
reproduced using the testcase.



 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Priority: Critical
 Fix For: 0.94.0


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine

2012-04-24 Thread Andrew Purtell (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260488#comment-13260488
 ] 

Andrew Purtell commented on HBASE-5732:
---

The only thing in security/ after the RPC engine merge should be the 
AccessController and TokenProvider coprocessors. TokenProvider will be needed 
for authenticating MR jobs when secure HBase RPC is enabled. AccessController 
is optional and provides per CF ACLs. 

Unfortunately one must .wrap(), the SASL layer requires it. In the event the 
user negotiates auth-conf and probably also auth-int then failure to wrap the 
payload will send junk.


 Remove the SecureRPCEngine and merge the security-related logic in the core 
 engine
 --

 Key: HBASE-5732
 URL: https://issues.apache.org/jira/browse/HBASE-5732
 Project: HBase
  Issue Type: Improvement
Reporter: Devaraj Das
Assignee: Devaraj Das
 Attachments: rpcengine-merge.3.patch, rpcengine-merge.patch


 Remove the SecureRPCEngine and merge the security-related logic in the core 
 engine. Follow up to HBASE-5727.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260541#comment-13260541
 ] 

Jonathan Hsieh commented on HBASE-5861:
---

There are two classes of changes I'm aware of currently:

1) Class becomes Interface - which means constructors for some things have 
moved to be impls.  Seems to be ok with reflection.
2) Enum moved to different new class, and original class exists for 
compatibility as a subclass of new class.  This seems to require recompile 
because this confuses class loader.

 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread Zhihong Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260557#comment-13260557
 ] 

Zhihong Yu commented on HBASE-4393:
---

This checkin might be related to:
{code}
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check 
(default) on project hbase: Too many unapproved licenses: 1 - [Help 1]
{code}

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus

2012-04-24 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260594#comment-13260594
 ] 

Jonathan Hsieh commented on HBASE-5853:
---

Are you running this as processes or unit tests?  

When you run 'hbase classpath' is hadoop-hdfs-*.jar part of your path or 
hadoop-core-*.jar part of your path?

 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
 --

 Key: HBASE-5853
 URL: https://issues.apache.org/jira/browse/HBASE-5853
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.1
 Environment: hadoop-0.23.1 hbase-0.92.1 
Reporter: jiafeng.zhang
 Fix For: 0.92.1, 0.94.0


 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error 
 reading responses on connection Thread[IPC Client (1260987126) connection to 
 server121/172.16.40.121:9000 from smp,5,main]
 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372)
   at 
 org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223)
   at 
 org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75)
   at 
 org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832)
   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756)
 Caused by: java.lang.ClassNotFoundException: Class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found
   at 
 org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151)
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368)
   ... 4 more
 2012-04-23 12:51:07,797 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 server124,60020,1335152900476: Replay of HLog required. Forcing server 
 shutdown
 org.apache.hadoop.hbase.DroppedSnapshotException: region: 
 hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a.
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: java.io.IOException: Failed on local exception: 
 java.io.IOException: Error reading responses; Host Details : local host is: 
 server124/172.16.40.124; destination host is: server121:9000; 
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724)
   at org.apache.hadoop.ipc.Client.call(Client.java:1094)
   at 
 org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193)
   at $Proxy10.getFileInfo(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65)
   at $Proxy10.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548)
   at 
 org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595)
   at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506)
   at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89)
   at 
 org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254)
   ... 6 more
 Caused by: java.io.IOException: Error reading responses
   at 

[jira] [Issue Comment Edited] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus

2012-04-24 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260594#comment-13260594
 ] 

Jonathan Hsieh edited comment on HBASE-5853 at 4/24/12 2:16 PM:


Are you running this (your client) as processes or as part of unit tests?  

When you run 'hbase classpath' is hadoop-hdfs-\*.jar part of your path or 
hadoop-core-\*.jar part of your path?

  was (Author: jmhsieh):
Are you running this as processes or unit tests?  

When you run 'hbase classpath' is hadoop-hdfs-*.jar part of your path or 
hadoop-core-*.jar part of your path?
  
 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
 --

 Key: HBASE-5853
 URL: https://issues.apache.org/jira/browse/HBASE-5853
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.1
 Environment: hadoop-0.23.1 hbase-0.92.1 
Reporter: jiafeng.zhang
 Fix For: 0.92.1, 0.94.0


 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error 
 reading responses on connection Thread[IPC Client (1260987126) connection to 
 server121/172.16.40.121:9000 from smp,5,main]
 java.lang.RuntimeException: readObject can't find class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372)
   at 
 org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223)
   at 
 org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75)
   at 
 org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832)
   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756)
 Caused by: java.lang.ClassNotFoundException: Class 
 org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found
   at 
 org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151)
   at 
 org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368)
   ... 4 more
 2012-04-23 12:51:07,797 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 server124,60020,1335152900476: Replay of HLog required. Forcing server 
 shutdown
 org.apache.hadoop.hbase.DroppedSnapshotException: region: 
 hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a.
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202)
   at 
 org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
   at java.lang.Thread.run(Thread.java:662)
 Caused by: java.io.IOException: Failed on local exception: 
 java.io.IOException: Error reading responses; Host Details : local host is: 
 server124/172.16.40.124; destination host is: server121:9000; 
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724)
   at org.apache.hadoop.ipc.Client.call(Client.java:1094)
   at 
 org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193)
   at $Proxy10.getFileInfo(Unknown Source)
   at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100)
   at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65)
   at $Proxy10.getFileInfo(Unknown Source)
   at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548)
   at 
 org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595)
   at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506)
   at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89)
   at 
 

[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260601#comment-13260601
 ] 

Jonathan Hsieh commented on HBASE-5861:
---

We shouldn't use JobContextImpl or TaskAttemptContextImpl since it is marked:

{code}
@InterfaceAudience.Private
{code}

My guess is that HBASE-5853 may be something else -- possibly a class path or 
jar compatibility problem.  At first glance it looks over there that only 
public apis are used.  

I'll move the previous patch to be a HadoopTestShim in hbase tests and use 
MapReduceTestUtil.  At the moment this seems to be isolated to tests.  If we 
need to have a shim for production code, we can make a new HadoopShim for those 
calls. 

 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5864:
--

Attachment: HBASE-5864_test.patch

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260634#comment-13260634
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

I have attached a sample testcase.  Not a regular test case.  If you apply the 
test case and in the logs search for the string 
The midLeafBlockOffset is  .
We should be getting 
{code}
2012-04-24 20:26:21,140 DEBUG [main] 
hfile.HFileBlockIndex$BlockIndexReader(553): The midLeafBlockOffset is  
7767124539785491489
{code}

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Jonathan Hsieh (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260644#comment-13260644
 ] 

Jonathan Hsieh commented on HBASE-5861:
---

Currently looks like InputSampler (a sampler for bulk loads) and friends are 
probably broken in 23 because it uses the  TaskAttemptContext constructor which 
is hadoop 1.x specific.  We didn't catch this because currently there are no 
tests exercising this code. If we use MapReduceTestUtil to get a dummy, we'd 
need to pull in the hadoop-test jar to compile.

Against hadoop 23, MapReduceTestUtil is included in the jars pulled in for 
compile and test.

 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260645#comment-13260645
 ] 

stack commented on HBASE-5864:
--

Good find lads.  I'm not sure I follow.  Is it fixable?

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260651#comment-13260651
 ] 

stack commented on HBASE-4393:
--

No.  This patch has a license.  The failure was because of the OOME.  RAT 
complaint is this:

{code}

Unapproved licenses:

  hs_err_pid23951.log

...
{code}

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5856) byte - String is not consistent between HBaseAdmin and HRegionInfo

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260661#comment-13260661
 ] 

stack commented on HBASE-5856:
--

Help me understand Binlijin.  This does not seem right to me but you are 
obviously on to something.  The String that is being passed in here is coming 
from the shell?  Is jruby shell doing the escaping before it passes the String 
to HBaseAdmin?  Thanks.

 byte - String is not consistent between HBaseAdmin and HRegionInfo
 

 Key: HBASE-5856
 URL: https://issues.apache.org/jira/browse/HBASE-5856
 Project: HBase
  Issue Type: Bug
  Components: client
Affects Versions: 0.90.6, 0.92.1
Reporter: binlijin
 Attachments: HBASE-5856-0.92.patch


 In HBaseAdmin 
   public void split(final String tableNameOrRegionName)
   throws IOException, InterruptedException {
 split(Bytes.toBytes(tableNameOrRegionName));  // string - byte 
   }
 In HRegionInfo
   this.regionNameStr = Bytes.toStringBinary(this.regionName);  // byte - 
 string
 Should we use Bytes.toBytesBinary in HBaseAdmin ?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5863:
-

   Resolution: Fixed
Fix Version/s: 0.94.1
   0.92.2
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to 0.92, 0.94 and trunk.  Thanks for the patch Harsh.

 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5864:
--

Attachment: HBASE-5864_1.patch

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260667#comment-13260667
 ] 

ramkrishna.s.vasudevan commented on HBASE-5848:
---

@Lars
I will commit the version as it is.

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260665#comment-13260665
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

The attached patch seems to fix the problem that we encountered.  Needs more 
testing and also with files with old format.
Will find out if something better is there.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260672#comment-13260672
 ] 

stack commented on HBASE-5864:
--

I'm not sure I follow what your patch is doing Ram.  And maybe we need a test 
around split of hfile?

What is this doing:

{code}
-final int ENTRY_COUNT = 1;
+final int ENTRY_COUNT = 5;
{code}

This is asking for too many entries?

Good stuff.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5625) Avoid byte buffer allocations when reading a value from a Result object

2012-04-24 Thread jirapos...@reviews.apache.org (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260677#comment-13260677
 ] 

jirapos...@reviews.apache.org commented on HBASE-5625:
--



bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/KeyValue.java, line 531
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97954#file97954line531
bq.  
bq.   How do we know this buffer is big enough?  Maybe should add an 
override that takes an offset into the buffer?
bq.  
bq.  Tudor Scurtu wrote:
bq.  Added overload. Added exception comment for when there is insufficient 
space remaining in the buffer.

So, the way this works, we just allocate N and hope that stuff fits inside N?  
If it doesn't we throw an exception?  There is no correlation between data that 
comes across and the N allocation?


bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/KeyValue.java, line 616
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97954#file97954line616
bq.  
bq.   How do I know the buffer is big enough?
bq.  
bq.  Tudor Scurtu wrote:
bq.  Added exception comment for when there is insufficient space remaining 
in the buffer. Is that what you meant?

I am not understanding how the allocation works.   It seems arbitrary unrelated 
to the actual result size that comes over from the server.  Is that so?  If so, 
it seems unfriendly throwing an exception when allocated size and what is 
returned from the server do not match.


bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/KeyValue.java, line 1138
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97954#file97954line1138
bq.  
bq.   Why not create a ByteBuffer?  or called ByteBuffer wrap?  Why not 
call it toByteBuffer?
bq.  
bq.  Tudor Scurtu wrote:
bq.  You need to be able to pass your own buffer when you have to compose 
multiple values.
bq.  I added a new method that uses 'ByteBuffer.wrap()'. I feel that this 
method should contain the word 'value' in its name so as not to create the 
impression that it is writing the entire underlying 'KeyValue' structure to the 
buffer, so I called it 'getValueAsByteBuffer()'. Please comment if it is 
inadequate.

Sounds good Tudor.


bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/Result.java, line 297
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97955#file97955line297
bq.  
bq.   This comment should be on the @return javadoc rather than here.
bq.  
bq.  Tudor Scurtu wrote:
bq.  This was copied from original. Should I remove both occurences?

Sorry.  I did not notice it was problem on original.   If you can fix it, 
that'd be sweet.


bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/Result.java, line 431
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97955#file97955line431
bq.  
bq.   Why we have both isNonEmptyColumn and isEmptyColumn?  Why not just 
one and then check return with a !?
bq.  
bq.  Tudor Scurtu wrote:
bq.  They are not complementary.
bq.containsColumn = value exists
bq.containsEmptyColumn = value exists  is empty byte array
bq.containsNonEmptyColumn = value exists  is not empty byte array
bq.  The value could be missing, in which case all methods would return 
false.

OK.  If not clear from comments, please add your notes above.  Will help those 
that come after.


bq.  On 2012-04-02 17:34:38, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/Result.java, line 478
bq.   https://reviews.apache.org/r/4607/diff/1/?file=97955#file97955line478
bq.  
bq.   Rename hasColumn or isColumn.
bq.  
bq.  Tudor Scurtu wrote:
bq.  This overloads an original method. A refactoring here would be a major 
non-backwards compatible change.

Ok.  Thanks for pointing this out.


- Michael


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4607/#review6623
---


On 2012-04-04 17:08:03, Tudor Scurtu wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4607/
bq.  ---
bq.  
bq.  (Updated 2012-04-04 17:08:03)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  When calling Result.getValue(), an extra dummy KeyValue and its associated 
underlying byte array are allocated, as well as a persistent buffer that will 
contain the returned value.
bq.  
bq.  These can be avoided by reusing a static array for the dummy object and by 
passing a 

[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5848:
--

Attachment: HBASE-5848_0.94.patch

Patch for 0.94 similar to trunk patch.

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5848:
--

Fix Version/s: 0.96.0

Committed to trunk and 0.94.
Thanks for the review Ted, Stack and Lars.

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5848:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260686#comment-13260686
 ] 

Hudson commented on HBASE-5863:
---

Integrated in HBase-TRUNK #2806 (See 
[https://builds.apache.org/job/HBase-TRUNK/2806/])
HBASE-5863 Improve the graceful_stop.sh CLI help (especially about reloads) 
(Revision 1329793)

 Result = SUCCESS
stack : 
Files : 
* /hbase/trunk/bin/graceful_stop.sh


 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status

2012-04-24 Thread rajeshbabu (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

rajeshbabu updated HBASE-5840:
--

Attachment: HBASE-5840.patch

Attached patch.Please review and provide your comments/suggestions.

 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing 
 the old status
 --

 Key: HBASE-5840
 URL: https://issues.apache.org/jira/browse/HBASE-5840
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5840.patch


 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will 
 keeps showing old status.
 This will miss leads the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5864:
--

Attachment: HBASE-5864_2.patch

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Zhihong Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260697#comment-13260697
 ] 

Zhihong Yu commented on HBASE-5864:
---

{code}
-DataInputStream nextBlockAsStream(BlockType blockType) throws IOException;
+HFileBlock nextBlockAsStream(BlockType blockType) throws IOException;
{code}
The method should be named nextBlock() because stream isn't returned.
{code}
+ * Read in the root-level index from the given input stream. Must match
{code}
'input stream' is no longer the input. HFileBlock is.
Please add @return to the javadoc.
For TestHFileWriterV2.java:
{code}
-final Compression.Algorithm COMPRESS_ALGO = Compression.Algorithm.GZ;
+final Compression.Algorithm COMPRESS_ALGO = Compression.Algorithm.NONE;
{code}
We should exercise both compression algorithms. Refactoring is needed.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260700#comment-13260700
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

Just added a test case to show how midKey() will throw IOException.
Stack what the patch does is,
For the checksum we add some additional bytes while forming the inputstream to 
read the block.
After reading the block we just check if the bytes available in stream is less 
than the MID_KEY_METADATA_SIZE.
But in this case (5 is many? but i get only one root level index), there is 
always some remaining bytes added as part of check sum and it is always more 
than MID_KEY_METADATA_SIZE.

The testcase that we carried out to get this problem was,
Created a table with no split keys.
Start pumping data to this region using parallel threads.
Allow a couple of flush/compaction.  
Then try to split the region.  We got this problem. 
I may be wrong, pls do correct me and feel free to update the patch also.



 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260701#comment-13260701
 ] 

ramkrishna.s.vasudevan commented on HBASE-5840:
---

+1..Test case looks good.
Lets wait for other's comments.

 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing 
 the old status
 --

 Key: HBASE-5840
 URL: https://issues.apache.org/jira/browse/HBASE-5840
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5840.patch


 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will 
 keeps showing old status.
 This will miss leads the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Assigned] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-5840:
-

Assignee: rajeshbabu  (was: ramkrishna.s.vasudevan)

 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing 
 the old status
 --

 Key: HBASE-5840
 URL: https://issues.apache.org/jira/browse/HBASE-5840
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: rajeshbabu
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5840.patch


 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will 
 keeps showing old status.
 This will miss leads the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260703#comment-13260703
 ] 

ramkrishna.s.vasudevan commented on HBASE-5864:
---

@Ted
Sorry just now saw your comments. Will update based on further reviews. Thanks. 
:)

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-24 Thread Jimmy Xiang (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-5620:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Convert the client protocol of HRegionInterface to PB
 -

 Key: HBASE-5620
 URL: https://issues.apache.org/jira/browse/HBASE-5620
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
 hbase-5620_v4.patch, hbase-5620_v4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5864:
--

Status: Patch Available  (was: Open)

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Zhihong Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260709#comment-13260709
 ] 

Zhihong Yu commented on HBASE-5864:
---

Patch v2 passes the new test.
{code}
+  private void writeDataAndReadFromHFile(Path hfilePath,
+  Algorithm COMPRESS_ALGO, int ENTRY_COUNT, boolean findMidKey) throws 
IOException {
{code}
Please don't use all upper case parameter names.

Please refactor the new readRootIndex() to re-use the existing method.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5851) TestProcessBasedCluster sometimes fails; currently disabled -- needs to be fixed and reenabled

2012-04-24 Thread Jimmy Xiang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260714#comment-13260714
 ] 

Jimmy Xiang commented on HBASE-5851:


@Stack, I tried to run the test in a row for 10 times and all passed.
.out is always empty for me.  There should be something in .log.
When it failed on you, anything in the .log?



 TestProcessBasedCluster sometimes fails; currently disabled -- needs to be 
 fixed and reenabled
 --

 Key: HBASE-5851
 URL: https://issues.apache.org/jira/browse/HBASE-5851
 Project: HBase
  Issue Type: Test
Reporter: Zhihong Yu
Assignee: Jimmy Xiang
 Attachments: disable.txt, hbase-5851.patch, hbase-5851_v2.patch, 
 metahang.txt, zkfail.txt


 TestProcessBasedCluster failed in 
 https://builds.apache.org/job/HBase-TRUNK-security/178
 Looks like cluster failed to start:
 {code}
 2012-04-21 14:22:32,666 INFO  [Thread-1] 
 util.ProcessBasedLocalHBaseCluster(176): Waiting for HBase to startup. 
 Retries left: 2
 java.io.IOException: Giving up trying to location region in meta: thread is 
 interrupted.
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1173)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:956)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:917)
   at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:252)
   at org.apache.hadoop.hbase.client.HTable.init(HTable.java:192)
   at 
 org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:174)
   at 
 org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
   at 
 org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
 java.lang.InterruptedException: sleep interrupted at 
 java.lang.Thread.sleep(Native Method)
   at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:134)
   at 
 org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:178)
   at 
 org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260719#comment-13260719
 ] 

Hudson commented on HBASE-5848:
---

Integrated in HBase-0.94 #143 (See 
[https://builds.apache.org/job/HBase-0.94/143/])
HBASE-5848 Create table with EMPTY_START_ROW passed as splitKey causes the 
HMaster to abort (Ram) (Revision 1329827)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java


 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)

2012-04-24 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260718#comment-13260718
 ] 

Hudson commented on HBASE-5863:
---

Integrated in HBase-0.94 #143 (See 
[https://builds.apache.org/job/HBase-0.94/143/])
HBASE-5863 Improve the graceful_stop.sh CLI help (especially about reloads) 
(Revision 1329794)

 Result = FAILURE
stack : 
Files : 
* /hbase/branches/0.94/bin/graceful_stop.sh


 Improve the graceful_stop.sh CLI help (especially about reloads)
 

 Key: HBASE-5863
 URL: https://issues.apache.org/jira/browse/HBASE-5863
 Project: HBase
  Issue Type: Improvement
  Components: scripts
Affects Versions: 0.94.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-5863.patch


 Right now, graceful_stop.sh prints:
 {code}
 Usage: graceful_stop.sh [--config conf-dir] [--restart] [--reload] 
 [--thrift] [--rest] hostname
  thrift  If we should stop/start thrift before/after the hbase stop/start
  restIf we should stop/start rest before/after the hbase stop/start
  restart If we should restart after graceful stop
  reload  Move offloaded regions back on to the stopped server
  debug   Move offloaded regions back on to the stopped server
  hostnameHostname of server we are to stop
 {code}
 This does not help us specify that reload is actually a sub/additive-option 
 to restart.
 Also, the debug line seems to still have an old copy/paste mistake.
 I've updated these two in the patch here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5862) After Region Close remove the Operation Metrics.

2012-04-24 Thread Elliott Clark (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260720#comment-13260720
 ] 

Elliott Clark commented on HBASE-5862:
--

@Stack yes our per region metrics will be messy if regions move or split or if 
tables are dropped.  

For operation metrics anything reading the jmx can just ignore metrics with 
_NumOps = 0.  That's messy but it works.  For something going forward that's a 
little more general would be nice.  I'll look into it.  

 After Region Close remove the Operation Metrics.
 

 Key: HBASE-5862
 URL: https://issues.apache.org/jira/browse/HBASE-5862
 Project: HBase
  Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Minor
 Attachments: HBASE-5862-0.patch


 If a region is closed then Hadoop metrics shouldn't still be reporting about 
 that region.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread Ramakrishna s vasudevan
The reason is because of HBASE-5848.  I forgot to do the submit patch option, 
so the hadoop QA did not run.
I will rectify the failures . Sorry for breaking the build.

Regards
Ram

From: Hudson (JIRA) [j...@apache.org]
Sent: Tuesday, April 24, 2012 10:49 PM
To: issues@hbase.apache.org
Subject: [jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW 
passed as splitKey causes the HMaster to abort

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260719#comment-13260719
 ]

Hudson commented on HBASE-5848:
---

Integrated in HBase-0.94 #143 (See 
[https://builds.apache.org/job/HBase-0.94/143/])
HBASE-5848 Create table with EMPTY_START_ROW passed as splitKey causes the 
HMaster to abort (Ram) (Revision 1329827)

 Result = FAILURE
ramkrishna :
Files :
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java


 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260725#comment-13260725
 ] 

Lars Hofhansl commented on HBASE-5861:
--

The comment in InputSampler also says that one should use the actual Hadoop 
code in 0.23+ (not the backported code), so I would guess that this would be OK 
as long as there is no compile time error.

If the only problem is TestHLogRecordReader, I should just fix up the test and 
be done with this.


 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260726#comment-13260726
 ] 

stack commented on HBASE-4393:
--

@Ted FYI, if you go under artifacts produced by the build into the target dir, 
you can see the rat.txt now.  Thats where I got the above from.  Thinking on 
it, I also went to change the build order so site goes first so we'll fail fast 
if a rat problem but it seems build is already this way -- must run unit tests 
up front anyways.

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread Ramkrishna.S.Vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260729#comment-13260729
 ] 

Ramkrishna.S.Vasudevan commented on HBASE-5848:
---

The reason is because of HBASE-5848.  I forgot to do the submit patch option, 
so the hadoop QA did not run.
I will rectify the failures . Sorry for breaking the build.

Regards
Ram


 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5848:
--

Status: Patch Available  (was: Reopened)

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, 
 HBASE-5848_0.94.patch, HBASE-5848_addendum.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Reopened] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reopened HBASE-5848:
---


 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, 
 HBASE-5848_0.94.patch, HBASE-5848_addendum.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort

2012-04-24 Thread ramkrishna.s.vasudevan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5848:
--

Attachment: HBASE-5848_addendum.patch

Addendum to pass the failed testcase 
https://builds.apache.org/job/HBase-0.94/143/testReport/junit/org.apache.hadoop.hbase/TestRegionRebalancing/testRebalanceOnRegionServerNumberChange/

 Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to 
 abort
 

 Key: HBASE-5848
 URL: https://issues.apache.org/jira/browse/HBASE-5848
 Project: HBase
  Issue Type: Bug
  Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5848.patch, HBASE-5848.patch, 
 HBASE-5848_0.94.patch, HBASE-5848_addendum.patch


 A coworker of mine just had this scenario. It does not make sense the 
 EMPTY_START_ROW as splitKey (since the region with the empty start key is 
 implicit), but it should not cause the HMaster to abort.
 The abort happens because it tries to bulk assign the same region twice and 
 then runs into race conditions with ZK.
 The same would (presumably) happen when two identical split keys are passed, 
 but the client blocks that. The simplest solution here is to also block 
 passed null or EMPTY_START_ROW as split key by the client.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Zhihong Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260743#comment-13260743
 ] 

Zhihong Yu commented on HBASE-5864:
---

The following computation assumes checksum is on:
{code}
+  int numBytes = (int) ChecksumUtil.numBytes(blk
+  .getOnDiskDataSizeWithHeader(), blk.getBytesPerChecksum());
{code}
If checksum is off, we would get 'divide by 0' exception.

I suggest using HFileBlock.totalChecksumBytes() in place of the above.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94

2012-04-24 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260744#comment-13260744
 ] 

Hadoop QA commented on HBASE-5864:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12524007/HBASE-5864_2.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 5 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestRegionRebalancing
  org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1627//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1627//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1627//console

This message is automatically generated.

 Error while reading from hfile in 0.94
 --

 Key: HBASE-5864
 URL: https://issues.apache.org/jira/browse/HBASE-5864
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0

 Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch, 
 HBASE-5864_test.patch


 Got the following stacktrace during region split.
 {noformat}
 2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store: 
 Failed getting store size for value
 java.io.IOException: Requested block is out of range: 2906737606134037404, 
 lastDataBlockOffset: 84764558
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
   at 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
   at 
 org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
   at 
 org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
   at 
 org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5865) test-util.sh broken with unittest updates

2012-04-24 Thread Jesse Yates (JIRA)
Jesse Yates created HBASE-5865:
--

 Summary: test-util.sh broken with unittest updates
 Key: HBASE-5865
 URL: https://issues.apache.org/jira/browse/HBASE-5865
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.96.0, 0.94.1
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
 Fix For: 0.96.0, 0.94.1


Since the default maven test is meant to be run on the server, this test script 
always fails. Needs to take into account the location of where the script is 
being run as well as some debugging options for future fixes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5865) test-util.sh broken with unittest updates

2012-04-24 Thread Jesse Yates (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-5865:
---

Attachment: sh_HBASE-5865-v0.patch

Attaching patch with the fix. Also adds the option to echo the command that is 
actually run and where the script is being run.

 test-util.sh broken with unittest updates
 -

 Key: HBASE-5865
 URL: https://issues.apache.org/jira/browse/HBASE-5865
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.96.0, 0.94.1
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
 Fix For: 0.96.0, 0.94.1

 Attachments: sh_HBASE-5865-v0.patch


 Since the default maven test is meant to be run on the server, this test 
 script always fails. Needs to take into account the location of where the 
 script is being run as well as some debugging options for future fixes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread Lars Francke (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260751#comment-13260751
 ] 

Lars Francke commented on HBASE-4393:
-

The Canary.java file has a wrong package definition.

Lives in org.apache.hadoop.hbase.tools but says {{package 
org.apache.hadoop.hbase.tool;}}

New issue?

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Created] (HBASE-5866) Canary in tool package but says its in tools.

2012-04-24 Thread stack (JIRA)
stack created HBASE-5866:


 Summary: Canary in tool package but says its in tools.
 Key: HBASE-5866
 URL: https://issues.apache.org/jira/browse/HBASE-5866
 Project: HBase
  Issue Type: Bug
Reporter: stack




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5866) Canary in tool package but says its in tools.

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5866:
-

Attachment: 5866.txt

Spotted by Lars Franke.  Moved Canary into tool package.

 Canary in tool package but says its in tools.
 -

 Key: HBASE-5866
 URL: https://issues.apache.org/jira/browse/HBASE-5866
 Project: HBase
  Issue Type: Bug
Reporter: stack
 Fix For: 0.94.0, 0.96.0

 Attachments: 5866.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5866) Canary in tool package but says its in tools.

2012-04-24 Thread Lars Francke (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260767#comment-13260767
 ] 

Lars Francke commented on HBASE-5866:
-

That was fast, thanks stack.

 Canary in tool package but says its in tools.
 -

 Key: HBASE-5866
 URL: https://issues.apache.org/jira/browse/HBASE-5866
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0, 0.96.0

 Attachments: 5866.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5866) Canary in tool package but says its in tools.

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5866.
--

   Resolution: Fixed
Fix Version/s: 0.96.0
   0.94.0
 Assignee: stack

Committed trunk and 0.94.

 Canary in tool package but says its in tools.
 -

 Key: HBASE-5866
 URL: https://issues.apache.org/jira/browse/HBASE-5866
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0, 0.96.0

 Attachments: 5866.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-4393) Implement a canary monitoring program

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260766#comment-13260766
 ] 

stack commented on HBASE-4393:
--

Let me take care of it Lars.  I should have seen that.  Thanks for pointing it 
out.  Fixed over in HBASE-5866.

 Implement a canary monitoring program
 -

 Key: HBASE-4393
 URL: https://issues.apache.org/jira/browse/HBASE-4393
 Project: HBase
  Issue Type: New Feature
  Components: monitoring
Affects Versions: 0.92.0
Reporter: Todd Lipcon
Assignee: Matteo Bertozzi
 Fix For: 0.94.0, 0.96.0

 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java


 This JIRA is to implement a standalone program that can be used to do canary 
 monitoring of a running HBase cluster. This program would gather a list of 
 the regions in the cluster, then iterate over them doing lightweight 
 operations (eg short scans) to provide metrics about latency as well as alert 
 on availability issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HBASE-5865) test-util.sh broken with unittest updates

2012-04-24 Thread stack (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-5865.
--

  Resolution: Fixed
Hadoop Flags: Reviewed

Tried it. Seems to work.  Committed trunk and 0.94.  Thanks for the patch Jesse.

 test-util.sh broken with unittest updates
 -

 Key: HBASE-5865
 URL: https://issues.apache.org/jira/browse/HBASE-5865
 Project: HBase
  Issue Type: Bug
  Components: scripts
Affects Versions: 0.96.0, 0.94.1
Reporter: Jesse Yates
Assignee: Jesse Yates
Priority: Minor
 Fix For: 0.96.0, 0.94.1

 Attachments: sh_HBASE-5865-v0.patch


 Since the default maven test is meant to be run on the server, this test 
 script always fails. Needs to take into account the location of where the 
 script is being run as well as some debugging options for future fixes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064

2012-04-24 Thread Jonathan Hsieh (JIRA)

 [ 
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5861:
--

Attachment: hbase-5861-v3.patch

v3 is a cleaned up version, and doesn't touch InputSampler.

 Hadoop 23 compile broken due to tests introduced in HBASE-5064 
 ---

 Key: HBASE-5861
 URL: https://issues.apache.org/jira/browse/HBASE-5861
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
 Fix For: 0.94.0, 0.96.0

 Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch, 
 hbase-5861-v3.patch


 When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of 
 compilation error messages:
 {code}
 jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
 ...
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 18.926s
 [INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
 [INFO] Final Memory: 55M/555M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile 
 (default-testCompile) on project hbase: Compilation failure: Compilation 
 failure:
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
  org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
 [ERROR] 
 [ERROR] 
 /home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
  org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be 
 instantiated
 [ERROR] - [Help 1]
 {code}
 Upon further investigation this issue is due to code introduced in HBASE-5064 
 and is also present in trunk.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status

2012-04-24 Thread stack (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260774#comment-13260774
 ] 

stack commented on HBASE-5840:
--

Patch looks good.  Is it just moving all of this (too long) method into a try 
block and then adding a finally that sets status.abort on the end (FYI, needs 
spacing around the operators in the status.abort line so its same code style as 
rest of file).  Do you have to convert the Exception to an IOE?  WHy is that?  
What does this method let out?  IOEs only?  If so, why we catch Exception?  In 
case its a non-checked exception?   On the test, it looks good too but in the 
finally you might want to use the new HRegion.closeHRegion(region) to clean up 
the wal log that gets made by the constructor.

 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing 
 the old status
 --

 Key: HBASE-5840
 URL: https://issues.apache.org/jira/browse/HBASE-5840
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: rajeshbabu
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5840.patch


 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will 
 keeps showing old status.
 This will miss leads the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




  1   2   3   >