date:20120415


 [ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5781:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Anoop, Kristam, thanks for finding this and hunting down the root cause of the 
problem.  Thanks for the quick review Lars.

Committed to 0.90/0.92/0.94.  Trunk is broken currently because of HBASE-5747, 
HBASE-5793.

 Zookeeper session got closed while trying to assign the region to RS using 
 hbck -fix
 

 Key: HBASE-5781
 URL: https://issues.apache.org/jira/browse/HBASE-5781
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5781.patch


 After running the hbck in the cluster ,it is found that one region is not 
 assigned
 So the hbck -fix is used to fix this 
 But the assignment didnt happen since the zookeeper session is closed
 Please find the attached trace for more details
 -
 Trying to fix unassigned region...
 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
 waiting for it to become assigned: {NAME = 
 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY = 
 '002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
 Closed zookeeper sessionid=0x236738a263a
 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
 ERROR: Region { meta = 
 ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs = 
 hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
 = } not deployed on any region server.
 Trying to fix unassigned region...
 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
 to set watcher on znode (/hbase)
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
 at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
 at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
 hconnection-0x236738a263a Received unexpected KeeperException, 
 re-throwing exception
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

2012-04-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254287#comment-13254287
 ] 

Jonathan Hsieh commented on HBASE-5781:
---

Oh -- but it is committed to trunk as well.

 Zookeeper session got closed while trying to assign the region to RS using 
 hbck -fix
 

 Key: HBASE-5781
 URL: https://issues.apache.org/jira/browse/HBASE-5781
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5781.patch


 After running the hbck in the cluster ,it is found that one region is not 
 assigned
 So the hbck -fix is used to fix this 
 But the assignment didnt happen since the zookeeper session is closed
 Please find the attached trace for more details
 -
 Trying to fix unassigned region...
 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
 waiting for it to become assigned: {NAME = 
 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY = 
 '002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
 Closed zookeeper sessionid=0x236738a263a
 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
 ERROR: Region { meta = 
 ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs = 
 hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
 = } not deployed on any region server.
 Trying to fix unassigned region...
 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
 to set watcher on znode (/hbase)
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
 at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
 at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
 hconnection-0x236738a263a Received unexpected KeeperException, 
 re-throwing exception
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix


[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254291#comment-13254291
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-TRUNK #2763 (See 
[https://builds.apache.org/job/HBase-TRUNK/2763/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326280)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


 Zookeeper session got closed while trying to assign the region to RS using 
 hbck -fix
 

 Key: HBASE-5781
 URL: https://issues.apache.org/jira/browse/HBASE-5781
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5781.patch


 After running the hbck in the cluster ,it is found that one region is not 
 assigned
 So the hbck -fix is used to fix this 
 But the assignment didnt happen since the zookeeper session is closed
 Please find the attached trace for more details
 -
 Trying to fix unassigned region...
 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
 waiting for it to become assigned: {NAME = 
 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY = 
 '002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
 Closed zookeeper sessionid=0x236738a263a
 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
 ERROR: Region { meta = 
 ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs = 
 hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
 = } not deployed on any region server.
 Trying to fix unassigned region...
 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
 to set watcher on znode (/hbase)
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
 at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
 at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
 hconnection-0x236738a263a Received unexpected KeeperException, 
 re-throwing exception
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix


[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254293#comment-13254293
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-0.94 #117 (See 
[https://builds.apache.org/job/HBase-0.94/117/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326281)

 Result = FAILURE
jmhsieh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


 Zookeeper session got closed while trying to assign the region to RS using 
 hbck -fix
 

 Key: HBASE-5781
 URL: https://issues.apache.org/jira/browse/HBASE-5781
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5781.patch


 After running the hbck in the cluster ,it is found that one region is not 
 assigned
 So the hbck -fix is used to fix this 
 But the assignment didnt happen since the zookeeper session is closed
 Please find the attached trace for more details
 -
 Trying to fix unassigned region...
 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
 waiting for it to become assigned: {NAME = 
 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY = 
 '002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
 Closed zookeeper sessionid=0x236738a263a
 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
 ERROR: Region { meta = 
 ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs = 
 hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
 = } not deployed on any region server.
 Trying to fix unassigned region...
 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
 to set watcher on znode (/hbase)
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
 at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
 at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
 hconnection-0x236738a263a Received unexpected KeeperException, 
 re-throwing exception
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at

[jira] [Updated] (HBASE-5787) Table owner can't disable/delete its own table

2012-04-15 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5787:
---

Attachment: HBASE-5787-tests-wrong-names.patch

Test seems ok, (tested against 0.94)
I've also attached a patch to fix some copy-paste code.
{code}
Running org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.659 sec
Running org.apache.hadoop.hbase.security.access.TestAccessControlFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.746 sec
Running org.apache.hadoop.hbase.security.access.TestAccessController
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.119 sec
Running org.apache.hadoop.hbase.security.access.TestTablePermissions
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.822 sec

Results :
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0
{code}

 Table owner can't disable/delete its own table
 --

 Key: HBASE-5787
 URL: https://issues.apache.org/jira/browse/HBASE-5787
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl, security
 Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch


 An user with CREATE privileges can create a table, but can not disable it, 
 because disable operation require ADMIN privileges. Also if a table is 
 already disabled, anyone can remove it.
 {code}
 public void preDeleteTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   requirePermission(Permission.Action.CREATE);
 }
 public void preDisableTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   /* TODO: Allow for users with global CREATE permission and the table owner 
 */
   requirePermission(Permission.Action.ADMIN);
 }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254301#comment-13254301
 ] 

Ferdy Galema commented on HBASE-2214:
-

Ok thanks for your comments. I just submitted new patch to review board. 
https://reviews.apache.org/r/4726/

So what's protocol here, can I add 'hbase' to reviewers group or just add 
individual reviewers?

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

2012-04-15 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254307#comment-13254307
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-0.92 #371 (See 
[https://builds.apache.org/job/HBase-0.92/371/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326282)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


 Zookeeper session got closed while trying to assign the region to RS using 
 hbck -fix
 

 Key: HBASE-5781
 URL: https://issues.apache.org/jira/browse/HBASE-5781
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5781.patch


 After running the hbck in the cluster ,it is found that one region is not 
 assigned
 So the hbck -fix is used to fix this 
 But the assignment didnt happen since the zookeeper session is closed
 Please find the attached trace for more details
 -
 Trying to fix unassigned region...
 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
 waiting for it to become assigned: {NAME = 
 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY = 
 '002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
 Closed zookeeper sessionid=0x236738a263a
 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
 ERROR: Region { meta = 
 ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs = 
 hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
 = } not deployed on any region server.
 Trying to fix unassigned region...
 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
 to set watcher on znode (/hbase)
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
 at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
 at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
 at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
 at 
 org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
 at 
 org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
 at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
 at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
 hconnection-0x236738a263a Received unexpected KeeperException, 
 re-throwing exception
 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
 = Session expired for /hbase
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
 at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
 at 
 org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
 at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
 at 
 org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
 at

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254329#comment-13254329
 ] 

jirapos...@reviews.apache.org commented on HBASE-2214:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4726/
---

(Updated 2012-04-15 13:38:31.680141)


Review request for hbase and Ted Yu.


Summary
---

HBASE-2214 per scan max buffersize.


This addresses bug HBASE-2214.
https://issues.apache.org/jira/browse/HBASE-2214


Diffs
-

  /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 
1326084 
  
/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
 1326084 

Diff: https://reviews.apache.org/r/4726/diff


Testing
---


Thanks,

ferdy



 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254330#comment-13254330
 ] 

Ferdy Galema commented on HBASE-2214:
-

Secondly, I'd like the uploaded diff to be compared to the 0.94 branch. How can 
I specify this? It seems it is always compared to trunk?

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254334#comment-13254334
 ] 

Zhihong Yu commented on HBASE-2214:
---

'hbase' group should be included in each review request.
Normally patch for trunk is generated first and gets reviewed on review board. 
Hadoop QA checks out patch against trunk.

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254335#comment-13254335
 ] 

Zhihong Yu commented on HBASE-2214:
---

I went over https://reviews.apache.org/r/4726 which looks clean.

If adding unit test is difficult, can you perform verification on real clsuter 
and let us know the results ?

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5787) Table owner can't disable/delete its own table


[ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254336#comment-13254336
 ] 

Zhihong Yu commented on HBASE-5787:
---

Thanks for the update.
Can you combine the two patches and run through tests again, including the 
following two ?
{code}
security/src/test//java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
security/src/test//java/org/apache/hadoop/hbase/security/token/TestZKSecretWatcher.java
{code}

 Table owner can't disable/delete its own table
 --

 Key: HBASE-5787
 URL: https://issues.apache.org/jira/browse/HBASE-5787
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl, security
 Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch


 An user with CREATE privileges can create a table, but can not disable it, 
 because disable operation require ADMIN privileges. Also if a table is 
 already disabled, anyone can remove it.
 {code}
 public void preDeleteTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   requirePermission(Permission.Action.CREATE);
 }
 public void preDisableTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   /* TODO: Allow for users with global CREATE permission and the table owner 
 */
   requirePermission(Permission.Action.ADMIN);
 }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5787) Table owner can't disable/delete its own table

2012-04-15 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5787:
---

Attachment: HBASE-5787-v1.patch

{code}
Running org.apache.hadoop.hbase.security.token.TestTokenAuthentication
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.765 sec
Running org.apache.hadoop.hbase.security.token.TestZKSecretWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.958 sec

Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
{code}

 Table owner can't disable/delete its own table
 --

 Key: HBASE-5787
 URL: https://issues.apache.org/jira/browse/HBASE-5787
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: acl, security
 Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch, 
 HBASE-5787-v1.patch


 An user with CREATE privileges can create a table, but can not disable it, 
 because disable operation require ADMIN privileges. Also if a table is 
 already disabled, anyone can remove it.
 {code}
 public void preDeleteTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   requirePermission(Permission.Action.CREATE);
 }
 public void preDisableTable(ObserverContextMasterCoprocessorEnvironment c,
 byte[] tableName) throws IOException {
   /* TODO: Allow for users with global CREATE permission and the table owner 
 */
   requirePermission(Permission.Action.ADMIN);
 }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254338#comment-13254338
 ] 

Ferdy Galema commented on HBASE-2214:
-

Sure I will do that.

Some patch segments do not apply to trunk so I will first create a patch for 
trunk.

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254339#comment-13254339
 ] 

Ferdy Galema commented on HBASE-2214:
-

(Nevermind the latest comment. It seems that with a fuzz factor it does apply.)

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
 Fix For: 0.94.1

 Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: 5795-v1.txt

Since only deserialization needs special handling, the attached patch adds a 
private method to read 0.92 RegionLoad.

Please comment.

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Attachments: 5794.txt, 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

2012-04-15 Thread Jimmy Xiang (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254360#comment-13254360
 ] 

Jimmy Xiang commented on HBASE-5620:


Thanks for reviewing. Both regular test suite and security test suite are green 
for me. I mean all tests in the suite.

 Convert the client protocol of HRegionInterface to PB
 -

 Key: HBASE-5620
 URL: https://issues.apache.org/jira/browse/HBASE-5620
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
 Fix For: 0.96.0

 Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
 hbase-5620_v4.patch, hbase-5620_v4.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB


[ 
https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254364#comment-13254364
 ] 

jirapos...@reviews.apache.org commented on HBASE-5621:
--



bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, 
line 2814
bq.   
https://reviews.apache.org/r/4714/diff/1/?file=101799#file101799line2814
bq.  
bq.   Where has allt his code gone?
bq.  
bq.  Jimmy Xiang wrote:
bq.  Some are moved to RegionServer as I did for 5620
bq.  
bq.  Michael Stack wrote:
bq.  But I did not see it in the patch?  Its already moved?

It should be in the patch, probably in the second page of touched files.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/ClientProtocol.java, line 29
bq.   https://reviews.apache.org/r/4714/diff/1/?file=101781#file101781line29
bq.  
bq.   Ditto
bq.  
bq.  Jimmy Xiang wrote:
bq.  Will move to client.
bq.  
bq.  Michael Stack wrote:
bq.  Is this only used out of the client package?  If so, yeah, move it 
there I'd say.

It is used out of the client package, and the server side implements it.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/AdminProtocol.java, line 28
bq.   https://reviews.apache.org/r/4714/diff/1/?file=101780#file101780line28
bq.  
bq.   Is this right?  It does more than talk to a regionserver?  You have 
to have really nice comments on your class now that you are at the top level 
Jimmy (smile)
bq.   
bq.   This class is only used by HBaseAdmin?  Is that right?  Or do other 
classes use it?  If only HBaseAdmin, maybe it should not be top level?
bq.  
bq.  Jimmy Xiang wrote:
bq.  Let me move it to client.
bq.  
bq.  Michael Stack wrote:
bq.  And it should not be public I'd say.

Yes, it is private.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java, line 
632
bq.   https://reviews.apache.org/r/4714/diff/1/?file=101782#file101782line632
bq.  
bq.   Do we have to have protobuf stuff sprinkled all over the code base?  
Its not too bad but just wondering.  Maybe we do but just wondering.  I suppose 
there is nothing to do about it.  If we did same operation over and over w/ 
some pb stuff and the output was a non-pb object, then yes, we could hide the 
pb stuff over in a class but what we do here is particular to this method 
Can't generalize.
bq.   
bq.   I do see that you have added some static methods into protobuf 
utils.  This one is not generic so doesn't deserve to go there?
bq.  
bq.  Jimmy Xiang wrote:
bq.  For getRegionInfo, I used to put it in the protobuf util since it is 
used lots of places.  I will move it to protobuf util.
bq.  
bq.  Michael Stack wrote:
bq.  Agree.  If used more than once, move it over to protobuf util 
otherwise I don't see a way around not importing protobuf classes when its a 
particular usage.

Cool, will fix.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.   src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 32
bq.   https://reviews.apache.org/r/4714/diff/1/?file=101784#file101784line32
bq.  
bq.   Did you say you were going to remove these from here?
bq.  
bq.  Jimmy Xiang wrote:
bq.  I tried but it is hard.  Let me think about it again.
bq.  
bq.  Michael Stack wrote:
bq.  Even if this stuff just moved to HCM, it'd give us a clean HConnection.

I was thinking to have a ProtocolFactory/Manager to manage the protocols, and 
let the connection manager to many connections only.
Let me move them to HCM for now.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4714/#review6922
---


On 2012-04-13 23:03:35, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4714/
bq.  ---
bq.  
bq.  (Updated 2012-04-13 23:03:35)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the admin part of HBase-5443.  AdminProtocol part.
bq.  
bq.  
bq.  This addresses bug HBASE-5621.
bq.  https://issues.apache.org/jira/browse/HBASE-5621
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/AdminProtocol.java PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/ClientProtocol.java PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 
408db79 
bq.src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java ee16e72 
bq.

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747


[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254369#comment-13254369
 ] 

Zhihong Yu commented on HBASE-5793:
---

TestHBaseFsck passes in PreCommit build #1530.
But there still was some other hanging test(s).

 TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
 -

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

2012-04-15 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254371#comment-13254371
 ] 

Zhihong Yu commented on HBASE-5792:
---

Interesting tool.
{code}
+region = HRegion.createHRegion(regionInfo, regionRootDir, getConf(), htd);
{code}
I don't see where region is closed.
There should be some cleanup method after the benchmarking is completed.

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5792) HLog Performance Evaluation Tool

2012-04-15 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5792:
---

Attachment: HBASE-5792-v1.patch

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.


 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Attachment: HBASE-5737_3.patch

Latest patch as per Stack's suggestion.

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
 HBASE-5737_2.patch, HBASE-5737_3.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5782) Not all the regions are getting assigned after the log splitting.

2012-04-15 Thread ramkrishna.s.vasudevan (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-5782:
-

Assignee: ramkrishna.s.vasudevan

 Not all the regions are getting assigned after the log splitting.
 -

 Key: HBASE-5782
 URL: https://issues.apache.org/jira/browse/HBASE-5782
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0


 Create a table with 1000 splits, after the region assignemnt, kill the 
 regionserver wich contains META table.
 Here few regions are missing after the log splitting and region assigment. 
 HBCK report shows multiple region holes are got created.
 Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.

2012-04-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254385#comment-13254385
 ] 

ramkrishna.s.vasudevan commented on HBASE-5782:
---

@LArs
Tomorrow i can try writing a test with similar scenario.  
bq.(Although that might be difficult)
I think it should be easier.  Hopefully tomorrow can come up with some initial 
analysis.

 Not all the regions are getting assigned after the log splitting.
 -

 Key: HBASE-5782
 URL: https://issues.apache.org/jira/browse/HBASE-5782
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0


 Create a table with 1000 splits, after the region assignemnt, kill the 
 regionserver wich contains META table.
 Here few regions are missing after the log splitting and region assigment. 
 HBCK report shows multiple region holes are got created.
 Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254393#comment-13254393
]

Hadoop QA commented on HBASE-5792:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522711/HBASE-5792-v1.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//console

This message is automatically generated.

HLog Performance Evaluation Tool

Key: HBASE-5792
URL: https://issues.apache.org/jira/browse/HBASE-5792
Project: HBase
Issue Type: Test
Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: performance, wal
Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch

Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5737) Minor Improvements related to balancer.

[
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254399#comment-13254399
]

Hadoop QA commented on HBASE-5737:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522713/HBASE-5737_3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//console

This message is automatically generated.

Minor Improvements related to balancer.
---

Key: HBASE-5737
URL: https://issues.apache.org/jira/browse/HBASE-5737
Project: HBase
Issue Type: Improvement
Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Attachments: HBASE-5737.patch, HBASE-5737_1.patch,
HBASE-5737_2.patch, HBASE-5737_3.patch

Currently in Am.getAssignmentByTable() we use a result map which is currenly
a hashmap. It could be better if we have a treeMap. Even in
MetaReader.fullScan we have the treeMap only so that we have the naming order
maintained. I felt this change could be very useful in cases where we are
extending the DefaultLoadBalancer.

[jira] [Updated] (HBASE-5256) Use WritableUtils.readVInt() in RegionLoad.readFields()

2012-04-15 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5256:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

This was committed, marking it accordingly.

 Use WritableUtils.readVInt() in RegionLoad.readFields()
 ---

 Key: HBASE-5256
 URL: https://issues.apache.org/jira/browse/HBASE-5256
 Project: HBase
  Issue Type: Task
Reporter: Zhihong Yu
Assignee: Mubarak Seyed
 Fix For: 0.94.0

 Attachments: HBASE-5256.trunk.v1.patch


 Currently in.readInt() is used in RegionLoad.readFields()
 More metrics would be added to RegionLoad in the future, we should utilize 
 WritableUtils.readVInt() to reduce the amount of data exchanged between 
 Master and region servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5795:
-

Fix Version/s: 0.94.0

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5794.txt, 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254414#comment-13254414
 ] 

Lars Hofhansl commented on HBASE-5795:
--

+1 om patch.

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5794.txt, 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254414#comment-13254414
 ] 

Lars Hofhansl edited comment on HBASE-5795 at 4/15/12 7:31 PM:
---

+1 on patch.

  was (Author: lhofhansl):
+1 om patch.
  
 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5794.txt, 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747

2012-04-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254420#comment-13254420
 ] 

Jonathan Hsieh commented on HBASE-5793:
---

Zhihong yeah, I broke this out to a separate issue because I was fixing other 
TestHBaseFsck problems, wanted test to pass, but didn't yet spend the effort to 
figure out the rest of the HBASE-5747 problems.

Since I got your confirmation and since this is a minor test update, I'm just 
going to commit.

 TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
 -

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased


 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retries 
increased  (was: TestHBaseFsck#TestNoHdfsTable test hangs after client retires 
increased)

 TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased


 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retires 
increased  (was: TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747)

 TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool


[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254426#comment-13254426
 ] 

Zhihong Yu commented on HBASE-5792:
---

Minor comments:
{code}
import org.apache.hadoop.conf.Configuration;
{code}
The above import was not used.
{code}
for (Thread t : threads) t.join();
{code}
join() may throw InterruptedException. Shall we catch it and proceed to the 
next Thread to be joined ?

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

2012-04-15 Thread Matteo Bertozzi (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254428#comment-13254428
 ] 

Matteo Bertozzi commented on HBASE-5792:


Don't know if it makes sense catch the InterruptException, since we're trying 
to test write performance with nThreads... if we lost one thread, the test 
doesn't reflect what we've asked for.

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5795:
-

Attachment: (was: 5794.txt)

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254429#comment-13254429
 ] 

Lars Hofhansl commented on HBASE-5795:
--

@Stack: I think you forgot to include the actual test in the patch :)

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254430#comment-13254430
 ] 

Lars Hofhansl commented on HBASE-5795:
--

Oh, you attached your 5794 patch to this issue. I removed to avoid confusing, 
when you get a chance could you attach your patch here? Then we can make a 
combined patch with that and Ted's fix.

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v1.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.


[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254433#comment-13254433
 ] 

Lars Hofhansl commented on HBASE-5782:
--

Thanks Ram!

 Not all the regions are getting assigned after the log splitting.
 -

 Key: HBASE-5782
 URL: https://issues.apache.org/jira/browse/HBASE-5782
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
 Fix For: 0.94.0


 Create a table with 1000 splits, after the region assignemnt, kill the 
 regionserver wich contains META table.
 Here few regions are missing after the log splitting and region assigment. 
 HBCK report shows multiple region holes are got created.
 Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool


[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254436#comment-13254436
 ] 

Zhihong Yu commented on HBASE-5792:
---

I wonder if it makes sense to persist benchmark results to HBase.
That may show us some trend w.r.t. HLog performance.

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

2012-04-15 Thread stack (Created) (JIRA)

Fix our abuse of IOE: see 
http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack


Lets make more context particular exceptions rather than throw IOEs everywhere. 
 See Benoît's rant: 
http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5795:
-

Attachment: 5795.unittest.txt

unit test

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v1.txt, 5795.unittest.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

2012-04-15 Thread stack (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254441#comment-13254441
 ] 

stack commented on HBASE-5795:
--

I looked at Ted's patch.  That should do it.  See if it makes the unit test 
pass I'd say.  I can test on cluster tomorrow morning (will also finish my 
rolling restart and kill of meta on a cluster w/ 1k regions too...)

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v1.txt, 5795.unittest.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased


 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

   Resolution: Fixed
Fix Version/s: 0.96.0
   0.94.0
   0.92.2
   0.90.7
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to 0.90/0.92/0.94/0.96/

 TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

2012-04-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254448#comment-13254448
 ] 

Jonathan Hsieh commented on HBASE-5747:
---

@stack Does removing the retries cause more errors to show up does it remain 
the same?  

 Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
 subdirectory of target/test
 

 Key: HBASE-5747
 URL: https://issues.apache.org/jira/browse/HBASE-5747
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.96.0

 Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, 
 5708v4.txt, 5708v4.txt


 Forward port as much as we can of Mikhail's hard-won test cleanups over on 
 0.89 branch  Will improve our being able to run unit tests in //.  He also 
 found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased


[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254461#comment-13254461
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-TRUNK #2765 (See 
[https://builds.apache.org/job/HBase-TRUNK/2765/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable test hangs after client retries 
increased (Revision 1326434)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased


[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254464#comment-13254464
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-0.94 #119 (See 
[https://builds.apache.org/job/HBase-0.94/119/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable tests hangs after client retries 
increased (Revision 1326435)

 Result = SUCCESS
jmhsieh : 
Files : 
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html


[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254470#comment-13254470
 ] 

Lars Hofhansl commented on HBASE-5796:
--

Could be as simple as a better hierarchy of classes extending IOException (like 
DoNotRetryException).

 Fix our abuse of IOE: see 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
 ---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack

 Lets make more context particular exceptions rather than throw IOEs 
 everywhere.  See Benoît's rant: 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: (was: 5795-v1.txt)

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795.unittest.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

2012-04-15 Thread Benoit Sigoure (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254474#comment-13254474
 ] 

Benoit Sigoure commented on HBASE-5796:
---

I propose the following, as a temporary stopgap solution:
# Create a new class that inherits from {{IOException}}, call it 
{{HBaseException}}.
# Change *all* HBase exceptions to inherit from them.  Replace all bare {{new 
IOException}} with {{new HBaseException}}.
# Classify all exceptions to be under two categories: what can be potentially 
retried, and what cannot (this partially exists today under 
{{DoNotRetryException}} as Lars points out).
# At this point, nothing should be a bare {{HBaseException}}, and nothing 
should be creating bare {{IOException}}s either.
# Update all APIs to specify exceptions thrown with a type as specific as 
possible.

I believe most (if not all) of the steps above can be done while preserving API 
compatibility (and maybe even ABI compatibility).

This would be a good first pass to partially clean up some of the mess we have 
today.  Also, because Hadoop is even worse than HBase, and because it's 
changing slowly, your code will still be crippled with {{IOException}} from 
Hadoop.  I'd recommend wrapping them all in a {{HadoopIOException}}, which 
would inherit from {{HBaseException}}.

Ultimately, I would *strongly* encourage you to move away from {{IOException}} 
altogether.  Not only because it's the wrong exception to use in most cases 
where it's used in Hadoop/HBase, but also because it's a checked exception.  As 
I wrote in my blog post, checked exceptions are like communism: they sound like 
a good idea, but they horribly fail in practice.  Here's a short but good 
article that conveys well two of the biggest show-stoppers with checked 
exceptions: http://www.artima.com/intv/handcuffsP.html – I'm so happy that 
Scala didn't pick this up from Java.  Checked exceptions almost always wind up 
defeating their own purpose.  I'm sure you guys have written enough code in big 
enough code bases to experience this too.

 Fix our abuse of IOE: see 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
 ---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack

 Lets make more context particular exceptions rather than throw IOEs 
 everywhere.  See Benoît's rant: 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: 5795-v2.txt

Patch v1 didn't make testHServerLoadVersioning pass.

Patch v2 does.
I found that the version of RegionLoad was actually serialized twice in 0.92: 
first by VersionedWritable.write(), followed by RegionLoad.write().
In patch v2, I removed the redundant write. readFields92() consumes the second 
copy of version.

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v2.txt, 5795.unittest.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility


 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Status: Patch Available  (was: Open)

 hbase-3927 breaks 0.92-0.94 compatibility
 ---

 Key: HBASE-5795
 URL: https://issues.apache.org/jira/browse/HBASE-5795
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: stack
 Fix For: 0.94.0

 Attachments: 5795-v2.txt, 5795.unittest.txt


 This commit broke our 0.92/0.94 compatibility:
 {code}
 
 r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
 HBASE-3927 display total uncompressed byte size of a region in web UI
 {code}
 I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
 cluster and rather than just digest version 1 of the HServerLoad, I get this:
 {code}
 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
 read call parameters for client 10.4.14.38
 java.io.IOException: Error in readFields
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
 at 
 org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
 at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 Caused by: A record version mismatch occured. Expecting v2, found v1
 at 
 org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
 at 
 org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
 at 
 org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
 at 
 org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
 ... 9 more
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html


[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254476#comment-13254476
 ] 

Zhihong Yu commented on HBASE-5796:
---

bq. all in a {{HadoopIOException}}, which would inherit from {{HBaseException}}.
I am afraid that some people would not feel comfortable with the above 
inheritance.

 Fix our abuse of IOE: see 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
 ---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack

 Lets make more context particular exceptions rather than throw IOEs 
 everywhere.  See Benoît's rant: 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

2012-04-15 Thread Benoit Sigoure (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254477#comment-13254477
 ] 

Benoit Sigoure commented on HBASE-5796:
---

Just to be clear: the wrapping of Hadoop's {{IOException}} would be for a first 
pass, to make sure everything HBase throws inherits from an {{HBaseException}}. 
 Ideally it would be better to pick a more specific type of exception, but I 
expect this will be difficult in various places where Hadoop suffers from the 
same problem and HBase itself has little to no visibility into what the 
exception from Hadoop really means.

Also I'd recommend to actually use {{HBaseIOException}} as a base class name, 
and keep the name {{HBaseException}} for when you guys switch to unchecked 
exceptions, if you're willing to consider going down that route eventually 
(that'd require a separate JIRA issue).

Since HBase 0.96 is the singularity, it's a good opportunity to break API 
compatibility.  In this case, every release up to and including 0.94.x would 
have {{HBaseIOException}} as the parent class, and in 0.96 the only thing you 
need to change is to make everything inherit from {{HBaseException}} (which 
itself would inherit from {{RuntimeException}}).

 Fix our abuse of IOE: see 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
 ---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack

 Lets make more context particular exceptions rather than throw IOEs 
 everywhere.  See Benoît's rant: 
 http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased


[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254482#comment-13254482
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-0.92 #372 (See 
[https://builds.apache.org/job/HBase-0.92/372/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable test hangs after client retries 
increased (Revision 1326436)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
 ---

 Key: HBASE-5793
 URL: https://issues.apache.org/jira/browse/HBASE-5793
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

 Attachments: hbase-5793.patch


 After the HBASE-5747 modification, this one particular case hangs.
 {code}
 mvn test -PlocalTests -Dtest=TestHBaseFsck
 {code}
 It was hanging on a scan of a table that the test deleted. It expected a call 
 to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
 larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5747) Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test

2012-04-15 Thread Zhihong Yu (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu resolved HBASE-5747.
---

Resolution: Fixed

Trunk build passes twice.

 Forward port hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
 subdirectory of target/test
 

 Key: HBASE-5747
 URL: https://issues.apache.org/jira/browse/HBASE-5747
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
Priority: Blocker
 Fix For: 0.96.0

 Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, 
 5708v4.txt, 5708v4.txt


 Forward port as much as we can of Mikhail's hard-won test cleanups over on 
 0.89 branch  Will improve our being able to run unit tests in //.  He also 
 found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility

[
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254484#comment-13254484
]

Hadoop QA commented on HBASE-5795:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522732/5795-v2.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//console

This message is automatically generated.

hbase-3927 breaks 0.92-0.94 compatibility
---

Key: HBASE-5795
URL: https://issues.apache.org/jira/browse/HBASE-5795
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.94.0

Attachments: 5795-v2.txt, 5795.unittest.txt

This commit broke our 0.92/0.94 compatibility:
{code}

r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
HBASE-3927 display total uncompressed byte size of a region in web UI
{code}
I just tried the new RC for 0.94. I brought up a 0.94 master on a 0.92
cluster and rather than just digest version 1 of the HServerLoad, I get this:
{code}
2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to
read call parameters for client 10.4.14.38
java.io.IOException: Error in readFields
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
at
org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: A record version mismatch occured. Expecting v2, found v1
at
org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
at
org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
at
org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
... 9 more
{code}

[jira] [Created] (HBASE-5797) Add the active master address on the web of regionserver

2012-04-15 Thread chunhui shen (Created) (JIRA)

Add the active master address on the web of regionserver


 Key: HBASE-5797
 URL: https://issues.apache.org/jira/browse/HBASE-5797
 Project: HBase
  Issue Type: Improvement
Reporter: chunhui shen
Assignee: chunhui shen


Add the active master address on the web of regionserver, so we can easily know 
the master server name and jump to master web from regionserver web.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

2012-04-15 Thread Todd Lipcon (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254496#comment-13254496
 ] 

Todd Lipcon commented on HBASE-5792:


bq. I wonder if it makes sense to persist benchmark results to HBase.

Let's not scope creep. The point of this tool is to have a quick benchmark you 
can run without having to set up an HBase cluster. Outputting results on the 
console is good enough to parse and store in whatever database you want. My 
preference would probably be just appending to a text file for easy graphing 
from Hudson.

 HLog Performance Evaluation Tool
 

 Key: HBASE-5792
 URL: https://issues.apache.org/jira/browse/HBASE-5792
 Project: HBase
  Issue Type: Test
  Components: wal
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
  Labels: performance, wal
 Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch


 Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
 It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
 to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5797) Add the active master address on the web of regionserver

2012-04-15 Thread chunhui shen (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen resolved HBASE-5797.
-

Resolution: Duplicate

Sorry, I find it has been done in trunk.

 Add the active master address on the web of regionserver
 

 Key: HBASE-5797
 URL: https://issues.apache.org/jira/browse/HBASE-5797
 Project: HBase
  Issue Type: Improvement
Reporter: chunhui shen
Assignee: chunhui shen

 Add the active master address on the web of regionserver, so we can easily 
 know the master server name and jump to master web from regionserver web.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism

2012-04-15 Thread Phabricator (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254498#comment-13254498
 ] 

Phabricator commented on HBASE-5104:


mbautin has commented on the revision [jira] [HBASE-5104] Provide a reliable 
intra-row pagination mechanism.

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/Get.java:470 Removed.
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:395 Done.
  src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide2.java:38 Done.

REVISION DETAIL
  https://reviews.facebook.net/D2799


 Provide a reliable intra-row pagination mechanism
 -

 Key: HBASE-5104
 URL: https://issues.apache.org/jira/browse/HBASE-5104
 Project: HBase
  Issue Type: Bug
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
 Attachments: D2799.1.patch, D2799.2.patch, testFilterList.rb


 Addendum:
 Doing pagination (retrieving at most limit number of KVs at a particular 
 offset) is currently supported via the ColumnPaginationFilter. However, it 
 is not a very clean way of supporting pagination.  Some of the problems with 
 it are:
 * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
 same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
 is not the case for ColumnPaginationFilter as its internal state gets updated 
 depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
 cell.
 * When this Filter is used in combination with other filters (e.g., doing AND 
 with another filter using FilterList), the behavior of the query depends on 
 the order of filters in the FilterList. This is not ideal.
 * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
 versions of the cell as separate values even if another filter upstream or 
 the ScanQueryMatcher is going to reject the value for other reasons.
 Seems like we need a reliable way to do pagination. The particular use case 
 that prompted this JIRA is pagination within the same rowKey. For example, 
 for a given row key R, get columns with prefix P, starting at offset X (among 
 columns which have prefix P) and limit Y. Some possible fixes might be:
 1) enhance ColumnPrefixFilter to support another constructor which supports 
 limit/offset.
 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
 as a filter) [Like SQL].
 Original Post:
 Thanks Jiakai Liu for reporting this issue and doing the initial 
 investigation. Email from Jiakai below:
 Assuming that we have an index column family with the following entries:
 tag0:001:thread1
 ...
 tag1:001:thread1
 tag1:002:thread2
 ...
 tag1:010:thread10
 ...
 tag2:001:thread1
 tag2:005:thread5
 ...
 To get threads with tag1 in range [5, 10), I tried the following code:
 ColumnPrefixFilter filter1 = new 
 ColumnPrefixFilter(Bytes.toBytes(tag1));
 ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
 */, 5 /* offset */);
 FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
 filters.addFilter(filter1);
 filters.addFilter(filter2);
 Get get = new Get(USER);
 get.addFamily(COLUMN_FAMILY);
 get.setMaxVersions(1);
 get.setFilter(filters);
 Somehow it didn't work as expected. It returned the entries as if the filter1 
 were not set.
 Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
 The FilterList filter does not handle this return code properly (treat it as 
 INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5104) Provide a reliable intra-row pagination mechanism

2012-04-15 Thread Phabricator (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5104:
---

Attachment: D2799.3.patch

mbautin updated the revision [jira] [HBASE-5104] Provide a reliable intra-row 
pagination mechanism.
Reviewers: madhuvaidya, lhofhansl, Kannan, tedyu, stack, todd, JIRA

  Fixing a bug with StoreScanner not resetting the current offset within the 
row/column family (countPerRow) to zero when transitioning to the next row. 
This is a porting error that is not present in Madhu's original fix for 89-fb. 
Also adding an independent test based on HRegion API that helped catch this bug 
(TestIntraRowPagination). I will post additional test results later.

REVISION DETAIL
  https://reviews.facebook.net/D2799

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/Get.java
  src/main/java/org/apache/hadoop/hbase/client/Result.java
  src/main/java/org/apache/hadoop/hbase/client/Scan.java
  src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
  src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  src/main/protobuf/Client.proto
  src/test/java/org/apache/hadoop/hbase/HTestConst.java
  src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPagination.java
  src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java


 Provide a reliable intra-row pagination mechanism
 -

 Key: HBASE-5104
 URL: https://issues.apache.org/jira/browse/HBASE-5104
 Project: HBase
  Issue Type: Bug
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
 Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, 
 testFilterList.rb


 Addendum:
 Doing pagination (retrieving at most limit number of KVs at a particular 
 offset) is currently supported via the ColumnPaginationFilter. However, it 
 is not a very clean way of supporting pagination.  Some of the problems with 
 it are:
 * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
 same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
 is not the case for ColumnPaginationFilter as its internal state gets updated 
 depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
 cell.
 * When this Filter is used in combination with other filters (e.g., doing AND 
 with another filter using FilterList), the behavior of the query depends on 
 the order of filters in the FilterList. This is not ideal.
 * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
 versions of the cell as separate values even if another filter upstream or 
 the ScanQueryMatcher is going to reject the value for other reasons.
 Seems like we need a reliable way to do pagination. The particular use case 
 that prompted this JIRA is pagination within the same rowKey. For example, 
 for a given row key R, get columns with prefix P, starting at offset X (among 
 columns which have prefix P) and limit Y. Some possible fixes might be:
 1) enhance ColumnPrefixFilter to support another constructor which supports 
 limit/offset.
 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
 as a filter) [Like SQL].
 Original Post:
 Thanks Jiakai Liu for reporting this issue and doing the initial 
 investigation. Email from Jiakai below:
 Assuming that we have an index column family with the following entries:
 tag0:001:thread1
 ...
 tag1:001:thread1
 tag1:002:thread2
 ...
 tag1:010:thread10
 ...
 tag2:001:thread1
 tag2:005:thread5
 ...
 To get threads with tag1 in range [5, 10), I tried the following code:
 ColumnPrefixFilter filter1 = new 
 ColumnPrefixFilter(Bytes.toBytes(tag1));
 ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
 */, 5 /* offset */);
 FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
 filters.addFilter(filter1);
 filters.addFilter(filter2);
 Get get = new Get(USER);
 get.addFamily(COLUMN_FAMILY);
 get.setMaxVersions(1);
 get.setFilter(filters);
 Somehow it didn't work as expected. It returned the entries as if the filter1 
 were not set.
 Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
 The FilterList filter does not handle this return code properly (treat it as 
 INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism

2012-04-15 Thread ramkrishna.s.vasudevan (Updated) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254501#comment-13254501
 ] 

Hadoop QA commented on HBASE-5104:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522735/D2799.3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 7 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1535//console

This message is automatically generated.

 Provide a reliable intra-row pagination mechanism
 -

 Key: HBASE-5104
 URL: https://issues.apache.org/jira/browse/HBASE-5104
 Project: HBase
  Issue Type: Bug
Reporter: Kannan Muthukkaruppan
Assignee: Madhuwanti Vaidya
 Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, 
 testFilterList.rb


 Addendum:
 Doing pagination (retrieving at most limit number of KVs at a particular 
 offset) is currently supported via the ColumnPaginationFilter. However, it 
 is not a very clean way of supporting pagination.  Some of the problems with 
 it are:
 * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
 same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
 is not the case for ColumnPaginationFilter as its internal state gets updated 
 depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
 cell.
 * When this Filter is used in combination with other filters (e.g., doing AND 
 with another filter using FilterList), the behavior of the query depends on 
 the order of filters in the FilterList. This is not ideal.
 * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
 versions of the cell as separate values even if another filter upstream or 
 the ScanQueryMatcher is going to reject the value for other reasons.
 Seems like we need a reliable way to do pagination. The particular use case 
 that prompted this JIRA is pagination within the same rowKey. For example, 
 for a given row key R, get columns with prefix P, starting at offset X (among 
 columns which have prefix P) and limit Y. Some possible fixes might be:
 1) enhance ColumnPrefixFilter to support another constructor which supports 
 limit/offset.
 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
 as a filter) [Like SQL].
 Original Post:
 Thanks Jiakai Liu for reporting this issue and doing the initial 
 investigation. Email from Jiakai below:
 Assuming that we have an index column family with the following entries:
 tag0:001:thread1
 ...
 tag1:001:thread1
 tag1:002:thread2
 ...
 tag1:010:thread10
 ...
 tag2:001:thread1
 tag2:005:thread5
 ...
 To get threads with tag1 in range [5, 10), I tried the following code:
 ColumnPrefixFilter filter1 = new 
 ColumnPrefixFilter(Bytes.toBytes(tag1));
 ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
 */, 5 /* offset */);
 FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
 filters.addFilter(filter1);
 filters.addFilter(filter2);
 Get get = new Get(USER);
 get.addFamily(COLUMN_FAMILY);
 get.setMaxVersions(1);
 get.setFilter(filters);
 Somehow it didn't work as expected. It returned the entries as if the filter1 
 were not set.
 Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
 The FilterList filter does not handle this return code properly (treat it as 
 INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.


 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Status: Open  (was: Patch Available)

 Minor Improvements related to balancer.
 ---

 Key: HBASE-5737
 URL: https://issues.apache.org/jira/browse/HBASE-5737
 Project: HBase
  Issue Type: Improvement
  Components: master
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
 HBASE-5737_2.patch, HBASE-5737_3.patch


 Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
 a hashmap.  It could be better if we have a treeMap.  Even in 
 MetaReader.fullScan we have the treeMap only so that we have the naming order 
 maintained. I felt this change could be very useful in cases where we are 
 extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92-0.94 compatibility