[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.

[Anoop Sam John (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254535#comment-13254535
 ] 

Anoop Sam John commented on HBASE-5635:
---

As per the patch the below variable is of no use now
{code}
this.zkretries = conf.getLong("hbase.splitlog.zk.retries", 3);
{code}


> If getTaskList() returns null splitlogWorker is down. It wont serve any 
> requests. 
> --
>
> Key: HBASE-5635
> URL: https://issues.apache.org/jira/browse/HBASE-5635
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.92.1
>Reporter: Kristam Subba Swathi
> Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch, HBASE-5635.patch
>
>
> During the hlog split operation if all the zookeepers are down ,then the 
> paths will be returned as null and the splitworker thread wil be exited
> Now this regionserver wil not be able to acquire any other tasks since the 
> splitworker thread is exited
> Please find the attached code for more details
> {code}
> private List getTaskList() {
> for (int i = 0; i < zkretries; i++) {
>   try {
> return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
> this.watcher.splitLogZNode));
>   } catch (KeeperException e) {
> LOG.warn("Could not get children of znode " +
> this.watcher.splitLogZNode, e);
> try {
>   Thread.sleep(1000);
> } catch (InterruptedException e1) {
>   LOG.warn("Interrupted while trying to get task list ...", e1);
>   Thread.currentThread().interrupt();
>   return null;
> }
>   }
> }
> {code}
> in the org.apache.hadoop.hbase.regionserver.SplitLogWorker 
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254531#comment-13254531
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-TRUNK-security #172 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/172/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326280)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254530#comment-13254530
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-TRUNK-security #172 (See 
[https://builds.apache.org/job/HBase-TRUNK-security/172/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable test hangs after client retries 
increased (Revision 1326434)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Status: Patch Available  (was: Open)

> Minor Improvements related to balancer.
> ---
>
> Key: HBASE-5737
> URL: https://issues.apache.org/jira/browse/HBASE-5737
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
> HBASE-5737_2.patch, HBASE-5737_3.patch
>
>
> Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
> a hashmap.  It could be better if we have a treeMap.  Even in 
> MetaReader.fullScan we have the treeMap only so that we have the naming order 
> maintained. I felt this change could be very useful in cases where we are 
> extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5747) Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test"

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254520#comment-13254520
 ] 

stack commented on HBASE-5747:
--

But I didn't change anything!  Does that mean Jon fixed it w/ his hbck commit?

@Jon Let me ask @Mikhail why he went to 100 retries...

> Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
> subdirectory of target/test"
> 
>
> Key: HBASE-5747
> URL: https://issues.apache.org/jira/browse/HBASE-5747
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, 
> 5708v4.txt, 5708v4.txt
>
>
> Forward port as much as we can of Mikhail's hard-won test cleanups over on 
> 0.89 branch  Will improve our being able to run unit tests in //.  He also 
> found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5654) [findbugs] Address dodgy bugs

[Uma Maheswara Rao G (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254512#comment-13254512
 ] 

Uma Maheswara Rao G commented on HBASE-5654:


Hey Ashutosh,

Looks there are some more(3) dodgy category bugs in report.
Also current bugs count in the report is lesser than the count updated in 
test-patch,properties.

For the comment in RSStatusTmplImpl, you may have to change in 
[RSStatusTmpl.jamon 
|http://svn.apache.org/repos/asf/hbase/trunk/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon]
 file.

> [findbugs] Address dodgy bugs
> -
>
> Key: HBASE-5654
> URL: https://issues.apache.org/jira/browse/HBASE-5654
> Project: HBase
>  Issue Type: Sub-task
>  Components: scripts
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Ashutosh Jindal
>  Labels: patch
> Fix For: 0.96.0
>
> Attachments: Hbase-5654.patch
>
>
> See 
> https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE
> This may be broken down further.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254510#comment-13254510
 ] 

Lars Hofhansl commented on HBASE-5795:
--

@Ted: I don't understand why that redundant write in 0.94 causes any problem. 
Can you elaborate? Was there an other problem in v1?


> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Status: Open  (was: Patch Available)

> Minor Improvements related to balancer.
> ---
>
> Key: HBASE-5737
> URL: https://issues.apache.org/jira/browse/HBASE-5737
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
> HBASE-5737_2.patch, HBASE-5737_3.patch
>
>
> Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
> a hashmap.  It could be better if we have a treeMap.  Even in 
> MetaReader.fullScan we have the treeMap only so that we have the naming order 
> maintained. I felt this change could be very useful in cases where we are 
> extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254501#comment-13254501
 ] 

Hadoop QA commented on HBASE-5104:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522735/D2799.3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 7 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1535//console

This message is automatically generated.

> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
>  Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, 
> testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular 
> "offset") is currently supported via the ColumnPaginationFilter. However, it 
> is not a very clean way of supporting pagination.  Some of the problems with 
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
> is not the case for ColumnPaginationFilter as its internal state gets updated 
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND 
> with another filter using FilterList), the behavior of the query depends on 
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
> versions of the cell as separate values even if another filter upstream or 
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case 
> that prompted this JIRA is pagination within the same rowKey. For example, 
> for a given row key R, get columns with prefix P, starting at offset X (among 
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports 
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial 
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new 
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1 
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
> The FilterList filter does not handle this return code properly (treat it as 
> INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5104) Provide a reliable intra-row pagination mechanism

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5104:
---

Attachment: D2799.3.patch

mbautin updated the revision "[jira] [HBASE-5104] Provide a reliable intra-row 
pagination mechanism".
Reviewers: madhuvaidya, lhofhansl, Kannan, tedyu, stack, todd, JIRA

  Fixing a bug with StoreScanner not resetting the current offset within the 
row/column family (countPerRow) to zero when transitioning to the next row. 
This is a porting error that is not present in Madhu's original fix for 89-fb. 
Also adding an independent test based on HRegion API that helped catch this bug 
(TestIntraRowPagination). I will post additional test results later.

REVISION DETAIL
  https://reviews.facebook.net/D2799

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/client/Get.java
  src/main/java/org/apache/hadoop/hbase/client/Result.java
  src/main/java/org/apache/hadoop/hbase/client/Scan.java
  src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
  src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
  src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  src/main/protobuf/Client.proto
  src/test/java/org/apache/hadoop/hbase/HTestConst.java
  src/test/java/org/apache/hadoop/hbase/client/TestIntraRowPagination.java
  src/test/java/org/apache/hadoop/hbase/client/TestScannersFromClientSide.java


> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
>  Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, 
> testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular 
> "offset") is currently supported via the ColumnPaginationFilter. However, it 
> is not a very clean way of supporting pagination.  Some of the problems with 
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
> is not the case for ColumnPaginationFilter as its internal state gets updated 
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND 
> with another filter using FilterList), the behavior of the query depends on 
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
> versions of the cell as separate values even if another filter upstream or 
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case 
> that prompted this JIRA is pagination within the same rowKey. For example, 
> for a given row key R, get columns with prefix P, starting at offset X (among 
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports 
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial 
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new 
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1 
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
> The FilterList filter does not handle this return code properly (treat it as 
> INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

   

[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254498#comment-13254498
 ] 

Phabricator commented on HBASE-5104:


mbautin has commented on the revision "[jira] [HBASE-5104] Provide a reliable 
intra-row pagination mechanism".

INLINE COMMENTS
  src/main/java/org/apache/hadoop/hbase/client/Get.java:470 Removed.
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:395 Done.
  src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide2.java:38 Done.

REVISION DETAIL
  https://reviews.facebook.net/D2799


> Provide a reliable intra-row pagination mechanism
> -
>
> Key: HBASE-5104
> URL: https://issues.apache.org/jira/browse/HBASE-5104
> Project: HBase
>  Issue Type: Bug
>Reporter: Kannan Muthukkaruppan
>Assignee: Madhuwanti Vaidya
> Attachments: D2799.1.patch, D2799.2.patch, testFilterList.rb
>
>
> Addendum:
> Doing pagination (retrieving at most "limit" number of KVs at a particular 
> "offset") is currently supported via the ColumnPaginationFilter. However, it 
> is not a very clean way of supporting pagination.  Some of the problems with 
> it are:
> * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have 
> same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This 
> is not the case for ColumnPaginationFilter as its internal state gets updated 
> depending on whether or not Filter(A) returns TRUE/FALSE for a particular 
> cell.
> * When this Filter is used in combination with other filters (e.g., doing AND 
> with another filter using FilterList), the behavior of the query depends on 
> the order of filters in the FilterList. This is not ideal.
> * ColumnPaginationFilter is a stateful filter which ends up counting multiple 
> versions of the cell as separate values even if another filter upstream or 
> the ScanQueryMatcher is going to reject the value for other reasons.
> Seems like we need a reliable way to do pagination. The particular use case 
> that prompted this JIRA is pagination within the same rowKey. For example, 
> for a given row key R, get columns with prefix P, starting at offset X (among 
> columns which have prefix P) and limit Y. Some possible fixes might be:
> 1) enhance ColumnPrefixFilter to support another constructor which supports 
> limit/offset.
> 2) Support pagination (limit/offset) at the Scan/Get API level (rather than 
> as a filter) [Like SQL].
> Original Post:
> Thanks Jiakai Liu for reporting this issue and doing the initial 
> investigation. Email from Jiakai below:
> Assuming that we have an index column family with the following entries:
> "tag0:001:thread1"
> ...
> "tag1:001:thread1"
> "tag1:002:thread2"
> ...
> "tag1:010:thread10"
> ...
> "tag2:001:thread1"
> "tag2:005:thread5"
> ...
> To get threads with "tag1" in range [5, 10), I tried the following code:
> ColumnPrefixFilter filter1 = new 
> ColumnPrefixFilter(Bytes.toBytes("tag1"));
> ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit 
> */, 5 /* offset */);
> FilterList filters = new FilterList(Operator.MUST_PASS_ALL);
> filters.addFilter(filter1);
> filters.addFilter(filter2);
> Get get = new Get(USER);
> get.addFamily(COLUMN_FAMILY);
> get.setMaxVersions(1);
> get.setFilter(filters);
> Somehow it didn't work as expected. It returned the entries as if the filter1 
> were not set.
> Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. 
> The FilterList filter does not handle this return code properly (treat it as 
> INCLUDE).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Todd Lipcon (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254496#comment-13254496
 ] 

Todd Lipcon commented on HBASE-5792:


bq. I wonder if it makes sense to persist benchmark results to HBase.

Let's not scope creep. The point of this tool is to have a quick benchmark you 
can run without having to set up an HBase cluster. Outputting results on the 
console is good enough to parse and store in whatever database you want. My 
preference would probably be just appending to a text file for easy graphing 
from Hudson.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5797) Add the active master address on the web of regionserver

[chunhui shen (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen resolved HBASE-5797.
-

Resolution: Duplicate

Sorry, I find it has been done in trunk.

> Add the active master address on the web of regionserver
> 
>
> Key: HBASE-5797
> URL: https://issues.apache.org/jira/browse/HBASE-5797
> Project: HBase
>  Issue Type: Improvement
>Reporter: chunhui shen
>Assignee: chunhui shen
>
> Add the active master address on the web of regionserver, so we can easily 
> know the master server name and jump to master web from regionserver web.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5797) Add the active master address on the web of regionserver

[chunhui shen (Created) (JIRA)]

Add the active master address on the web of regionserver


 Key: HBASE-5797
 URL: https://issues.apache.org/jira/browse/HBASE-5797
 Project: HBase
  Issue Type: Improvement
Reporter: chunhui shen
Assignee: chunhui shen


Add the active master address on the web of regionserver, so we can easily know 
the master server name and jump to master web from regionserver web.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254484#comment-13254484
 ] 

Hadoop QA commented on HBASE-5795:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522732/5795-v2.txt
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

-1 tests included.  The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 4 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1534//console

This message is automatically generated.

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5747) Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test"

[Zhihong Yu (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu resolved HBASE-5747.
---

Resolution: Fixed

Trunk build passes twice.

> Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
> subdirectory of target/test"
> 
>
> Key: HBASE-5747
> URL: https://issues.apache.org/jira/browse/HBASE-5747
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, 
> 5708v4.txt, 5708v4.txt
>
>
> Forward port as much as we can of Mikhail's hard-won test cleanups over on 
> 0.89 branch  Will improve our being able to run unit tests in //.  He also 
> found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254482#comment-13254482
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-0.92 #372 (See 
[https://builds.apache.org/job/HBase-0.92/372/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable test hangs after client retries 
increased (Revision 1326436)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

[Benoit Sigoure (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254477#comment-13254477
 ] 

Benoit Sigoure commented on HBASE-5796:
---

Just to be clear: the wrapping of Hadoop's {{IOException}} would be for a first 
pass, to make sure everything HBase throws inherits from an {{HBaseException}}. 
 Ideally it would be better to pick a more specific type of exception, but I 
expect this will be difficult in various places where Hadoop suffers from the 
same problem and HBase itself has little to no visibility into what the 
exception from Hadoop really means.

Also I'd recommend to actually use {{HBaseIOException}} as a base class name, 
and keep the name {{HBaseException}} for when you guys switch to unchecked 
exceptions, if you're willing to consider going down that route eventually 
(that'd require a separate JIRA issue).

Since HBase 0.96 is the "singularity", it's a good opportunity to break API 
compatibility.  In this case, every release up to and including 0.94.x would 
have {{HBaseIOException}} as the parent class, and in 0.96 the only thing you 
need to change is to make everything inherit from {{HBaseException}} (which 
itself would inherit from {{RuntimeException}}).

> Fix our abuse of IOE: see 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
> ---
>
> Key: HBASE-5796
> URL: https://issues.apache.org/jira/browse/HBASE-5796
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>
> Lets make more context particular exceptions rather than throw IOEs 
> everywhere.  See Benoît's rant: 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254476#comment-13254476
 ] 

Zhihong Yu commented on HBASE-5796:
---

bq. all in a {{HadoopIOException}}, which would inherit from {{HBaseException}}.
I am afraid that some people would not feel comfortable with the above 
inheritance.

> Fix our abuse of IOE: see 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
> ---
>
> Key: HBASE-5796
> URL: https://issues.apache.org/jira/browse/HBASE-5796
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>
> Lets make more context particular exceptions rather than throw IOEs 
> everywhere.  See Benoît's rant: 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Status: Patch Available  (was: Open)

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: 5795-v2.txt

Patch v1 didn't make testHServerLoadVersioning pass.

Patch v2 does.
I found that the version of RegionLoad was actually serialized twice in 0.92: 
first by VersionedWritable.write(), followed by RegionLoad.write().
In patch v2, I removed the redundant write. readFields92() consumes the second 
copy of version.

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v2.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: (was: 5795-v1.txt)

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

[Benoit Sigoure (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254474#comment-13254474
 ] 

Benoit Sigoure commented on HBASE-5796:
---

I propose the following, as a temporary stopgap solution:
# Create a new class that inherits from {{IOException}}, call it 
{{HBaseException}}.
# Change *all* HBase exceptions to inherit from them.  Replace all bare {{new 
IOException}} with {{new HBaseException}}.
# Classify all exceptions to be under two categories: what can be potentially 
retried, and what cannot (this partially exists today under 
{{DoNotRetryException}} as Lars points out).
# At this point, nothing should be a bare {{HBaseException}}, and nothing 
should be creating bare {{IOException}}s either.
# Update all APIs to specify exceptions thrown with a type as specific as 
possible.

I believe most (if not all) of the steps above can be done while preserving API 
compatibility (and maybe even ABI compatibility).

This would be a good first pass to partially clean up some of the mess we have 
today.  Also, because Hadoop is even worse than HBase, and because it's 
changing slowly, your code will still be crippled with {{IOException}} from 
Hadoop.  I'd recommend wrapping them all in a {{HadoopIOException}}, which 
would inherit from {{HBaseException}}.

Ultimately, I would *strongly* encourage you to move away from {{IOException}} 
altogether.  Not only because it's the wrong exception to use in most cases 
where it's used in Hadoop/HBase, but also because it's a checked exception.  As 
I wrote in my blog post, checked exceptions are like communism: they sound like 
a good idea, but they horribly fail in practice.  Here's a short but good 
article that conveys well two of the biggest show-stoppers with checked 
exceptions: http://www.artima.com/intv/handcuffsP.html – I'm so happy that 
Scala didn't pick this up from Java.  Checked exceptions almost always wind up 
defeating their own purpose.  I'm sure you guys have written enough code in big 
enough code bases to experience this too.

> Fix our abuse of IOE: see 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
> ---
>
> Key: HBASE-5796
> URL: https://issues.apache.org/jira/browse/HBASE-5796
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>
> Lets make more context particular exceptions rather than throw IOEs 
> everywhere.  See Benoît's rant: 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254470#comment-13254470
 ] 

Lars Hofhansl commented on HBASE-5796:
--

Could be as simple as a better hierarchy of classes extending IOException (like 
DoNotRetryException).

> Fix our abuse of IOE: see 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
> ---
>
> Key: HBASE-5796
> URL: https://issues.apache.org/jira/browse/HBASE-5796
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>
> Lets make more context particular exceptions rather than throw IOEs 
> everywhere.  See Benoît's rant: 
> http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254464#comment-13254464
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-0.94 #119 (See 
[https://builds.apache.org/job/HBase-0.94/119/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable tests hangs after client retries 
increased (Revision 1326435)

 Result = SUCCESS
jmhsieh : 
Files : 
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254461#comment-13254461
 ] 

Hudson commented on HBASE-5793:
---

Integrated in HBase-TRUNK #2765 (See 
[https://builds.apache.org/job/HBase-TRUNK/2765/])
HBASE-5793 TestHBaseFsck#testNoHdfsTable test hangs after client retries 
increased (Revision 1326434)

 Result = SUCCESS
jmhsieh : 
Files : 
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5747) Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a subdirectory of target/test"

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254448#comment-13254448
 ] 

Jonathan Hsieh commented on HBASE-5747:
---

@stack Does removing the retries cause more errors to show up does it remain 
the same?  

> Forward port "hbase-5708 [89-fb] Make MiniMapRedCluster directory a 
> subdirectory of target/test"
> 
>
> Key: HBASE-5747
> URL: https://issues.apache.org/jira/browse/HBASE-5747
> Project: HBase
>  Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.96.0
>
> Attachments: 5474.txt, 5474v2.txt, 5474v3 (1).txt, 5474v3.txt, 
> 5708v4.txt, 5708v4.txt
>
>
> Forward port as much as we can of Mikhail's hard-won test cleanups over on 
> 0.89 branch  Will improve our being able to run unit tests in //.  He also 
> found a few bugs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

   Resolution: Fixed
Fix Version/s: 0.96.0
   0.94.0
   0.92.2
   0.90.7
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to 0.90/0.92/0.94/0.96/

> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[stack (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254441#comment-13254441
 ] 

stack commented on HBASE-5795:
--

I looked at Ted's patch.  That should do it.  See if it makes the unit test 
pass I'd say.  I can test on cluster tomorrow morning (will also finish my 
rolling restart and kill of meta on a cluster w/ 1k regions too...)

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v1.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[stack (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5795:
-

Attachment: 5795.unittest.txt

unit test

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v1.txt, 5795.unittest.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5796) Fix our abuse of IOE: see http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

[stack (Created) (JIRA)]

Fix our abuse of IOE: see 
http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html
---

 Key: HBASE-5796
 URL: https://issues.apache.org/jira/browse/HBASE-5796
 Project: HBase
  Issue Type: Task
Reporter: stack


Lets make more context particular exceptions rather than throw IOEs everywhere. 
 See Benoît's rant: 
http://blog.tsunanet.net/2012/04/apache-hadoop-abuse-ioexception.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254436#comment-13254436
 ] 

Zhihong Yu commented on HBASE-5792:
---

I wonder if it makes sense to persist benchmark results to HBase.
That may show us some trend w.r.t. HLog performance.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254433#comment-13254433
 ] 

Lars Hofhansl commented on HBASE-5782:
--

Thanks Ram!

> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.0
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254430#comment-13254430
 ] 

Lars Hofhansl commented on HBASE-5795:
--

Oh, you attached your 5794 patch to this issue. I removed to avoid confusing, 
when you get a chance could you attach your patch here? Then we can make a 
combined patch with that and Ted's fix.

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254429#comment-13254429
 ] 

Lars Hofhansl commented on HBASE-5795:
--

@Stack: I think you forgot to include the actual test in the patch :)

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5795:
-

Attachment: (was: 5794.txt)

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Matteo Bertozzi (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254428#comment-13254428
 ] 

Matteo Bertozzi commented on HBASE-5792:


Don't know if it makes sense catch the InterruptException, since we're trying 
to test write performance with nThreads... if we lost one thread, the test 
doesn't reflect what we've asked for.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254426#comment-13254426
 ] 

Zhihong Yu commented on HBASE-5792:
---

Minor comments:
{code}
import org.apache.hadoop.conf.Configuration;
{code}
The above import was not used.
{code}
for (Thread t : threads) t.join();
{code}
join() may throw InterruptedException. Shall we catch it and proceed to the 
next Thread to be joined ?

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retires 
increased  (was: TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747)

> TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5793:
--

Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retries 
increased  (was: TestHBaseFsck#TestNoHdfsTable test hangs after client retires 
increased)

> TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
> ---
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254420#comment-13254420
 ] 

Jonathan Hsieh commented on HBASE-5793:
---

Zhihong yeah, I broke this out to a separate issue because I was fixing other 
TestHBaseFsck problems, wanted test to pass, but didn't yet spend the effort to 
figure out the rest of the HBASE-5747 problems.

Since I got your confirmation and since this is a minor test update, I'm just 
going to commit.

> TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
> -
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Issue Comment Edited) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254414#comment-13254414
 ] 

Lars Hofhansl edited comment on HBASE-5795 at 4/15/12 7:31 PM:
---

+1 on patch.

  was (Author: lhofhansl):
+1 om patch.
  
> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5794.txt, 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254414#comment-13254414
 ] 

Lars Hofhansl commented on HBASE-5795:
--

+1 om patch.

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5794.txt, 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5256) Use WritableUtils.readVInt() in RegionLoad.readFields()

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5256:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

This was committed, marking it accordingly.

> Use WritableUtils.readVInt() in RegionLoad.readFields()
> ---
>
> Key: HBASE-5256
> URL: https://issues.apache.org/jira/browse/HBASE-5256
> Project: HBase
>  Issue Type: Task
>Reporter: Zhihong Yu
>Assignee: Mubarak Seyed
> Fix For: 0.94.0
>
> Attachments: HBASE-5256.trunk.v1.patch
>
>
> Currently in.readInt() is used in RegionLoad.readFields()
> More metrics would be added to RegionLoad in the future, we should utilize 
> WritableUtils.readVInt() to reduce the amount of data exchanged between 
> Master and region servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Lars Hofhansl (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5795:
-

Fix Version/s: 0.94.0

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.0
>
> Attachments: 5794.txt, 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5737) Minor Improvements related to balancer.

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254399#comment-13254399
 ] 

Hadoop QA commented on HBASE-5737:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522713/HBASE-5737_3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 3 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1533//console

This message is automatically generated.

> Minor Improvements related to balancer.
> ---
>
> Key: HBASE-5737
> URL: https://issues.apache.org/jira/browse/HBASE-5737
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
> HBASE-5737_2.patch, HBASE-5737_3.patch
>
>
> Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
> a hashmap.  It could be better if we have a treeMap.  Even in 
> MetaReader.fullScan we have the treeMap only so that we have the naming order 
> maintained. I felt this change could be very useful in cases where we are 
> extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Hadoop QA (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254393#comment-13254393
 ] 

Hadoop QA commented on HBASE-5792:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522711/HBASE-5792-v1.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

-1 findbugs.  The patch appears to introduce 3 new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

 -1 core tests.  The patch failed these unit tests:
   org.apache.hadoop.hbase.master.TestSplitLogManager

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/1532//console

This message is automatically generated.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5782) Not all the regions are getting assigned after the log splitting.

[ramkrishna.s.vasudevan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254385#comment-13254385
 ] 

ramkrishna.s.vasudevan commented on HBASE-5782:
---

@LArs
Tomorrow i can try writing a test with similar scenario.  
bq.(Although that might be difficult)
I think it should be easier.  Hopefully tomorrow can come up with some initial 
analysis.

> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.0
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5782) Not all the regions are getting assigned after the log splitting.

[ramkrishna.s.vasudevan (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan reassigned HBASE-5782:
-

Assignee: ramkrishna.s.vasudevan

> Not all the regions are getting assigned after the log splitting.
> -
>
> Key: HBASE-5782
> URL: https://issues.apache.org/jira/browse/HBASE-5782
> Project: HBase
>  Issue Type: Bug
>  Components: wal
>Affects Versions: 0.94.0
>Reporter: Gopinathan A
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.0
>
>
> Create a table with 1000 splits, after the region assignemnt, kill the 
> regionserver wich contains META table.
> Here few regions are missing after the log splitting and region assigment. 
> HBCK report shows multiple region holes are got created.
> Same scenario was verified mulitple times in 0.92.1, no issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.

[ramkrishna.s.vasudevan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5737:
--

Attachment: HBASE-5737_3.patch

Latest patch as per Stack's suggestion.

> Minor Improvements related to balancer.
> ---
>
> Key: HBASE-5737
> URL: https://issues.apache.org/jira/browse/HBASE-5737
> Project: HBase
>  Issue Type: Improvement
>  Components: master
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Minor
> Attachments: HBASE-5737.patch, HBASE-5737_1.patch, 
> HBASE-5737_2.patch, HBASE-5737_3.patch
>
>
> Currently in Am.getAssignmentByTable()  we use a result map which is currenly 
> a hashmap.  It could be better if we have a treeMap.  Even in 
> MetaReader.fullScan we have the treeMap only so that we have the naming order 
> maintained. I felt this change could be very useful in cases where we are 
> extending the DefaultLoadBalancer.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5792) HLog Performance Evaluation Tool

[Matteo Bertozzi (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5792:
---

Attachment: HBASE-5792-v1.patch

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch, HBASE-5792-v1.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5792) HLog Performance Evaluation Tool

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254371#comment-13254371
 ] 

Zhihong Yu commented on HBASE-5792:
---

Interesting tool.
{code}
+region = HRegion.createHRegion(regionInfo, regionRootDir, getConf(), htd);
{code}
I don't see where region is closed.
There should be some cleanup method after the benchmarking is completed.

> HLog Performance Evaluation Tool
> 
>
> Key: HBASE-5792
> URL: https://issues.apache.org/jira/browse/HBASE-5792
> Project: HBase
>  Issue Type: Test
>  Components: wal
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: performance, wal
> Attachments: HBASE-5792-v0.patch
>
>
> Related to HDFS-3280 and the HBase WAL slowdown on 0.23+
> It would be nice to have a simple tool like HFilePerformanceEvaluation, ...
> to be able to check easily the HLog performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254369#comment-13254369
 ] 

Zhihong Yu commented on HBASE-5793:
---

TestHBaseFsck passes in PreCommit build #1530.
But there still was some other hanging test(s).

> TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
> -
>
> Key: HBASE-5793
> URL: https://issues.apache.org/jira/browse/HBASE-5793
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 0.96.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Attachments: hbase-5793.patch
>
>
> After the HBASE-5747 modification, this one particular case hangs.
> {code}
> mvn test -PlocalTests -Dtest=TestHBaseFsck
> {code}
> It was hanging on a scan of a table that the test deleted. It expected a call 
> to thrown an exception after a timeout.  HBASE-5747 changed the timeout to a 
> larger number of retries which caused mvn to fail the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254364#comment-13254364
 ] 

jirapos...@reviews.apache.org commented on HBASE-5621:
--



bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, 
line 2814
bq.  > 

bq.  >
bq.  > Where has allt his code gone?
bq.  
bq.  Jimmy Xiang wrote:
bq.  Some are moved to RegionServer as I did for 5620
bq.  
bq.  Michael Stack wrote:
bq.  But I did not see it in the patch?  Its already moved?

It should be in the patch, probably in the second page of touched files.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/ClientProtocol.java, line 29
bq.  > 
bq.  >
bq.  > Ditto
bq.  
bq.  Jimmy Xiang wrote:
bq.  Will move to client.
bq.  
bq.  Michael Stack wrote:
bq.  Is this only used out of the client package?  If so, yeah, move it 
there I'd say.

It is used out of the client package, and the server side implements it.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/AdminProtocol.java, line 28
bq.  > 
bq.  >
bq.  > Is this right?  It does more than talk to a regionserver?  You have 
to have really nice comments on your class now that you are at the top level 
Jimmy (smile)
bq.  > 
bq.  > This class is only used by HBaseAdmin?  Is that right?  Or do other 
classes use it?  If only HBaseAdmin, maybe it should not be top level?
bq.  
bq.  Jimmy Xiang wrote:
bq.  Let me move it to client.
bq.  
bq.  Michael Stack wrote:
bq.  And it should not be public I'd say.

Yes, it is private.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java, line 
632
bq.  > 
bq.  >
bq.  > Do we have to have protobuf stuff sprinkled all over the code base?  
Its not too bad but just wondering.  Maybe we do but just wondering.  I suppose 
there is nothing to do about it.  If we did same operation over and over w/ 
some pb stuff and the output was a non-pb object, then yes, we could hide the 
pb stuff over in a class but what we do here is particular to this method 
Can't generalize.
bq.  > 
bq.  > I do see that you have added some static methods into protobuf 
utils.  This one is not generic so doesn't deserve to go there?
bq.  
bq.  Jimmy Xiang wrote:
bq.  For getRegionInfo, I used to put it in the protobuf util since it is 
used lots of places.  I will move it to protobuf util.
bq.  
bq.  Michael Stack wrote:
bq.  Agree.  If used more than once, move it over to protobuf util 
otherwise I don't see a way around not importing protobuf classes when its a 
particular usage.

Cool, will fix.


bq.  On 2012-04-13 23:59:40, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 32
bq.  > 
bq.  >
bq.  > Did you say you were going to remove these from here?
bq.  
bq.  Jimmy Xiang wrote:
bq.  I tried but it is hard.  Let me think about it again.
bq.  
bq.  Michael Stack wrote:
bq.  Even if this stuff just moved to HCM, it'd give us a clean HConnection.

I was thinking to have a ProtocolFactory/Manager to manage the protocols, and 
let the connection manager to many connections only.
Let me move them to HCM for now.


- Jimmy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4714/#review6922
---


On 2012-04-13 23:03:35, Jimmy Xiang wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4714/
bq.  ---
bq.  
bq.  (Updated 2012-04-13 23:03:35)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  This is the admin part of HBase-5443.  AdminProtocol part.
bq.  
bq.  
bq.  This addresses bug HBASE-5621.
bq.  https://issues.apache.org/jira/browse/HBASE-5621
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.src/main/java/org/apache/hadoop/hbase/AdminProtocol.java PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/ClientProtocol.java PRE-CREATION 
bq.src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 
408db79 
bq.src/main/java/org/apache/hadoop/hbase/client/HBase

[jira] [Commented] (HBASE-5620) Convert the client protocol of HRegionInterface to PB

[Jimmy Xiang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254360#comment-13254360
 ] 

Jimmy Xiang commented on HBASE-5620:


Thanks for reviewing. Both regular test suite and security test suite are green 
for me. I mean all tests in the suite.

> Convert the client protocol of HRegionInterface to PB
> -
>
> Key: HBASE-5620
> URL: https://issues.apache.org/jira/browse/HBASE-5620
> Project: HBase
>  Issue Type: Sub-task
>  Components: ipc, master, migration, regionserver
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5620-sec.patch, hbase-5620_v3.patch, 
> hbase-5620_v4.patch, hbase-5620_v4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5795) hbase-3927 breaks 0.92<->0.94 compatibility

[Zhihong Yu (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5795:
--

Attachment: 5795-v1.txt

Since only deserialization needs special handling, the attached patch adds a 
private method to read 0.92 RegionLoad.

Please comment.

> hbase-3927 breaks 0.92<->0.94 compatibility
> ---
>
> Key: HBASE-5795
> URL: https://issues.apache.org/jira/browse/HBASE-5795
> Project: HBase
>  Issue Type: Bug
>Reporter: stack
>Assignee: stack
> Attachments: 5794.txt, 5795-v1.txt
>
>
> This commit broke our 0.92/0.94 compatibility:
> {code}
> 
> r1136686 | stack | 2011-06-16 14:18:08 -0700 (Thu, 16 Jun 2011) | 1 line
> HBASE-3927 display total uncompressed byte size of a region in web UI
> {code}
> I just tried the new RC for 0.94.  I brought up a 0.94 master on a 0.92 
> cluster and rather than just digest version 1 of the HServerLoad, I get this:
> {code}
> 2012-04-14 22:47:59,752 WARN org.apache.hadoop.ipc.HBaseServer: Unable to 
> read call parameters for client 10.4.14.38
> java.io.IOException: Error in readFields
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:684)
> at 
> org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1269)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1184)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:722)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:513)
> at 
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:488)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: A record version mismatch occured. Expecting v2, found v1
> at 
> org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
> at 
> org.apache.hadoop.hbase.HServerLoad$RegionLoad.readFields(HServerLoad.java:379)
> at 
> org.apache.hadoop.hbase.HServerLoad.readFields(HServerLoad.java:686)
> at 
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:681)
> ... 9 more
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Ferdy Galema (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254339#comment-13254339
 ] 

Ferdy Galema commented on HBASE-2214:
-

(Nevermind the latest comment. It seems that with a fuzz factor it does apply.)

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5787) Table owner can't disable/delete its own table

[Matteo Bertozzi (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5787:
---

Attachment: HBASE-5787-v1.patch

{code}
Running org.apache.hadoop.hbase.security.token.TestTokenAuthentication
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.765 sec
Running org.apache.hadoop.hbase.security.token.TestZKSecretWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.958 sec

Results :
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
{code}

> Table owner can't disable/delete its own table
> --
>
> Key: HBASE-5787
> URL: https://issues.apache.org/jira/browse/HBASE-5787
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Affects Versions: 0.92.1, 0.94.0, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: acl, security
> Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch, 
> HBASE-5787-v1.patch
>
>
> An user with CREATE privileges can create a table, but can not disable it, 
> because disable operation require ADMIN privileges. Also if a table is 
> already disabled, anyone can remove it.
> {code}
> public void preDeleteTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   requirePermission(Permission.Action.CREATE);
> }
> public void preDisableTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   /* TODO: Allow for users with global CREATE permission and the table owner 
> */
>   requirePermission(Permission.Action.ADMIN);
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Ferdy Galema (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254338#comment-13254338
 ] 

Ferdy Galema commented on HBASE-2214:
-

Sure I will do that.

Some patch segments do not apply to trunk so I will first create a patch for 
trunk.

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5787) Table owner can't disable/delete its own table

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254336#comment-13254336
 ] 

Zhihong Yu commented on HBASE-5787:
---

Thanks for the update.
Can you combine the two patches and run through tests again, including the 
following two ?
{code}
security/src/test//java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java
security/src/test//java/org/apache/hadoop/hbase/security/token/TestZKSecretWatcher.java
{code}

> Table owner can't disable/delete its own table
> --
>
> Key: HBASE-5787
> URL: https://issues.apache.org/jira/browse/HBASE-5787
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Affects Versions: 0.92.1, 0.94.0, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: acl, security
> Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch
>
>
> An user with CREATE privileges can create a table, but can not disable it, 
> because disable operation require ADMIN privileges. Also if a table is 
> already disabled, anyone can remove it.
> {code}
> public void preDeleteTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   requirePermission(Permission.Action.CREATE);
> }
> public void preDisableTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   /* TODO: Allow for users with global CREATE permission and the table owner 
> */
>   requirePermission(Permission.Action.ADMIN);
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254335#comment-13254335
 ] 

Zhihong Yu commented on HBASE-2214:
---

I went over https://reviews.apache.org/r/4726 which looks clean.

If adding unit test is difficult, can you perform verification on real clsuter 
and let us know the results ?

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Zhihong Yu (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254334#comment-13254334
 ] 

Zhihong Yu commented on HBASE-2214:
---

'hbase' group should be included in each review request.
Normally patch for trunk is generated first and gets reviewed on review board. 
Hadoop QA checks out patch against trunk.

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Ferdy Galema (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254330#comment-13254330
 ] 

Ferdy Galema commented on HBASE-2214:
-

Secondly, I'd like the uploaded diff to be compared to the 0.94 branch. How can 
I specify this? It seems it is always compared to trunk?

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254329#comment-13254329
 ] 

jirapos...@reviews.apache.org commented on HBASE-2214:
--


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4726/
---

(Updated 2012-04-15 13:38:31.680141)


Review request for hbase and Ted Yu.


Summary
---

HBASE-2214 per scan max buffersize.


This addresses bug HBASE-2214.
https://issues.apache.org/jira/browse/HBASE-2214


Diffs
-

  /src/main/java/org/apache/hadoop/hbase/client/Scan.java 1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 
1326084 
  /src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java 
1326084 
  
/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
 1326084 

Diff: https://reviews.apache.org/r/4726/diff


Testing
---


Thanks,

ferdy



> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254307#comment-13254307
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-0.92 #371 (See 
[https://builds.apache.org/job/HBase-0.92/371/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326282)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/branches/0.92/CHANGES.txt
* 
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKee

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

[Ferdy Galema (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254301#comment-13254301
 ] 

Ferdy Galema commented on HBASE-2214:
-

Ok thanks for your comments. I just submitted new patch to review board. 
https://reviews.apache.org/r/4726/

So what's protocol here, can I add 'hbase' to reviewers group or just add 
individual reviewers?

> Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
> properly
> -
>
> Key: HBASE-2214
> URL: https://issues.apache.org/jira/browse/HBASE-2214
> Project: HBase
>  Issue Type: New Feature
>Reporter: stack
>Assignee: Ferdy Galema
> Fix For: 0.94.1
>
> Attachments: HBASE-2214-0.94.txt, HBASE-2214_with_broken_TestShell.txt
>
>
> The notion that you set size rather than row count specifying how many rows a 
> scanner should return in each cycle was raised over in hbase-1966.  Its a 
> good one making hbase "regular" though the data under it may vary.  
> HBase-1966 was committed but the patch was constrained by the fact that it 
> needed to not change RPC interface.  This issue is about doing hbase-1966 for 
> 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5787) Table owner can't disable/delete its own table

[Matteo Bertozzi (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5787:
---

Attachment: HBASE-5787-tests-wrong-names.patch

Test seems ok, (tested against 0.94)
I've also attached a patch to fix some "copy-paste" code.
{code}
Running org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.659 sec
Running org.apache.hadoop.hbase.security.access.TestAccessControlFilter
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.746 sec
Running org.apache.hadoop.hbase.security.access.TestAccessController
Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.119 sec
Running org.apache.hadoop.hbase.security.access.TestTablePermissions
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.822 sec

Results :
Tests run: 27, Failures: 0, Errors: 0, Skipped: 0
{code}

> Table owner can't disable/delete its own table
> --
>
> Key: HBASE-5787
> URL: https://issues.apache.org/jira/browse/HBASE-5787
> Project: HBase
>  Issue Type: Bug
>  Components: security
>Affects Versions: 0.92.1, 0.94.0, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
>  Labels: acl, security
> Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch
>
>
> An user with CREATE privileges can create a table, but can not disable it, 
> because disable operation require ADMIN privileges. Also if a table is 
> already disabled, anyone can remove it.
> {code}
> public void preDeleteTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   requirePermission(Permission.Action.CREATE);
> }
> public void preDisableTable(ObserverContext c,
> byte[] tableName) throws IOException {
>   /* TODO: Allow for users with global CREATE permission and the table owner 
> */
>   requirePermission(Permission.Action.ADMIN);
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254293#comment-13254293
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-0.94 #117 (See 
[https://builds.apache.org/job/HBase-0.94/117/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326281)

 Result = FAILURE
jmhsieh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> or

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254291#comment-13254291
 ] 

Hudson commented on HBASE-5781:
---

Integrated in HBase-TRUNK #2763 (See 
[https://builds.apache.org/job/HBase-TRUNK/2763/])
HBASE-5781 Zookeeper session got closed while trying to assign the region 
to RS using hbck -fix (Revision 1326280)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java


> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apa

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Jonathan Hsieh (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254287#comment-13254287
 ] 

Jonathan Hsieh commented on HBASE-5781:
---

Oh -- but it is committed to trunk as well.

> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hba

[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Jonathan Hsieh (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5781:
--

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Anoop, Kristam, thanks for finding this and hunting down the root cause of the 
problem.  Thanks for the quick review Lars.

Committed to 0.90/0.92/0.94.  Trunk is broken currently because of HBASE-5747, 
HBASE-5793.

> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectio

[jira] [Commented] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix

[Anoop Sam John (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254283#comment-13254283
 ] 

Anoop Sam John commented on HBASE-5781:
---

Yes finally code can be removed I think... [Any way HBCK is short living 
process also] Thanks Jon for the patch. Sorry I could not check it yday night.
@Lars we are eagerly waiting for the good news from you regarding 94 release..:)

> Zookeeper session got closed while trying to assign the region to RS using 
> hbck -fix
> 
>
> Key: HBASE-5781
> URL: https://issues.apache.org/jira/browse/HBASE-5781
> Project: HBase
>  Issue Type: Bug
>  Components: hbck
>Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
>Reporter: Kristam Subba Swathi
>Assignee: Jonathan Hsieh
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: hbase-5781.patch
>
>
> After running the hbck in the cluster ,it is found that one region is not 
> assigned
> So the hbck -fix is used to fix this 
> But the assignment didnt happen since the zookeeper session is closed
> Please find the attached trace for more details
> -
> Trying to fix unassigned region...
> 12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition, 
> waiting for it to become assigned: {NAME => 
> 'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY => 
> '002300', ENDKEY => '002311', ENCODED => 00871fbd7583512e12c4eb38e900be8d,}
> 12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation: 
> Closed zookeeper sessionid=0x236738a263a
> 12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
> ERROR: Region { meta => 
> ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs => 
> hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed 
> => } not deployed on any region server.
> Trying to fix unassigned region...
> 12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable 
> to set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
> at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
> at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
> at 
> org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
> at 
> org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
> 12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher: 
> hconnection-0x236738a263a Received unexpected KeeperException, 
> re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode 
> = Session expired for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
> at 
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
> at 
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNo