date:20120315

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229944#comment-13229944
]

Phabricator commented on HBASE-5542:

sc has commented on the revision HBASE-5542 [jira] Unify
HRegion.mutateRowsWithLocks() and HRegion.processRow().

Thanks for the review.

What do you think if I take your patch and try to fit doMiniBatchPut into it
and then update the patch accordingly (if needed) - before we commit the patch?

Sure. Please make it better before you check it in.
But you will still go through your major work in the other JIRA, right?
https://issues.apache.org/jira/browse/HBASE-5565
We will do the review over there. There are too much information in this
thread.

INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:238 Sure. Let
me change it to 1 minute.
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4309 Thanks.
That's a good catch!
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:4286 I see.
We can still make remember some state after calling preProcess() and change the
output of getRowsToLock(). I like making getRowsToLock() returns the rows
rather than preProcess() because it seems more clear to me.

REVISION DETAIL
https://reviews.facebook.net/D2217

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.10.patch,
HBASE-5542.D2217.11.patch, HBASE-5542.D2217.12.patch,
HBASE-5542.D2217.13.patch, HBASE-5542.D2217.14.patch,
HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch,
HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch,
HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt

mutateRowsWithLocks() does atomic mutations on multiple rows.
processRow() does atomic read-modify-writes on a single row.
It will be useful to generalize both and have a
processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()


 [ 
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5542:
---

Attachment: HBASE-5542.D2217.15.patch

sc updated the revision HBASE-5542 [jira] Unify HRegion.mutateRowsWithLocks() 
and HRegion.processRow().
Reviewers: tedyu, lhofhansl, JIRA

  Addressed review comments from Lars, thanks!

REVISION DETAIL
  https://reviews.facebook.net/D2217

AFFECTED FILES
  
src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRowProcessorEndpoint.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/RowProcessorProtocol.java
  src/main/java/org/apache/hadoop/hbase/regionserver/BaseRowProcessor.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  
src/main/java/org/apache/hadoop/hbase/regionserver/MultiRowMutationProcessor.java
  src/main/java/org/apache/hadoop/hbase/regionserver/RowProcessor.java
  src/main/java/org/apache/hadoop/hbase/coprocessor/RowProcessor.java
  src/test/java/org/apache/hadoop/hbase/coprocessor/TestProcessRowEndpoint.java
  
src/test/java/org/apache/hadoop/hbase/coprocessor/TestRowProcessorEndpoint.java


 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
 

 Key: HBASE-5542
 URL: https://issues.apache.org/jira/browse/HBASE-5542
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5542.D2217.1.patch, HBASE-5542.D2217.10.patch, 
 HBASE-5542.D2217.11.patch, HBASE-5542.D2217.12.patch, 
 HBASE-5542.D2217.13.patch, HBASE-5542.D2217.14.patch, 
 HBASE-5542.D2217.15.patch, HBASE-5542.D2217.2.patch, 
 HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch, HBASE-5542.D2217.5.patch, 
 HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch, HBASE-5542.D2217.8.patch, 
 HBASE-5542.D2217.9.patch, HBASE-5542.txt


 mutateRowsWithLocks() does atomic mutations on multiple rows.
 processRow() does atomic read-modify-writes on a single row.
 It will be useful to generalize both and have a
 processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
 This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

2012-03-15 Thread Scott Chen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Chen updated HBASE-5542:
--

Attachment: HBASE-5542.2.txt

 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
 

 Key: HBASE-5542
 URL: https://issues.apache.org/jira/browse/HBASE-5542
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5542.2.txt, HBASE-5542.D2217.1.patch, 
 HBASE-5542.D2217.10.patch, HBASE-5542.D2217.11.patch, 
 HBASE-5542.D2217.12.patch, HBASE-5542.D2217.13.patch, 
 HBASE-5542.D2217.14.patch, HBASE-5542.D2217.15.patch, 
 HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch, 
 HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch, 
 HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt


 mutateRowsWithLocks() does atomic mutations on multiple rows.
 processRow() does atomic read-modify-writes on a single row.
 It will be useful to generalize both and have a
 processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
 This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229949#comment-13229949
 ] 

Phabricator commented on HBASE-5542:


sc has commented on the revision HBASE-5542 [jira] Unify 
HRegion.mutateRowsWithLocks() and HRegion.processRow().

  I am also manually upload a patch on JIRA.
  I think reviews.facebook.net doesn't generates the correct patch.
  Please use this one if you need to apply patch.
  https://issues.apache.org/jira/secure/attachment/12518431/HBASE-5542.2.txt

REVISION DETAIL
  https://reviews.facebook.net/D2217


 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
 

 Key: HBASE-5542
 URL: https://issues.apache.org/jira/browse/HBASE-5542
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5542.2.txt, HBASE-5542.D2217.1.patch, 
 HBASE-5542.D2217.10.patch, HBASE-5542.D2217.11.patch, 
 HBASE-5542.D2217.12.patch, HBASE-5542.D2217.13.patch, 
 HBASE-5542.D2217.14.patch, HBASE-5542.D2217.15.patch, 
 HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch, 
 HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch, 
 HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt


 mutateRowsWithLocks() does atomic mutations on multiple rows.
 processRow() does atomic read-modify-writes on a single row.
 It will be useful to generalize both and have a
 processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
 This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5571) Table will be disabling forever


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229966#comment-13229966
 ] 

ramkrishna.s.vasudevan commented on HBASE-5571:
---

Patch v2 does not include changes in v1? is it intended?

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229967#comment-13229967
]

Hadoop QA commented on HBASE-5542:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12518431/HBASE-5542.2.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 160 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1192//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1192//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1192//console

This message is automatically generated.

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

Attachments: HBASE-5542.2.txt, HBASE-5542.D2217.1.patch,
HBASE-5542.D2217.10.patch, HBASE-5542.D2217.11.patch,
HBASE-5542.D2217.12.patch, HBASE-5542.D2217.13.patch,
HBASE-5542.D2217.14.patch, HBASE-5542.D2217.15.patch,
HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch,
HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch,
HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt

[jira] [Commented] (HBASE-5571) Table will be disabling forever

2012-03-15 Thread chunhui shen (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229968#comment-13229968
 ] 

chunhui shen commented on HBASE-5571:
-

As stack's comment, patch v2 use zk's split info to check whether region is 
splitting, needn't HRegionServer keeping splitting-region list.

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values

2012-03-15 Thread Commented


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229970#comment-13229970
 ] 

Daniel Gómez Ferro commented on HBASE-5579:
---

0 wasn't counting before the change. There were 4 versions (0 to 3), the test 
removed version 1 and assumed 0 doesn't count so it expected 2 versions.

With this fix, 0 does indeed count and it expects 3 versions correctly.

If we set/check -1 we would have the same problem with version -1, a delete 
with version  -1 would also mask values inserted at timestamp -1.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5571) Table will be disabling forever


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229975#comment-13229975
 ] 

ramkrishna.s.vasudevan commented on HBASE-5571:
---

Sorry Chunhui.. Did not see Stack's comments fully.  Will review and get back 
to this.
Thanks

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5571) Table will be disabling forever

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229988#comment-13229988
 ] 

ramkrishna.s.vasudevan commented on HBASE-5571:
---

Handling disabling when nodeDeleted is needed as we try doing in 
SplitRegionHandler.

{code}
catch (NotServingRegionException nsre) {
   LOG.info(Server  + server +  returned  + nsre +  for  +
 region.getRegionNameAsString());
-  // Presume that master has stale data.  Presume remote side just split.
-  // Presume that the split message when it comes in will fix up the 
master's
-  // in memory cluster state.
+  RegionState regionState = regionsInTransition.get(encodedName);
+  if (regionState != null
+   (regionState.isSplitting() || regionState.isSplit())) {
+LOG.info(Region  + region.getRegionNameAsString()
++  is splitting, return);
+return;
+  } else {
+cancelClosingRegionIfDisabling(region);
+  }
{code}
Is this the correct place to add the code.  I feel NSRE will be a remote 
exception and this check should come in the below catch(Throwable) where we 
check for NSRE.
I remember happening lik that when i was working on some earlier issues.  
Correct me if am wrong.  BTW, thanks for the patch.

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5571) Table will be disabling forever

2012-03-15 Thread chunhui shen (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230003#comment-13230003
 ] 

chunhui shen commented on HBASE-5571:
-

In our testing with 90 version, close region rpc return NSRE rather than a 
remote exception if region is already been closed, so that table will be 
diabling all the time.

Maybe calling unassign(rs.getRegion()) in the nodeDeleted() will affect zk 
event. 
What about define a new state for RegionState.state which will retry closing if 
return NSRE.

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5571) Table will be disabling forever

2012-03-15 Thread chunhui shen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chunhui shen updated HBASE-5571:


Attachment: HBASE-5571v3.patch

patchv3 change unassign() in nodeDeleted() to invokeUnassign().

 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch, HBASE-5571v3.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5572) KeeperException.SessionExpiredException management could be improved in Master

2012-03-15 Thread nkeywal (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230035#comment-13230035
 ] 

nkeywal commented on HBASE-5572:


@stack: it seems the patch could be committed?

 KeeperException.SessionExpiredException management could be improved in Master
 --

 Key: HBASE-5572
 URL: https://issues.apache.org/jira/browse/HBASE-5572
 Project: HBase
  Issue Type: Improvement
  Components: master
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5572.v1.patch, 5572.v2.patch, 5572.v2.patch, 
 5572.v2.patch


 Synthesis:
  1) TestMasterZKSessionRecovery distinguish two cases on 
 SessionExpiredException. One is explicitly not managed. However, is seems 
 that there is no reason for this.
  2) The issue lies in ActiveMasterManager#blockUntilBecomingActiveMaster, a 
 quite complex function, with a useless recursive call.
  3) TestMasterZKSessionRecovery#testMasterZKSessionRecoverySuccess is 
 equivalent to TestZooKeeper#testMasterSessionExpired
  4) TestMasterZKSessionRecovery#testMasterZKSessionRecoveryFailure can be 
 removed if we merge the two cases mentioned above.
 Changes are:
  2) Changing ActiveMasterManager#blockUntilBecomingActiveMaster to have a 
 single case and remove recursion
  1) Removing TestMasterZKSessionRecovery
 Detailed justification:
 testMasterZKSessionRecoveryFailure says:
 {noformat}
   /**
* Negative test of master recovery from zk session expiry.
*
* Starts with one master. Fakes the master zk session expired.
* Ensures the master cannot recover the expired zk session since
* the master zk node is still there.
*/
   public void testMasterZKSessionRecoveryFailure() throws Exception {
 MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
 HMaster m = cluster.getMaster();
 m.abort(Test recovery from zk session expired,
   new KeeperException.SessionExpiredException());
 assertTrue(m.isStopped());
   }
 {noformat}
 This tests works, i.e. the assertion is always verified.
 But do we really want this behavior?
 When looking at the code, we see that this what's happening is strange:
 - HMaster#abort calls Master#abortNow. If HMaster#abortNow returns false 
 HMaster#abort stops the master.
 - HMaster#abortNow checks the exception type. As it's a 
 SessionExpiredException it will try to recover, calling 
 HMaster#tryRecoveringExpiredZKSession. If it cannot, it will return false 
 (and that will make HMaster#abort stopping the master)
 - HMaster#tryRecoveringExpiredZKSession recreates a ZooKeeperConnection and 
 then try to become the active master. If it cannot, it will return false (and 
 that will make HMaster#abort stopping the master).
 - HMaster#becomeActiveMaster returns the result of 
 ActiveMasterManager#blockUntilBecomingActiveMaster. 
 blockUntilBecomingActiveMaster says it will return false if there is any 
 error preventing it to become the active master.
 - ActiveMasterManager#blockUntilBecomingActiveMaster reads ZK for the master 
 address. If it's the same port  host, it deletes the nodes, that will start 
 a recursive call to blockUntilBecomingActiveMaster. This second call succeeds 
 (we became the active master) and return true. This result is ignored by the 
 first blockUntilBecomingActiveMaster: it return false (even if we actually 
 became the active master), hence the whole suite call returns false and 
 HMaster#abort stops the master.
 In other words, the comment says Ensures the master cannot recover the 
 expired zk session since the master zk node is still there. but we're 
 actually doing a check just for this and deleting the node. If we were not 
 ignoring the result, we would return true, so we would not stop the master, 
 so the test would fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble

2012-03-15 Thread nkeywal (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230040#comment-13230040
 ] 

nkeywal commented on HBASE-5399:


I've actually been able to reproduce the issue on a previous trunk version, so 
it's not linked to my changes, and that's a great step forward in term of 
rationality.

Sometimes when we trigger a zk session expiry the zk connection remains in the 
connected state. I don't have any explication for this, but at least it's not 
a new behavior. I will manage this case explicitly in the version I'm writing 
for #5549.

 Cut the link between the client and the zookeeper ensemble
 --

 Key: HBASE-5399
 URL: https://issues.apache.org/jira/browse/HBASE-5399
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.94.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 
 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 
 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 
 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 
 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 
 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 
 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 
 5399_inprogress.v9.patch, nochange.patch


 The link is often considered as an issue, for various reasons. One of them 
 being that there is a limit on the number of connection that ZK can manage. 
 Stack was suggesting as well to remove the link to master from HConnection.
 There are choices to be made considering the existing API (that we don't want 
 to break).
 The first patches I will submit on hadoop-qa should not be committed: they 
 are here to show the progress on the direction taken.
 ZooKeeper is used for:
 - public getter, to let the client do whatever he wants, and close ZooKeeper 
 when closing the connection = we have to deprecate this but keep it.
 - read get master address to create a master = now done with a temporary 
 zookeeper connection
 - read root location = now done with a temporary zookeeper connection, but 
 questionable. Used in public function locateRegion. To be reworked.
 - read cluster id = now done once with a temporary zookeeper connection.
 - check if base done is available = now done once with a zookeeper 
 connection given as a parameter
 - isTableDisabled/isTableAvailable = public functions, now done with a 
 temporary zookeeper connection.
  - Called internally from HBaseAdmin and HTable
 - getCurrentNrHRS(): public function to get the number of region servers and 
 create a pool of thread = now done with a temporary zookeeper connection
 -
 Master is used for:
 - getMaster public getter, as for ZooKeeper = we have to deprecate this but 
 keep it.
 - isMasterRunning(): public function, used internally by HMerge  HBaseAdmin
 - getHTableDescriptor*: public functions offering access to the master.  = 
 we could make them using a temporary master connection as well.
 Main points are:
 - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a 
 strongly coupled architecture ;-). This can be changed, but requires a lot of 
 modifications in these classes (likely adding a class in the middle of the 
 hierarchy, something like that). Anyway, non connected client will always be 
 really slower, because it's a tcp connection, and establishing a tcp 
 connection is slow.
 - having a link between ZK and all the client seems to make sense for some 
 Use Cases. However, it won't scale if a TCP connection is required for every 
 client
 - if we move the table descriptor part away from the client, we need to find 
 a new place for it.
 - we will have the same issue if HBaseAdmin (for both ZK  Master), may be we 
 can put a timeout on the connection. That would make the whole system less 
 deterministic however.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5399) Cut the link between the client and the zookeeper ensemble

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230050#comment-13230050
 ] 

stack commented on HBASE-5399:
--

I should have said, given you've repro'd in an older version -- freeing up this 
commit of blame -- AND that you are going to fix it anyways in 5549, I'd say 
its good to close this issue again.  Good stuff.

 Cut the link between the client and the zookeeper ensemble
 --

 Key: HBASE-5399
 URL: https://issues.apache.org/jira/browse/HBASE-5399
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.94.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 
 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 
 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 
 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 
 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 
 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 
 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 
 5399_inprogress.v9.patch, nochange.patch


 The link is often considered as an issue, for various reasons. One of them 
 being that there is a limit on the number of connection that ZK can manage. 
 Stack was suggesting as well to remove the link to master from HConnection.
 There are choices to be made considering the existing API (that we don't want 
 to break).
 The first patches I will submit on hadoop-qa should not be committed: they 
 are here to show the progress on the direction taken.
 ZooKeeper is used for:
 - public getter, to let the client do whatever he wants, and close ZooKeeper 
 when closing the connection = we have to deprecate this but keep it.
 - read get master address to create a master = now done with a temporary 
 zookeeper connection
 - read root location = now done with a temporary zookeeper connection, but 
 questionable. Used in public function locateRegion. To be reworked.
 - read cluster id = now done once with a temporary zookeeper connection.
 - check if base done is available = now done once with a zookeeper 
 connection given as a parameter
 - isTableDisabled/isTableAvailable = public functions, now done with a 
 temporary zookeeper connection.
  - Called internally from HBaseAdmin and HTable
 - getCurrentNrHRS(): public function to get the number of region servers and 
 create a pool of thread = now done with a temporary zookeeper connection
 -
 Master is used for:
 - getMaster public getter, as for ZooKeeper = we have to deprecate this but 
 keep it.
 - isMasterRunning(): public function, used internally by HMerge  HBaseAdmin
 - getHTableDescriptor*: public functions offering access to the master.  = 
 we could make them using a temporary master connection as well.
 Main points are:
 - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a 
 strongly coupled architecture ;-). This can be changed, but requires a lot of 
 modifications in these classes (likely adding a class in the middle of the 
 hierarchy, something like that). Anyway, non connected client will always be 
 really slower, because it's a tcp connection, and establishing a tcp 
 connection is slow.
 - having a link between ZK and all the client seems to make sense for some 
 Use Cases. However, it won't scale if a TCP connection is required for every 
 client
 - if we move the table descriptor part away from the client, we need to find 
 a new place for it.
 - we will have the same issue if HBaseAdmin (for both ZK  Master), may be we 
 can put a timeout on the connection. That would make the whole system less 
 deterministic however.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5399) Cut the link between the client and the zookeeper ensemble

2012-03-15 Thread nkeywal (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal resolved HBASE-5399.


Resolution: Fixed

 Cut the link between the client and the zookeeper ensemble
 --

 Key: HBASE-5399
 URL: https://issues.apache.org/jira/browse/HBASE-5399
 Project: HBase
  Issue Type: Improvement
  Components: client
Affects Versions: 0.94.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5399.v27.patch, 5399.v38.patch, 5399.v39.patch, 
 5399.v40.patch, 5399.v41.patch, 5399.v42.patch, 5399.v42.patch, 
 5399.v42.patch, 5399.v42.patch, 5399_inprogress.patch, 
 5399_inprogress.v14.patch, 5399_inprogress.v16.patch, 
 5399_inprogress.v18.patch, 5399_inprogress.v20.patch, 
 5399_inprogress.v21.patch, 5399_inprogress.v23.patch, 
 5399_inprogress.v3.patch, 5399_inprogress.v32.patch, 
 5399_inprogress.v9.patch, nochange.patch


 The link is often considered as an issue, for various reasons. One of them 
 being that there is a limit on the number of connection that ZK can manage. 
 Stack was suggesting as well to remove the link to master from HConnection.
 There are choices to be made considering the existing API (that we don't want 
 to break).
 The first patches I will submit on hadoop-qa should not be committed: they 
 are here to show the progress on the direction taken.
 ZooKeeper is used for:
 - public getter, to let the client do whatever he wants, and close ZooKeeper 
 when closing the connection = we have to deprecate this but keep it.
 - read get master address to create a master = now done with a temporary 
 zookeeper connection
 - read root location = now done with a temporary zookeeper connection, but 
 questionable. Used in public function locateRegion. To be reworked.
 - read cluster id = now done once with a temporary zookeeper connection.
 - check if base done is available = now done once with a zookeeper 
 connection given as a parameter
 - isTableDisabled/isTableAvailable = public functions, now done with a 
 temporary zookeeper connection.
  - Called internally from HBaseAdmin and HTable
 - getCurrentNrHRS(): public function to get the number of region servers and 
 create a pool of thread = now done with a temporary zookeeper connection
 -
 Master is used for:
 - getMaster public getter, as for ZooKeeper = we have to deprecate this but 
 keep it.
 - isMasterRunning(): public function, used internally by HMerge  HBaseAdmin
 - getHTableDescriptor*: public functions offering access to the master.  = 
 we could make them using a temporary master connection as well.
 Main points are:
 - hbase class for ZooKeeper; ZooKeeperWatcher is really designed for a 
 strongly coupled architecture ;-). This can be changed, but requires a lot of 
 modifications in these classes (likely adding a class in the middle of the 
 hierarchy, something like that). Anyway, non connected client will always be 
 really slower, because it's a tcp connection, and establishing a tcp 
 connection is slow.
 - having a link between ZK and all the client seems to make sense for some 
 Use Cases. However, it won't scale if a TCP connection is required for every 
 client
 - if we move the table descriptor part away from the client, we need to find 
 a new place for it.
 - we will have the same issue if HBaseAdmin (for both ZK  Master), may be we 
 can put a timeout on the connection. That would make the whole system less 
 deterministic however.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5571) Table will be disabling forever


[ 
https://issues.apache.org/jira/browse/HBASE-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230057#comment-13230057
 ] 

ramkrishna.s.vasudevan commented on HBASE-5571:
---

@Chunhui
+1 on making invokeUnassign().  But NSRE is my doubt. I will try out once if am 
able to reproduce it.  


 Table will be disabling forever
 ---

 Key: HBASE-5571
 URL: https://issues.apache.org/jira/browse/HBASE-5571
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: BASE-5571v2.patch, HBASE-5571.patch, HBASE-5571v3.patch


 If we restart master when it is disabling one table, the table will be 
 disabling forever.
 In current logic, Region CLOSE RPC will always returned 
 NotServingRegionException because RS has already closed the region before we 
 restart master. So table will be disabling forever because the region will in 
 RIT all along.
 In another case, when AssignmentManager#rebuildUserRegions(), it will put 
 parent regions to AssignmentManager.regions, so we can't close these parent 
 regions until it is purged by CatalogJanitor if we execute disabling the 
 table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions

2012-03-15 Thread ramkrishna.s.vasudevan (Created) (JIRA)

Master restart on create table with splitkeys does not recreate table with all 
the splitkey regions
---

 Key: HBASE-5583
 URL: https://issues.apache.org/jira/browse/HBASE-5583
 Project: HBase
  Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0


- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of regions 
than specified in splitkeys

Anyway client will get an exception if i had called sync create table.  But 
table exists or not check will say table exists. 
Is this scenario to be handled by client only or can we have some mechanism on 
the master side for this? Pls suggest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup

2012-03-15 Thread jirapos...@reviews.apache.org (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230148#comment-13230148
]

jirapos...@reviews.apache.org commented on HBASE-5209:
--

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3892/
---

(Updated 2012-03-15 13:16:01.439710)

Review request for hbase.

Changes
---

Addressed Jon's and Stack's comments with addendum patch. Passed unit tests
and I threw it onto a setup to make sure everything looked OK.

Summary
---

Problem:
There is no method in the HBase client-facing APIs to determine which of the
masters is currently active. This can be especially useful in setups with
multiple backup masters.

Solution:
Augment ClusterStatus to return the currently active master and the list of
backup masters.

Notes:
* I uncovered a race condition in ActiveMasterManager, between when it
determines that it did not win the original race to be the active master, and
when it reads the ServerName of the active master. If the active master goes
down in that time, the read to determine the active master's ServerName will
fail ungracefully and the candidate master will abort. The solution
incorporated in this patch is to check to see if the read of the ServerName
succeeded before trying to use it.
* I fixed some minor formatting issues while going through the code. I can
take these changes out if it is considered improper to commit such non-related
changes with the main changes.

This addresses bug HBASE-5209.
https://issues.apache.org/jira/browse/HBASE-5209

Diffs (updated)
-

src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 9df4c10
src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 7416ae2
src/main/java/org/apache/hadoop/hbase/master/HMaster.java e2bbbd0

Diff: https://reviews.apache.org/r/3892/diff

Testing
---

* Ran mvn -P localTests test multiple times - no new tests fail
* Ran mvn -P localTests -Dtest=TestActiveMasterManager test multiple runs - no
failures
* Ran mvn -P localTests -Dtest=TestMasterFailover test multiple runs - no
failures
* Started active and multiple backup masters, then killed active master, then
brought it back up (will now be a backup master)
* Did the following before and after killing
* hbase hbck -details - checked output to see that active and backup
masters are reported properly
* zk_dump - checked that active and backup masters are reported properly
* Started cluster with no backup masters to make sure change operates correctly
that way
* Tested build with this diff vs. build without this diff, in all combinations
of client and server
* Verified that new client can run against old servers without incident and
with the defaults applied.
* Note that old clients get an error when running against new servers,
because the old readFields() code in ClusterStatus does not handle exceptions
of any kind. This is not solvable, at least in the scope of this change.

12/02/15 15:15:38 INFO zookeeper.ClientCnxn: Session establishment complete on
server haus02.sf.cloudera.com/172.29.5.33:30181, sessionid = 0x135834c75e20008,
negotiated timeout = 5000
12/02/15 15:15:39 ERROR io.HbaseObjectWritable: Error in readFields
A record version mismatch occured. Expecting v2, found v3
at
org.apache.hadoop.io.VersionedWritable.readFields(VersionedWritable.java:46)
at
org.apache.hadoop.hbase.ClusterStatus.readFields(ClusterStatus.java:247)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:583)
at
org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:297)

* Ran dev-support/test-patch.sh - no new issues fail:

-1 overall.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 7 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -136 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version )
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Thanks,

David

HConnection/HMasterInterface should allow for way to get hostname of
currently active master in multi-master HBase setup

Key: HBASE-5209
URL: https://issues.apache.org/jira/browse/HBASE-5209
Project: HBase
Issue Type: Improvement
Components: master

[jira] [Created] (HBASE-5584) Coprocessor hooks can be called in the respective handlers

2012-03-15 Thread ramkrishna.s.vasudevan (Created) (JIRA)

Coprocessor hooks can be called in the respective handlers
--

 Key: HBASE-5584
 URL: https://issues.apache.org/jira/browse/HBASE-5584
 Project: HBase
  Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0


Following points can be changed w.r.t to coprocessors

- Call preCreate, postCreate, preEnable, postEnable, etc. in their respective 
handlers
- Currently it is called in the HMaster thus making the postApis async w.r.t 
the handlers
- Similar is the case with the balancer.

with current behaviour once we are in the postEnable(for eg) we any way need to 
wait for the main enable handler to 
be completed.
We should ensure that we dont wait in the main thread so again we need to spawn 
a thread and wait on that.
On the other hand if the pre and post api is called on the handlers then only 
that handler thread will be
used in the pre/post apis

If the above said plan is ok i can prepare a patch for all such related changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5584) Coprocessor hooks can be called in the respective handlers

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230157#comment-13230157
 ] 

ramkrishna.s.vasudevan commented on HBASE-5584:
---

One more reason for doing this could be on master failover we call 
EnableTableHandler and DisableTableHandler for partially enabled/disabled 
tables.  But we don't call the coprocessor hooks in such scenarios.



 Coprocessor hooks can be called in the respective handlers
 --

 Key: HBASE-5584
 URL: https://issues.apache.org/jira/browse/HBASE-5584
 Project: HBase
  Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0


 Following points can be changed w.r.t to coprocessors
 - Call preCreate, postCreate, preEnable, postEnable, etc. in their 
 respective handlers
 - Currently it is called in the HMaster thus making the postApis async w.r.t 
 the handlers
 - Similar is the case with the balancer.
 with current behaviour once we are in the postEnable(for eg) we any way need 
 to wait for the main enable handler to 
 be completed.
 We should ensure that we dont wait in the main thread so again we need to 
 spawn a thread and wait on that.
 On the other hand if the pre and post api is called on the handlers then only 
 that handler thread will be
 used in the pre/post apis
 If the above said plan is ok i can prepare a patch for all such related 
 changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays


 [ 
https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5358:
--

Fix Version/s: 0.92.1

 HBaseObjectWritable should be able to serialize/deserialize generic arrays
 --

 Key: HBASE-5358
 URL: https://issues.apache.org/jira/browse/HBASE-5358
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors, io
Affects Versions: 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 0.92.1, 0.94.0

 Attachments: 5358-92.txt, HBASE-5358_v3.patch


 HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where 
 A extends Writable. This becomes an issue for example when adding a 
 coprocessor method which takes A[] (see HBASE-5352). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5569) TestAtomicOperation.testMultiRowMutationMultiThreads fails occasionally

2012-03-15 Thread Lars Hofhansl (Issue Comment Edited) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230207#comment-13230207
 ] 

Lars Hofhansl commented on HBASE-5569:
--

Ran all test in TestAtomicOperation more that 3000 times without a failure.

 TestAtomicOperation.testMultiRowMutationMultiThreads fails occasionally
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (HBASE-5569) TestAtomicOperation.testMultiRowMutationMultiThreads fails occasionally


[ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230207#comment-13230207
 ] 

Lars Hofhansl edited comment on HBASE-5569 at 3/15/12 2:42 PM:
---

Ran all tests in TestAtomicOperation more that 3000 times without a failure.

  was (Author: lhofhansl):
Ran all test in TestAtomicOperation more that 3000 times without a failure.
  
 TestAtomicOperation.testMultiRowMutationMultiThreads fails occasionally
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.

2012-03-15 Thread Lars Hofhansl (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-5569:
-

Description:
I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
fails rarely.
The solution is similar to HBASE-2856, where expired KVs are not collected when
in use by a scanner.

---
What I pieced together so far is that it is the *scanning* side that has
problems sometimes.

Every time I see a assertion failure in the log I see this before:
{quote}
2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
Storescanner.peek() is changed where before =
rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
{quote}
The order of if the Put and Delete is sometimes reversed.

The test threads should always see exactly one KV, if the before was the Put
the thread see 0 KVs, if the before was the Delete the threads see 2 KVs.

This debug message comes from StoreScanner to checkReseek. It seems we still
some consistency issue with scanning sometimes :(

was:
What I pieced together so far is that it is the *scanning* side that has
problems sometimes.

The test threads should always see exactly one KV, if the before was the Put
the thread see 0 KVs, if the before was the Delete the threads see 2 KVs.

This debug message comes from StoreScanner to checkReseek. It seems we still
some consistency issue with scanning sometimes :(

Priority: Major (was: Minor)
Summary: Do not collect deleted KVs when they are still in use by a
scanner. (was: TestAtomicOperation.testMultiRowMutationMultiThreads fails
occasionally)

Do not collect deleted KVs when they are still in use by a scanner.
---

Key: HBASE-5569
URL: https://issues.apache.org/jira/browse/HBASE-5569
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0, 0.96.0

Attachments: 5569-v2.txt, 5569.txt,
TestAtomicOperation-output.trunk_120313.rar

I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
fails rarely.
The solution is similar to HBASE-2856, where expired KVs are not collected
when in use by a scanner.
---
What I pieced together so far is that it is the *scanning* side that has
problems sometimes.
Every time I see a assertion failure in the log I see this before:
{quote}
2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
Storescanner.peek() is changed where before =
rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
{quote}
The order of if the Put and Delete is sometimes reversed.
The test threads should always see exactly one KV, if the before was the
Put the thread see 0 KVs, if the before was the Delete the threads see 2
KVs.
This debug message comes from StoreScanner to checkReseek. It seems we still
some consistency issue with scanning sometimes :(

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230227#comment-13230227
 ] 

Lars Hofhansl commented on HBASE-5579:
--

I see. Makes sense.
Maybe use Long.MIN_VALUE?

I'm fine with the patch, just a bit of hesitation, since the more state 
variables we have the easier it will be in the future to mess this up.


 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

2012-03-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230236#comment-13230236
 ] 

Jonathan Hsieh commented on HBASE-5563:
---

Hey Chenhui, looked at your code and had a question (similar to stack's) -- did 
you intend to have newer regions (larger timestamps) sorted *before* older 
HRegionInfos?  If so why?  My intuition says to to have newer regions sorted 
after older ones.  At the moment nothing depends on one order vs another, so I 
would like to add some simple unit tests to make the desired behavior this 
explicitly tested.  

Also, I have a few runs with the broken test passing now (a bunch of tests seem 
to be flakey on my setup). 

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values

2012-03-15 Thread Commented


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230247#comment-13230247
 ] 

Daniel Gómez Ferro commented on HBASE-5579:
---

Then it could bit someone who inserts values with Long.MIN_VALUE, not something 
that should happen often but if it happens it is going to be hard to debug.

If you are worried about state variables we could use a Long and reset() it to 
null, but I think that's not very clean.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5542:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12517842/HBASE-5542.D2217.4.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -120 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 158 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1155//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1155//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1155//console

This message is automatically generated.)

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5542:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12518084/HBASE-5542.D2217.9.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 159 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1168//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1168//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1168//console

This message is automatically generated.)

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5542:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12518068/HBASE-5542.D2217.8.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

-1 javadoc. The javadoc tool appears to have generated -120 warning
messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 158 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1166//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1166//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1166//console

This message is automatically generated.)

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

[jira] [Updated] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

[
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhihong Yu updated HBASE-5542:
--

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12518295/HBASE-5542.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 160 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1183//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1183//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1183//console

This message is automatically generated.)

Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()

Key: HBASE-5542
URL: https://issues.apache.org/jira/browse/HBASE-5542
Project: HBase
Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230257#comment-13230257
 ] 

Lars Hofhansl commented on HBASE-5579:
--

Fair enough. +1 on patch.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5542) Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()


[ 
https://issues.apache.org/jira/browse/HBASE-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230267#comment-13230267
 ] 

Zhihong Yu commented on HBASE-5542:
---

That would be done in HBASE-5565, right ?

 Unify HRegion.mutateRowsWithLocks() and HRegion.processRow()
 

 Key: HBASE-5542
 URL: https://issues.apache.org/jira/browse/HBASE-5542
 Project: HBase
  Issue Type: Improvement
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5542.2.txt, HBASE-5542.D2217.1.patch, 
 HBASE-5542.D2217.10.patch, HBASE-5542.D2217.11.patch, 
 HBASE-5542.D2217.12.patch, HBASE-5542.D2217.13.patch, 
 HBASE-5542.D2217.14.patch, HBASE-5542.D2217.15.patch, 
 HBASE-5542.D2217.2.patch, HBASE-5542.D2217.3.patch, HBASE-5542.D2217.4.patch, 
 HBASE-5542.D2217.5.patch, HBASE-5542.D2217.6.patch, HBASE-5542.D2217.7.patch, 
 HBASE-5542.D2217.8.patch, HBASE-5542.D2217.9.patch, HBASE-5542.txt


 mutateRowsWithLocks() does atomic mutations on multiple rows.
 processRow() does atomic read-modify-writes on a single row.
 It will be useful to generalize both and have a
 processRowsWithLocks() that does atomic read-modify-writes on multiple rows.
 This also helps reduce some redundancy in the codes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.

2012-03-15 Thread ramkrishna.s.vasudevan (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230295#comment-13230295
]

Lars Hofhansl commented on HBASE-5569:
--

3500 test runs, no failures. Going to commit if nobody objects.

Do not collect deleted KVs when they are still in use by a scanner.
---

Key: HBASE-5569
URL: https://issues.apache.org/jira/browse/HBASE-5569
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0, 0.96.0

Attachments: 5569-v2.txt, 5569.txt,
TestAtomicOperation-output.trunk_120313.rar

[jira] [Commented] (HBASE-5312) Closed parent region present in Hlog.lastSeqWritten


[ 
https://issues.apache.org/jira/browse/HBASE-5312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230334#comment-13230334
 ] 

ramkrishna.s.vasudevan commented on HBASE-5312:
---

@Todd/Jimmy
Do you feel HBASE-5568 solves the problem for your scenario also? If so we can 
close this issue i feel?

 Closed parent region present in Hlog.lastSeqWritten
 ---

 Key: HBASE-5312
 URL: https://issues.apache.org/jira/browse/HBASE-5312
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: ramkrishna.s.vasudevan
 Fix For: 0.90.7


 This is in reference to the mail sent in the dev mailing list
 Closed parent region present in Hlog.lastSeqWritten.
 The sceanrio described is
 We had a region that was split into two daughters.  When the hlog roll tried 
 to flush the region there was an entry in the HLog.lastSeqWritten that was 
 not flushed or removed from the lastSeqWritten during the parent close.
 Because this flush was not happening subsequent flushes were getting blocked
 {code}
  05:06:44,422 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=122, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:06:44,422 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 requester=null
  05:10:48,666 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=123, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:10:48,666 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 requester=null
  05:14:46,075 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=124, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:14:46,075 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 requester=null
  05:15:41,584 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: Too many
  hlogs: logs=125, maxlogs=32; forcing flush of 1 regions(s):
  2acaf8e3acfd2e8a5825a1f6f0aca4a8
  05:15:41,584 WARN org.apache.hadoop.hbase.regionserver.LogRoller: Failed
  to schedule flush of 2acaf8e3acfd2e8a5825a1f6f0aca4a8r=null,
 {code}
 Lets see what happened for the region 2acaf8e3acfd2e8a5825a1f6f0aca4a8
 {code}
 2012-01-06 00:30:55,214 INFO org.apache.hadoop.hbase.regionserver.Store: 
 Renaming flushed file at 
 hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/.tmp/1755862026714756815
  to 
 hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123
 2012-01-06 00:30:58,946 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
 Instantiated 
 Htable_UFDR_016,049790700093168-0456520,1325809837958.0ebe5bd7fcbc09ee074d5600b9d4e062.
 2012-01-06 00:30:59,614 INFO org.apache.hadoop.hbase.regionserver.Store: 
 Added 
 hdfs://192.168.1.103:9000/hbase/Htable_UFDR_031/2acaf8e3acfd2e8a5825a1f6f0aca4a8/value/973789709483406123,
  entries=7537, sequenceid=20312223, memsize=4.2m, filesize=2.9m
 2012-01-06 00:30:59,787 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: 
 Finished snapshotting, commencing flushing stores
 2012-01-06 00:30:59,787 INFO org.apache.hadoop.hbase.regionserver.HRegion: 
 Finished memstore flush of ~133.5m for region 
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. in 
 21816ms, sequenceid=20312223, compaction requested=true
 2012-01-06 00:30:59,787 DEBUG 
 org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested 
 for Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8. 
 because regionserver20020.cacheFlusher; priority=0, compaction queue size=5840
 {code}
 A user triggered split has been issued to this region which can be seen in 
 the above logs.
 The flushing of this region has resulted in a seq id 20312223.
 The region has been splitted and the parent region has been closed
 {code}
 00:31:12,607 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: 
 Starting split of region 
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing 
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.: 
 disabling compactions  flushes
 00:31:13,694 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates 
 disabled for region 
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 00:31:13,718 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed 
 Htable_UFDR_031,00332,1325808823997.2acaf8e3acfd2e8a5825a1f6f0aca4a8.
 00:31:39,552 INFO

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230360#comment-13230360
 ] 

Zhihong Yu commented on HBASE-4608:
---

w.r.t. Lars' comment: 
https://issues.apache.org/jira/browse/HBASE-4608?focusedCommentId=13229010page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13229010

I think it makes sense.
How about introducing an enum CompressionType with values of NONE and 
DICTIONARY ?
HConstants.ENABLE_WAL_COMPRESSION would be replaced by another String: 
hbase.regionserver.wal.compressiontype
If hbase.regionserver.wal.compressiontype doesn't appear in conf, 
CompressionType.NONE is assumed.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: stack
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5584) Coprocessor hooks can be called in the respective handlers

2012-03-15 Thread Mingjie Lai (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230368#comment-13230368
]

Mingjie Lai commented on HBASE-5584:

Ramkrishna.
One thing you need to take into consideration is hbase ACL. Right now table
level permission control (table creation, enabling, etc) is performed at master
preCreate, preEnable e.g. hooks, and a request gets denied by throwing an
exception.
If you want to move preXXX to handler threads, the ACL exception cannot be
handled properly to indicate a permission rejection. So in this case, if a user
tries to delete someone else's tables, she won't see any exception although the
request has been rejected.

Coprocessor hooks can be called in the respective handlers
--

Key: HBASE-5584
URL: https://issues.apache.org/jira/browse/HBASE-5584
Project: HBase
Issue Type: Improvement
Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0

Following points can be changed w.r.t to coprocessors
- Call preCreate, postCreate, preEnable, postEnable, etc. in their
respective handlers
- Currently it is called in the HMaster thus making the postApis async w.r.t
the handlers
- Similar is the case with the balancer.
with current behaviour once we are in the postEnable(for eg) we any way need
to wait for the main enable handler to
be completed.
We should ensure that we dont wait in the main thread so again we need to
spawn a thread and wait on that.
On the other hand if the pre and post api is called on the handlers then only
that handler thread will be
used in the pre/post apis
If the above said plan is ok i can prepare a patch for all such related
changes.

[jira] [Created] (HBASE-5585) Add a feature in shell to allow adding entries into META easily

2012-03-15 Thread Shrijeet Paliwal (Created) (JIRA)

Add a feature in shell to allow adding entries into META easily
---

 Key: HBASE-5585
 URL: https://issues.apache.org/jira/browse/HBASE-5585
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Shrijeet Paliwal
Priority: Minor


From IRC 

{noformat}
sulabhc St^Ack: that fixed most of the issue, just one last question, I removed 
some META entries how do I put it back ?
sulabhc St^Ack: putting from Hbase-shell does not seem to be possible
St^Ack  sulabhc: it is but the content in .META. is serialized Writable of 
HRegionInfo.
St^Ack  Can you find the region in the fs that you want to put back?
St^Ack  If so, cat its .regioninfo
St^Ack  it has binary format of the HRegionInfo that needs to be in .META. and 
a txt version.
St^Ack  You'll need to create an HRegionInfo, serialize it as bytes, and then 
put that into .META.
St^Ack  Look around in the catalog package, the MetaEditor class, for how to do 
this from java.
St^Ack  We should really make this easier to do in shell...
St^Ack  or look in bin to see how we do some of this via (j)ruby
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5584) Coprocessor hooks can be called in the respective handlers

2012-03-15 Thread Andrew Purtell (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230385#comment-13230385
 ] 

Andrew Purtell commented on HBASE-5584:
---

Ram,

What Mingjie said. The coprocessor hooks can't be asynchronous with respect to 
client action.

Would adding additional hooks for handler actions work for you here?

 Coprocessor hooks can be called in the respective handlers
 --

 Key: HBASE-5584
 URL: https://issues.apache.org/jira/browse/HBASE-5584
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0


 Following points can be changed w.r.t to coprocessors
 - Call preCreate, postCreate, preEnable, postEnable, etc. in their 
 respective handlers
 - Currently it is called in the HMaster thus making the postApis async w.r.t 
 the handlers
 - Similar is the case with the balancer.
 with current behaviour once we are in the postEnable(for eg) we any way need 
 to wait for the main enable handler to 
 be completed.
 We should ensure that we dont wait in the main thread so again we need to 
 spawn a thread and wait on that.
 On the other hand if the pre and post api is called on the handlers then only 
 that handler thread will be
 used in the pre/post apis
 If the above said plan is ok i can prepare a patch for all such related 
 changes.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires

2012-03-15 Thread nkeywal (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5549:
---

Attachment: 5549.v6.patch

Contains the patch for HBASE-5572

 Master can fail if ZooKeeper session expires
 

 Key: HBASE-5549
 URL: https://issues.apache.org/jira/browse/HBASE-5549
 Project: HBase
  Issue Type: Bug
  Components: master, zookeeper
Affects Versions: 0.96.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5549.v6.patch


 There is a retry mechanism in RecoverableZooKeeper, but when the session 
 expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism 
 does not work in this case. This is why a sleep is needed in 
 TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher 
 to be recreated before using the connection.
 This can happen in real life, it can happen when:
 - master  zookeeper starts
 - zookeeper connection is cut
 - master enters the retry loop
 - in the meantime the session expires
 - the network comes back, the session is recreated
 - the retries continues, but on the wrong object, hence fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5549) Master can fail if ZooKeeper session expires

2012-03-15 Thread nkeywal (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5549:
---

Status: Patch Available  (was: Open)

 Master can fail if ZooKeeper session expires
 

 Key: HBASE-5549
 URL: https://issues.apache.org/jira/browse/HBASE-5549
 Project: HBase
  Issue Type: Bug
  Components: master, zookeeper
Affects Versions: 0.96.0
 Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5549.v6.patch


 There is a retry mechanism in RecoverableZooKeeper, but when the session 
 expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism 
 does not work in this case. This is why a sleep is needed in 
 TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher 
 to be recreated before using the connection.
 This can happen in real life, it can happen when:
 - master  zookeeper starts
 - zookeeper connection is cut
 - master enters the retry loop
 - in the meantime the session expires
 - the network comes back, the session is recreated
 - the retries continues, but on the wrong object, hence fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.

2012-03-15 Thread Lars Hofhansl (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5569:
-

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to 0.94 and trunk

 Do not collect deleted KVs when they are still in use by a scanner.
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads 
 fails rarely.
 The solution is similar to HBASE-2856, where expired KVs are not collected 
 when in use by a scanner.
 ---
 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230427#comment-13230427
 ] 

Zhihong Yu commented on HBASE-5579:
---

Will integrate the patch this evening if there is no objection.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.


[ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230428#comment-13230428
 ] 

stack commented on HBASE-5569:
--

@Lars Good one.

 Do not collect deleted KVs when they are still in use by a scanner.
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads 
 fails rarely.
 The solution is similar to HBASE-2856, where expired KVs are not collected 
 when in use by a scanner.
 ---
 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230445#comment-13230445
 ] 

stack commented on HBASE-5579:
--

+1 on patch

This was messing up bulk loaded data?

How did you figure it Daniel?  Nice digging.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230449#comment-13230449
 ] 

stack commented on HBASE-4608:
--

I'll commit v30 then.

Thanks all for reviews, etc.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: stack
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values

2012-03-15 Thread Commented


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230450#comment-13230450
 ] 

Daniel Gómez Ferro commented on HBASE-5579:
---

It was messing a YCSB experiment where we do the initial load with TS = 0

We spent a good time trying to figure out who was removing data from our 
cluster!

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230456#comment-13230456
 ] 

stack commented on HBASE-4608:
--

Now this is in, does that mean we can cut a 0.94RC0?

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4608) HLog Compression

2012-03-15 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-4608:
-

  Resolution: Fixed
Assignee: Li Pi  (was: stack)
Release Note: Adds a custom dictionary-based compression on WAL.  Off by 
default.  To enable, set hbase.regionserver.wal.enablecompression to true in 
hbase-site.xml.  (was: Patch for WAL Compression.)
  Status: Resolved  (was: Patch Available)

Committed 0.94 branch and trunk.  Thanks for the patch Li and to all others who 
hacked on this patch and reviewed it.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires

[
https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230457#comment-13230457
]

Hadoop QA commented on HBASE-5549:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12518500/5549.v6.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 20 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 161 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.client.TestMultiParallel

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1193//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1193//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1193//console

This message is automatically generated.

Master can fail if ZooKeeper session expires

Key: HBASE-5549
URL: https://issues.apache.org/jira/browse/HBASE-5549
Project: HBase
Issue Type: Bug
Components: master, zookeeper
Affects Versions: 0.96.0
Environment: all
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Attachments: 5549.v6.patch

There is a retry mechanism in RecoverableZooKeeper, but when the session
expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism
does not work in this case. This is why a sleep is needed in
TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher
to be recreated before using the connection.
This can happen in real life, it can happen when:
- master zookeeper starts
- zookeeper connection is cut
- master enters the retry loop
- in the meantime the session expires
- the network comes back, the session is recreated
- the retries continues, but on the wrong object, hence fails.

[jira] [Updated] (HBASE-5579) A Delete Version could mask other values


 [ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5579:
--

Fix Version/s: 0.96.0
   0.94.0
 Hadoop Flags: Reviewed

This should go to 0.94, I assume

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230490#comment-13230490
 ] 

Lars Hofhansl commented on HBASE-4608:
--

Yeah! And, yes, time for an RC.

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.


[ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230492#comment-13230492
 ] 

Hudson commented on HBASE-5569:
---

Integrated in HBase-0.94 #32 (See 
[https://builds.apache.org/job/HBase-0.94/32/])
HBASE-5569  Do not collect deleted KVs when they are still in use by a 
scanner. (Revision 1301138)

 Result = SUCCESS
larsh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java


 Do not collect deleted KVs when they are still in use by a scanner.
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads 
 fails rarely.
 The solution is similar to HBASE-2856, where expired KVs are not collected 
 when in use by a scanner.
 ---
 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230495#comment-13230495
 ] 

Zhihong Yu commented on HBASE-5579:
---

Integrated to TRUNK.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230493#comment-13230493
 ] 

Hudson commented on HBASE-4608:
---

Integrated in HBase-0.94 #32 (See 
[https://builds.apache.org/job/HBase-0.94/32/])
HBASE-4608 HLog Compression (Revision 1301167)

 Result = SUCCESS
stack : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/Compressor.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/Dictionary.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/KeyValueCompression.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/LRUDictionary.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCompressor.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestKeyValueCompression.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLRUDictionary.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplayCompressed.java


 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression

2012-03-15 Thread Li Pi (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230500#comment-13230500
 ] 

Li Pi commented on HBASE-4608:
--

Woohoo! It's in!

 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230536#comment-13230536
 ] 

Hudson commented on HBASE-5579:
---

Integrated in HBase-TRUNK #2683 (See 
[https://builds.apache.org/job/HBase-TRUNK/2683/])
HBASE-5579 A Delete Version could mask other values (Daniel Ferro) 
(Revision 1301173)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanDeleteTracker.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanDeleteTracker.java


 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5569) Do not collect deleted KVs when they are still in use by a scanner.


[ 
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230537#comment-13230537
 ] 

Hudson commented on HBASE-5569:
---

Integrated in HBase-TRUNK #2683 (See 
[https://builds.apache.org/job/HBase-TRUNK/2683/])
HBASE-5569  Do not collect deleted KVs when they are still in use by a 
scanner. (Revision 1301135)

 Result = FAILURE
larsh : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestAtomicOperation.java


 Do not collect deleted KVs when they are still in use by a scanner.
 ---

 Key: HBASE-5569
 URL: https://issues.apache.org/jira/browse/HBASE-5569
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Fix For: 0.94.0, 0.96.0

 Attachments: 5569-v2.txt, 5569.txt, 
 TestAtomicOperation-output.trunk_120313.rar


 I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads 
 fails rarely.
 The solution is similar to HBASE-2856, where expired KVs are not collected 
 when in use by a scanner.
 ---
 What I pieced together so far is that it is the *scanning* side that has 
 problems sometimes.
 Every time I see a assertion failure in the log I see this before:
 {quote}
 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499): 
 Storescanner.peek() is changed where before = 
 rowB/colfamily11:qual1/75366/Put/vlen=6,and after = 
 rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
 {quote}
 The order of if the Put and Delete is sometimes reversed.
 The test threads should always see exactly one KV, if the before was the 
 Put the thread see 0 KVs, if the before was the Delete the threads see 2 
 KVs.
 This debug message comes from StoreScanner to checkReseek. It seems we still 
 some consistency issue with scanning sometimes :(

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4608) HLog Compression


[ 
https://issues.apache.org/jira/browse/HBASE-4608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230538#comment-13230538
 ] 

Hudson commented on HBASE-4608:
---

Integrated in HBase-TRUNK #2683 (See 
[https://builds.apache.org/job/HBase-TRUNK/2683/])
HBASE-4608 HLog Compression (Revision 1301165)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/Compressor.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/Dictionary.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/KeyValueCompression.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/LRUDictionary.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALEdit.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestCompressor.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestKeyValueCompression.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLRUDictionary.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplayCompressed.java


 HLog Compression
 

 Key: HBASE-4608
 URL: https://issues.apache.org/jira/browse/HBASE-4608
 Project: HBase
  Issue Type: New Feature
Reporter: Li Pi
Assignee: Li Pi
 Fix For: 0.94.0

 Attachments: 4608-v19.txt, 4608-v20.txt, 4608-v22.txt, 4608v1.txt, 
 4608v13.txt, 4608v13.txt, 4608v14.txt, 4608v15.txt, 4608v16.txt, 4608v17.txt, 
 4608v18.txt, 4608v23.txt, 4608v24.txt, 4608v25.txt, 4608v27.txt, 4608v29.txt, 
 4608v30.txt, 4608v5.txt, 4608v6.txt, 4608v7.txt, 4608v8fixed.txt, 
 hbase-4608-v28-delta.txt, hbase-4608-v28.txt, hbase-4608-v28.txt


 The current bottleneck to HBase write speed is replicating the WAL appends 
 across different datanodes. We can speed up this process by compressing the 
 HLog. Current plan involves using a dictionary to compress table name, region 
 id, cf name, and possibly other bits of repeated data. Also, HLog format may 
 be changed in other ways to produce a smaller HLog.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5521) Move compression/decompression to an encoder specific encoding context


 [ 
https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5521:
---

Attachment: HBASE-5521.D2097.6.patch

heyongqiang updated the revision HBASE-5521 [jira] Move 
compression/decompression to an encoder specific encoding context.
Reviewers: JIRA, dhruba, tedyu, sc, mbautin

  rebase with trunk

REVISION DETAIL
  https://reviews.facebook.net/D2097

AFFECTED FILES
  src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
  src/test/java/org/apache/hadoop/hbase/io/encoding/TestDataBlockEncoders.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
  src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileDataBlockEncoder.java
  
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDecodingContext.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/EncodedDataBlock.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultDecodingContext.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockEncodingContext.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/PrefixKeyDeltaEncoder.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/HFileBlockDefaultEncodingContext.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoder.java
  
src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/CopyKeyDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DiffKeyDeltaEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
  src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java


 Move compression/decompression to an encoder specific encoding context
 --

 Key: HBASE-5521
 URL: https://issues.apache.org/jira/browse/HBASE-5521
 Project: HBase
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang
 Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch, 
 HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch, 
 HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch


 As part of working on HBASE-5313, we want to add a new columnar 
 encoder/decoder. It makes sense to move compression to be part of 
 encoder/decoder:
 1) a scanner for a columnar encoded block can do lazy decompression to a 
 specific part of a key value object
 2) avoid an extra bytes copy from encoder to hblock-writer. 
 If there is no encoder specified for a writer, the HBlock.Writer will use a 
 default compression-context to do something very similar to today's code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3440) Clean out load_table.rb and make sure all roads lead to completebulkload tool

2012-03-15 Thread Jean-Daniel Cryans (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans updated HBASE-3440:
--

Fix Version/s: (was: 0.90.2)
   0.92.0
 Hadoop Flags: Incompatible change

Doing a few edits to this jira:

 - it was committed to 0.92 (trunk back then), not 0.90.2
 - the adding of the Exception on LoadIncrementalHFiles's constructor breaks 
the code that depends on it, so I marked it as an incompatible change

 Clean out load_table.rb and make sure all roads lead to completebulkload tool
 -

 Key: HBASE-3440
 URL: https://issues.apache.org/jira/browse/HBASE-3440
 Project: HBase
  Issue Type: Bug
Reporter: stack
Assignee: Vidhyashankar Venkataraman
 Fix For: 0.92.0

 Attachments: 3440-v6.txt, blkload-v3.patch, blkload-v4.patch, 
 blkload-v5.patch


 Up on list Vidhya tried using load_table.rb with 0.90 and new master and it 
 don't work any more now we assign differently.  Clean out this script.  Make 
 sure all doc points at completebulkload tool instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5586) [replication] NPE in ReplicationSource when creating a stream to an inexistent cluster

2012-03-15 Thread Jean-Daniel Cryans (Created) (JIRA)

[replication] NPE in ReplicationSource when creating a stream to an inexistent 
cluster
--

 Key: HBASE-5586
 URL: https://issues.apache.org/jira/browse/HBASE-5586
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
 Fix For: 0.90.7, 0.92.2, 0.94.0


This is from 0.92.1-ish:

{noformat}
2012-03-15 09:52:16,589 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource, currentPath=null
java.lang.NullPointerException
   at 
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.chooseSinks(ReplicationSource.java:223)
   at 
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.connectToPeers(ReplicationSource.java:442)
   at 
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:246)
{noformat}

I wanted to add a replication stream to a cluster that wasn't existing yet so 
that the logs would be buffered until then. This should just be treated as if 
there was no region servers.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails


 [ 
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5581:
---

Attachment: D2343.1.patch

binujohn7 requested code review of [jira] [HBASE-5581] [89-fb] Creating a 
table with invalid syntax does not give an error message when it fails.
Reviewers: JIRA, Kannan

  Creating a table with invalid syntax does not give an error message when it 
fails. In this case, it doesn't actually create the CF requested, but doesn't 
give any indication to the user that it failed. Throw an error in case of 
invalid table creation syntax.

  Fix a typo in the error message when NUMREGIONS = 1.

TEST PLAN
  Try to create a table with an invalid command as follows -
  create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, NUMREGIONS 
= 20, SPLITALGO = HexStringSplit, COMPRESSION = 'LZO', BLOOMFILTER = 
'ROW'}

  This should report an error.

  Verify that the table is created correctly when the column family 
configuration is specified in a separate clause as shown below

  create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, 
COMPRESSION = 'LZO', BLOOMFILTER = 'ROW'}, {NUMREGIONS = 20, SPLITALGO = 
HexStringSplit}

REVISION DETAIL
  https://reviews.facebook.net/D2343

AFFECTED FILES
  src/main/ruby/hbase/admin.rb

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/5193/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


 Creating a table with invalid syntax does not give an error message when it 
 fails
 -

 Key: HBASE-5581
 URL: https://issues.apache.org/jira/browse/HBASE-5581
 Project: HBase
  Issue Type: Bug
  Components: shell
Reporter: Binu John
Priority: Minor
 Attachments: D2343.1.patch


 Creating a table with invalid syntax does not give an error message when it 
 fails. In this case, it doesn't actually create the CF requested, but doesn't 
 give any indication to the user that it failed.
 create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, NUMREGIONS 
 = 20, SPLITALGO = HexStringSplit, COMPRESSION = 'LZO', BLOOMFILTER = 
 'ROW'}
 0 row(s) in 3.0930 seconds
 hbase(main):002:0 describe 'test'
 DESCRIPTION   
   ENABLED
  {NAME = 'test', FAMILIES = []} 
   true   
 1 row(s) in 0.1430 seconds
 
 Putting {NUMREGIONS = 20, SPLITALGO = HexStringSplit} into a separate 
 stanza works fine, so the feature is fine. 
 create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, 
 COMPRESSION = 'LZO', BLOOMFILTER = 'ROW'}, {NUMREGIONS = 20, SPLITALGO = 
 HexStringSplit}
 0 row(s) in 2.7860 seconds
 hbase(main):002:0 describe 'test'
 DESCRIPTION   
   ENABLED
  {NAME = 'test', FAMILIES = [{NAME = 'test', DATA_BLOCK_ENCODING = 
 'NONE',  true   
  BLOOMFILTER = 'ROW', REPLICATION_SCOPE = '0', COMPRESSION = 'LZO', 
 VERSIONS
   = '1', TTL = '2147483647', BLOCKSIZE = '65536', BLOOMFILTER_ERRORRATE = 
 '
  0.01', ENCODE_ON_DISK = 'true', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]}  
 
 We should throw an error if we can't create the CF so it's clear to the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230618#comment-13230618
 ] 

Lars Hofhansl commented on HBASE-5579:
--

bq. This should go to 0.94, I assume

Yes. Want me to do it, Ted?

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5587) Remove dns.interface configuration options

2012-03-15 Thread Eli Collins (Created) (JIRA)

Remove dns.interface configuration options 
---

 Key: HBASE-5587
 URL: https://issues.apache.org/jira/browse/HBASE-5587
 Project: HBase
  Issue Type: Improvement
Reporter: Eli Collins


Are the {{hbase.*.dns.interface}} configuration options used or needed?  Per 
HBASE-4109 it looks like these never really worked, at least in cases where the 
hostname with a trailing dot doesn't resolve. The reason I asked is that while 
these were introduced in Hadoop, I don't think they're actually used, nor am I 
convinced bypassing the host for DNS lookups is a good idea (leads to painful 
bugs where default Java DNS lookups differ with these lookups). HBase started 
using these via a similar feature in HBASE-1279 and HBASE-1279.

I filed HADOOP-8156 to remove the API which HBase uses, which is obviously an 
incompatible change and would need to be worked around here if you wanted to 
keep this functionality in HBase, ie *if* that were to get checked into Hadoop 
we'd first need to get you on your own DNS class. Either way I'll update DNS' 
InterfaceAudience annotation to indicate HBase is a user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230625#comment-13230625
 ] 

Zhihong Yu commented on HBASE-5579:
---

Integrated to 0.94 as well.

Thanks for the patch Daniel.

Thanks for the review, Stack and Lars.

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5579) A Delete Version could mask other values


 [ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5579:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
Assignee: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94

2012-03-15 Thread Lars Hofhansl (Resolved) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl resolved HBASE-5551.
--

Resolution: Fixed

Committed to 0.94.

 Some functions should not be used by customer code and must be deprecated in 
 0.94
 -

 Key: HBASE-5551
 URL: https://issues.apache.org/jira/browse/HBASE-5551
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: nkeywal
Assignee: nkeywal
 Fix For: 0.94.0

 Attachments: 5551.092.patch


 They are:
 HBaseAdmin#getMaster
 HConnection#getZooKeeperWatcher
 HConnection#getMaster

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5521) Move compression/decompression to an encoder specific encoding context

[
https://issues.apache.org/jira/browse/HBASE-5521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230637#comment-13230637
]

Hadoop QA commented on HBASE-5521:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12518539/HBASE-5521.D2097.6.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 15 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 166 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2
org.apache.hadoop.hbase.coprocessor.TestProcessRowEndpoint
org.apache.hadoop.hbase.io.hfile.TestHFile
org.apache.hadoop.hbase.io.hfile.TestReseekTo

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1194//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1194//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1194//console

This message is automatically generated.

Move compression/decompression to an encoder specific encoding context
--

Key: HBASE-5521
URL: https://issues.apache.org/jira/browse/HBASE-5521
Project: HBase
Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang
Attachments: HBASE-5521.1.patch, HBASE-5521.D2097.1.patch,
HBASE-5521.D2097.2.patch, HBASE-5521.D2097.3.patch, HBASE-5521.D2097.4.patch,
HBASE-5521.D2097.5.patch, HBASE-5521.D2097.6.patch

As part of working on HBASE-5313, we want to add a new columnar
encoder/decoder. It makes sense to move compression to be part of
encoder/decoder:
1) a scanner for a columnar encoded block can do lazy decompression to a
specific part of a key value object
2) avoid an extra bytes copy from encoder to hblock-writer.
If there is no encoder specified for a writer, the HBlock.Writer will use a
default compression-context to do something very similar to today's code.

[jira] [Commented] (HBASE-5587) Remove dns.interface configuration options

[
https://issues.apache.org/jira/browse/HBASE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230642#comment-13230642
]

stack commented on HBASE-5587:
--

Thanks Eli for looking out for us. Could do this in 0.96. 0.96 is going to be
the big hbase singularity where we take on ugly, needed change.

This will be 0.24 hadoop (or cdh4)? We'll need to add a shim to figure what
the underlying hadoop supplies? (We expect 0.96 to run on hadoop's as old as
1.0.x)

Remove dns.interface configuration options
---

Key: HBASE-5587
URL: https://issues.apache.org/jira/browse/HBASE-5587
Project: HBase
Issue Type: Improvement
Reporter: Eli Collins

Are the {{hbase.*.dns.interface}} configuration options used or needed? Per
HBASE-4109 it looks like these never really worked, at least in cases where
the hostname with a trailing dot doesn't resolve. The reason I asked is that
while these were introduced in Hadoop, I don't think they're actually used,
nor am I convinced bypassing the host for DNS lookups is a good idea (leads
to painful bugs where default Java DNS lookups differ with these lookups).
HBase started using these via a similar feature in HBASE-1279 and HBASE-1279.
I filed HADOOP-8156 to remove the API which HBase uses, which is obviously an
incompatible change and would need to be worked around here if you wanted to
keep this functionality in HBase, ie *if* that were to get checked into
Hadoop we'd first need to get you on your own DNS class. Either way I'll
update DNS' InterfaceAudience annotation to indicate HBase is a user.

[jira] [Commented] (HBASE-5579) A Delete Version could mask other values


[ 
https://issues.apache.org/jira/browse/HBASE-5579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230661#comment-13230661
 ] 

Hudson commented on HBASE-5579:
---

Integrated in HBase-0.94 #33 (See 
[https://builds.apache.org/job/HBase-0.94/33/])
HBASE-5579 A Delete Version could mask other values (Daniel Ferro) 
(Revision 1301240)

 Result = SUCCESS
tedyu : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ScanDeleteTracker.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
* 
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanDeleteTracker.java


 A Delete Version could mask other values
 

 Key: HBASE-5579
 URL: https://issues.apache.org/jira/browse/HBASE-5579
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Daniel Gómez Ferro
Assignee: Daniel Gómez Ferro
 Fix For: 0.94.0, 0.96.0

 Attachments: HBASE-5579.patch


 A Delete Version operation mask values that have version = 0. The problem 
 happens at ScanDeleteTracker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5551) Some functions should not be used by customer code and must be deprecated in 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230660#comment-13230660
 ] 

Hudson commented on HBASE-5551:
---

Integrated in HBase-0.94 #33 (See 
[https://builds.apache.org/job/HBase-0.94/33/])
HBASE-5551  Some functions should not be used by customer code and must be 
deprecated in 0.94 (nkeywal) (Revision 1301243)

 Result = SUCCESS
larsh : 
Files : 
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* 
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnection.java


 Some functions should not be used by customer code and must be deprecated in 
 0.94
 -

 Key: HBASE-5551
 URL: https://issues.apache.org/jira/browse/HBASE-5551
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.0
Reporter: nkeywal
Assignee: nkeywal
 Fix For: 0.94.0

 Attachments: 5551.092.patch


 They are:
 HBaseAdmin#getMaster
 HConnection#getZooKeeperWatcher
 HConnection#getMaster

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0


[ 
https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230665#comment-13230665
 ] 

stack commented on HBASE-5559:
--

Sujee Since you are learning how to contrib., how about your figuring how to 
add a test for your change.  Your life testing might be easier if you made 
getSplits static and passed in this.presplitRegions as an argument (you can 
have getSplits call it).  In your test you'd exercise the different 
combinations passing a this.presplitRegions with no elements and then with a 
few different values for R?

Patch is looking good otherwise.

 --presplit option creates a first split with rowkey-end=0
 -

 Key: HBASE-5559
 URL: https://issues.apache.org/jira/browse/HBASE-5559
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Sujee Maniyam
Assignee: Sujee Maniyam
Priority: Trivial
  Labels: benchmark
 Attachments: 5559_v1.patch, HBASE-5559-v2.patch


 HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
 when the splits are generated, the first split has row-end-key=0 (zero).  
 Hence this split doesn't get any data.
 For example, 
 if total keyspace is 100, and splits requested are 5, 
 generated splits = [0, 20, 40, 60, 80]
 it should be = [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3873) Mavenize Hadoop Snappy JAR/SOs project dependencies

2012-03-15 Thread Francisco Cruz (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230664#comment-13230664
 ] 

Francisco Cruz commented on HBASE-3873:
---

Hi all, I am having the following problem when building the hadoop snappy:


 [exec] config.status: executing libtool commands
 [exec] depbase=`echo 
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo | sed 
's|[^/]*$|.deps/|;s|\.lo$||'`;\
 [exec] /bin/bash 
./lsrc/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c: In function 
'Java_org_apache_hadoop_io_compress_snappy_SnappyCompreibtool --tag=CC   
--mode=compile gcc -DHAVE_CONFIG_H -I.  -I/usr/lib/jvm/java-7-oracle/include 
-I/usr/lib/jvm/java-7-oracle/inclssor_initIDs':
 [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:64:49: 
error: expected expression before ',' token
 [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c: In 
function 
'Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_compressBytesDirect':
 [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:117:3: 
warning: passing argument 4 of 'dlsym_snappy_compress' from incompatible 
pointer type [enabled by default]
 [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:117:3: 
note: expected 'size_t *' but argument is of type 'jint *'
 [exec] make: *** 
[src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo] Error 1
 [exec] ude/linux -I/home/gsd/hadoop-snappy-read-only/src/main/native/src 
-Isrc/org/apache/hadoop/io/compress/snappy -I/usr/local//include -g -Wall -fPIC 
-O2 -m64 -g -O2 -MT 
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo -MD -MP -MF 
$depbase.Tpo -c -o src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo 
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c \
 [exec] mv -f $depbase.Tpo $depbase.Plo
 [exec] libtool: compile:  gcc -DHAVE_CONFIG_H -I. 
-I/usr/lib/jvm/java-7-oracle/include -I/usr/lib/jvm/java-7-oracle/include/linux 
-I/home/gsd/hadoop-snappy-read-only/src/main/native/src 
-Isrc/org/apache/hadoop/io/compress/snappy -I/usr/local//include -g -Wall -fPIC 
-O2 -m64 -g -O2 -MT 
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo -MD -MP -MF 
src/org/apache/hadoop/io/compress/snappy/.deps/SnappyCompressor.Tpo -c 
src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c  -fPIC -DPIC -o 
src/org/apache/hadoop/io/compress/snappy/.libs/SnappyCompressor.o

Does anyone have any idea why is this happening?
Thanks for your help

 Mavenize Hadoop Snappy JAR/SOs project dependencies
 ---

 Key: HBASE-3873
 URL: https://issues.apache.org/jira/browse/HBASE-3873
 Project: HBase
  Issue Type: Improvement
  Components: build
Affects Versions: 0.90.2
 Environment: Linux
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
  Labels: build
 Fix For: 0.92.0

 Attachments: HBASE-3873.patch, HBASE-3873.patch


 (This JIRA builds on HBASE-3691)
 I'm working on simplifying how to use Hadoop Snappy from other based maven 
 projects. The idea is that hadoop-snappy JAR and the SOs (snappy and 
 hadoop-snappy) would be picked up from a Maven repository (like any other 
 dependencies). SO files will be picked up based on the architecture where the 
 build is running (32 or 64 bits).
 For Hbase this would remove the need to manually copy snappy JAR and SOs 
 (snappy and hadoop-snappy) into HADOOP_HOME/lib or HBASE_HOME/lib and 
 hadoop-snappy would be handled as a regular maven dependency (with a trick 
 for the SOs file).
 The changes would affect only the pom.xml and the would be in a 'snappy' 
 profile, thus requiring '-Dsnappy' option in Maven invocations to trigger the 
 including of snappy JAR and SOs.
 Because hadoop-snappy (JAR and SOs) are not currently avail in public Maven 
 repos, until that happens, Hbase developer would have to checkout and 'mvn 
 install' hadoop-snappy. Which is (IMO) simpler than what will have to be done 
 in once HBASE-3691 is committed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5270) Handle potential data loss due to concurrent processing of processFaileOver and ServerShutdownHandler

[
https://issues.apache.org/jira/browse/HBASE-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230668#comment-13230668
]

stack commented on HBASE-5270:
--

Thanks Chunhui. Will apply after 0.92.1 leaves the gate so its there for
0.92.2 (unless 0.92.1RC0 gets squashed and then I'll include it).

Handle potential data loss due to concurrent processing of processFaileOver
and ServerShutdownHandler
-

Key: HBASE-5270
URL: https://issues.apache.org/jira/browse/HBASE-5270
Project: HBase
Issue Type: Sub-task
Components: master
Reporter: Zhihong Yu
Assignee: chunhui shen
Fix For: 0.92.2

Attachments: 5270-90-testcase.patch, 5270-90-testcasev2.patch,
5270-90.patch, 5270-90v2.patch, 5270-90v3.patch, 5270-testcase.patch,
5270-testcasev2.patch, HBASE-5270-92v11.patch, HBASE-5270v11.patch,
hbase-5270.patch, hbase-5270v10.patch, hbase-5270v2.patch,
hbase-5270v4.patch, hbase-5270v5.patch, hbase-5270v6.patch,
hbase-5270v7.patch, hbase-5270v8.patch, hbase-5270v9.patch, sampletest.txt

This JIRA continues the effort from HBASE-5179. Starting with Stack's
comments about patches for 0.92 and TRUNK:
Reviewing 0.92v17
isDeadServerInProgress is a new public method in ServerManager but it does
not seem to be used anywhere.
Does isDeadRootServerInProgress need to be public? Ditto for meta version.
This method param names are not right 'definitiveRootServer'; what is meant
by definitive? Do they need this qualifier?
Is there anything in place to stop us expiring a server twice if its carrying
root and meta?
What is difference between asking assignment manager isCarryingRoot and this
variable that is passed in? Should be doc'd at least. Ditto for meta.
I think I've asked for this a few times - onlineServers needs to be
explained... either in javadoc or in comment. This is the param passed into
joinCluster. How does it arise? I think I know but am unsure. God love the
poor noob that comes awandering this code trying to make sense of it all.
It looks like we get the list by trawling zk for regionserver znodes that
have not checked in. Don't we do this operation earlier in master setup? Are
we doing it again here?
Though distributed split log is configured, we will do in master single
process splitting under some conditions with this patch. Its not explained in
code why we would do this. Why do we think master log splitting 'high
priority' when it could very well be slower. Should we only go this route if
distributed splitting is not going on. Do we know if concurrent distributed
log splitting and master splitting works?
Why would we have dead servers in progress here in master startup? Because a
servershutdownhandler fired?
This patch is different to the patch for 0.90. Should go into trunk first
with tests, then 0.92. Should it be in this issue? This issue is really hard
to follow now. Maybe this issue is for 0.90.x and new issue for more work on
this trunk patch?
This patch needs to have the v18 differences applied.

[jira] [Updated] (HBASE-5578) NPE when regionserver reported server load, caused rs stop.

2012-03-15 Thread stack (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-5578:
-

Priority: Critical  (was: Major)

Thats pretty nasty.  Can you reproduce easily Storm?  Is it happening all the 
time?  Is this fresh start up or something?  You have no content under /hbase?

 NPE when regionserver reported server load, caused rs stop.
 ---

 Key: HBASE-5578
 URL: https://issues.apache.org/jira/browse/HBASE-5578
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.0
 Environment: centos6.2 hadoop-1.0.0 hbase-0.92.0
Reporter: Storm Lee
Priority: Critical

 The regeionserver log:
 2012-03-11 11:55:37,808 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 data3,60020,1331286604591: Unhandled exception: null
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.regionserver.Store.getTotalStaticIndexSize(Store.java:1788)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:994)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:800)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:776)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:678)
   at java.lang.Thread.run(Thread.java:662)
 2012-03-11 11:55:37,808 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: 
 loaded coprocessors are: []
 2012-03-11 11:55:37,808 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: 
 requestsPerSecond=1687, numberOfOnlineRegions=37, numberOfStores=37, 
 numberOfStorefiles=144, storefileIndexSizeMB=2, rootIndexSizeKB=2362, 
 totalStaticIndexSizeKB=229808, totalStaticBloomSizeKB=2166296, 
 memstoreSizeMB=2854, readRequestsCount=1352673, writeRequestsCount=113137586, 
 compactionQueueSize=8, flushQueueSize=3, usedHeapMB=7359, maxHeapMB=12999, 
 blockCacheSizeMB=32.31, blockCacheFreeMB=3867.52, blockCacheCount=38, 
 blockCacheHitCount=87713, blockCacheMissCount=22144560, 
 blockCacheEvictedCount=122, blockCacheHitRatio=0%, 
 blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=100
 2012-03-11 11:55:37,992 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unhandled 
 exception: null

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

2012-03-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230681#comment-13230681
 ] 

Jonathan Hsieh commented on HBASE-5563:
---

This patch also makes older regions smaller than newer regions (reversing 
chenhui's sort order).  This makes fixing the test cases easier.

Oddly, TestAdmin.testEnableTableRoundRobinAssignement seems to fail for me 
locally but pass in my jenkins test runners.

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId


 [ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5563:
--

Status: Patch Available  (was: Open)

Updated Patch to add test case and to fix broken test cases.  

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId


 [ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5563:
--

Attachment: hbase-5563-v3.patch

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

2012-03-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230682#comment-13230682
 ] 

Jonathan Hsieh commented on HBASE-5563:
---

This will probably be needed for hbck in HBASE-5128 for 0.92, 0.94, and 
trunk/0.96.  It was not required for 0.90.x hbck updates.

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId


 [ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5563:
--

Affects Version/s: 0.96.0
   0.94.0
   0.90.5
   0.92.0

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2462) Review compaction heuristic and move compaction code out so standalone and independently testable

2012-03-15 Thread stack (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-2462:
-

Attachment: 2462v2.txt

Not done yet. Had to stand up an instance of Store and even that requires a
mocked HRegion instance at least because this all is too tangled (I need
instance of StoreScanner -- I tried hard to undo SS's dependence on Store but
had to give up). I have it so have a Tool that you can pass a directory of
files too. Was going to finish this up. At a minium its turning up some
little improvements on Store -- I'm able to remove some duplicated and
unnecessary data members, etc.

Review compaction heuristic and move compaction code out so standalone and
independently testable
-

Key: HBASE-2462
URL: https://issues.apache.org/jira/browse/HBASE-2462
Project: HBase
Issue Type: Improvement
Components: performance
Reporter: stack
Assignee: Jonathan Gray
Priority: Critical
Labels: moved_from_0_20_5
Attachments: 2462v2.txt, standalone.txt

Anything that improves our i/o profile makes hbase run smoother. Over in
HBASE-2457, good work has been done already describing the tension between
minimizing compactions versus minimizing count of store files. This issue is
about following on from what has been done in 2457 but also, breaking the
hard-to-read compaction code out of Store.java out to a standalone class that
can be the easier tested (and easily analyzed for its performance
characteristics).
If possible, in the refactor, we'd allow specification of alternate merge
sort implementations.

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

[
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230688#comment-13230688
]

Hadoop QA commented on HBASE-5563:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12518565/hbase-5563-v3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 161 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestColumnSeeking

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1195//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1195//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1195//console

This message is automatically generated.

HRegionInfo#compareTo add the comparison of regionId

Key: HBASE-5563
URL: https://issues.apache.org/jira/browse/HBASE-5563
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
Attachments: HBASE-5563.patch, HBASE-5563v2.patch,
HBASE-5563v2.patch, hbase-5563-v3.patch

In the one region multi assigned case, we could find that two regions have
the same table name, same startKey, same endKey, and different regionId, so
these two regions are same in TreeMap but different in HashMap.

[jira] [Commented] (HBASE-5581) Creating a table with invalid syntax does not give an error message when it fails


[ 
https://issues.apache.org/jira/browse/HBASE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230691#comment-13230691
 ] 

Phabricator commented on HBASE-5581:


Kannan has accepted the revision [jira] [HBASE-5581] [89-fb] Creating a table 
with invalid syntax does not give an error message when it fails.

  Thank! Lgtm...

REVISION DETAIL
  https://reviews.facebook.net/D2343

BRANCH
  table_create_error


 Creating a table with invalid syntax does not give an error message when it 
 fails
 -

 Key: HBASE-5581
 URL: https://issues.apache.org/jira/browse/HBASE-5581
 Project: HBase
  Issue Type: Bug
  Components: shell
Reporter: Binu John
Priority: Minor
 Attachments: D2343.1.patch


 Creating a table with invalid syntax does not give an error message when it 
 fails. In this case, it doesn't actually create the CF requested, but doesn't 
 give any indication to the user that it failed.
 create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, NUMREGIONS 
 = 20, SPLITALGO = HexStringSplit, COMPRESSION = 'LZO', BLOOMFILTER = 
 'ROW'}
 0 row(s) in 3.0930 seconds
 hbase(main):002:0 describe 'test'
 DESCRIPTION   
   ENABLED
  {NAME = 'test', FAMILIES = []} 
   true   
 1 row(s) in 0.1430 seconds
 
 Putting {NUMREGIONS = 20, SPLITALGO = HexStringSplit} into a separate 
 stanza works fine, so the feature is fine. 
 create 'test', {NAME = 'test', VERSIONS = 1, BLOCKCACHE = true, 
 COMPRESSION = 'LZO', BLOOMFILTER = 'ROW'}, {NUMREGIONS = 20, SPLITALGO = 
 HexStringSplit}
 0 row(s) in 2.7860 seconds
 hbase(main):002:0 describe 'test'
 DESCRIPTION   
   ENABLED
  {NAME = 'test', FAMILIES = [{NAME = 'test', DATA_BLOCK_ENCODING = 
 'NONE',  true   
  BLOOMFILTER = 'ROW', REPLICATION_SCOPE = '0', COMPRESSION = 'LZO', 
 VERSIONS
   = '1', TTL = '2147483647', BLOCKSIZE = '65536', BLOOMFILTER_ERRORRATE = 
 '
  0.01', ENCODE_ON_DISK = 'true', IN_MEMORY = 'false', BLOCKCACHE = 
 'true'}]}  
 
 We should throw an error if we can't create the CF so it's clear to the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230693#comment-13230693
 ] 

stack commented on HBASE-5563:
--

+1 on committing everywhere, even into 0.92 (I've tagged RC0; if we do an RC1 
it should be included.  Mark fix in 0.92.2 I'd say for now)

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5559) --presplit option creates a first split with rowkey-end=0

2012-03-15 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230699#comment-13230699
 ] 

Lars Hofhansl commented on HBASE-5559:
--

Patch looks good.
(If I wanted to split hairs I'd say that if the number of splits is approaching 
the size of the keyspace you'd run into funky integer arithmetic.
Say the keyspace is 100 and presplits is 30. You'd get 29 regions of size 3 and 
one of size 10... But do not want to split hairs :) )


 --presplit option creates a first split with rowkey-end=0
 -

 Key: HBASE-5559
 URL: https://issues.apache.org/jira/browse/HBASE-5559
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Sujee Maniyam
Assignee: Sujee Maniyam
Priority: Trivial
  Labels: benchmark
 Attachments: 5559_v1.patch, HBASE-5559-v2.patch


 HBASE-4440 adds a 'presplit' option to PerformanceEvaluation utility.
 when the splits are generated, the first split has row-end-key=0 (zero).  
 Hence this split doesn't get any data.
 For example, 
 if total keyspace is 100, and splits requested are 5, 
 generated splits = [0, 20, 40, 60, 80]
 it should be = [20, 40, 60, 80, 100]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

2012-03-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230702#comment-13230702
 ] 

Jonathan Hsieh commented on HBASE-5563:
---


TestColumnSeeking passes for me locally, first time its ever failed for me.

{code}
jon@swoop:~/proj/hbase-trunk$ mvn test -PlocalTests -Dtest=TestColumnSeeking
..
---
 T E S T S
---
Running org.apache.hadoop.hbase.regionserver.TestColumnSeeking
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.061 sec

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
{code}

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5445) Add PB-based calls to HMasterInterface


[ 
https://issues.apache.org/jira/browse/HBASE-5445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230709#comment-13230709
 ] 

jirapos...@reviews.apache.org commented on HBASE-5445:
--



bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   Looks good. Just a few comments below.

Thanks for the review.  Will address in next patch.


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/HMasterProtocol.proto, line 85
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91002#file91002line85
bq.  
bq.   These are in the master protocol but are they used elsewhere?  If 
so, should they be in here?  Maybe they are not and its just me confused.

I don't think they are used elsewhere, will check.


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/HMasterProtocol.proto, line 112
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91002#file91002line112
bq.  
bq.   Is this class deprecated (HServerInfo?)

This is equivalent to HServerLoad.


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/hbase.proto, line 21
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91003#file91003line21
bq.  
bq.   The package in first class is protobuf.generated.  Here its 
generated.protobuf.  Did you mean that?

Good catch.


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/hbase.proto, line 119
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91003#file91003line119
bq.  
bq.   This is ServerName?  Why call it ServerSpecifier?  It should be a 
different name?

I was just matching it up so it would be consistent with RegionSpecifier -- 
I'll figure out something better for this.


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/HMasterProtocol.proto, line 22
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91002#file91002line22
bq.  
bq.   Is this a good name for this class?  Drop the H?  Can we change it?

I'll drop the H everywhere I can :).


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/HMasterProtocol.proto, line 19
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91002#file91002line19
bq.  
bq.   Reference your nice PDF?
bq.   
bq.   BTW, in your PDF, you have one method that is 'not used' so you 
don't include it.  Do we need to deprecate it in 0.94 so its safe to remove by 
the time 0.96 comes around?  If so, mind filing an issue or just me know and I 
can take care of it.
bq.   
bq.   Ditto for loadBalancerSwitchIs(LoadBalancerSwitchIsRequest).  This 
method is just broke from its name and to how it works.  Can we deprecate it in 
0.94?  Add a method that makes sense in 0.94, one that makes sense that you can 
redo in pb?
bq.   
bq.   Otherwise, the pdf looks good.

I got rid of public void assign(final byte [] regionName, final boolean 
force) which is already deprecated, so nothing to do for 0.94.

Do you have any recommendations for:
public boolean balanceSwitch(final boolean b) and public boolean 
synchronousBalanceSwitch(final boolean b)?
which is what I combined into loadBalancerSwitchIs.  I thought switch was 
confusing because it is unclear if that is a verb (i.e. it switches whether the 
load balancer is running, so non-idempotent), vs a noun (it is a switch like an 
oven switch [on/off], so idempotent).  I think a noun is the correct way to 
think about it; I thought SwitchIs makes it clearer that its a noun.  On 
second though, I think I'd just call it loadBalancerIs(final bool on,...).  
Should I add that method and deprecate the existing two methods above in 0.94?  
I thought we were going to break everything in 0.96 anyway?


bq.  On 2012-03-13 05:33:05, Michael Stack wrote:
bq.   src/main/proto/hbase.proto, line 149
bq.   https://reviews.apache.org/r/4283/diff/1/?file=91003#file91003line149
bq.  
bq.   Should ColumnMetadata and TableMetadata have common type?

They could be.  Having them separate lets us evolve them independently, though.


- Gregory


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4283/#review5868
---


On 2012-03-10 02:09:45, Gregory Chanan wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4283/
bq.  ---
bq.  
bq.  (Updated 2012-03-10 02:09:45)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  First draft of the protobufs specification for HMasterInterface.
bq.  This is relatively close to a one-to-one mapping with the existing 
interface.  A pdf listing the existing-to-protobufs mapping is available on the 
JIRA:

[jira] [Created] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition

2012-03-15 Thread Jonathan Hsieh (Created) (JIRA)

Deprecate/remove AssignmentManager#clearRegionFromTransition


 Key: HBASE-5588
 URL: https://issues.apache.org/jira/browse/HBASE-5588
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.92.0, 0.90.5, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh


This method is essentially a dupe of Assignment#regionOffline.  As suggested in 
early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5589) Add of the offline call to the Master Interface

2012-03-15 Thread Jonathan Hsieh (Created) (JIRA)

Add of the offline call to the Master Interface
---

 Key: HBASE-5589
 URL: https://issues.apache.org/jira/browse/HBASE-5589
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 0.92.0, 0.90.6, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh


Hbck from HBASE-5128 requires an offline method on the master to properly 
cleanup state during certain assignment repair operations.  This will this 
method will be added to recent and older versions of HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId

2012-03-15 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230725#comment-13230725
 ] 

Jonathan Hsieh commented on HBASE-5563:
---

I have full tests suites running for the trunk, 0.94, and 0.92 versions.  If 
they come back clean, I'll commit first thing tommorrow am.

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId


 [ 
https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5563:
--

Attachment: hbase-5563-v3-0.92.patch

hbase-5563-v3.patch applies cleanly to 0.94 branch.  

Attached 0.92 version -- it passes the trouble cases from v1/v2.

 HRegionInfo#compareTo add the comparison of regionId
 

 Key: HBASE-5563
 URL: https://issues.apache.org/jira/browse/HBASE-5563
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, 
 HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch


 In the one region multi assigned case,  we could find that two regions have 
 the same table name, same startKey, same endKey, and different regionId, so 
 these two regions are same in TreeMap but different in HashMap.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations


 [ 
https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HBASE-5335:
---

Attachment: D2247.2.patch

nspiegelberg updated the revision [jira] [HBASE-5335] Dynamic Schema Config.
Reviewers: JIRA, Kannan, stack, mbautin, Liyin

  Addressed Ted  Stack's peer reviews.  Added unit tests for new 
functionality.  Fixed bugs found when writing tests.

REVISION DETAIL
  https://reviews.facebook.net/D2247

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  src/main/java/org/apache/hadoop/hbase/regionserver/SplitRequest.java
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
  src/main/java/org/apache/hadoop/hbase/util/CompoundConfiguration.java
  src/main/ruby/hbase/admin.rb
  src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java
  src/test/java/org/apache/hadoop/hbase/util/TestCompoundConfiguration.java


 Dynamic Schema Configurations
 -

 Key: HBASE-5335
 URL: https://issues.apache.org/jira/browse/HBASE-5335
 Project: HBase
  Issue Type: New Feature
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
  Labels: configuration, schema
 Attachments: D2247.1.patch, D2247.2.patch


 Currently, the ability for a core developer to add per-table  per-CF 
 configuration settings is very heavyweight.  You need to add a reserved 
 keyword all the way up the stack  you have to support this variable 
 long-term if you're going to expose it explicitly to the user.  This has 
 ended up with using Configuration.get() a lot because it is lightweight and 
 you can tweak settings while you're trying to understand system behavior 
 [since there are many config params that may never need to be tuned].  We 
 need to add the ability to put  read arbitrary KV settings in the HBase 
 schema.  Combined with online schema change, this will allow us to safely 
 iterate on configuration settings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5335) Dynamic Schema Configurations


[ 
https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230753#comment-13230753
 ] 

Phabricator commented on HBASE-5335:


nspiegelberg has commented on the revision [jira] [HBASE-5335] Dynamic Schema 
Config.

INLINE COMMENTS
  src/test/java/org/apache/hadoop/hbase/util/TestCompoundConfiguration.java:1 
will add license header before commit

REVISION DETAIL
  https://reviews.facebook.net/D2247


 Dynamic Schema Configurations
 -

 Key: HBASE-5335
 URL: https://issues.apache.org/jira/browse/HBASE-5335
 Project: HBase
  Issue Type: New Feature
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
  Labels: configuration, schema
 Attachments: D2247.1.patch, D2247.2.patch


 Currently, the ability for a core developer to add per-table  per-CF 
 configuration settings is very heavyweight.  You need to add a reserved 
 keyword all the way up the stack  you have to support this variable 
 long-term if you're going to expose it explicitly to the user.  This has 
 ended up with using Configuration.get() a lot because it is lightweight and 
 you can tweak settings while you're trying to understand system behavior 
 [since there are many config params that may never need to be tuned].  We 
 need to add the ability to put  read arbitrary KV settings in the HBase 
 schema.  Combined with online schema change, this will allow us to safely 
 iterate on configuration settings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5590) Add more server mode state to jmx output.

2012-03-15 Thread Jonathan Hsieh (Created) (JIRA)

Add more server mode state to jmx output.
-

 Key: HBASE-5590
 URL: https://issues.apache.org/jira/browse/HBASE-5590
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Reporter: Jonathan Hsieh


Related to HBASE-5325, and HBASE-5533 there is more state information that 
would be good to expose in a machine readable fashion.

Some suggestions for state information include:
* the balancer is on or off.
* if a master is active or a backup.
* If a we are in hlog recovery mode
* tasks distributed from distributed log splitting

More suggestions are welcome.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5587) Remove dns.interface configuration options

2012-03-15 Thread Eli Collins (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13230760#comment-13230760
]

Eli Collins commented on HBASE-5587:

Depends on you guys, and the Hadoop jira. I'm happy to leave in the code (which
will be dead code in Hadoop) in 23/cdh4 if that makes maintaining compatibility
much easier for you. You guys could also insulate yourselves by copying the
relevant methods into your branches if you want. I'd be curious to know if you
guys have people using this feature.

Remove dns.interface configuration options
---

Key: HBASE-5587
URL: https://issues.apache.org/jira/browse/HBASE-5587
Project: HBase
Issue Type: Improvement
Reporter: Eli Collins

[jira] [Commented] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId