date:20120625


 [ 
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laxman updated HBASE-6253:
--

Status: Patch Available  (was: Open)

Moving to PA.

 Dont allow user to disable/drop _acl_ table
 ---

 Key: HBASE-6253
 URL: https://issues.apache.org/jira/browse/HBASE-6253
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
  Labels: security
 Fix For: 0.94.1

 Attachments: HBASE-6253.patch, HBASE-6253.patch


 Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_ 
 table name, due to this user can able to disable/enable/drop/create the acl 
 table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6253) Dont allow user to disable/drop _acl_ table


[ 
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400294#comment-13400294
 ] 

Laxman commented on HBASE-6253:
---

should we consider disallowing all DDL operations (add/delete/modify column)?
with online schema change, its not mandatory that we need to diable the table.
that means, we can drop the columns of acl table even now.


 Dont allow user to disable/drop _acl_ table
 ---

 Key: HBASE-6253
 URL: https://issues.apache.org/jira/browse/HBASE-6253
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
  Labels: security
 Fix For: 0.94.1

 Attachments: HBASE-6253.patch, HBASE-6253.patch


 Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_ 
 table name, due to this user can able to disable/enable/drop/create the acl 
 table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6253) Dont allow user to disable/drop _acl_ table


[ 
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400296#comment-13400296
 ] 

Laxman commented on HBASE-6253:
---

by DDL i consider : AddColumn, ModifyColumn, DeleteColumn, EnableTable, 
DisableTable, ModifyTable, DeleteTable

Reference: ACL matrix available @ HBASE-6192


 Dont allow user to disable/drop _acl_ table
 ---

 Key: HBASE-6253
 URL: https://issues.apache.org/jira/browse/HBASE-6253
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
  Labels: security
 Fix For: 0.94.1

 Attachments: HBASE-6253.patch, HBASE-6253.patch


 Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_ 
 table name, due to this user can able to disable/enable/drop/create the acl 
 table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6253) Dont allow user to disable/drop _acl_ table


[ 
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400297#comment-13400297
 ] 

ramkrishna.s.vasudevan commented on HBASE-6253:
---

I think its better we fix keeping online schema changes also.

 Dont allow user to disable/drop _acl_ table
 ---

 Key: HBASE-6253
 URL: https://issues.apache.org/jira/browse/HBASE-6253
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
  Labels: security
 Fix For: 0.94.1

 Attachments: HBASE-6253.patch, HBASE-6253.patch


 Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_ 
 table name, due to this user can able to disable/enable/drop/create the acl 
 table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix


[ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400303#comment-13400303
 ] 

Anoop Sam John commented on HBASE-6200:
---

{code}
-  if (lcolumnlength == 0  ltype == Type.Minimum.getCode()) {
+  if (lfamilylength == 0  lqualifierlength == 0
+   ltype == Type.Minimum.getCode()) {
{code}
May be only lfamilylength ==0 check is enough. When the family is missing from 
KV there can not be a qualifier.
Same with
{code}
-  if (rcolumnlength == 0  rtype == Type.Minimum.getCode()) {
+  if (rfamilylength == 0  rqualifierlength == 0
+   rtype == Type.Minimum.getCode()) {
{code}



 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6253) Dont allow user to disable/drop _acl_ table

[
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400317#comment-13400317
]

Hadoop QA commented on HBASE-6253:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533259/HBASE-6253.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 11 new Findbugs (version
1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2247//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2247//console

This message is automatically generated.

Dont allow user to disable/drop _acl_ table
---

Key: HBASE-6253
URL: https://issues.apache.org/jira/browse/HBASE-6253
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
Labels: security
Fix For: 0.94.1

Attachments: HBASE-6253.patch, HBASE-6253.patch

Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_
table name, due to this user can able to disable/enable/drop/create the acl
table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix

2012-06-25 Thread Jieshan Bean (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400323#comment-13400323
 ] 

Jieshan Bean commented on HBASE-6200:
-

@Anoop:
Thanks for your carefully review. 
Only the createLastOnRow methods using Type.Minimum. Likes below:
{noformat}
 public static KeyValue createLastOnRow(final byte[] row) {
return new KeyValue(row, null, null, HConstants.LATEST_TIMESTAMP, 
Type.Minimum);
 }
{noformat}
At this time, the length of CF (or Qualifier) should be 0. 
Actually, we can create one KeyValue with CF as empty but Qualifier is not 
empty. And if we call KeyValue#getQualifier, we can get the correct value. So I 
think this check should be strict with both CF == 0 and Qualifier == 0.


 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400359#comment-13400359
 ] 

Anoop Sam John commented on HBASE-4379:
---

@Jon
I am working on this. Will provide a patch based on 94 version soon.

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.


[ 
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400381#comment-13400381
 ] 

ramkrishna.s.vasudevan commented on HBASE-5360:
---

Small nit
{code}
  p.add(HConstants.CATALOG_FAMILY, HConstants.SPLITA_QUALIFIER,
Writables.getBytes(a));
  p.add(HConstants.CATALOG_FAMILY, HConstants.SPLITA_QUALIFIER,
Writables.getBytes(b));
{code}

In the testcase, the second put should be for SPLITB?


 [uberhbck] Add options for how to handle offline split parents. 
 

 Key: HBASE-5360
 URL: https://issues.apache.org/jira/browse/HBASE-5360
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 
 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path


 In a recent case, we attempted to repair a cluster that suffered from 
 HBASE-4238 that had about 6-7 generations of leftover split data.  The hbck 
 repair options in an development version of HBASE-5128 treat HDFS as ground 
 truth but didn't check SPLIT and OFFLINE flags only found in meta.  The net 
 effect was that it essentially attempted to merge many regions back into its 
 eldest geneneration's parent's range.  
 More safe guards to prevent mega-merges are being added on HBASE-5128.
 This issue would automate the handling of the mega-merge avoiding cases 
 such as lingering grandparents.  The strategy here would be to add more 
 checks against .META., and perform part of the catalog janitor's 
 responsibilities for lingering grandparents.  This would potentially include 
 options to sideline regions, deleting grandparent regions, min size for 
 sidelining, and mechanisms for cleaning .META..  
 Note: There already exists an mechanism to reload these regions -- the bulk 
 loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents 
 (automatically splitting them if necessary) to HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

2012-06-25 Thread Flavio Junqueira (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400406#comment-13400406
]

Flavio Junqueira commented on HBASE-2315:
-

Hi Jonathan, A BK backend for logging can potentially deal with the issues
you're raising. The main issue is that it doesn't expose a file system
interface, and HBase currently seems to rely upon one. If we have an interface
that matches more naturally the BK api, then we will possibly be able to deal
with the issues you're raising without much effort.

I was actually trying to implement a BookKeeperFS class that extends
FileSystem, but it doesn't feel natural. I was wondering if we should try to
work on the interface first instead of hacking it in by implementing an
HLog.Reader and an HLog.Writer.

BookKeeper for write-ahead logging
--

Key: HBASE-2315
URL: https://issues.apache.org/jira/browse/HBASE-2315
Project: HBase
Issue Type: New Feature
Components: regionserver
Reporter: Flavio Junqueira
Attachments: HBASE-2315.patch, bookkeeperOverview.pdf,
zookeeper-dev-bookkeeper.jar

BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high
throughput write-ahead logging service. This issue provides an implementation
of write-ahead logging for hbase using BookKeeper. Apart from expected
throughput improvements, BookKeeper also has stronger durability guarantees
compared to the implementation currently used by hbase.

[jira] [Commented] (HBASE-6265) Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed

2012-06-25 Thread Lars George (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400422#comment-13400422
 ] 

Lars George commented on HBASE-6265:


I think one easy non-intrusive fix is to reset the timestampCache variable on 
calling updateLatestStamp(). If you agree I would provide a patch doing that?

 Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed
 -

 Key: HBASE-6265
 URL: https://issues.apache.org/jira/browse/HBASE-6265
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Lars George

 There is an issue when you call getTimestamp() on any KV handed into a 
 Coprocessor's prePut(). It initializes the internal timestampCache 
 variable. 
 When you then pass it to the normal processing, the region server sets the 
 time to the server time in case you have left it unset from the client side 
 (updateLatestStamp() call). 
 The TimeRangeTracker then calls getTimestamp() later on to see if it has to 
 include the KV, but instead of getting the proper time it sees the cached 
 timestamp from the prePut() call.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6265) Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed

2012-06-25 Thread Lars George (JIRA)

Lars George created HBASE-6265:
--

 Summary: Calling getTimestamp() on a KV in cp.prePut() causes KV 
not to be flushed
 Key: HBASE-6265
 URL: https://issues.apache.org/jira/browse/HBASE-6265
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Lars George


There is an issue when you call getTimestamp() on any KV handed into a 
Coprocessor's prePut(). It initializes the internal timestampCache variable. 

When you then pass it to the normal processing, the region server sets the time 
to the server time in case you have left it unset from the client side 
(updateLatestStamp() call). 

The TimeRangeTracker then calls getTimestamp() later on to see if it has to 
include the KV, but instead of getting the proper time it sees the cached 
timestamp from the prePut() call.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


 [ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John updated HBASE-4379:
--

Attachment: HBASE-4379_94.patch

Patch for 94.

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400426#comment-13400426
 ] 

Anoop Sam John commented on HBASE-4379:
---

{code}
public boolean checkRegionChain(TableIntegrityErrorHandler handler) throws 
IOException {
+  // When table is disabled no need to check for the region chain. Some of 
the regions
+  // accidently if deployed, this below code might report some issues like 
missing start
+  // or end regions or region hole in chain and may try to fix which is 
unwanted.
+  if (disabledTables.contains(this.tableName.getBytes())) {
+return true;
+  }
{code}

When the table is disabled, the regions in this need not undergo the check for 
consistency I feel. Suppose a table is disabled and 1st and 3rd regions of it 
are online. The HBCK will fix this making the regions offline. But this 
checkRegionChain can report a hole between the 2 regions and may try to fix 
this creating a new region. I just added this check at this place. May be at a 
broader scope need to add. Pls give you feedback. I will debug more and add 
changes if needed.

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400427#comment-13400427
 ] 

Hadoop QA commented on HBASE-4379:
--

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12533281/HBASE-4379_94.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 3 new or modified tests.

-1 patch.  The patch command could not apply the patch.

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/2248//console

This message is automatically generated.

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6266) Updation issue in hbase.

2012-06-25 Thread shailendra kumar (JIRA)

shailendra kumar created HBASE-6266:
---

 Summary: Updation issue in hbase.
 Key: HBASE-6266
 URL: https://issues.apache.org/jira/browse/HBASE-6266
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.19.3
 Environment: Linux
Reporter: shailendra kumar
Priority: Critical
 Fix For: 0.19.3



Hi,
please find the below logs for same scenario:

java.lang.NullPointerException
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.getStringData(SchedulerMR.java:1149)
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.updateJobTable(SchedulerMR.java:945)
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.commitTask(SchedulerMR.java:803)
at org.apache.hadoop.mapred.Task.commit(Task.java:721)
at org.apache.hadoop.mapred.Task.done(Task.java:644)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:460)
at org.apache.hadoop.mapred.Child.main(Child.java:158)
java.lang.NullPointerException
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.getStringData(SchedulerMR.java:1149)
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.updateJobTableForReporting(SchedulerMR.java:1055)
at 
com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.commitTask(SchedulerMR.java:804)
at org.apache.hadoop.mapred.Task.commit(Task.java:721)
at org.apache.hadoop.mapred.Task.done(Task.java:644)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:460)
at org.apache.hadoop.mapred.Child.main(Child.java:158)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5630) hbck should disable the balancer using synchronousBalanceSwitch.


 [ 
https://issues.apache.org/jira/browse/HBASE-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5630:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 hbck should disable the balancer using synchronousBalanceSwitch.
 

 Key: HBASE-5630
 URL: https://issues.apache.org/jira/browse/HBASE-5630
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0, 0.94.1

 Attachments: 5630-0.94-v3.txt, 5630-trunk-v3.patch, 
 HBASE-5630-94-v2.patch, HBASE-5630-94.patch, HBASE-5630-trunk-v2.patch, 
 HBASE-5630.patch


 hbck disable the balancer using admin.balanceSwith(bool) when it would be 
 preferable to use the newer synchronusBalanceSwitch method found in 0.94 and 
 trunk branches.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix

2012-06-25 Thread Jieshan Bean (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jieshan Bean updated HBASE-6200:


Attachment: PerformanceTestCase-6200-94.patch

This is how I did this test:
1. Test comparing between 'famia:qualia' with 'famib:qualia'. Run for 100*2 
times. Calculate the time consumed(Using System.currentTimeMillis() to get 
current time).
2. Test comparing between 'fami:qualia' with 'fami:qualib'. Run for 1,000,000*2 
times. Calculate the time consumed.
3. Repeats 1~2 for 20 times. Accumulate the total consumed time at step 1 and 
step 2, and then calculate for the average time.
4. Repeats 1~3 for 3 groups.

Please find the attached file PerformanceTestCase-6200-94.patch for details.

And this is the test results in my machine(Machine info: 64-bit Suse, CPU: 16 * 
1.6GHz, Memory: 32GB):

[With patch 6200]
{noformat}
Group-1:
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2425, average 
- 121
Compare {fami:qualia} with {fami:qualib} used time(totally) - 3282, average - 
164

Group-2:
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2421, average 
- 121
Compare {fami:qualia} with {fami:qualib} used time(totally) - 3279, average - 
163

Group-3:
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2417, average 
- 120
Compare {fami:qualia} with {fami:qualib} used time(totally) - 3279, average - 
163
{noformat}

[Without patch 6200]
{noformat}
Group-1:
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2015, average 
- 100
Compare {fami:qualia} with {fami:qualib} used time(totally) - 2191, average - 
109

Group-12:
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2014, average 
- 100
Compare {fami:qualia} with {fami:qualib} used time(totally) - 2186, average - 
109

Group-13: 
Compare {famia:qualia} with {famib:qualia} used time(totally) - 2012, average 
- 100
Compare {fami:qualia} with {fami:qualib} used time(totally) - 2186, average - 
109
{noformat}

Plz share your comments:)

 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix

[
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400443#comment-13400443
]

Hadoop QA commented on HBASE-6200:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12533286/PerformanceTestCase-6200-94.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2249//console

This message is automatically generated.

KeyComparator.compareWithoutRow can be wrong when families have the same
prefix
---

Key: HBASE-6200
URL: https://issues.apache.org/jira/browse/HBASE-6200
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch,
HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch,
HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch,
HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch

As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird
behavior when some families share the same prefix. He posted a link to his
code to show how it fails, http://pastebin.com/7TBA1XGh
Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families
and qualifiers so f:a is said to be bigger than f1:, which is false. Then
what happens is that the KVs are returned in the right order from the RS but
then doing {{Result.binarySearch}} it uses
{{KeyComparator.compareWithoutRow}} which has a different sorting so the end
result is undetermined.
I added some debug and I can see that the data is returned in the right order
but {{Arrays.binarySearch}} returned the wrong KV, which is then verified
agains the passed family and qualifier which fails so null is returned.
I don't know how frequent it is for users to have families with the same
prefix, but those that do have that and that use those families at the same
time will have big correctness issues. This is why I mark this as a blocker.

[jira] [Commented] (HBASE-3743) Throttle major compaction

2012-06-25 Thread Lars George (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400453#comment-13400453
 ] 

Lars George commented on HBASE-3743:


+1 on a feature like that. We need some script/tool/thread that can run major 
compactions based on load and abort if the load goes over a certain threshold. 
Once the low load is resumed we can continue where left off. Do this region by 
region, with a configurable number, i.e. one per cluster, one per node, and so 
on.

We should also add a JMX/API call that returns the compaction status per 
server. It should list the various compaction queues, live compactions, their 
scope, and region/cf they work on. Maybe put this into the ServerInfo?

 Throttle major compaction
 -

 Key: HBASE-3743
 URL: https://issues.apache.org/jira/browse/HBASE-3743
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Joep Rottinghuis

 Add the ability to throttle major compaction.
 For those use cases when a stop-the-world approach is not practical, it is 
 useful to be able to throttle the impact that major compaction has on the 
 cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix

[
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HBASE-6200:
-

Comment: was deleted

(was: -1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12533286/PerformanceTestCase-6200-94.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2249//console

This message is automatically generated.)

KeyComparator.compareWithoutRow can be wrong when families have the same
prefix
---

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix


[ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400462#comment-13400462
 ] 

Lars Hofhansl commented on HBASE-6200:
--

So if I read that right 1.000.000 KV compares take 100ms without the patch and 
121ms with the patch. I think that would be within the noise of any real 
operation.


 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server

[
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400481#comment-13400481
]

Zhihong Ted Yu commented on HBASE-5875:
---

I think so.

Process RIT and Master restart may remove an online server considering it as
a dead server
--

Key: HBASE-5875
URL: https://issues.apache.org/jira/browse/HBASE-5875
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.1

Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch,
HBASE-5875_0.94_1.patch, HBASE-5875_trunk.patch, HBASE-5875_trunk.patch,
HBASE-5875v2.patch

If on master restart it finds the ROOT/META to be in RIT state, master tries
to assign the ROOT region through ProcessRIT.
Master will trigger the assignment and next will try to verify the Root
Region Location.
Root region location verification is done seeing if the RS has the region in
its online list.
If the master triggered assignment has not yet been completed in RS then the
verify root region location will fail.
Because it failed
{code}
splitLogAndExpireIfOnline(currentRootServer);
{code}
we do split log and also remove the server from online server list. Ideally
here there is nothing to do in splitlog as no region server was restarted.
So master, though the server is online, master just invalidates the region
server.
In a special case, if i have only one RS then my cluster will become non
operative.

[jira] [Commented] (HBASE-6195) Increment data will be lost when the memstore is flushed

2012-06-25 Thread Xing Shi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400505#comment-13400505
 ] 

Xing Shi commented on HBASE-6195:
-

@Ted and @ram:
This problem will simply occur when one KeyValue have same row, family, 
qualifier, timestamp and different memstoreTS.

There are losts of optimisation for memstoreTS for storage:

1. The flush will set memstoreTS to 0 not just Increment but Put, code in 
Store.internalFlushCache():
{code}
  if (kv.getMemstoreTS() = smallestReadPoint) {
// let us not change the original KV. It could be in the memstore
// changing its memstoreTS could affect other threads/scanners.
kv = kv.shallowCopy();
kv.setMemstoreTS(0);
  }
{code}
If the versions of the same row with same TimeStamp flushed to StoreFiles, the 
get will choose the latest version by
{code}
// Negate this comparison so later edits show up first
  return -Longs.compare(left.getMemstoreTS(), right.getMemstoreTS());
{code}

Because the TimeStamps(in one millionsecond) and memstoreTSs are all the 
same(0) in StoreFiles, so we didn't know which one is the newest.

2. Besides this, in StoreFileScanner, there is an optimisation in 
HBASE-4346(code through HBASE-2856)
{code}
if (cur.getMemstoreTS() = readPoint) {
  cur.setMemstoreTS(0);
}
{code}

So, even though we set memstoreTS progressively increases when 
Increment(memstoreTS will always 0) or Put, if we flushed two records(all the 
same excepts memstoreTS, sf1.row.memstoreTS  sf2.row.memstoreTS) into two 
StoreFiles. The memstoreTSs will also be set to 0, and we may got the old 
record sf1.row


3. Why I can't get all the records for different memstoreTS?
In the Scanner, the ExplicitColumnTracker will be used for tracking. And there 
are such code in ExplicitColumnTracker.checkColumn():
{code}
  //If column matches, check if it is a duplicate timestamp
  if (sameAsPreviousTS(timestamp)) {
//If duplicate, skip this Key
return ScanQueryMatcher.MatchCode.SKIP;
  }
{code}

So the Get returns just one result although they are different for memstoreTS.

4. How to resolve this?
There are some optimization through the memstoreTS makes the solution complex, 
I still don't find a solution for this problem and still thinking how to, may 
be remove some optimization.

 Increment data will be lost when the memstore is flushed
 

 Key: HBASE-6195
 URL: https://issues.apache.org/jira/browse/HBASE-6195
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Xing Shi
Assignee: ShiXing
 Fix For: 0.96.0, 0.94.1

 Attachments: 6195-trunk-V7.patch, 6195.addendum, 
 HBASE-6195-trunk-V2.patch, HBASE-6195-trunk-V3.patch, 
 HBASE-6195-trunk-V4.patch, HBASE-6195-trunk-V5.patch, 
 HBASE-6195-trunk-V6.patch, HBASE-6195-trunk.patch


 There are two problems in increment() now:
 First:
 I see that the timestamp(the variable now) in HRegion's Increment() is 
 generated before got the rowLock, so when there are multi-thread increment 
 the same row, although it generate earlier, it may got the lock later. 
 Because increment just store one version, so till now, the result will still 
 be right.
 When the region is flushing, these increment will read the kv from snapshot 
 and memstore with whose timestamp is larger, and write it back to memstore. 
 If the snapshot's timestamp larger than the memstore, the increment will got 
 the old data and then do the increment, it's wrong.
 Secondly:
 Also there is a risk in increment. Because it writes the memstore first and 
 then HLog, so if it writes HLog failed, the client will also read the 
 incremented value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


 [ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-4379:
--

Assignee: Anoop Sam John  (was: Jonathan Hsieh)

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Anoop Sam John
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix


[ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400529#comment-13400529
 ] 

Zhihong Ted Yu commented on HBASE-6200:
---

@Lars:
What other tests do you think should be performed ?

 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400545#comment-13400545
 ] 

Jonathan Hsieh edited comment on HBASE-6230 at 6/25/12 3:41 PM:


It's been pointed out to me that oracle has a feature called flashback (more 
specifically a flashback archive) which is seems functionally closer to we've 
for the use cases I think we've been talking about.

http://docs.oracle.com/cd/B28359_01/appdev.111/b28424/adfns_flashback.htm
http://www.oracle.com/technetwork/database/features/availability/flashback-overview-082751.html
http://www.orafaq.com/node/50

  was (Author: jmhsieh):
It's been pointed out to me that oracle has a feature called flashback 
(more specifically a flashback archive which is seems functionally closer to 
we've for the use cases I think we've been talking about.

http://docs.oracle.com/cd/B28359_01/appdev.111/b28424/adfns_flashback.htm
http://www.oracle.com/technetwork/database/features/availability/flashback-overview-082751.html
http://www.orafaq.com/node/50
  
 [brainstorm] Restore snapshots for HBase 0.96
 ---

 Key: HBASE-6230
 URL: https://issues.apache.org/jira/browse/HBASE-6230
 Project: HBase
  Issue Type: Brainstorming
Reporter: Jesse Yates
Assignee: Matteo Bertozzi

 Discussion ticket around the definitions/expectations of different parts of 
 snapshot restoration.  This is complementary, but separate from the _how_ of 
 taking a snapshot of a table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96


[ 
https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400545#comment-13400545
 ] 

Jonathan Hsieh commented on HBASE-6230:
---

It's been pointed out to me that oracle has a feature called flashback (more 
specifically a flashback archive which is seems functionally closer to we've 
for the use cases I think we've been talking about.

http://docs.oracle.com/cd/B28359_01/appdev.111/b28424/adfns_flashback.htm
http://www.oracle.com/technetwork/database/features/availability/flashback-overview-082751.html
http://www.orafaq.com/node/50

 [brainstorm] Restore snapshots for HBase 0.96
 ---

 Key: HBASE-6230
 URL: https://issues.apache.org/jira/browse/HBASE-6230
 Project: HBase
  Issue Type: Brainstorming
Reporter: Jesse Yates
Assignee: Matteo Bertozzi

 Discussion ticket around the definitions/expectations of different parts of 
 snapshot restoration.  This is complementary, but separate from the _how_ of 
 taking a snapshot of a table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5699) Run with 1 WAL in HRegionServer


[ 
https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400559#comment-13400559
 ] 

Lars Hofhansl commented on HBASE-5699:
--

Assuming Datanodes and RegionServers are colocated no more bits will have to 
cross the (aggregate) wires. Further assuming good load balancing within 
HBase the net bandwidth is still spread over the cluster (but with lower 
latency at each RegionServer).
So I do not believe that HBASE-6116 will actually hurt performance.

The key question is whether WAL writing is mostly bound by latency or bandwidth 
(And I do not know.)
Do we get 35-40mb throughput from writing the WAL? If not, it is likely bound 
by latency.

 Run with  1 WAL in HRegionServer
 -

 Key: HBASE-5699
 URL: https://issues.apache.org/jira/browse/HBASE-5699
 Project: HBase
  Issue Type: Improvement
Reporter: binlijin
Assignee: Li Pi
 Attachments: PerfHbase.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6266) Updation issue in hbase.


 [ 
https://issues.apache.org/jira/browse/HBASE-6266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Daniel Cryans resolved HBASE-6266.
---

Resolution: Invalid

The null pointer exception is in your code, we can't fix it for you :)

Closing this jira as invalid.

 Updation issue in hbase.
 

 Key: HBASE-6266
 URL: https://issues.apache.org/jira/browse/HBASE-6266
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.19.3
 Environment: Linux
Reporter: shailendra kumar
Priority: Critical
 Fix For: 0.19.3


 Hi,
 please find the below logs for same scenario:
 java.lang.NullPointerException
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.getStringData(SchedulerMR.java:1149)
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.updateJobTable(SchedulerMR.java:945)
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.commitTask(SchedulerMR.java:803)
   at org.apache.hadoop.mapred.Task.commit(Task.java:721)
   at org.apache.hadoop.mapred.Task.done(Task.java:644)
   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:460)
   at org.apache.hadoop.mapred.Child.main(Child.java:158)
 java.lang.NullPointerException
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.getStringData(SchedulerMR.java:1149)
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.updateJobTableForReporting(SchedulerMR.java:1055)
   at 
 com.onmobile.bmg.scheduler.mr.scheduler.SchedulerMR$SchedulerOutputCommitter.commitTask(SchedulerMR.java:804)
   at org.apache.hadoop.mapred.Task.commit(Task.java:721)
   at org.apache.hadoop.mapred.Task.done(Task.java:644)
   at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:460)
   at org.apache.hadoop.mapred.Child.main(Child.java:158)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6265) Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed


[ 
https://issues.apache.org/jira/browse/HBASE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400585#comment-13400585
 ] 

Zhihong Ted Yu commented on HBASE-6265:
---

Sounds good to me.
A new test case would be nice to show this problem.

 Calling getTimestamp() on a KV in cp.prePut() causes KV not to be flushed
 -

 Key: HBASE-6265
 URL: https://issues.apache.org/jira/browse/HBASE-6265
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Lars George

 There is an issue when you call getTimestamp() on any KV handed into a 
 Coprocessor's prePut(). It initializes the internal timestampCache 
 variable. 
 When you then pass it to the normal processing, the region server sets the 
 time to the server time in case you have left it unset from the client side 
 (updateLatestStamp() call). 
 The TimeRangeTracker then calls getTimestamp() later on to see if it has to 
 include the KV, but instead of getting the proper time it sees the cached 
 timestamp from the prePut() call.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5352) ACL improvements

[
https://issues.apache.org/jira/browse/HBASE-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400591#comment-13400591
]

Laxman commented on HBASE-5352:
---

Patch available for review : HBASE-6257
Please review.

ACL improvements

Key: HBASE-5352
URL: https://issues.apache.org/jira/browse/HBASE-5352
Project: HBase
Issue Type: Improvement
Components: security
Affects Versions: 0.92.1, 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar

In this issue I would like to open discussion for a few minor ACL related
improvements. The proposed changes are as follows:
1. Introduce something like
AccessControllerProtocol.checkPermissions(Permission[] permissions) API, so
that clients can check access rights before carrying out the operations. We
need this kind of operation for HCATALOG-245, which introduces authorization
providers for hbase over hcat. We cannot use getUserPermissions() since it
requires ADMIN permissions on the global/table level.
2. getUserPermissions(tableName)/grant/revoke and drop/modify table
operations should not check for global CREATE/ADMIN rights, but table
CREATE/ADMIN rights. The reasoning is that if a user is able to admin or read
from a table, she should be able to read the table's permissions. We can
choose whether we want only READ or ADMIN permissions for
getUserPermission(). Since we check for global permissions first for table
permissions, configuring table access using global permissions will continue
to work.
3. Grant/Revoke global permissions - HBASE-5342 (included for completeness)
From all 3, we may want to backport the first one to 0.92 since without it,
Hive/Hcatalog cannot use Hbase's authorization mechanism effectively.
I will create subissues and convert HBASE-5342 to a subtask when we get some
feedback, and opinions for going further.

[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.

[
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400592#comment-13400592
]

Jimmy Xiang commented on HBASE-5360:

Good catch. I will fix it in a separate jira. Thanks.

[uberhbck] Add options for how to handle offline split parents.

Key: HBASE-5360
URL: https://issues.apache.org/jira/browse/HBASE-5360
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch,
5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path

In a recent case, we attempted to repair a cluster that suffered from
HBASE-4238 that had about 6-7 generations of leftover split data. The hbck
repair options in an development version of HBASE-5128 treat HDFS as ground
truth but didn't check SPLIT and OFFLINE flags only found in meta. The net
effect was that it essentially attempted to merge many regions back into its
eldest geneneration's parent's range.
More safe guards to prevent mega-merges are being added on HBASE-5128.
This issue would automate the handling of the mega-merge avoiding cases
such as lingering grandparents. The strategy here would be to add more
checks against .META., and perform part of the catalog janitor's
responsibilities for lingering grandparents. This would potentially include
options to sideline regions, deleting grandparent regions, min size for
sidelining, and mechanisms for cleaning .META..
Note: There already exists an mechanism to reload these regions -- the bulk
loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents
(automatically splitting them if necessary) to HBase.

[jira] [Updated] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix


 [ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-6200:
--

Status: Open  (was: Patch Available)

 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0, 0.92.1, 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

2012-06-25 Thread Jesse Yates (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400616#comment-13400616
 ] 

Jesse Yates commented on HBASE-2315:


personally, I'd prefer to see a logical higher-level WAL interface and then 
have an FsWAL subclass that might have some more specifics to make an easier 
match to an underlying FS implementation (hadoop or otherwise). Maybe a bit 
more pain upfront, but will (likely) pay off in the end).

 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6240) Race in HCM.getMaster stalls clients


[ 
https://issues.apache.org/jira/browse/HBASE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400643#comment-13400643
 ] 

Jean-Daniel Cryans commented on HBASE-6240:
---

Ah yeah I completely overlooked that. Did you see how we get this exception? It 
looks so dirty in the code and now having it twice would look a lot worse.

 Race in HCM.getMaster stalls clients
 

 Key: HBASE-6240
 URL: https://issues.apache.org/jira/browse/HBASE-6240
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.94.1

 Attachments: HBASE-6240.patch, HBASE-6240_1_0.94.patch


 I found this issue trying to run YCSB on 0.94, I don't think it exists on any 
 other branch. I believe that this was introduced in HBASE-5058 Allow 
 HBaseAdmin to use an existing connection.
 The issue is that in HCM.getMaster it does this recipe:
  # Check if the master is null and runs (if so, return)
  # Grab a lock on masterLock
  # nullify this.master
  # try to get a new master
 The issue happens at 3, it should re-run 1 since while you're waiting on the 
 lock someone else could have already fixed it for you. What happens right now 
 is that the threads are all able to set the master to null before others are 
 able to get out of getMaster and it's a complete mess.
 Figuring it out took me some time because it doesn't manifest itself right 
 away, silent retries are done in the background. Basically the first clue was 
 this:
 {noformat}
 Error doing get: org.apache.hadoop.hbase.client.RetriesExhaustedException: 
 Failed after attempts=10, exceptions:
 Tue Jun 19 23:40:46 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:47 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:48 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:49 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:51 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:53 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:57 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:01 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:09 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:25 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 {noformat}
 This was caused by the little dance up in HBaseAdmin where it deletes stale 
 connections... which are not stale at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6253) Dont allow user to disable/drop _acl_ table


[ 
https://issues.apache.org/jira/browse/HBASE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400665#comment-13400665
 ] 

Andrew Purtell commented on HBASE-6253:
---

+1 on patch

 Dont allow user to disable/drop _acl_ table
 ---

 Key: HBASE-6253
 URL: https://issues.apache.org/jira/browse/HBASE-6253
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 0.94.0, 0.96.0, 0.94.1
Reporter: Gopinathan A
  Labels: security
 Fix For: 0.94.1

 Attachments: HBASE-6253.patch, HBASE-6253.patch


 Currently HTableDescriptor.isLegalTableName API doesn't check for the _acl_ 
 table name, due to this user can able to disable/enable/drop/create the acl 
 table. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6192) Document ACL matrix in the book


 [ 
https://issues.apache.org/jira/browse/HBASE-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laxman updated HBASE-6192:
--

Attachment: HBase Security-ACL Matrix.xls
HBase Security-ACL Matrix.pdf

@Stack, attached the updated matrix. Please review.


 Document ACL matrix in the book
 ---

 Key: HBASE-6192
 URL: https://issues.apache.org/jira/browse/HBASE-6192
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, security
Affects Versions: 0.96.0, 0.94.1
Reporter: Enis Soztutar
Assignee: Laxman
  Labels: documentaion, security
 Attachments: HBase Security-ACL Matrix.pdf, HBase Security-ACL 
 Matrix.pdf, HBase Security-ACL Matrix.pdf, HBase Security-ACL Matrix.xls, 
 HBase Security-ACL Matrix.xls, HBase Security-ACL Matrix.xls


 We have an excellent matrix at 
 https://issues.apache.org/jira/secure/attachment/12531252/Security-ACL%20Matrix.pdf
  for ACL. Once the changes are done, we can adapt that and put it in the 
 book, also add some more documentation about the new authorization features. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400674#comment-13400674
 ] 

Jonathan Hsieh commented on HBASE-4379:
---

Anoop, 

In the case you are talking about -- is disabling in progress as hbck is being 
run or it is just in a funky state?  I believe this repair only happens if the 
user specifies the option to repair these holes right? (fixHdfsHoles or 
fixMetaHoles options).

Can you also at least do a port to trunk? Generally it is best to start there 
and then backport to older versions.  This guarantees that it makes it to all 
future releases. :)

{code}
+  @Test
+  public void testMissingLastRegion() throws Exception {
+String table = testMissingFirstRegion;
+
{code}

nit: please change testMissingFirstRegion.



 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Anoop Sam John
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.

[
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400592#comment-13400592
]

Jimmy Xiang edited comment on HBASE-5360 at 6/25/12 5:12 PM:
-

Good catch. Fixed as an addendum. Thanks.

was (Author: jxiang):
Good catch. I will fix it in a separate jira. Thanks.

[uberhbck] Add options for how to handle offline split parents.

Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch,
5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path

[jira] [Created] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default

Andrew Purtell created HBASE-6267:
-

 Summary: hbase.store.delete.expired.storefile should be true by 
default
 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Andrew Purtell
 Fix For: 0.96.0, 0.94.1


HBASE-5199 introduces this logic into Store:

{code}
+  // Delete the expired store files before the compaction selection.
+  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
+   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
+CompactSelection expiredSelection = compactSelection
+.selectExpiredStoreFilesToCompact(
+EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
+
+// If there is any expired store files, delete them  by compaction.
+if (expiredSelection != null) {
+  return expiredSelection;
+}
+  }
{code}

Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default


 [ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-6267:
--

Affects Version/s: 0.94.1
   0.96.0
Fix Version/s: (was: 0.94.1)
   (was: 0.96.0)

 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default

2012-06-25 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400693#comment-13400693
 ] 

stack commented on HBASE-6267:
--

+1 for true (if above does what I think its doing -- is there a test Andy?)

 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default

2012-06-25 Thread stack (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400695#comment-13400695
 ] 

stack commented on HBASE-6267:
--

Is expiredSelection a good name for what is returned out of 
selectExpiredStoreFilesToCompact.  Its a bit hard to read that pasted section 
in isolation.

 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6200) KeyComparator.compareWithoutRow can be wrong when families have the same prefix


[ 
https://issues.apache.org/jira/browse/HBASE-6200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400698#comment-13400698
 ] 

Zhihong Ted Yu commented on HBASE-6200:
---

@Stack:
Do you have further comments on the latest patches ?

 KeyComparator.compareWithoutRow can be wrong when families have the same 
 prefix
 ---

 Key: HBASE-6200
 URL: https://issues.apache.org/jira/browse/HBASE-6200
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.90.6, 0.92.1, 0.94.0
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Blocker
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 6200-trunk-v2.patch, HBASE-6200-90-v2.patch, 
 HBASE-6200-90.patch, HBASE-6200-92-v2.patch, HBASE-6200-92.patch, 
 HBASE-6200-94-v2.patch, HBASE-6200-94.patch, HBASE-6200-trunk-v2.patch, 
 HBASE-6200-trunk.patch, PerformanceTestCase-6200-94.patch


 As reported by Desert Rose on IRC and on the ML, {{Result}} has a weird 
 behavior when some families share the same prefix. He posted a link to his 
 code to show how it fails, http://pastebin.com/7TBA1XGh
 Basically {{KeyComparator.compareWithoutRow}} doesn't differentiate families 
 and qualifiers so f:a is said to be bigger than f1:, which is false. Then 
 what happens is that the KVs are returned in the right order from the RS but 
 then doing {{Result.binarySearch}} it uses 
 {{KeyComparator.compareWithoutRow}} which has a different sorting so the end 
 result is undetermined.
 I added some debug and I can see that the data is returned in the right order 
 but {{Arrays.binarySearch}} returned the wrong KV, which is then verified 
 agains the passed family and qualifier which fails so null is returned.
 I don't know how frequent it is for users to have families with the same 
 prefix, but those that do have that and that use those families at the same 
 time will have big correctness issues. This is why I mark this as a blocker.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default


[ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400714#comment-13400714
 ] 

Andrew Purtell commented on HBASE-6267:
---

bq. (if above does what I think its doing – is there a test Andy?)

TestStore#testDeleteExpiredStoreFiles, see 
https://issues.apache.org/jira/secure/attachment/12514057/hbase-5199.patch

bq. Is expiredSelection a good name for what is returned out of 
selectExpiredStoreFilesToCompact.

That's a question for Liyin the original author I think.

If we are game I can put in a trivial patch to make this on by default.




 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6240) Race in HCM.getMaster stalls clients


[ 
https://issues.apache.org/jira/browse/HBASE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400715#comment-13400715
 ] 

ramkrishna.s.vasudevan commented on HBASE-6240:
---

bq.Did you see how we get this exception?
When we try to do master.isRunning(), the call goes to the master itself thro 
the RPC layer.  So as the master is down we get this exception.  Did you mean 
something else? 
Sorry, if my answer is not clear.

 Race in HCM.getMaster stalls clients
 

 Key: HBASE-6240
 URL: https://issues.apache.org/jira/browse/HBASE-6240
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.94.1

 Attachments: HBASE-6240.patch, HBASE-6240_1_0.94.patch


 I found this issue trying to run YCSB on 0.94, I don't think it exists on any 
 other branch. I believe that this was introduced in HBASE-5058 Allow 
 HBaseAdmin to use an existing connection.
 The issue is that in HCM.getMaster it does this recipe:
  # Check if the master is null and runs (if so, return)
  # Grab a lock on masterLock
  # nullify this.master
  # try to get a new master
 The issue happens at 3, it should re-run 1 since while you're waiting on the 
 lock someone else could have already fixed it for you. What happens right now 
 is that the threads are all able to set the master to null before others are 
 able to get out of getMaster and it's a complete mess.
 Figuring it out took me some time because it doesn't manifest itself right 
 away, silent retries are done in the background. Basically the first clue was 
 this:
 {noformat}
 Error doing get: org.apache.hadoop.hbase.client.RetriesExhaustedException: 
 Failed after attempts=10, exceptions:
 Tue Jun 19 23:40:46 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:47 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:48 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:49 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:51 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:53 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:57 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:01 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:09 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:25 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 {noformat}
 This was caused by the little dance up in HBaseAdmin where it deletes stale 
 connections... which are not stale at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400722#comment-13400722
 ] 

Jonathan Hsieh commented on HBASE-2315:
---

+1 to what Jesse said.  The HLog class is relatively well encapsulated.  It 
already has its own Reader and Writer interfaces (I'd probably ignore the FS 
argument or some how hide BK specific stuff into a constructor), and a Metric 
helper class.  Excluding its constructors, HLog only has a few hdfs-specific 
public methods (hsync, hflush, sync) and some public but static methods.

 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6227) SSH and cluster startup causes data loss


[ 
https://issues.apache.org/jira/browse/HBASE-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400723#comment-13400723
 ] 

ramkrishna.s.vasudevan commented on HBASE-6227:
---

I am planning to commit this tomorrow.  Pls provide your comments/suggestions. 
@Chunhui
Could you provide a patch for 0.94 also?

 SSH and cluster startup  causes data loss
 -

 Key: HBASE-6227
 URL: https://issues.apache.org/jira/browse/HBASE-6227
 Project: HBase
  Issue Type: Bug
  Components: master
Reporter: chunhui shen
Assignee: chunhui shen
 Attachments: HBASE-6227.patch, HBASE-6227v2.patch


 In AssignmentManager#processDeadServersAndRegionsInTransition, if 
 servershutdownhandler is processing and master consider it cluster startup, 
 master will assign all user regions, however, servershutdownhandler has not 
 completed splitting logs.
 Let me describe it in more detail.
 Suppose there are two regionservers A1 and B1, A1 carried META and ROOT
 1.master restart and completed assignRootAndMeta
 2.A1 and B1 are both restarted, new regionservers are A2 and B2
 3.SSH which processed for A1 completed assigning ROOT and META
 4.master do rebuilding user regions and no regions added to master's region 
 list
 5.master consider it as a cluster startup, and assign all user regions
 6.SSH which processed for B1 start to split B1's logs
 7.All regions' data carried by B1 would loss.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400728#comment-13400728
 ] 

Anoop Sam John commented on HBASE-4379:
---

Jon
I will make patch for trunk tomorrow..  


 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Anoop Sam John
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus


 [ 
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5933:
--

Attachment: HBASE-5933-v3.patch

 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
 -

 Key: HBASE-5933
 URL: https://issues.apache.org/jira/browse/HBASE-5933
 Project: HBase
  Issue Type: Task
  Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch


 From Stack's comments on HBASE-5444:
 We need this import?  Its for cp.  Thats ok I'd say One day we can hide 
 that too..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server

[
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-5875:
--

Attachment: HBASE-5875_0.94_2.patch

PAtch for 0.94 ready for commit.

Process RIT and Master restart may remove an online server considering it as
a dead server
--

Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch,
HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch,
HBASE-5875_trunk.patch, HBASE-5875v2.patch

[jira] [Updated] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server

[
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-5875:
--

Attachment: HBASE-5875_trunk_1.patch

Patch for trunk, ready for commit.

Process RIT and Master restart may remove an online server considering it as
a dead server
--

Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch,
HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch,
HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400738#comment-13400738
]

Zhihong Ted Yu commented on HBASE-6205:
---

From
http://www.oracle.com/technetwork/database/features/availability/flashback-overview-082751.html
(Jon Hsieh referred to):

Flashback Drop: recover an accidentally dropped table. It restores the dropped
table, and all of its indexes, constraints, and triggers, from the Recycle Bin
(a logical container of all dropped objects).

Support an option to keep data of dropped table for some time
-

Key: HBASE-6205
URL: https://issues.apache.org/jira/browse/HBASE-6205
Project: HBase
Issue Type: New Feature
Affects Versions: 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

User may drop table accidentally because of error code or other uncertain
reasons.
Unfortunately, it happens in our environment because one user make a mistake
between production cluster and testing cluster.
So, I just give a suggestion, do we need to support an option to keep data of
dropped table for some time, e.g. 1 day
In the patch:
We make a new dir named .trashtables in the rood dir.
In the DeleteTableHandler, we move files in dropped table's dir to trash
table dir instead of deleting them directly.
And Create new class TrashCleaner which will clean dropped tables if it is
time out with a period check.
Default keep time for dropped tables is 1 day, and check period is 1 hour.

[jira] [Commented] (HBASE-4379) [hbck] Does not complain about tables with no end region [Z,]


[ 
https://issues.apache.org/jira/browse/HBASE-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400739#comment-13400739
 ] 

Anoop Sam John commented on HBASE-4379:
---

The case I was mentioning is one table having 3 regions is disabled now. Then 
suppose 2 regions (1st and last) are deployed in some RS. Now when the HBCK is 
run, we expect it will fix this making these 2 regions offline.
But when running the checkRegionChain, it might report as a hole and if the 
fixHdfsHoles option is ON, it might create a new region. [This is unwanted I 
think]

In testRegionShouldNotBeDeployed() if we make the region [ddd,) also deployed 
on some RS, we can see it report a region hole error and trying to fix the same.

 [hbck] Does not complain about tables with no end region [Z,]
 -

 Key: HBASE-4379
 URL: https://issues.apache.org/jira/browse/HBASE-4379
 Project: HBase
  Issue Type: Bug
  Components: hbck
Affects Versions: 0.90.5, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Anoop Sam John
 Attachments: 
 0001-HBASE-4379-hbck-does-not-complain-about-tables-with-.patch, 
 HBASE-4379_94.patch, hbase-4379.v2.patch


 hbck does not detect or have an error condition when the last region of a 
 table is missing (end key != '').

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server

[
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-5875:
--

Resolution: Fixed
Status: Resolved (was: Patch Available)

Committed to 0.94.1 and 0.96.
Thanks to Rajesh for the patch.
Thanks Ted, Chunhui and Stack for the review.

Process RIT and Master restart may remove an online server considering it as
a dead server
--

Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch,
HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch,
HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch

[jira] [Updated] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server


 [ 
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-5875:
--

Fix Version/s: 0.96.0

 Process RIT and Master restart may remove an online server considering it as 
 a dead server
 --

 Key: HBASE-5875
 URL: https://issues.apache.org/jira/browse/HBASE-5875
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch, 
 HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch, 
 HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch


 If on master restart it finds the ROOT/META to be in RIT state, master tries 
 to assign the ROOT region through ProcessRIT.
 Master will trigger the assignment and next will try to verify the Root 
 Region Location.
 Root region location verification is done seeing if the RS has the region in 
 its online list.
 If the master triggered assignment has not yet been completed in RS then the 
 verify root region location will fail.
 Because it failed 
 {code}
 splitLogAndExpireIfOnline(currentRootServer);
 {code}
 we do split log and also remove the server from online server list. Ideally 
 here there is nothing to do in splitlog as no region server was restarted.
 So master, though the server is online, master just invalidates the region 
 server.
 In a special case, if i have only one RS then my cluster will become non 
 operative.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

2012-06-25 Thread Jesse Yates (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400752#comment-13400752
 ] 

Jesse Yates commented on HBASE-6205:


Is it possible to do an hbase-flashback by just rebuilding from the files in 
hdfs /trash? The main thing I'm concerned with is having the deleted files end 
up in a bunch of places.

 Support an option to keep data of dropped table for some time
 -

 Key: HBASE-6205
 URL: https://issues.apache.org/jira/browse/HBASE-6205
 Project: HBase
  Issue Type: New Feature
Affects Versions: 0.94.0, 0.96.0
Reporter: chunhui shen
Assignee: chunhui shen
 Fix For: 0.96.0

 Attachments: HBASE-6205.patch, HBASE-6205v2.patch, 
 HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch


 User may drop table accidentally because of error code or other uncertain 
 reasons.
 Unfortunately, it happens in our environment because one user make a mistake 
 between production cluster and testing cluster.
 So, I just give a suggestion, do we need to support an option to keep data of 
 dropped table for some time, e.g. 1 day
 In the patch:
 We make a new dir named .trashtables in the rood dir.
 In the DeleteTableHandler, we move files in dropped table's dir to trash 
 table dir instead of deleting them directly.
 And Create new class TrashCleaner which will clean dropped tables if it is 
 time out with a period check.
 Default keep time for dropped tables is 1 day, and check period is 1 hour.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

2012-06-25 Thread Enis Soztutar (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400756#comment-13400756
]

Enis Soztutar commented on HBASE-6205:
--

+1 on using the hdfs trash, it has all the properties we need (configurable,
easy to use, and works). We just need a way to reconstruct the table.

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400764#comment-13400764
]

Jimmy Xiang commented on HBASE-6205:

hbck can be used to reconstruct the table.

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Comment Edited] (HBASE-6205) Support an option to keep data of dropped table for some time

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400764#comment-13400764
]

Jimmy Xiang edited comment on HBASE-6205 at 6/25/12 6:42 PM:
-

hbck can be used to get the table back after all the data files are recovered
from hdfs trash.

was (Author: jxiang):
hbck can be used to reconstruct the table.

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Commented] (HBASE-6240) Race in HCM.getMaster stalls clients


[ 
https://issues.apache.org/jira/browse/HBASE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400772#comment-13400772
 ] 

Jean-Daniel Cryans commented on HBASE-6240:
---

bq. Did you mean something else? 

You answered right. It seems to me that if we get an exception, then either the 
master is down or it is unreachable. Unfortunately we don't have a concept for 
the latter.

Looking at the code we do have a isMasterRunning() that encapsulates some of 
logic about looking up a master but strangely enough you'll never get *false* 
getting out of there, it's true or MasterNotRunningException!

I think we should refactor this in a followup jira. In the meantime +1 on your 
patch Ram.

 Race in HCM.getMaster stalls clients
 

 Key: HBASE-6240
 URL: https://issues.apache.org/jira/browse/HBASE-6240
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Jean-Daniel Cryans
Priority: Critical
 Fix For: 0.94.1

 Attachments: HBASE-6240.patch, HBASE-6240_1_0.94.patch


 I found this issue trying to run YCSB on 0.94, I don't think it exists on any 
 other branch. I believe that this was introduced in HBASE-5058 Allow 
 HBaseAdmin to use an existing connection.
 The issue is that in HCM.getMaster it does this recipe:
  # Check if the master is null and runs (if so, return)
  # Grab a lock on masterLock
  # nullify this.master
  # try to get a new master
 The issue happens at 3, it should re-run 1 since while you're waiting on the 
 lock someone else could have already fixed it for you. What happens right now 
 is that the threads are all able to set the master to null before others are 
 able to get out of getMaster and it's a complete mess.
 Figuring it out took me some time because it doesn't manifest itself right 
 away, silent retries are done in the background. Basically the first clue was 
 this:
 {noformat}
 Error doing get: org.apache.hadoop.hbase.client.RetriesExhaustedException: 
 Failed after attempts=10, exceptions:
 Tue Jun 19 23:40:46 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:47 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:48 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:49 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:51 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:53 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:40:57 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:01 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:09 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 Tue Jun 19 23:41:25 UTC 2012, 
 org.apache.hadoop.hbase.client.HTable$3@571a4bd4, java.io.IOException: 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2eb0a3f5
  closed
 {noformat}
 This was caused by the little dance up in HBaseAdmin where it deletes stale 
 connections... which are not stale at all.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server


[ 
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400775#comment-13400775
 ] 

Hudson commented on HBASE-5875:
---

Integrated in HBase-0.94 #280 (See 
[https://builds.apache.org/job/HBase-0.94/280/])
HBASE-5875 Process RIT and Master restart may remove an online server 
considering it as a dead server

Submitted by:Rajesh
Reviewed by:Ram, Ted, Stack (Revision 1353690)

 Result = FAILURE
ramkrishna : 
Files : 
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 Process RIT and Master restart may remove an online server considering it as 
 a dead server
 --

 Key: HBASE-5875
 URL: https://issues.apache.org/jira/browse/HBASE-5875
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch, 
 HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch, 
 HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch


 If on master restart it finds the ROOT/META to be in RIT state, master tries 
 to assign the ROOT region through ProcessRIT.
 Master will trigger the assignment and next will try to verify the Root 
 Region Location.
 Root region location verification is done seeing if the RS has the region in 
 its online list.
 If the master triggered assignment has not yet been completed in RS then the 
 verify root region location will fail.
 Because it failed 
 {code}
 splitLogAndExpireIfOnline(currentRootServer);
 {code}
 we do split log and also remove the server from online server list. Ideally 
 here there is nothing to do in splitlog as no region server was restarted.
 So master, though the server is online, master just invalidates the region 
 server.
 In a special case, if i have only one RS then my cluster will become non 
 operative.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus

[
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400779#comment-13400779
]

Hadoop QA commented on HBASE-5933:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533346/HBASE-5933-v3.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 15 new or modified tests.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2250//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2250//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2250//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2250//console

This message is automatically generated.

Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
-

Key: HBASE-5933
URL: https://issues.apache.org/jira/browse/HBASE-5933
Project: HBase
Issue Type: Task
Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
Fix For: 0.96.0

Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch

From Stack's comments on HBASE-5444:
We need this import? Its for cp. Thats ok I'd say One day we can hide
that too..

[jira] [Commented] (HBASE-6116) Allow parallel HDFS writes for HLogs.


[ 
https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400801#comment-13400801
 ] 

Andrew Purtell commented on HBASE-6116:
---

bq. I'm going to do this again tomorrow on cluster compute instances. The 
results should be cleaner.

I should have realized this earlier, but CC instances don't support instance 
store volumes, only EBS. EBS is IMO a crappy storage subsystem, in my 
experience instance store is about 2x slower than real hardware, but EBS can be 
10x+ slower and highly variable. So this completes what I can do with EC2.

 Allow parallel HDFS writes for HLogs.
 -

 Key: HBASE-6116
 URL: https://issues.apache.org/jira/browse/HBASE-6116
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 6116-v1.txt, pipelined-vs-parallel-comparison.zip


 In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk.
 This issue will include the necessary reflection changes to optionally enable 
 this for the WALs in HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default

2012-06-25 Thread Liyin Tang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400822#comment-13400822
 ] 

Liyin Tang commented on HBASE-6267:
---

+ 1 to enable it by default. Let me know if you have a better name  :)


 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default


[ 
https://issues.apache.org/jira/browse/HBASE-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400825#comment-13400825
 ] 

Andrew Purtell commented on HBASE-6267:
---

Will commit as soon as local tests for 0.94 complete. Should be in a few hours.

 hbase.store.delete.expired.storefile should be true by default
 --

 Key: HBASE-6267
 URL: https://issues.apache.org/jira/browse/HBASE-6267
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Affects Versions: 0.96.0, 0.94.1
Reporter: Andrew Purtell

 HBASE-5199 introduces this logic into Store:
 {code}
 +  // Delete the expired store files before the compaction selection.
 +  if (conf.getBoolean(hbase.store.delete.expired.storefile, false)
 +   (ttl != Long.MAX_VALUE)  (this.scanInfo.minVersions == 0)) {
 +CompactSelection expiredSelection = compactSelection
 +.selectExpiredStoreFilesToCompact(
 +EnvironmentEdgeManager.currentTimeMillis() - this.ttl);
 +
 +// If there is any expired store files, delete them  by compaction.
 +if (expiredSelection != null) {
 +  return expiredSelection;
 +}
 +  }
 {code}
 Is there any reason why that should not be default {{true}}?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6226) move DataBlockEncoding and related classes to hbase-common module

2012-06-25 Thread Mikhail Bautin (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400830#comment-13400830
]

Mikhail Bautin commented on HBASE-6226:
---

Hi Matt,

The module breakdown you have described makes sense. Unfortunately, I can't see
much from the patch because a lot of files are being moved around. Would you
mind posting it at https://reviews.facebook.com? You need to run mvn -Darc
initialize, then (assuming you have a local git-svn checkout) run arc diff
--only when your changes correspond to the latest local commit, copy-paste the
URL arc gives you into the browser, and fill out some fields such as title and
summary.

Thanks,
Mikhail

move DataBlockEncoding and related classes to hbase-common module
-

Key: HBASE-6226
URL: https://issues.apache.org/jira/browse/HBASE-6226
Project: HBase
Issue Type: Improvement
Components: io, regionserver
Affects Versions: 0.96.0
Reporter: Matt Corgan
Assignee: Matt Corgan
Attachments: HBASE-6226-v1.patch

In order to isolate the implementation details of HBASE-4676 (PrefixTrie
encoding) and other DataBlockEncoders by putting them in modules, this pulls
up the DataBlockEncoding related interfaces into hbase-common.
No tests are moved in this patch. The only notable change was trimming a few
dependencies on HFileBlock which adds dependencies to much of the
regionserver.
The test suite passes locally for me.
I tried to keep it as simple as possible... let me know if there are any
concerns.

[jira] [Commented] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server


[ 
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400840#comment-13400840
 ] 

Hudson commented on HBASE-5875:
---

Integrated in HBase-TRUNK #3070 (See 
[https://builds.apache.org/job/HBase-TRUNK/3070/])
HBASE-5875 Process RIT and Master restart may remove an online server 
considering it as a dead server (Rajesh)

Submitted by:Rajesh 
Reviewed by:Ram Ted, Stack (Revision 1353688)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 Process RIT and Master restart may remove an online server considering it as 
 a dead server
 --

 Key: HBASE-5875
 URL: https://issues.apache.org/jira/browse/HBASE-5875
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch, 
 HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch, 
 HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch


 If on master restart it finds the ROOT/META to be in RIT state, master tries 
 to assign the ROOT region through ProcessRIT.
 Master will trigger the assignment and next will try to verify the Root 
 Region Location.
 Root region location verification is done seeing if the RS has the region in 
 its online list.
 If the master triggered assignment has not yet been completed in RS then the 
 verify root region location will fail.
 Because it failed 
 {code}
 splitLogAndExpireIfOnline(currentRootServer);
 {code}
 we do split log and also remove the server from online server list. Ideally 
 here there is nothing to do in splitlog as no region server was restarted.
 So master, though the server is online, master just invalidates the region 
 server.
 In a special case, if i have only one RS then my cluster will become non 
 operative.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6231) Minor Typos in HBase Book

[
https://issues.apache.org/jira/browse/HBASE-6231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400861#comment-13400861
]

Jean-Marc Spaggiari commented on HBASE-6231:

http://hbase.apache.org/book/important_configurations.html#balancer_config
Under 2.8.3.1 The balancer is periodic operation run on the master to
redistribute regions on the cluster. - I think this reads a little better -
The balancer is a periodic operation which is ran on the master to
redistribute regions on the cluster.
I agree with the first modification, I'm not sure for the 2nd one. I will
suggest The balancer is a periodic operation run on the master to redistribute
regions on the cluster.

http://hbase.apache.org/book/master.html#master.processes.loadbalancer
Under 9.5.4.1 Periodically, and when there are not any regions in transition,
a load balancer will run and move regions around to balance cluster load. - I
think this reads a little better - Periodically, and when there are not any
regions in transition, a load balancer will run and move regions around to
balance the cluster's load.
I will propose to change by ... a load balancer will run and move regions
around to balance cluster's load. I don't think the the is required here.

http://hbase.apache.org/book/node.management.html
Under 14.3.2 Effect repairs if inconsistent. I believe this should be
Expect repairs if inconsistent
I think this one is correct. It says $ ./bin/hbase hbck will effect some
repairs if the cluster is inconsistent.

I will attach a patch with the fixes I'm proposing here.

Minor Typos in HBase Book
-

Key: HBASE-6231
URL: https://issues.apache.org/jira/browse/HBASE-6231
Project: HBase
Issue Type: Improvement
Components: documentation
Reporter: Chad Gorshing
Priority: Trivial
Labels: documentation

I have other minor things like this to submit, what is the best method to
submit these?
http://hbase.apache.org/book/important_configurations.html#balancer_config
Under 2.8.3.1 The balancer is periodic operation run on the master to
redistribute regions on the cluster. - I think this reads a little better -
The balancer is a periodic operation which is ran on the master to
redistribute regions on the cluster.
http://hbase.apache.org/book/master.html#master.processes.loadbalancer
Under 9.5.4.1 Periodically, and when there are not any regions in
transition, a load balancer will run and move regions around to balance
cluster load. - I think this reads a little better - Periodically, and when
there are not any regions in transition, a load balancer will run and move
regions around to balance the cluster's load.
http://hbase.apache.org/book/node.management.html
Under 14.3.2 Effect repairs if inconsistent. I believe this should be
Expect repairs if inconsistent

[jira] [Updated] (HBASE-6231) Minor Typos in HBase Book


 [ 
https://issues.apache.org/jira/browse/HBASE-6231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Marc Spaggiari updated HBASE-6231:
---

Attachment: hbase_JIRA_62.31.patch

Patch for JIRA 6231 (documenation wording).

 Minor Typos in HBase Book
 -

 Key: HBASE-6231
 URL: https://issues.apache.org/jira/browse/HBASE-6231
 Project: HBase
  Issue Type: Improvement
  Components: documentation
Reporter: Chad Gorshing
Priority: Trivial
  Labels: documentation
 Attachments: hbase_JIRA_62.31.patch


 I have other minor things like this to submit, what is the best method to 
 submit these?
 http://hbase.apache.org/book/important_configurations.html#balancer_config
 Under 2.8.3.1 The balancer is periodic operation run on the master to 
 redistribute regions on the cluster. - I think this reads a little better - 
 The balancer is a periodic operation which is ran on the master to 
 redistribute regions on the cluster.
 http://hbase.apache.org/book/master.html#master.processes.loadbalancer
 Under 9.5.4.1 Periodically, and when there are not any regions in 
 transition, a load balancer will run and move regions around to balance 
 cluster load. - I think this reads a little better - Periodically, and when 
 there are not any regions in transition, a load balancer will run and move 
 regions around to balance the cluster's load.
 http://hbase.apache.org/book/node.management.html
 Under 14.3.2 Effect repairs if inconsistent. I believe this should be 
 Expect repairs if inconsistent

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

2012-06-25 Thread Flavio Junqueira (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400864#comment-13400864
 ] 

Flavio Junqueira commented on HBASE-2315:
-

If I understand Jesse's proposal, the idea is to have a WAL interface and 
essentially bring all hdfs dependent WAL code under FsWAL? We would then 
implement an equivalent BkWAL for a BookKeeper backend? 

I didn't understand the point about the HLog class being relatively well 
encapsulated, Jonathan. Do you mean to say that it would be relatively easy to 
implement FsWAL by using HLog?



 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

2012-06-25 Thread Jesse Yates (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400870#comment-13400870
 ] 

Jesse Yates commented on HBASE-2315:


@Flavio  - yup, that's what I was getting at.

 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4858) hbase-site.xml example in quickstart doesn't work in Linux


[ 
https://issues.apache.org/jira/browse/HBASE-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400872#comment-13400872
 ] 

Jean-Marc Spaggiari commented on HBASE-4858:


Hi Bryce,

So what you are saying is that you used this example instead?
{code:xml} 
?xml version=1.0?
?xml-stylesheet type=text/xsl href=configuration.xsl?
configuration
  property
namehbase.rootdir/name
!-- Depending on your platform, this may create a 'file:' directory
 in hbase home instead of the desired behavior. Try using a standard
 platform specific absolute path instead. --
value/DIRECTORY/hbase/value
  /property
/configuration
{code} 

On a fully-distributed environment, it should be something like hdfs:// 
But file://... should still be working even under linux. At least, it was 
worling for me. Are you sure there is no typo? Can you give it another try? I 
will retry too on my side.

 hbase-site.xml example in quickstart doesn't work in Linux
 --

 Key: HBASE-4858
 URL: https://issues.apache.org/jira/browse/HBASE-4858
 Project: HBase
  Issue Type: Bug
  Components: documentation
 Environment: java version 1.6.0_23
 OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre11-1)
 OpenJDK Client VM (build 20.0-b11, mixed mode, sharing)
Reporter: Bryce Allen
Priority: Minor

 Under Linux with OpenJDK 1.6, using a file:///XX URL in the config file 
 creates a directory called 'file:' in the hbase root directory. If I use a 
 standard Unix absolute path, it works as expected. This may work on other 
 platforms, but it would be good to add a note in the example:
 {code}
 ?xml version=1.0?
 ?xml-stylesheet type=text/xsl href=configuration.xsl?
 configuration
   property
 namehbase.rootdir/name
 !-- Depending on your platform, this may create a 'file:' directory
  in hbase home instead of the desired behavior. Try using a standard
  platform specific absolute path instead. --
 valuefile:///DIRECTORY/hbase/value
   /property
 /configuration
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3760) Package info docs missing from org.apache.hadoop.hbase.rest


[ 
https://issues.apache.org/jira/browse/HBASE-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400878#comment-13400878
 ] 

Jean-Marc Spaggiari commented on HBASE-3760:


stargate package seems to not exist anymore in the 0.94 version. Neither in 
0.95. Should this issue been closed?

 Package info docs missing from org.apache.hadoop.hbase.rest
 ---

 Key: HBASE-3760
 URL: https://issues.apache.org/jira/browse/HBASE-3760
 Project: HBase
  Issue Type: Task
  Components: documentation, rest
Reporter: Gary Helmling
Priority: Trivial

 It looks like the old stargate package-info javadocs somehow got dropped when 
 it replaced the REST code in 0.90.  Was pointed out on IRC that the package 
 docs existed here:
 http://hbase.apache.org/docs/r0.20.6/api/org/apache/hadoop/hbase/stargate/package-summary.html
 We should pull them out of 0.20 into 0.90 and make whatever updates 
 necessary.  Seems to be causing some confusion with users getting started.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6158) Data loss if the words 'merges' or 'splits' are used as Column Family name

2012-06-25 Thread Aditya Kishore (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400884#comment-13400884
 ] 

Aditya Kishore commented on HBASE-6158:
---

1. Would like to clarify that this change affects CF names and not user tables 
names.

{code}
\-HBase Root
 +-\user_table
   +-\region_name
 +-\merges

and

\-HBase Root
 +-\user_table
   +-\region_name
 +-\splits
{code}

2. Prior to this fix, it was impossible for any table to have a column family 
with name merges or splits *ON DISK* since these folders get deleted 
whenever a region is opened. If a table has a defined schema with merges or 
splits as CF name, it will continue to accept puts until they are to be 
flushed to disk at which point the flush fails with the following exception

ERROR: org.apache.hadoop.ipc.RemoteException: 
org.apache.hadoop.hbase.DroppedSnapshotException: region: region_name
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:999)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:904)
at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:856)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.flushRegion(HRegionServer.java:2192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
Caused by: java.io.FileNotFoundException: File does not exist: 
hdfs://server:port/path_to_region/merges
at org.apache.hadoop.hdfs.DistributedFileSystem(DistributedFileSystem.java:519)
at org.apache.hadoop.hdfs.DistributedFileSystem(DistributedFileSystem.java:504)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.getUniqueFile(StoreFile.java:580)
at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:493)
at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:448)
at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:81)
at 
org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1519)
at 
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:977)
... 9 more

3. You are right about the merges/MERGEDIR not being created anymore. Looks 
like this is a leftover code from original region merge code which was 
[modified|http://svn.apache.org/viewvc/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java?r1=638612r2=639775pathrev=1342855diff_format=h]
 as part of HBASE-483. However the delete code still exist. I think the 
constant along with the code which deletes the MERGEDIR folder can be safely 
removed.

4. Which means that the only folder that we may even need to consider during 
upgrade is splits. However, it is a transient folder which should exist only 
until the parent region is clean up. And this does get cleaned up when the 
corresponding region is opened.

 Data loss if the words 'merges' or 'splits' are used as Column Family name
 --

 Key: HBASE-6158
 URL: https://issues.apache.org/jira/browse/HBASE-6158
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Affects Versions: 0.94.0
Reporter: Aditya Kishore
Assignee: Aditya Kishore
 Fix For: 0.92.2, 0.94.1

 Attachments: HBASE-6158_94.patch, HBASE-6158_trunk.patch


 If a table is creates with either 'merges' or 'splits' as one of the Column 
 Family name it can never be flushed to the disk even though the table 
 creation (and data population) succeeds.
 The reason for this is that these two are used as temporary directory names 
 inside the region folder or merge and splits respectively and hence conflicts 
 with the directories created for CF with same name.
 A simple fix would be to uses .merges' and .splits as the working folder 
 (patch attached). This will also be consistent with other work folder names. 
 An alternate fix would be to declare these words (and other similar) as 
 reserve words and throw exception when they are used. However, I do find the 
 alternate approach as unnecessarily restrictive.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400891#comment-13400891
 ] 

Jonathan Hsieh commented on HBASE-2315:
---

@Flavio - with the encapsulation comment -- basically if you look at the 
interface there is very little that is hdfs-specific about it -- append, open, 
close, roll operations all basically take hbase constructs and could have any 
implementation behind it.   The hdfs-specifics have been contained in its 
constructor and the init functions of the reader/writer classes.

 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus


[ 
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400902#comment-13400902
 ] 

Zhihong Ted Yu commented on HBASE-5933:
---

Integrated to trunk.

Thanks for the patch, Gregory.

Thanks for the review, Elliot.

 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
 -

 Key: HBASE-5933
 URL: https://issues.apache.org/jira/browse/HBASE-5933
 Project: HBase
  Issue Type: Task
  Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch


 From Stack's comments on HBASE-5444:
 We need this import?  Its for cp.  Thats ok I'd say One day we can hide 
 that too..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging

2012-06-25 Thread Flavio Junqueira (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400908#comment-13400908
]

Flavio Junqueira commented on HBASE-2315:
-

@Jonathan My issue has been exactly with the initialization of the log reader
and writer. They take a FileSystem and a Path as input parameters. As I
mentioned above, but I've been looking at implementing a BK FileSystem, and it
looks a bit hacky.

If I get your suggestion, you're saying that we could reuse most of HLog and
focus on generalizing its constructors and the init methods. Is it right?

BookKeeper for write-ahead logging
--

[jira] [Commented] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus


[ 
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400933#comment-13400933
 ] 

Hudson commented on HBASE-5933:
---

Integrated in HBase-TRUNK #3071 (See 
[https://builds.apache.org/job/HBase-TRUNK/3071/])
HBASE-5933 Add RegionLoad.java (Gregory) (Revision 1353743)
HBASE-5933 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from 
ClusterStatus, remove HServerLoad.java
   (Gregory) (Revision 1353742)
HBASE-5933 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from 
ClusterStatus (Gregory) (Revision 1353740)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/RegionLoad.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HServerLoad.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HServerLoad.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java
* /hbase/trunk/hbase-server/src/main/protobuf/hbase.proto
* /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerLoad.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java


 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
 -

 Key: HBASE-5933
 URL: https://issues.apache.org/jira/browse/HBASE-5933
 Project: HBase
  Issue Type: Task
  Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch


 From Stack's comments on HBASE-5444:
 We need this import?  Its for cp.  Thats ok I'd say One day we can hide 
 that too..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HBASE-6009) Changes for HBASE-5209 are technically incompatible

[
https://issues.apache.org/jira/browse/HBASE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh resolved HBASE-6009.
---

Resolution: Won't Fix
Assignee: David S. Wang
Release Note: HBASE-5209 was introduced in 0.92.1 and 0.94.0 but introduced
an incompatibility from a 0.92.0 client. Since this had maded it into two
releases already, we've decided to leave it in.
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)

It seems like from discussion and inaction, we've decided to keep this in the
acknowledge the incompatiblity between 0.92.0 and 0.92.1+ series, and keep this
in 0.94.0 series allowing for compatibility between 0.92.1+ and 0.94.0

Changes for HBASE-5209 are technically incompatible
---

Key: HBASE-6009
URL: https://issues.apache.org/jira/browse/HBASE-6009
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: David S. Wang
Assignee: David S. Wang

The additions to add backup masters to ClusterStatus are technically
incompatible between clients and servers. Older clients will basically not
read the extra bits that the newer server pushes for the backup masters, thus
screwing up the serialization for the next blob in the pipe.
For the Writable, we can add a total size field for ClusterStatus at the
beginning, or we can have start and end markers. I can make a patch for
either approach; interested in whatever folks have to suggest. Would be good
to get this in soon to limit the damage to 0.92.1 (don't know if we can get
this in in time for 0.94.0).
Either change will make us forward-compatible starting with when the change
goes in, but will not fix the backwards incompatibility, which we will have
to mark with a release note as there have already been releases with this
change.
Hopefully we can do this in a cleaner way when wire compat rolls around in
0.96.

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

2012-06-25 Thread Devaraj Das (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400944#comment-13400944
]

Devaraj Das commented on HBASE-6205:

Would this work:
1. Have the delete command simply disable the table (flag with disable_delete
or something and record it in ZK)
2. Have the master periodically check for disable_delete table flags and if a
table was disabled for more than 24 hours (say), then the master deletes the
table
3. If (1) was done in error, then an 'undelete' command could simply reenable
the table within the 24 hour period delete the ZK entry (of course these
operations needs to be done carefully to avoid races).

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

2012-06-25 Thread Enis Soztutar (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400946#comment-13400946
]

Enis Soztutar commented on HBASE-6205:
--

After an offline conversation with Ted, and Jitendra, it seems that hdfs trash
works only for shell. One other concern is that trash is not exposed as hadoop
filesystem feature, so we have to use the shell-equivalent commands to
accomplish this, and it will work only on hdfs, not other file systems.

The question of whether to implement an hbase-thrash boils down to whether we
want this to work with file systems other than hdfs, and have more control on
the retention policy.

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Commented] (HBASE-6205) Support an option to keep data of dropped table for some time

[
https://issues.apache.org/jira/browse/HBASE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400958#comment-13400958
]

Zhihong Ted Yu commented on HBASE-6205:
---

Over in HBASE-2315, Flavio proposed basing WAL on Bookkeeper.
Before WAL can have separate file system other than that used by HFiles, we
should accommodate more file system choices beyond hdfs.

Support an option to keep data of dropped table for some time
-

Attachments: HBASE-6205.patch, HBASE-6205v2.patch,
HBASE-6205v3.patch, HBASE-6205v4.patch, HBASE-6205v5.patch

[jira] [Commented] (HBASE-5967) ServerLoad OpenDataException


[ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400960#comment-13400960
 ] 

Gregory Chanan commented on HBASE-5967:
---

Attached HBASE-5967.patch, fixes issue by renaming the offending function.

 ServerLoad OpenDataException
 

 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5967.patch, master.log


 I saw this error in the master log:
 Caused by: java.lang.IllegalArgumentException: Method 
 org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
 return type that cannot be translated into an open type
 at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.init(MBeanAnalyzer.java:99)
 ... 14 more
 Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
 java.util.Mapjava.lang.String, org.apache.hadoop.hbase.ServerLoad
 at 
 com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5967) ServerLoad OpenDataException


 [ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5967:
--

Status: Patch Available  (was: Open)

 ServerLoad OpenDataException
 

 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5967.patch, master.log


 I saw this error in the master log:
 Caused by: java.lang.IllegalArgumentException: Method 
 org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
 return type that cannot be translated into an open type
 at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.init(MBeanAnalyzer.java:99)
 ... 14 more
 Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
 java.util.Mapjava.lang.String, org.apache.hadoop.hbase.ServerLoad
 at 
 com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus


 [ 
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Ted Yu updated HBASE-5933:
--

Resolution: Fixed
Status: Resolved  (was: Patch Available)

 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
 -

 Key: HBASE-5933
 URL: https://issues.apache.org/jira/browse/HBASE-5933
 Project: HBase
  Issue Type: Task
  Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch


 From Stack's comments on HBASE-5444:
 We need this import?  Its for cp.  Thats ok I'd say One day we can hide 
 that too..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2315) BookKeeper for write-ahead logging


[ 
https://issues.apache.org/jira/browse/HBASE-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400967#comment-13400967
 ] 

Jonathan Hsieh commented on HBASE-2315:
---

@Flavio

I'm suggesting you use the Factory pattern to create the HLog, and have in the 
config file that would have it instantiate a new BKHLog (doesn't even use FS or 
Path) that returns its own instances of BKReader/BKWriters. 

I only see a handful of non-test places where HLogs are constructed so this 
part shouldn't be too hard to make factory pluggable:

https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L1267
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3719
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HMerge.java#L161
(I think there are a few more)..

My guess is that it will take a little bit of work but won't be too onerous to 
do.

 BookKeeper for write-ahead logging
 --

 Key: HBASE-2315
 URL: https://issues.apache.org/jira/browse/HBASE-2315
 Project: HBase
  Issue Type: New Feature
  Components: regionserver
Reporter: Flavio Junqueira
 Attachments: HBASE-2315.patch, bookkeeperOverview.pdf, 
 zookeeper-dev-bookkeeper.jar


 BookKeeper, a contrib of the ZooKeeper project, is a fault tolerant and high 
 throughput write-ahead logging service. This issue provides an implementation 
 of write-ahead logging for hbase using BookKeeper. Apart from expected 
 throughput improvements, BookKeeper also has stronger durability guarantees 
 compared to the implementation currently used by hbase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5967) ServerLoad OpenDataException


[ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400978#comment-13400978
 ] 

Zhihong Ted Yu commented on HBASE-5967:
---

How about switching to another verb so that the method name reveals what the 
method does ?
http://www.synonym.com/synonyms/get/

Consider the following:
induce, obtain

 ServerLoad OpenDataException
 

 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5967.patch, master.log


 I saw this error in the master log:
 Caused by: java.lang.IllegalArgumentException: Method 
 org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
 return type that cannot be translated into an open type
 at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.init(MBeanAnalyzer.java:99)
 ... 14 more
 Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
 java.util.Mapjava.lang.String, org.apache.hadoop.hbase.ServerLoad
 at 
 com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6116) Allow parallel HDFS writes for HLogs.


[ 
https://issues.apache.org/jira/browse/HBASE-6116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401016#comment-13401016
 ] 

Lars Hofhansl commented on HBASE-6116:
--

@Andy: Fair enough :)
BTW, do you still have the HBase-0.94 patch you made (so I do not have to do 
the same work)?


 Allow parallel HDFS writes for HLogs.
 -

 Key: HBASE-6116
 URL: https://issues.apache.org/jira/browse/HBASE-6116
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
 Attachments: 6116-v1.txt, pipelined-vs-parallel-comparison.zip


 In HDFS-1783 I adapted Dhrubas changes to be used in Hadoop trunk.
 This issue will include the necessary reflection changes to optionally enable 
 this for the WALs in HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5904) is_enabled from shell returns differently from pre- and post- HBASE-5155

[
https://issues.apache.org/jira/browse/HBASE-5904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401019#comment-13401019
]

Jonathan Hsieh commented on HBASE-5904:
---

I'm testing this patch currently (which is a revert of HBASE-5155) and plan on
committing for 0.90.7 if it is clean unless I hear any complaints.

is_enabled from shell returns differently from pre- and post- HBASE-5155

Key: HBASE-5904
URL: https://issues.apache.org/jira/browse/HBASE-5904
Project: HBase
Issue Type: Bug
Components: zookeeper
Affects Versions: 0.90.6
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Blocker
Fix For: 0.90.7

Attachments: HBASE-5904.patch

If I launch an hbase shell that uses HBase and ZooKeeper without HBASE-5155,
against HBase servers with HBASE-5155, then is_enabled for a table always
returns false even if the table is considered enabled by the servers from the
logs. If I then do the same thing but with an HBase shell and ZooKeeper with
HBASE-5155, then is_enabled returns as expected.
If I launch an HBase shell that uses HBase and ZooKeeper without HBASE-5155,
against HBase servers also without HBASE-5155, then is_enabled works as you'd
expect. But if I then do the same thing but with an HBase shell and
ZooKeeper with HBASE-5155, then is_enabled returns false even though the
table is considered enabled by the servers from the logs.
Additionally, if I then try to enable the table from the
HBASE-5155-containing shell, it hangs because the ZooKeeper code waits for
the ZNode to be updated with ENABLED in the data field, but what actually
happens is that the ZNode gets deleted since the servers are running without
HBASE-5155.
I think the culprit is that the indication of how a table is considered
enabled inside ZooKeeper has changed with HBASE-5155. Before HBASE-5155, a
table was considered enabled if the ZNode for it did not exist. After
HBASE-5155, a table is considered enabled if the ZNode for it exists and has
ENABLED in its data. I think the current code is incompatible when running
clients and servers where one side has HBASE-5155 and the other side does not.

[jira] [Commented] (HBASE-6226) move DataBlockEncoding and related classes to hbase-common module

2012-06-25 Thread Matt Corgan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401033#comment-13401033
 ] 

Matt Corgan commented on HBASE-6226:


Hi Mikhail,

I installed arcanist from scratch and haven't used it before.  Do you know how 
to fix this error?

{quote}
Usage Exception: Failed to load phutil library at location '.arc_jira_lib'. 
This library is specified by the phutil_libraries setting in .arcconfig. 
Check that the setting is correct and the library is located in the right place.
{quote}


here are the contents of my hbase/.arcconfig file:
{code}
{
  project_id : hbase,
  conduit_uri : https://reviews.facebook.net/;,
  copyright_holder : Apache Software Foundation,
  phutil_libraries : {
arclib : .arc_jira_lib
  },
  arcanist_configuration : ArcJIRAConfiguration,
  jira_project : HBASE,
  jira_api_url : https://issues.apache.org/jira/si/;,
  lint_engine : JavaLintEngine,
  max_line_length : 100
}
{code}

 move DataBlockEncoding and related classes to hbase-common module
 -

 Key: HBASE-6226
 URL: https://issues.apache.org/jira/browse/HBASE-6226
 Project: HBase
  Issue Type: Improvement
  Components: io, regionserver
Affects Versions: 0.96.0
Reporter: Matt Corgan
Assignee: Matt Corgan
 Attachments: HBASE-6226-v1.patch


 In order to isolate the implementation details of HBASE-4676 (PrefixTrie 
 encoding) and other DataBlockEncoders by putting them in modules, this pulls 
 up the DataBlockEncoding related interfaces into hbase-common.
 No tests are moved in this patch.  The only notable change was trimming a few 
 dependencies on HFileBlock which adds dependencies to much of the 
 regionserver.
 The test suite passes locally for me.
 I tried to keep it as simple as possible... let me know if there are any 
 concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5967) ServerLoad OpenDataException


 [ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated HBASE-5967:
--

Attachment: HBASE-5967-v2.patch

* Attached HBASE-5967-v2.patch *

Thanks for the review, Ted.  Changed to obtainServerLoadPB

 ServerLoad OpenDataException
 

 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5967-v2.patch, HBASE-5967.patch, master.log


 I saw this error in the master log:
 Caused by: java.lang.IllegalArgumentException: Method 
 org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
 return type that cannot be translated into an open type
 at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.init(MBeanAnalyzer.java:99)
 ... 14 more
 Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
 java.util.Mapjava.lang.String, org.apache.hadoop.hbase.ServerLoad
 at 
 com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5967) ServerLoad OpenDataException


[ 
https://issues.apache.org/jira/browse/HBASE-5967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401043#comment-13401043
 ] 

Zhihong Ted Yu commented on HBASE-5967:
---

Thanks for the quick response.

I ran patch v1 locally and didn't see any OpenDataException in test output.

Will integrate after Hadoop QA run.

 ServerLoad OpenDataException
 

 Key: HBASE-5967
 URL: https://issues.apache.org/jira/browse/HBASE-5967
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jimmy Xiang
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: HBASE-5967-v2.patch, HBASE-5967.patch, master.log


 I saw this error in the master log:
 Caused by: java.lang.IllegalArgumentException: Method 
 org.apache.hadoop.hbase.master.MXBean.getRegionServers has parameter or 
 return type that cannot be translated into an open type
 at com.sun.jmx.mbeanserver.ConvertingMethod.from(ConvertingMethod.java:32)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:63)
 at 
 com.sun.jmx.mbeanserver.MXBeanIntrospector.mFrom(MXBeanIntrospector.java:33)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.initMaps(MBeanAnalyzer.java:118)
 at com.sun.jmx.mbeanserver.MBeanAnalyzer.init(MBeanAnalyzer.java:99)
 ... 14 more
 Caused by: javax.management.openmbean.OpenDataException: Cannot convert type: 
 java.util.Mapjava.lang.String, org.apache.hadoop.hbase.ServerLoad
 at 
 com.sun.jmx.mbeanserver.OpenConverter.openDataException(OpenConverter.jav

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server


[ 
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401046#comment-13401046
 ] 

Hudson commented on HBASE-5875:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #68 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/68/])
HBASE-5875 Process RIT and Master restart may remove an online server 
considering it as a dead server (Rajesh)

Submitted by:Rajesh 
Reviewed by:Ram Ted, Stack (Revision 1353688)

 Result = FAILURE
ramkrishna : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 Process RIT and Master restart may remove an online server considering it as 
 a dead server
 --

 Key: HBASE-5875
 URL: https://issues.apache.org/jira/browse/HBASE-5875
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 0.96.0, 0.94.1

 Attachments: HBASE-5875.patch, HBASE-5875_0.94.patch, 
 HBASE-5875_0.94_1.patch, HBASE-5875_0.94_2.patch, HBASE-5875_trunk.patch, 
 HBASE-5875_trunk.patch, HBASE-5875_trunk_1.patch, HBASE-5875v2.patch


 If on master restart it finds the ROOT/META to be in RIT state, master tries 
 to assign the ROOT region through ProcessRIT.
 Master will trigger the assignment and next will try to verify the Root 
 Region Location.
 Root region location verification is done seeing if the RS has the region in 
 its online list.
 If the master triggered assignment has not yet been completed in RS then the 
 verify root region location will fail.
 Because it failed 
 {code}
 splitLogAndExpireIfOnline(currentRootServer);
 {code}
 we do split log and also remove the server from online server list. Ideally 
 here there is nothing to do in splitlog as no region server was restarted.
 So master, though the server is online, master just invalidates the region 
 server.
 In a special case, if i have only one RS then my cluster will become non 
 operative.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.


[ 
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401047#comment-13401047
 ] 

Hudson commented on HBASE-5360:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #68 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/68/])
HBASE-5360 [uberhbck] Add options for how to handle offline split parents, 
addendum (Revision 1353658)

 Result = FAILURE
jxiang : 
Files : 
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 [uberhbck] Add options for how to handle offline split parents. 
 

 Key: HBASE-5360
 URL: https://issues.apache.org/jira/browse/HBASE-5360
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
 Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1

 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 
 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path


 In a recent case, we attempted to repair a cluster that suffered from 
 HBASE-4238 that had about 6-7 generations of leftover split data.  The hbck 
 repair options in an development version of HBASE-5128 treat HDFS as ground 
 truth but didn't check SPLIT and OFFLINE flags only found in meta.  The net 
 effect was that it essentially attempted to merge many regions back into its 
 eldest geneneration's parent's range.  
 More safe guards to prevent mega-merges are being added on HBASE-5128.
 This issue would automate the handling of the mega-merge avoiding cases 
 such as lingering grandparents.  The strategy here would be to add more 
 checks against .META., and perform part of the catalog janitor's 
 responsibilities for lingering grandparents.  This would potentially include 
 options to sideline regions, deleting grandparent regions, min size for 
 sidelining, and mechanisms for cleaning .META..  
 Note: There already exists an mechanism to reload these regions -- the bulk 
 loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents 
 (automatically splitting them if necessary) to HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5933) Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus


[ 
https://issues.apache.org/jira/browse/HBASE-5933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401045#comment-13401045
 ] 

Hudson commented on HBASE-5933:
---

Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #68 (See 
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/68/])
HBASE-5933 Add RegionLoad.java (Gregory) (Revision 1353743)
HBASE-5933 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from 
ClusterStatus, remove HServerLoad.java
   (Gregory) (Revision 1353742)
HBASE-5933 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from 
ClusterStatus (Gregory) (Revision 1353740)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/RegionLoad.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HServerLoad.java

tedyu : 
Files : 
* 
/hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HServerLoad.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ServerLoad.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java
* /hbase/trunk/hbase-server/src/main/protobuf/hbase.proto
* /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestSerialization.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/TestServerLoad.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java
* 
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java


 Hide HBaseProtos.ServerLoad and HBaseProtos.RegionLoad from ClusterStatus
 -

 Key: HBASE-5933
 URL: https://issues.apache.org/jira/browse/HBASE-5933
 Project: HBase
  Issue Type: Task
  Components: ipc
Affects Versions: 0.96.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Priority: Minor
 Fix For: 0.96.0

 Attachments: 5933-v2.txt, HBASE-5933-v3.patch, HBASE-5933.patch


 From Stack's comments on HBASE-5444:
 We need this import?  Its for cp.  Thats ok I'd say One day we can hide 
 that too..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-6268) Can't enable a table on a 0.94 cluster from a 0.92 client

Jean-Daniel Cryans created HBASE-6268:
-

 Summary: Can't enable a table on a 0.94 cluster from a 0.92 client
 Key: HBASE-6268
 URL: https://issues.apache.org/jira/browse/HBASE-6268
 Project: HBase
  Issue Type: Bug
Reporter: Jean-Daniel Cryans
 Fix For: 0.92.2


In 0.92 we know a table's enabled by doing this in HCM.isEnabledTable:

bq. return getTableState(zkw, tableName) == null;

In 0.94 we do:

bq. return getTableState(zkw, tableName) == TableState.ENABLED;

So what happens is that the the 0.92 client will hang forever since the znode 
contains ENABLED instead of being absent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6267) hbase.store.delete.expired.storefile should be true by default