date:20120328

[
https://issues.apache.org/jira/browse/HBASE-5564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240345#comment-13240345
]

Hadoop QA commented on HBASE-5564:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12520255/HBASE-5564_trunk.4_final.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1326//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1326//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1326//console

This message is automatically generated.

Bulkload is discarding duplicate records

Key: HBASE-5564
URL: https://issues.apache.org/jira/browse/HBASE-5564
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.96.0
Environment: HBase 0.92
Reporter: Laxman
Assignee: Laxman
Labels: bulkloader
Fix For: 0.96.0

Attachments: 5564.lint, HBASE-5564_trunk.1.patch,
HBASE-5564_trunk.1.patch, HBASE-5564_trunk.2.patch, HBASE-5564_trunk.3.patch,
HBASE-5564_trunk.4_final.patch, HBASE-5564_trunk.patch

Duplicate records are getting discarded when duplicate records exists in same
input file and more specifically if they exists in same split.
Duplicate records are considered if the records are from diffrent different
splits.
Version under test: HBase 0.92

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Status: Patch Available  (was: Open)

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Status: Open  (was: Patch Available)

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Attachment: 5573.v7.patch

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads

[
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240373#comment-13240373
]

Hadoop QA commented on HBASE-5573:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520262/5573.v7.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 12 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1327//console

This message is automatically generated.

Replace client ZooKeeper watchers by simple ZooKeeper reads
---

Key: HBASE-5573
URL: https://issues.apache.org/jira/browse/HBASE-5573
Project: HBase
Issue Type: Improvement
Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch,
5573.v6.patch, 5573.v7.patch

Some code in the package needs to read data in ZK. This could be done by a
simple read, but is actually implemented with a watcher. This holds ZK
resources.
Fixing this could also be an opportunity to remove the need for the client to
provide the master address and port.

[jira] [Created] (HBASE-5663) MultithreadedTableMapper doesn't work.

2012-03-28 Thread Takuya Ueshin (Created) (JIRA)

MultithreadedTableMapper doesn't work.
--

 Key: HBASE-5663
 URL: https://issues.apache.org/jira/browse/HBASE-5663
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.92.1
Reporter: Takuya Ueshin


MapReduce job using MultithreadedTableMapper goes down throwing the following 
Exception:

{noformat}
java.io.IOException: java.lang.NoSuchMethodException: 
org.apache.hadoop.mapreduce.Mapper$Context.init(org.apache.hadoop.conf.Configuration,
 org.apache.hadoop.mapred.TaskAttemptID, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, 
org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter,
 org.apache.hadoop.hbase.mapreduce.TableSplit)
at 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.init(MultithreadedTableMapper.java:260)
at 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper.run(MultithreadedTableMapper.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.NoSuchMethodException: 
org.apache.hadoop.mapreduce.Mapper$Context.init(org.apache.hadoop.conf.Configuration,
 org.apache.hadoop.mapred.TaskAttemptID, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter, 
org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter,
 org.apache.hadoop.hbase.mapreduce.TableSplit)
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getConstructor(Class.java:1657)
at 
org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.init(MultithreadedTableMapper.java:241)
... 8 more
{noformat}

This occured when the tasks are creating MapRunner threads.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.

2012-03-28 Thread Takuya Ueshin (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240393#comment-13240393
 ] 

Takuya Ueshin commented on HBASE-5636:
--

I filed the new issue of the bug.

By the way, the TestCase name 'TestMulitthreadedTableMapper' is typo, isn't it?
I will rename it to 'TestMultithreadedTableMapper'.

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1
Reporter: Takuya Ueshin

 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row

2012-03-28 Thread Anoop Sam John (Created) (JIRA)

CP hooks in Scan flow for fast forward when filter filters out a row


 Key: HBASE-5664
 URL: https://issues.apache.org/jira/browse/HBASE-5664
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
 Fix For: 0.96.0


In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies 
filter condition. When Filter filters out the current fetched row we call 
nextRow(byte [] currentRow) before going with the next row.
{code}  
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row

2012-03-28 Thread Anoop Sam John (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John reassigned HBASE-5664:
-

Assignee: Anoop Sam John

 CP hooks in Scan flow for fast forward when filter filters out a row
 

 Key: HBASE-5664
 URL: https://issues.apache.org/jira/browse/HBASE-5664
 Project: HBase
  Issue Type: Improvement
  Components: coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
 Fix For: 0.96.0


 In HRegion.nextInternal(int limit, String metric)
   We have while(true) loop so as to fetch a next result which satisfies 
 filter condition. When Filter filters out the current fetched row we call 
 nextRow(byte [] currentRow) before going with the next row.
 {code}
 if (results.isEmpty() || filterRow()) {
 // this seems like a redundant step - we already consumed the row
 // there're no left overs.
 // the reasons for calling this method are:
 // 1. reset the filters.
 // 2. provide a hook to fast forward the row (used by subclasses)
 nextRow(currentRow);
 {code}
 // 2. provide a hook to fast forward the row (used by subclasses)
 We can provide same feature of fast forward support for the CP also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5607) Implement scanner caching throttling to prevent too big responses

2012-03-28 Thread Ferdy Galema (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240412#comment-13240412
 ] 

Ferdy Galema commented on HBASE-5607:
-

Ok sure. I'm currently not into HBase development at all but I'm willing to 
give it a shot.

 Implement scanner caching throttling to prevent too big responses 
 --

 Key: HBASE-5607
 URL: https://issues.apache.org/jira/browse/HBASE-5607
 Project: HBase
  Issue Type: Improvement
Reporter: Ferdy Galema

 When a misconfigured client retrieves fat rows with a scanner caching value 
 set too high, there is a big chance the regionserver cannot handle the 
 response buffers. (See log example below). Also see the mailing list thread: 
 http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/24819
 This issue is for tracking a solution that throttles the scanner caching 
 value in the case the response buffers are too big.
 A few possible solutions:
 a) Is a response (repeatedly) over 100MB (configurable), then reduce the 
 scanner-caching by half its size. (In either server or client).
 b) Introduce a property that defines a fixed (target) response size, instead 
 of defining the numbers of rows to cache.
 2012-03-20 07:57:40,092 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 5 on 60020, responseTooLarge for: next(4438820558358059204, 1000) 
 from 172.23.122.15:50218: Size: 105.0m
 2012-03-20 07:57:53,226 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 3 on 60020, responseTooLarge for: next(-7429189123174849941, 1000) 
 from 172.23.122.15:50218: Size: 214.4m
 2012-03-20 07:57:57,839 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 5 on 60020, responseTooLarge for: next(-7429189123174849941, 1000) 
 from 172.23.122.15:50218: Size: 103.2m
 2012-03-20 07:57:59,442 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 2 on 60020, responseTooLarge for: next(-7429189123174849941, 1000) 
 from 172.23.122.15:50218: Size: 101.8m
 2012-03-20 07:58:20,025 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 6 on 60020, responseTooLarge for: next(9033159548564260857, 1000) 
 from 172.23.122.15:50218: Size: 107.2m
 2012-03-20 07:58:27,273 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 3 on 60020, responseTooLarge for: next(9033159548564260857, 1000) 
 from 172.23.122.15:50218: Size: 100.1m
 2012-03-20 07:58:52,783 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 1 on 60020, responseTooLarge for: next(-8611621895979000997, 1000) 
 from 172.23.122.15:50218: Size: 101.7m
 2012-03-20 07:59:02,541 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 0 on 60020, responseTooLarge for: next(-511305750191148153, 1000) 
 from 172.23.122.15:50218: Size: 120.9m
 2012-03-20 07:59:25,346 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 6 on 60020, responseTooLarge for: next(1570572538285935733, 1000) 
 from 172.23.122.15:50218: Size: 107.8m
 2012-03-20 07:59:46,805 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 3 on 60020, responseTooLarge for: next(-727080724379055435, 1000) 
 from 172.23.122.15:50218: Size: 102.7m
 2012-03-20 08:00:00,138 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 3 on 60020, responseTooLarge for: next(-3701270248575643714, 1000) 
 from 172.23.122.15:50218: Size: 122.1m
 2012-03-20 08:00:21,232 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 6 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 157.5m
 2012-03-20 08:00:23,199 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 9 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 160.7m
 2012-03-20 08:00:28,174 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 2 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 160.8m
 2012-03-20 08:00:32,643 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 7 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 182.4m
 2012-03-20 08:00:36,826 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 9 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 237.2m
 2012-03-20 08:00:40,850 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 3 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 212.7m
 2012-03-20 08:00:44,736 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 1 on 60020, responseTooLarge for: next(5831907615409186602, 1000) 
 from 172.23.122.15:50218: Size: 232.9m
 2012-03-20 08:00:49,471 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server 
 handler 7 on 60020,

[jira] [Updated] (HBASE-5663) MultithreadedTableMapper doesn't work.


 [ 
https://issues.apache.org/jira/browse/HBASE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takuya Ueshin updated HBASE-5663:
-

Affects Version/s: (was: 0.92.1)
   0.94.0

 MultithreadedTableMapper doesn't work.
 --

 Key: HBASE-5663
 URL: https://issues.apache.org/jira/browse/HBASE-5663
 Project: HBase
  Issue Type: Bug
  Components: mapreduce
Affects Versions: 0.94.0
Reporter: Takuya Ueshin

 MapReduce job using MultithreadedTableMapper goes down throwing the following 
 Exception:
 {noformat}
 java.io.IOException: java.lang.NoSuchMethodException: 
 org.apache.hadoop.mapreduce.Mapper$Context.init(org.apache.hadoop.conf.Configuration,
  org.apache.hadoop.mapred.TaskAttemptID, 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader,
  
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter,
  org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter,
  org.apache.hadoop.hbase.mapreduce.TableSplit)
   at 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.init(MultithreadedTableMapper.java:260)
   at 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper.run(MultithreadedTableMapper.java:133)
   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
   at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
   at org.apache.hadoop.mapred.Child.main(Child.java:249)
 Caused by: java.lang.NoSuchMethodException: 
 org.apache.hadoop.mapreduce.Mapper$Context.init(org.apache.hadoop.conf.Configuration,
  org.apache.hadoop.mapred.TaskAttemptID, 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordReader,
  
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapRecordWriter,
  org.apache.hadoop.hbase.mapreduce.TableOutputCommitter, 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$SubMapStatusReporter,
  org.apache.hadoop.hbase.mapreduce.TableSplit)
   at java.lang.Class.getConstructor0(Class.java:2706)
   at java.lang.Class.getConstructor(Class.java:1657)
   at 
 org.apache.hadoop.hbase.mapreduce.MultithreadedTableMapper$MapRunner.init(MultithreadedTableMapper.java:241)
   ... 8 more
 {noformat}
 This occured when the tasks are creating MapRunner threads.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Status: Open  (was: Patch Available)

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly

2012-03-28 Thread Ferdy Galema (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferdy Galema updated HBASE-2214:


Assignee: Ferdy Galema

As mentioned in HBASE-5607, I will pick up this issue.

First of all:
I looked at the patch and I noticed that is huge. A lot of unnecessary changes 
are in it, whitespace/reformatting and such. It is very difficult to work with 
this when applying it with regular svn patch tools. Therefore, if you don't 
mind, I want to start over from scratch. (Of course it has been a while so a 
lot of changes won't apply anyway).

Secondly, does this feature replaces the old number of rows caching? I like to 
propose that it is additional. So: A user specifying 100 rows and 10MB for a 
Scan will get chunks that are either capped at 100 rows, or 10MB, whichever 
limit comes first. Do you agree?

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
  Labels: noob
 Fix For: 0.96.0

 Attachments: HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Attachment: 5573.v8.patch

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch, 5573.v8.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads


 [ 
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-5573:
---

Status: Patch Available  (was: Open)

 Replace client ZooKeeper watchers by simple ZooKeeper reads
 ---

 Key: HBASE-5573
 URL: https://issues.apache.org/jira/browse/HBASE-5573
 Project: HBase
  Issue Type: Improvement
  Components: client, zookeeper
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: nkeywal
Priority: Minor
 Attachments: 5573.v1.patch, 5573.v2.patch, 5573.v4.patch, 
 5573.v6.patch, 5573.v7.patch, 5573.v8.patch


 Some code in the package needs to read data in ZK. This could be done by a 
 simple read, but is actually implemented with a watcher. This holds ZK 
 resources.
 Fixing this could also be an opportunity to remove the need for the client to 
 provide the master address and port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly


[ 
https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240432#comment-13240432
 ] 

Zhihong Yu commented on HBASE-2214:
---

Feel free to start from scratch.

For #2, size to return is optional setting. In case both size to return and 
number of rows are specified, your description makes sense.

 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- 
 properly
 -

 Key: HBASE-2214
 URL: https://issues.apache.org/jira/browse/HBASE-2214
 Project: HBase
  Issue Type: New Feature
Reporter: stack
Assignee: Ferdy Galema
  Labels: noob
 Fix For: 0.96.0

 Attachments: HBASE-2214_with_broken_TestShell.txt


 The notion that you set size rather than row count specifying how many rows a 
 scanner should return in each cycle was raised over in hbase-1966.  Its a 
 good one making hbase regular though the data under it may vary.  
 HBase-1966 was committed but the patch was constrained by the fact that it 
 needed to not change RPC interface.  This issue is about doing hbase-1966 for 
 0.21 in a clean, unconstrained way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work properly.


 [ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takuya Ueshin updated HBASE-5636:
-

Affects Version/s: 0.94.0

I'm sorry, TestMulitthreadedTableMapper was not included in 0.92.x.

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin

 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4565) Maven HBase build broken on cygwin with copynativelib.sh call.

2012-03-28 Thread Suraj Varma (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240448#comment-13240448
 ] 

Suraj Varma commented on HBASE-4565:


@Laxman - Yes, please go ahead.

@Lars - Ok, no problem.

 Maven HBase build broken on cygwin with copynativelib.sh call.
 --

 Key: HBASE-4565
 URL: https://issues.apache.org/jira/browse/HBASE-4565
 Project: HBase
  Issue Type: Bug
  Components: build
Affects Versions: 0.92.0
 Environment: cygwin (on xp and win7)
Reporter: Suraj Varma
Assignee: Suraj Varma
  Labels: build, maven
 Fix For: 0.96.0

 Attachments: HBASE-4565-0.92.patch, HBASE-4565-v2.patch, 
 HBASE-4565-v3-0.92.patch, HBASE-4565-v3.patch, HBASE-4565.patch


 This is broken in both 0.92 as well as trunk pom.xml
 Here's a sample maven log snippet from trunk (from Mayuresh on user mailing 
 list)
 [INFO] [antrun:run {execution: package}]
 [INFO] Executing tasks
 main:
[mkdir] Created dir: 
 D:\workspace\mkshirsa\hbase-trunk\target\hbase-0.93-SNAPSHOT\hbase-0.93-SNAPSHOT\lib\native\${build.platform}
 [exec] ls: cannot access D:workspacemkshirsahbase-trunktarget/nativelib: 
 No such file or directory
 [exec] tar (child): Cannot connect to D: resolve failed
 [INFO] 
 
 [ERROR] BUILD ERROR
 [INFO] 
 
 [INFO] An Ant BuildException has occured: exec returned: 3328
 There are two issues: 
 1) The ant run task below doesn't resolve the windows file separator returned 
 by the project.build.directory - this causes the above resolve failed.
 !-- Using Unix cp to preserve symlinks, using script to handle wildcards --
 echo file=${project.build.directory}/copynativelibs.sh
 if [ `ls ${project.build.directory}/nativelib | wc -l` -ne 0]; then
 2) The tar argument value below also has a similar issue in that the path arg 
 doesn't resolve right.
 !-- Using Unix tar to preserve symlinks --
 exec executable=tar failonerror=yes 
 dir=${project.build.directory}/${project.artifactId}-${project.version}
 arg value=czf/
 arg 
 value=/cygdrive/c/workspaces/hbase-0.92-svn/target/${project.artifactId}-${project.version}.tar.gz/
 arg value=./
 /exec
 In both cases, the fix would probably be to use a cross-platform way to 
 handle the directory locations. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work properly.


 [ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takuya Ueshin updated HBASE-5636:
-

Attachment: HBASE-5636.patch

I attached a patch file for trunk.
This includes patches for TestMulitthreadedTableMapper.

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-4818) HBase Shell - Add support for formatting row keys before output

2012-03-28 Thread Ben West (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240451#comment-13240451
 ] 

Ben West commented on HBASE-4818:
-

The ReverseIDFormatter in that patch overrides the default formatter to display 
row keys in reverse order.

Something which we will have to think about is how we can maintain usability 
with these new formatters. Scans, for example, might not go in the order the 
user predicts because the stored format is different from the displayed one. 
Similarly with where regions split and so forth. Maybe we should require sort 
order to be constant across formatted and unformatted row keys (which would 
make the ReverseIDFormatter and probably most formatters impossible).

I'm not super familiar with the web UI, but it looks like the only spots we 
display row keys is when we specify the start and end rows of each region, and 
when we issue splits/compactions. So that shouldn't be too bad to change.

 HBase Shell - Add support for formatting row keys before output
 ---

 Key: HBASE-4818
 URL: https://issues.apache.org/jira/browse/HBASE-4818
 Project: HBase
  Issue Type: Improvement
  Components: shell
Reporter: Eran Kampf
Priority: Trivial
 Attachments: format3.patch, hbase-4818.patch

   Original Estimate: 24h
  Remaining Estimate: 24h

 As many HBase users use binary row keys rather than strings to optimize 
 memory consumption displaying an escaped string in the HBase shell isn't 
 useful (and takes a lot of screen space)
 Allowing user to provide a row key formatter as part of the scan\get commands 
 would allow developers to display the row key in a way thats makes sense for 
 them.
 Example:
 scan 'stats', { ROWFORMATTER = MyRowFormatter.new }
 The row formatter simply gets the bytes array key and formats it to a string.
 Its an easy change tomake with simple monkey-patching of the shell commands 
 but I would be happy to see it as part of the shell itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5573) Replace client ZooKeeper watchers by simple ZooKeeper reads

2012-03-28 Thread Uma Maheswara Rao G (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240453#comment-13240453
]

Hadoop QA commented on HBASE-5573:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520266/5573.v8.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 12 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1328//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1328//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1328//console

This message is automatically generated.

Replace client ZooKeeper watchers by simple ZooKeeper reads
---

[jira] [Commented] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings

2012-03-28 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240467#comment-13240467
 ] 

Jonathan Hsieh commented on HBASE-5642:
---

The o.a.h.h.generated.master code looks to be generated from jsp's/jamon code 
and seems more potentially serious.  From the warning there, that the string 
needs to be sanitized before being re-displayed (as opposed to being ignored).



 [findbugs] Exclude Thrift and Protobuf warnings
 ---

 Key: HBASE-5642
 URL: https://issues.apache.org/jira/browse/HBASE-5642
 Project: HBase
  Issue Type: Sub-task
  Components: build
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
 Attachments: HBASE-5642.patch, HBASE-5642.patch


 Exclude thrift and protobuf warnings since these are machine generated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings


[ 
https://issues.apache.org/jira/browse/HBASE-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240472#comment-13240472
 ] 

Uma Maheswara Rao G commented on HBASE-5642:


I got you, Thanks a lot Jon for explanation.

Lets committ the current patch in this JIRA if your reviews are ok.

 [findbugs] Exclude Thrift and Protobuf warnings
 ---

 Key: HBASE-5642
 URL: https://issues.apache.org/jira/browse/HBASE-5642
 Project: HBase
  Issue Type: Sub-task
  Components: build
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
 Attachments: HBASE-5642.patch, HBASE-5642.patch


 Exclude thrift and protobuf warnings since these are machine generated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings

2012-03-28 Thread Jonathan Hsieh (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240474#comment-13240474
 ] 

Jonathan Hsieh commented on HBASE-5642:
---

Yup, I'm in the process of doing a quick test and committing.  Thanks Uma!


 [findbugs] Exclude Thrift and Protobuf warnings
 ---

 Key: HBASE-5642
 URL: https://issues.apache.org/jira/browse/HBASE-5642
 Project: HBase
  Issue Type: Sub-task
  Components: build
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
 Attachments: HBASE-5642.patch, HBASE-5642.patch


 Exclude thrift and protobuf warnings since these are machine generated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings

2012-03-28 Thread Jonathan Hsieh (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-5642:
--

   Resolution: Fixed
Fix Version/s: 0.96.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk

 [findbugs] Exclude Thrift and Protobuf warnings
 ---

 Key: HBASE-5642
 URL: https://issues.apache.org/jira/browse/HBASE-5642
 Project: HBase
  Issue Type: Sub-task
  Components: build
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Uma Maheswara Rao G
 Fix For: 0.96.0

 Attachments: HBASE-5642.patch, HBASE-5642.patch


 Exclude thrift and protobuf warnings since these are machine generated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5128) [uber hbck] Online automated repair of table integrity and region consistency problems

2012-03-28 Thread Jonathan Hsieh (Updated) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Hsieh updated HBASE-5128:
--

Release Note:
HBaseFsck (hbck) has been updated with new repair capabilities. hbck is a tool
for checking the region consistency and the table integrity invariants of a
running HBase cluster. Checking region consistency verifies that .META.,
region deployment on region servers and the state of data in HDFS (.regioninfo
files) all are in accordance. Table integrity checks verify that all possible
row keys resolve to exactly one region of a table -- e.g. there are no
individual degenerate or backwards regions; no holes between regions; and no
overlapping regions. Previously hbck had the ability to diagnose
inconsistencies but only had the ability to repair deployment region
consistency problems. The updated version now has been augmented with the
ability repair region consistency problems in .META. (by patching holes),
repair overlapping regions (via merging), patch region holes (by fabricating
new regions), and detecting and adopting orphaned regions (by fabricating new
.regioninfo file if it is missing in a region's dir).

Caveats:
* The new hbck selects repairs assuming that HDFS as ground truth, the previous
version treated .META. as ground truth.
* The hbck '-fix' option is present but deprecated and replaced with
'-fixAssignments' option.
* This tool adds APIs in 0.90.7, 0.92.2 and 0.94.0 for clean repairs. The 0.90
version of the tool is compatible with HBase 0.90+, but may require restarting
the master or individual individual regionserver for table
enable/disable/delete commands to work properly.

was:
HBaseFsck (hbck) has been updated with new repair capabilities. hbck is a tool
for checking the region consistency and the table integrity invariants of a
running HBase cluster. Checking region consistency verifies that .META.,
region deployment on region servers and the state of data in HDFS (.regioninfo
files) all are in accordance. Table integrity checks verify that all possible
row keys resolve to exactly one region of a table -- e.g. there are no
individual degenerate or backwards regions; no holes between regions; and no
overlapping regions. Previously hbck had the ability to diagnose
inconsistencies but only had the ability to repair deployment region
consistency problems. The updated version now has been augmented with the
ability repair region consistency problems in .META. (by patching holes),
repair overlapping regions (via merging), patch region holes (by fabricating
new regions), and detecting and adopting orphaned regions (by fabricating new
.regioninfo file if it is missing in a region's dir).

Caveats:
* The new hbck selects repairs assuming that HDFS as ground truth, the previous
version treated .META. as ground truth.
* The hbck '-fix' option is present but deprecated and replaced with
-fixAssignments option.
* This tool adds APIs in 0.90.7, 0.92.2 and 0.94.0 for clean repairs. The 0.90
version of he tool is compatible with HBase 0.90+, but may require restarting
the master or individual individual regionserver for table
enable/disable/delete commands to work properly.

[uber hbck] Online automated repair of table integrity and region consistency
problems
--

Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0

Attachments: 5128-trunk.addendum, hbase-5128-0.90-v2.patch,
hbase-5128-0.90-v2b.patch, hbase-5128-0.90-v4.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch, hbase-5128-0.94-v2.patch,
hbase-5128-0.94-v4.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch,
hbase-5128-v3.patch, hbase-5128-v4.patch

The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.


[ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240506#comment-13240506
 ] 

Zhihong Yu commented on HBASE-5636:
---

Since TestMultithreadedTableMapper.java is a new file, can you outline the 
changes you made on top of TestMulitthreadedTableMapper ?
I noticed this:
{code}
+UTIL.loadTable(table, INPUT_FAMILY);
{code}

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work properly.


 [ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5636:
--

Status: Patch Available  (was: Open)

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.

[
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240513#comment-13240513
]

Hadoop QA commented on HBASE-5636:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520270/HBASE-5636.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 10 new or modified tests.

-1 patch. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1329//console

This message is automatically generated.

TestTableMapReduce doesn't work properly.
-

Key: HBASE-5636
URL: https://issues.apache.org/jira/browse/HBASE-5636
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
Attachments: HBASE-5636.patch

No map function is called because there are no test data put before test
starts.
The following three tests are in the same situation:
- org.apache.hadoop.hbase.mapred.TestTableMapReduce
- org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
- org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.

2012-03-28 Thread Takuya Ueshin (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240514#comment-13240514
 ] 

Takuya Ueshin commented on HBASE-5636:
--

Oh, I'm very sorry.

Here is the diff of TestMulitthreadedTableMapper.java:

{noformat}
diff --git 
a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
 
b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
index cc5b1df..ad34dd2 100644
--- 
a/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
+++ 
b/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMulitthreadedTableMapper.java
@@ -19,6 +19,7 @@ package org.apache.hadoop.hbase.mapreduce;
 
 import java.io.File;
 import java.io.IOException;
+import java.util.Iterator;
 import java.util.Map;
 import java.util.NavigableMap;
 
@@ -28,7 +29,6 @@ import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.FileUtil;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hbase.*;
-import org.apache.hadoop.hbase.client.HBaseAdmin;
 import org.apache.hadoop.hbase.client.HTable;
 import org.apache.hadoop.hbase.client.Put;
 import org.apache.hadoop.hbase.client.Result;
@@ -56,19 +56,17 @@ public class TestMulitthreadedTableMapper {
   private static final Log LOG = 
LogFactory.getLog(TestMulitthreadedTableMapper.class);
   private static final HBaseTestingUtility UTIL =
   new HBaseTestingUtility();
-  static final String MULTI_REGION_TABLE_NAME = mrtest;
+  static final byte[] MULTI_REGION_TABLE_NAME = Bytes.toBytes(mrtest);
   static final byte[] INPUT_FAMILY = Bytes.toBytes(contents);
   static final byte[] OUTPUT_FAMILY = Bytes.toBytes(text);
   static final intNUMBER_OF_THREADS = 10;
 
   @BeforeClass
   public static void beforeClass() throws Exception {
-HTableDescriptor desc = new HTableDescriptor(MULTI_REGION_TABLE_NAME);
-desc.addFamily(new HColumnDescriptor(INPUT_FAMILY));
-desc.addFamily(new HColumnDescriptor(OUTPUT_FAMILY));
 UTIL.startMiniCluster();
-HBaseAdmin admin = new HBaseAdmin(UTIL.getConfiguration());
-admin.createTable(desc, HBaseTestingUtility.KEYS);
+HTable table = UTIL.createTable(MULTI_REGION_TABLE_NAME, new byte[][] 
{INPUT_FAMILY, OUTPUT_FAMILY});
+UTIL.createMultiRegions(table, INPUT_FAMILY);
+UTIL.loadTable(table, INPUT_FAMILY);
 UTIL.startMiniMapReduceCluster();
   }
 
@@ -149,7 +147,7 @@ public class TestMulitthreadedTableMapper {
   IdentityTableReducer.class, job);
   FileOutputFormat.setOutputPath(job, new Path(test));
   LOG.info(Started  + Bytes.toString(table.getTableName()));
-  job.waitForCompletion(true);
+  assertTrue(job.waitForCompletion(true));
   LOG.info(After map/reduce completion);
   // verify map-reduce results
   verify(Bytes.toString(table.getTableName()));
@@ -203,7 +201,10 @@ public class TestMulitthreadedTableMapper {
 scan.addFamily(OUTPUT_FAMILY);
 ResultScanner scanner = table.getScanner(scan);
 try {
-  for (Result r : scanner) {
+  IteratorResult itr = scanner.iterator();
+  assertTrue(itr.hasNext());
+  while(itr.hasNext()) {
+Result r = itr.next();
 if (LOG.isDebugEnabled()) {
   if (r.size()  2 ) {
 throw new IOException(Too many results, expected 2 got  +
{noformat}


 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.


[ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240517#comment-13240517
 ] 

Zhihong Yu commented on HBASE-5636:
---

Changes for mapred/TestTableMapReduce.java and 
mapreduce/TestTableMapReduce.java couldn't apply cleanly.
Can you upload a new patch for trunk ?

Thanks

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5544) Add metrics to HRegion.processRow()


 [ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-5544:
--

Attachment: HBASE-5544.D2457.2.patch

Re-attaching patch v2

 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5636) TestTableMapReduce doesn't work properly.


 [ 
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takuya Ueshin updated HBASE-5636:
-

Attachment: HBASE-5636-v2.patch

I attached a patch v2.
Maybe I forgot --no-prefix option of git-diff command.

 TestTableMapReduce doesn't work properly.
 -

 Key: HBASE-5636
 URL: https://issues.apache.org/jira/browse/HBASE-5636
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 0.92.1, 0.94.0
Reporter: Takuya Ueshin
 Attachments: HBASE-5636-v2.patch, HBASE-5636.patch


 No map function is called because there are no test data put before test 
 starts.
 The following three tests are in the same situation:
 - org.apache.hadoop.hbase.mapred.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
 - org.apache.hadoop.hbase.mapreduce.TestMulitthreadedTableMapper

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5649) [findbugs] Fix security warning

2012-03-28 Thread Laxman (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laxman reassigned HBASE-5649:
-

Assignee: Laxman

 [findbugs] Fix security warning
 ---

 Key: HBASE-5649
 URL: https://issues.apache.org/jira/browse/HBASE-5649
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: Laxman

 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_SECURITY
 Fix possible XSS Vuln.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5650) [findbugs] Address extra synchronization on CLSM, Atomic*

2012-03-28 Thread Laxman (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laxman reassigned HBASE-5650:
-

Assignee: Laxman

 [findbugs] Address extra synchronization on CLSM, Atomic*
 -

 Key: HBASE-5650
 URL: https://issues.apache.org/jira/browse/HBASE-5650
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: Laxman

 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS
 Fix/exclude class JLM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5636) TestTableMapReduce doesn't work properly.

[
https://issues.apache.org/jira/browse/HBASE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240540#comment-13240540
]

Hadoop QA commented on HBASE-5636:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520285/HBASE-5636-v2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 10 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestColumnSeeking

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1331//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1331//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1331//console

This message is automatically generated.

TestTableMapReduce doesn't work properly.
-

[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs


 [ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-3996:
--

Attachment: 3996-v6.txt

Patch v6 is same as Eran's patch v5, formatted to be accepted by review board.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240568#comment-13240568
 ] 

Zhihong Yu commented on HBASE-3996:
---

Currently TableSplit class is marked Stable. With the addition of Scan member, 
I plan to change the label to evolving.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5544) Add metrics to HRegion.processRow()

[
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240569#comment-13240569
]

Hadoop QA commented on HBASE-5544:
--

-1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12520284/HBASE-5544.D2457.2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1330//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1330//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1330//console

This message is automatically generated.

Add metrics to HRegion.processRow()
---

Key: HBASE-5544
URL: https://issues.apache.org/jira/browse/HBASE-5544
Project: HBase
Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
Fix For: 0.96.0

Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch,
HBASE-5544.D2457.2.patch

Add metrics of
1. time for waiting for the lock
2. processing time (scan time)
3. time spent while holding the lock
4. total call time
5. number of failures / calls

[jira] [Assigned] (HBASE-5648) [findbugs] Fix final/protected/constant declarations.

2012-03-28 Thread Mubarak Seyed (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mubarak Seyed reassigned HBASE-5648:


Assignee: Mubarak Seyed

 [findbugs] Fix final/protected/constant declarations.
 -

 Key: HBASE-5648
 URL: https://issues.apache.org/jira/browse/HBASE-5648
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: Mubarak Seyed

 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
 Fix warnings from class MS

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5643) [findbugs] Fix compareTo/equals/hashcode warnings

2012-03-28 Thread Mubarak Seyed (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mubarak Seyed reassigned HBASE-5643:


Assignee: Mubarak Seyed

 [findbugs] Fix compareTo/equals/hashcode warnings
 -

 Key: HBASE-5643
 URL: https://issues.apache.org/jira/browse/HBASE-5643
 Project: HBase
  Issue Type: Sub-task
  Components: scripts
Reporter: Jonathan Hsieh
Assignee: Mubarak Seyed

 See 
 https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
 Fix code to eliminate [Eq,ES,HE] categories.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs


 [ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhihong Yu updated HBASE-3996:
--

Attachment: 3996-v7.txt

Patch v7 introduces versioning for TableSplit, using the same tactic used for 
HLogKey.

Since most of enum Version code is copied, we may want to factor the base enum 
to its own class. Would org.apache.hadoop.hbase.util be a good namespace for 
the enum class ?

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread Todd Lipcon (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240592#comment-13240592
 ] 

Todd Lipcon commented on HBASE-3996:


Rather than do manual versioning, why not switch this to a protobuf? Then you 
avoid the manual serialization and you don't have to worry about versioning.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

[
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240596#comment-13240596
]

Hadoop QA commented on HBASE-3996:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520290/3996-v6.txt
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

-1 core tests. The patch failed these unit tests:

org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1332//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1332//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1332//console

This message is automatically generated.

Support multiple tables and scanners as input to the mapper in map/reduce jobs
--

Key: HBASE-3996
URL: https://issues.apache.org/jira/browse/HBASE-3996
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
Fix For: 0.96.0

Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt,
3996-v6.txt, 3996-v7.txt, HBase-3996.patch

It seems that in many cases feeding data from multiple tables or multiple
scanners on a single table can save a lot of time when running map/reduce
jobs.
I propose a new MultiTableInputFormat class that would allow doing this.

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread Todd Lipcon (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240599#comment-13240599
 ] 

Todd Lipcon commented on HBASE-3996:


The other question is whether we need version compatibility at all for this 
enum. The split object is created when you submit the job, and then only used 
by that one job, right? i.e it's never persisted or transferred over the wire 
to some other process, is it?

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240608#comment-13240608
 ] 

Zhihong Yu commented on HBASE-3996:
---

Test failure for patch v6 was due to MAPREDUCE-3583:
{code}
attempt_20120328173919786_0001_m_000100_1:  at 
org.apache.hadoop.mapred.Child.main(Child.java:249)
java.lang.NumberFormatException: For input string: 18446743988103913343
at 
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at 
org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
at 
org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
at 
org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
at org.apache.hadoop.mapred.Task.initialize(Task.java:536)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
{code}

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5665) Repeated split causes HRegionServer failures and breaks table

2012-03-28 Thread Cosmin Lehene (Created) (JIRA)

Repeated split causes HRegionServer failures and breaks table 
--

 Key: HBASE-5665
 URL: https://issues.apache.org/jira/browse/HBASE-5665
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 0.92.1, 0.92.0
Reporter: Cosmin Lehene
Priority: Blocker


Repeated splits on large tables (2 consecutive would suffice) will essentially 
break the table (and the cluster), unrecoverable.
The regionserver doing the split dies and the master will get into an infinite 
loop trying to assign regions that seem to have the files missing from HDFS.

The table can be disabled once. upon trying to re-enable it, it will remain in 
an intermediary state forever.

I was able to reproduce this on a smaller table consistently.

{code}
hbase(main):030:0 (0..1).each{|x| put 't1', #{x}, 'f1:t', 'dd'}
hbase(main):030:0 (0..1000).each{|x| split 't1', #{x*10}}
{code}

Running overlapping splits in parallel (e.g. #{x*10+1}, #{x*10+2}... ) will 
reproduce the issue almost instantly and consistently. 

{code}
2012-03-28 10:57:16,320 INFO org.apache.hadoop.hbase.catalog.MetaEditor: 
Offlined parent region t1,,1332957435767.2fb0473f4e71339e88dab0ee0d4dffa1. in 
META
2012-03-28 10:57:16,321 DEBUG 
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for 
t1,5,1332957435767.648d30de55a5cec6fc2f56dcb3c7eee1..  compaction_queue=(0:1), 
split_queue=10
2012-03-28 10:57:16,343 INFO org.apache.hadoop.hbase.regionserver.SplitRequest: 
Running rollback/cleanup of failed split of 
t1,,1332957435767.2fb0473f4e71339e88dab0ee0d4dffa1.; Failed 
ld2,60020,1332957343833-daughterOpener=2469c5650ea2aeed631eb85d3cdc3124
java.io.IOException: Failed 
ld2,60020,1332957343833-daughterOpener=2469c5650ea2aeed631eb85d3cdc3124
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughters(SplitTransaction.java:363)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:451)
at 
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException: File does not exist: 
/hbase/t1/589c44cabba419c6ad8c9b427e5894e3.2fb0473f4e71339e88dab0ee0d4dffa1/f1/d62a852c25ad44e09518e102ca557237
at 
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1822)
at 
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.init(DFSClient.java:1813)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:544)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:456)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:341)
at 
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1008)
at 
org.apache.hadoop.hbase.io.HalfStoreFileReader.init(HalfStoreFileReader.java:65)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548)
at 
org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:284)
at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:221)
at 
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2511)
at 
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:450)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3229)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughterRegion(SplitTransaction.java:504)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction$DaughterOpener.run(SplitTransaction.java:484)
... 1 more
2012-03-28 10:57:16,345 FATAL 
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
ld2,60020,1332957343833: Abort; we got an error after point-of-no-return
{code}


http://hastebin.com/diqinibajo.avrasm

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-50) Snapshot of table

2012-03-28 Thread Jesse Yates (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240625#comment-13240625
]

Jesse Yates commented on HBASE-50:
--

bq. maybe fully consistent across the whole table isn't necessary

This was just something that has come up in multiple conversations - is full
table, consistent snapshots necessary? The default answer seems to be yes,
because that is what we get with mysql/postgres/oracle, with the implication
that we _have to have it_ for production HBase. However, its inherently a
different storage model and it may be that we just need to change the
pointy-haired managers' ideas of what's necessary. If instead we can say its is
consistent view, per regionserver, at some point from 2:51:00 to 2:51:30 pm on
friday, then we can save a lot of pain and can be done with _very little
downtime_, to the point where you can mask it in a 'slow request'. If that
isn't acceptable, then the only way I see to do it is to drop write
availability while the snapshot is taking place (fastest region has to to wait
on the slowest region to finish) to get the fully consistent view.

No reason with the current design that we couldn't allow both, it would just be
another flag to pass in while making the snapshot and really only touches about
5 lines of the guts of the implementation.

Currently, we only guarantee row-level consistency. Getting into cross-RS
consistency means we bump up against CAP and have to give up even more
availability (for a fully consistent view, all RS needs to block writes, which
could take as long as the timeout (30sec - 1min in the worst case) or slowest
region - whichever is faster, which in many cases is unacceptable).

However, if you can take point-in-time within a window as acceptable (sloppy
snapshot) - maybe the window is thirty seconds - when each region blocks
writes for the time each region takes to make the snapshot (max of maybe a few
seconds as no data is being copied, but rather just a few references created
and counts incremented) you keep availability high without really sacrificing
any of the consistency guarantees we currently make.

Clearly, this breaks the multi-put situation where two puts go to different
servers but that is potentially acceptable since we don't make guarantees there
either, just that on return, all of the puts have completed. Same problem as
with doing any of the current backup solutions (replication not included). If
you are worried about cross-RS transactions, you have to use something like
Omid (or similar) to track transactions, and that system can then also decide
when a snapshot is allowable to
ensure all parts of the multi-put complete.

bq. If we're not reusing much of Chongxin's code, we should put discussion into
a new JIRA

A lot of the infrastructure we have has changed (eg. zookeeper utilities,
locking on the RS), but the new features - reference counting, possibly
importing/exporting snapshots, etc - will definitely be reused exactly or only
slightly modified. So at 50/50 on what is kept and what is tossed, at least
right now.

We have also gone through like 3 different stops and starts on this ticket. I
worry moving to a new ticket will cause even worse fragmentation, at least
until the code being used doesn't even resemble Chongxin's :)

Snapshot of table
-

Key: HBASE-50
URL: https://issues.apache.org/jira/browse/HBASE-50
Project: HBase
Issue Type: New Feature
Affects Versions: 0.96.0
Reporter: Billy Pearson
Assignee: Li Chongxin
Priority: Minor
Labels: gsoc
Fix For: 0.96.0

Attachments: HBase Snapshot Design Report V2.pdf, HBase Snapshot
Design Report V3.pdf, HBase Snapshot Implementation Plan.pdf, Snapshot Class
Diagram.png

Havening an option to take a snapshot of a table would be vary useful in
production.
What I would like to see this option do is do a merge of all the data into
one or more files stored in the same folder on the dfs. This way we could
save data in case of a software bug in hadoop or user code.
The other advantage would be to be able to export a table to multi locations.
Say I had a read_only table that must be online. I could take a snapshot of
it when needed and export it to a separate data center and have it loaded
there and then i would have it online at multi data centers for load
balancing and failover.
I understand that hadoop takes the need out of havening backup to protect
from failed servers, but this does not protect use from software bugs that
might delete or alter data in ways we did not plan. We should have a way we
can roll back a dataset.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240628#comment-13240628
 ] 

Zhihong Yu commented on HBASE-3996:
---

w.r.t. Todd's question above, the versioning is trying to solve the following 
scenario:
hbase jar on job tracker is updated to include the versioning mechanism but the 
job client has pre-versioning hbase jar.


 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread Todd Lipcon (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240632#comment-13240632
 ] 

Todd Lipcon commented on HBASE-3996:


bq. hbase jar on job tracker is updated to include the versioning mechanism but 
the job client has pre-versioning hbase jar.

The jar on the JT doesn't matter. Split computation and interpretation happens 
only in the user code -- i.e on the client machine and inside the tasks 
themselves. So you don't need HBase installed on the JT at all. As for the TTs, 
it's possible to configure the TTs to put an hbase jar on the classpath, but I 
usually recommend against it for the exact reason you're mentioning - if the 
jars differ in version, and they're not 100% API compatible, you can get nasty  
errors. The recommended deployment is to _not_ put hbase on the TT classpath, 
and instead ship the HBase dependencies as part of the MR job, using the 
provided utility function in TableMapReduceUtil.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HBASE-5547) Don't delete HFiles when in backup mode

2012-03-28 Thread Jesse Yates (Assigned) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates reassigned HBASE-5547:
--

Assignee: Jesse Yates

 Don't delete HFiles when in backup mode
 -

 Key: HBASE-5547
 URL: https://issues.apache.org/jira/browse/HBASE-5547
 Project: HBase
  Issue Type: New Feature
Reporter: Lars Hofhansl
Assignee: Jesse Yates

 This came up in a discussion I had with Stack.
 It would be nice if HBase could be notified that a backup is in progress (via 
 a znode for example) and in that case either:
 1. rename HFiles to be delete to file.bck
 2. rename the HFiles into a special directory
 3. rename them to a general trash directory (which would not need to be tied 
 to backup mode).
 That way it should be able to get a consistent backup based on HFiles (HDFS 
 snapshots or hard links would be better options here, but we do not have 
 those).
 #1 makes cleanup a bit harder.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread jirapos...@reviews.apache.org (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240651#comment-13240651
 ] 

Zhihong Yu commented on HBASE-3996:
---

I agree if HBase dependencies are shipped as part of the MR job jar, there is 
no need to worry about versioning of TableSplit.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-28 Thread Matteo Bertozzi (Created) (JIRA)

RegionServer doesn't retry to check if base node is available
-

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
hbase-zookeeper.log

I've a script that starts hbase and a couple of region servers in distributed 
mode (hbase.cluster.distributed = true)
{code}
$HBASE_HOME/bin/start-hbase.sh
$HBASE_HOME/bin/local-regionservers.sh start 1 2 3
{code}

but the region servers are not able to start...
It seems that during the RS start the the znode is still not available, and 
HRegionServer.initializeZooKeeper() check just once if the base not is 
available.
{code}
2012-03-28 21:54:05,013 INFO 
org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
configured in 'zookeeper.znode.parent'. There could be a mismatch with the one 
configured in the master.
2012-03-28 21:54:08,598 FATAL 
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
localhost,60202,133296824: Initialization of RS failed.  Hence aborting RS.
java.io.IOException: Received the shutdown message while waiting.
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
at java.lang.Thread.run(Thread.java:662)
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-28 Thread Matteo Bertozzi (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-5666:
---

Attachment: hbase-zookeeper.log
hbase-regionserver.log
hbase-master.log
hbase-3-regionserver.log
hbase-2-regionserver.log
hbase-1-regionserver.log

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5451) Switch RPC call envelope/headers to PBs


[ 
https://issues.apache.org/jira/browse/HBASE-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240722#comment-13240722
 ] 

jirapos...@reviews.apache.org commented on HBASE-5451:
--



bq.  On 2012-03-24 07:38:03, Benoit Sigoure wrote:
bq.   
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java,
 line 102
bq.   https://reviews.apache.org/r/4096/diff/2/?file=86903#file86903line102
bq.  
bq.   Argh, no, don't change this!  I got other HBase devs to promise to 
not change this as it makes backwards compatible clients impossibly complicated.
bq.  
bq.  Devaraj Das wrote:
bq.  I see. This was the basis of the graceful failure for current 
clients that are not aware of PB (clients would bail out if the versions of RPC 
don't match, right). The response to your comment below I don't see how this 
is graceful. is actually this change in the version.
bq.  
bq.  Michael Stack wrote:
bq.  Benoit's point is that this mechanism doesn't work so his point is 
lets not bother changing the version.  Previous, if you volunteered a hrpc 
version other than what is expected, the connection was closed by the server 
w/o saying what was wrong.  We fixed hbase so it at least throws an exception 
but it doesn't say what version its expecting.
bq.  
bq.  Devaraj Das wrote:
bq.  Stack, if we don't change the server version number then even the 
exception you're referring to won't be thrown. The exception/error will happen 
later on in the processing of the RPC... Are we sure we want this as the 
behavior? Please let me know.
bq.  
bq.  Michael Stack wrote:
bq.  On ...The exception/error will happen later on in the processing of 
the RPC... Are we sure we want this as the behavior? Please let me know., its 
useless as is.   Can we make this rationale?  Like if version is bumped, it 
tells client what version server is?

Hi Stack, not sure what you meant in your last comment. The VersionMismatch 
exception that is sent to the client has an accompanying message that says 
something like - Server IPC version current-version cannot communicate .. . 
By parsing the exception the client can know what's wrong (hacky but works). 

Once we have PB in the RPC we can actually remove this version check since 
clients/servers talk PB and PB will handle compatibility in the RPC messages. 
But I want to change things with more thought and as such want to keep the 
version number around for at least this jira.

Given the above, I am not sure what to do: to me version change seems 
sufficient to catch non-compliant clients early (and since the RPC is changing 
in a major by switching to PB, makes sense to me to change the version number). 
If on the other hand, we let the client pass this initial step by not changing 
the version number, we'll let old clients pass this initial step. It'll fail 
later on. 

Thoughts?


- Devaraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4096/#review6302
---


On 2012-03-01 03:40:14, Devaraj Das wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/4096/
bq.  ---
bq.  
bq.  (Updated 2012-03-01 03:40:14)
bq.  
bq.  
bq.  Review request for .
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Switch RPC call envelope/headers to PBs
bq.  
bq.  
bq.  This addresses bug HBASE-5451.
bq.  https://issues.apache.org/jira/browse/HBASE-5451
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.http://svn.apache.org/repos/asf/hbase/trunk/pom.xml 1294899 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/DataOutputOutputStream.java
 1294899 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
 1294899 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java
 1294899 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/security/User.java
 1294899 
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/proto/RPCMessageProto.proto
 PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/4096/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Devaraj
bq.  
bq.



 Switch RPC call envelope/headers to PBs
 ---

 Key: HBASE-5451
 URL: https://issues.apache.org/jira/browse/HBASE-5451
 Project: HBase
  Issue Type: Sub-task
  Components: ipc, master, migration, regionserver
Affects Versions:

[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-28 Thread David Arthur (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Arthur updated HBASE-5667:


Status: Patch Available  (was: Open)

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-28 Thread David Arthur (Created) (JIRA)

RegexStringComparator supports java.util.regex.Pattern flags


 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff

* Add constructor that takes in a Pattern
* Add Pattern's flags to Writable fields, and actually use them when 
recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-28 Thread David Arthur (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Arthur updated HBASE-5667:


Attachment: HBASE-5667.diff

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available


[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240751#comment-13240751
 ] 

Zhihong Yu commented on HBASE-5666:
---

@Matteo:
You're suggesting addition of a loop to wait for base node to become available 
(in place of what we have now below) ?
{code}
if (false == tracker.checkIfBaseNodeAvailable()) {
{code}

Ramkrishna added the check.
@Ram:
What do you think ?

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags


[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240753#comment-13240753
 ] 

Zhihong Yu commented on HBASE-5667:
---

This is an incompatible change, right ?
{code}
-this.pattern = Pattern.compile(expr);
+int flags = in.readInt();
+this.pattern = Pattern.compile(expr, flags);
{code}
What's the advantage of passing Pattern directly ?

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5666) RegionServer doesn't retry to check if base node is available

2012-03-28 Thread Matteo Bertozzi (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240757#comment-13240757
 ] 

Matteo Bertozzi commented on HBASE-5666:


@Zhihong Yu:
Probably a retry loop is the simplest solution...
But retry for how long? an infinite loop hoping that the node become available 
seems wrong, maybe we can add another parameter to define a ZK_RETRY_TIMEOUT.

 RegionServer doesn't retry to check if base node is available
 -

 Key: HBASE-5666
 URL: https://issues.apache.org/jira/browse/HBASE-5666
 Project: HBase
  Issue Type: Bug
  Components: regionserver, zookeeper
Reporter: Matteo Bertozzi
 Attachments: hbase-1-regionserver.log, hbase-2-regionserver.log, 
 hbase-3-regionserver.log, hbase-master.log, hbase-regionserver.log, 
 hbase-zookeeper.log


 I've a script that starts hbase and a couple of region servers in distributed 
 mode (hbase.cluster.distributed = true)
 {code}
 $HBASE_HOME/bin/start-hbase.sh
 $HBASE_HOME/bin/local-regionservers.sh start 1 2 3
 {code}
 but the region servers are not able to start...
 It seems that during the RS start the the znode is still not available, and 
 HRegionServer.initializeZooKeeper() check just once if the base not is 
 available.
 {code}
 2012-03-28 21:54:05,013 INFO 
 org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Check the value 
 configured in 'zookeeper.znode.parent'. There could be a mismatch with the 
 one configured in the master.
 2012-03-28 21:54:08,598 FATAL 
 org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
 localhost,60202,133296824: Initialization of RS failed.  Hence aborting 
 RS.
 java.io.IOException: Received the shutdown message while waiting.
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:626)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:596)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:558)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672)
   at java.lang.Thread.run(Thread.java:662)
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread Ming Ma (Commented) (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240765#comment-13240765
]

Ming Ma commented on HBASE-3996:

Appreciate if anyone can clarify the type of applications that could benefit
from this.

1. Does this work try to help with hbase map reduce job performance? If so,
Eran, do you have any data for that? Couple months I tried scanning multiple
regions in one mapper task, that only helps if the mapper task takes less than
couple minutes and thus map reduce task scheduling becomes the overhead.

2. In the multitable scenario, if we assume different tables have different
schemes, does that mean the application mapper implementation need to take care
of input from different tables?

Support multiple tables and scanners as input to the mapper in map/reduce jobs
--

Key: HBASE-3996
URL: https://issues.apache.org/jira/browse/HBASE-3996
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
Fix For: 0.96.0

Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt,
3996-v6.txt, 3996-v7.txt, HBase-3996.patch

[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

[
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240778#comment-13240778
]

Hadoop QA commented on HBASE-5667:
--

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520321/HBASE-5667.diff
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 3 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1334//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1334//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1334//console

This message is automatically generated.

RegexStringComparator supports java.util.regex.Pattern flags

Key: HBASE-5667
URL: https://issues.apache.org/jira/browse/HBASE-5667
Project: HBase
Issue Type: Improvement
Components: filters
Reporter: David Arthur
Priority: Minor
Attachments: HBASE-5667.diff

* Add constructor that takes in a Pattern
* Add Pattern's flags to Writable fields, and actually use them when
recomposing the Filter

[jira] [Commented] (HBASE-5544) Add metrics to HRegion.processRow()


[ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240784#comment-13240784
 ] 

Zhihong Yu commented on HBASE-5544:
---

A search among findbugs report for the classes touched by the patch didn't 
reveal anything.

Going to integrate later tonight if there is no objection.

 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-3996) Support multiple tables and scanners as input to the mapper in map/reduce jobs

2012-03-28 Thread Bohdan Mushkevych (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240791#comment-13240791
 ] 

Bohdan Mushkevych commented on HBASE-3996:
--

@Ming Ma: 
Described functionality is essential to make JOINs; or to process multiple 
regions from the same table.
While trying to merge 2+ datasets together, you better be aware what structures 
you are processing.

 Support multiple tables and scanners as input to the mapper in map/reduce jobs
 --

 Key: HBASE-3996
 URL: https://issues.apache.org/jira/browse/HBASE-3996
 Project: HBase
  Issue Type: Improvement
  Components: mapreduce
Reporter: Eran Kutner
Assignee: Eran Kutner
 Fix For: 0.96.0

 Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, 3996-v5.txt, 
 3996-v6.txt, 3996-v7.txt, HBase-3996.patch


 It seems that in many cases feeding data from multiple tables or multiple 
 scanners on a single table can save a lot of time when running map/reduce 
 jobs.
 I propose a new MultiTableInputFormat class that would allow doing this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5544) Add metrics to HRegion.processRow()

2012-03-28 Thread Scott Chen (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240797#comment-13240797
 ] 

Scott Chen commented on HBASE-5544:
---

Thanks, Ted

 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5668) HRegionServer.checkFileSystem() should only abort() after fs is down for some time

2012-03-28 Thread Prakash Khemani (Created) (JIRA)

HRegionServer.checkFileSystem() should only abort() after fs is down for some 
time
--

 Key: HBASE-5668
 URL: https://issues.apache.org/jira/browse/HBASE-5668
 Project: HBase
  Issue Type: Improvement
Reporter: Prakash Khemani


When checkFileSystem() fails then the region server should wait for sometime 
before aborting. By default, the timeout can be same as zookeeper session 
timeout.

When say a rack switch reboots or fails for a few minutes, and all the traffic 
to the region server dies ... then we don't want the region servers to 
unnecessarily kill themselves when ongoing compactions or flushes fail.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5665) Repeated split causes HRegionServer failures and breaks table

2012-03-28 Thread Cosmin Lehene (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cosmin Lehene updated HBASE-5665:
-

Description: 
Repeated splits on large tables (2 consecutive would suffice) will essentially 
break the table (and the cluster), unrecoverable.
The regionserver doing the split dies and the master will get into an infinite 
loop trying to assign regions that seem to have the files missing from HDFS.

The table can be disabled once. upon trying to re-enable it, it will remain in 
an intermediary state forever.

I was able to reproduce this on a smaller table consistently.

{code}
hbase(main):030:0 (0..1).each{|x| put 't1', #{x}, 'f1:t', 'dd'}
hbase(main):030:0 (0..1000).each{|x| split 't1', #{x*10}}
{code}

Running overlapping splits in parallel (e.g. #{x*10+1}, #{x*10+2}... ) will 
reproduce the issue almost instantly and consistently. 

{code}
2012-03-28 10:57:16,320 INFO org.apache.hadoop.hbase.catalog.MetaEditor: 
Offlined parent region t1,,1332957435767.2fb0473f4e71339e88dab0ee0d4dffa1. in 
META
2012-03-28 10:57:16,321 DEBUG 
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for 
t1,5,1332957435767.648d30de55a5cec6fc2f56dcb3c7eee1..  compaction_queue=(0:1), 
split_queue=10
2012-03-28 10:57:16,343 INFO org.apache.hadoop.hbase.regionserver.SplitRequest: 
Running rollback/cleanup of failed split of 
t1,,1332957435767.2fb0473f4e71339e88dab0ee0d4dffa1.; Failed 
ld2,60020,1332957343833-daughterOpener=2469c5650ea2aeed631eb85d3cdc3124
java.io.IOException: Failed 
ld2,60020,1332957343833-daughterOpener=2469c5650ea2aeed631eb85d3cdc3124
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughters(SplitTransaction.java:363)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:451)
at 
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.FileNotFoundException: File does not exist: 
/hbase/t1/589c44cabba419c6ad8c9b427e5894e3.2fb0473f4e71339e88dab0ee0d4dffa1/f1/d62a852c25ad44e09518e102ca557237
at 
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1822)
at 
org.apache.hadoop.hdfs.DFSClient$DFSInputStream.init(DFSClient.java:1813)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:544)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:456)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:341)
at 
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1008)
at 
org.apache.hadoop.hbase.io.HalfStoreFileReader.init(HalfStoreFileReader.java:65)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:467)
at 
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548)
at 
org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:284)
at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:221)
at 
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2511)
at 
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:450)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3229)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction.openDaughterRegion(SplitTransaction.java:504)
at 
org.apache.hadoop.hbase.regionserver.SplitTransaction$DaughterOpener.run(SplitTransaction.java:484)
... 1 more
2012-03-28 10:57:16,345 FATAL 
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server 
ld2,60020,1332957343833: Abort; we got an error after point-of-no-return
{code}

http://hastebin.com/diqinibajo.avrasm

later edit:

(I'm using the last 4 characters from each string)
Region 94e3 has storefile 7237
Region 94e3 gets splited in daughters a: ffa1 and b: eee1
Daughter region ffa1 get's splitted in daughters a: 3124 and b: dc77
ffa1 has a reference: 7237.94e3 for it's store file
when ffa1 gets splited it will create another reference: 7237.94e3.ffa1
when SplitTransaction will execute() it will try to open that (openDaughters 
above) and it will match it from left to right [storefile].[region] 
{code}
^([0-9a-f]+)(?:\\.(.+))?$
{code}
and will attempt to go to /hbase/t1/[region] which resolves to 
/hbase/t1/94e3.ffa1/f1/7237 - which obviously doesn't exist and will fail. 

This seems like a design problem: we should either stop from splitting if the 
path is reference or be able to recursively resolve reference paths (e.g. parse 
right to left

[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags

2012-03-28 Thread David Arthur (Commented) (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240818#comment-13240818
 ] 

David Arthur commented on HBASE-5667:
-

@Zhihong, yes it would be incompatible if it tried to read Filters that were 
serialized with the current code. I'm assuming that these Filters are ephemeral 
and aren't stored anywhere (could be a very wrong assumption). 

Otherwise, the purpose of this patch is to expose the ability to add regex 
flags to the comparator. E.g., if I want a case-insensitive match I could 
construct a Filter like:

{code}
new SingleColumnValueFilter(COLUMN_FAMILY, COLUMN_QUALIFIER, CompareOp.EQUAL,
  new RegexStringComparator(Pattern.compile(foo, Pattern.CASE_INSENSITIVE | 
Pattern.DOTALL)));
{code}

Also, in the current code, DOTALL is added to the underlying Pattern, but 
doesn't appear to be applied when deserializing. 

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5669) AggregationClient fails validation for open stoprow scan

2012-03-28 Thread Brian Rogers (Created) (JIRA)

AggregationClient fails validation for open stoprow scan


 Key: HBASE-5669
 URL: https://issues.apache.org/jira/browse/HBASE-5669
 Project: HBase
  Issue Type: Bug
  Components: coprocessors
Affects Versions: 0.92.1
 Environment: n/a
Reporter: Brian Rogers
Priority: Minor
 Fix For: 0.92.2


AggregationClient.validateParameters throws an exception when the Scan has a 
valid startrow but an unset endrow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5667) RegexStringComparator supports java.util.regex.Pattern flags


[ 
https://issues.apache.org/jira/browse/HBASE-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240839#comment-13240839
 ] 

Zhihong Yu commented on HBASE-5667:
---

Maintaining backward compatibility would be desirable.

Would moving the read of flags to after the following line be better ?
{code}
final String charset = in.readUTF();
{code}

 RegexStringComparator supports java.util.regex.Pattern flags
 

 Key: HBASE-5667
 URL: https://issues.apache.org/jira/browse/HBASE-5667
 Project: HBase
  Issue Type: Improvement
  Components: filters
Reporter: David Arthur
Priority: Minor
 Attachments: HBASE-5667.diff


 * Add constructor that takes in a Pattern
 * Add Pattern's flags to Writable fields, and actually use them when 
 recomposing the Filter

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5544) Add metrics to HRegion.processRow()


[ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240848#comment-13240848
 ] 

Zhihong Yu commented on HBASE-5544:
---

Patch integrated to TRUNK.

Thanks for the patch, Scott.

 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5544) Add metrics to HRegion.processRow()

2012-03-28 Thread Scott Chen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Chen updated HBASE-5544:
--

Release Note: Collects the time spent on each step in HRegion.processRow()

 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5544) Add metrics to HRegion.processRow()

2012-03-28 Thread Scott Chen (Updated) (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Chen updated HBASE-5544:
--

Release Note: 
Collects the time spent on each step in HRegion.processRow()
Here are the metrics collected

rowprocessor.name.nano
rowprocessor.name.error.nano
rowprocessor.name. acquirelock.nano
rowprocessor.name. occupylock.nano
rowprocessor.name. sync.nano

where name is the value of RowProcessor.getName().

  was:Collects the time spent on each step in HRegion.processRow()


 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-5670) Have Mutation implement the Row interface.

2012-03-28 Thread Lars Hofhansl (Created) (JIRA)

Have Mutation implement the Row interface.
--

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial


In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.

In a discussion with a co-worker I noticed that Put/Delete/Append still 
implement the Row interface, but Mutation does not.

In a trivial change I would like to move that interface up to Mutation, along 
with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
(HConnection.processBatch takes List? extends Row already anyway), so that 
HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5670) Have Mutation implement the Row interface.


 [ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5670:
-

Fix Version/s: 0.94.1

 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.94.1


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5670) Have Mutation implement the Row interface.


 [ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5670:
-

Attachment: 5670-0.94.txt

Simple patch that does just that. Mutation implements Row, and HTable.batch 
takes List? extends Row

 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.94.1

 Attachments: 5670-0.94.txt, 5670-trunk.txt


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5670) Have Mutation implement the Row interface.


 [ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5670:
-

Attachment: 5670-trunk.txt

Same for trunk

 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.94.1

 Attachments: 5670-0.94.txt, 5670-trunk.txt


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-5670) Have Mutation implement the Row interface.


 [ 
https://issues.apache.org/jira/browse/HBASE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-5670:
-

Status: Patch Available  (was: Open)

 Have Mutation implement the Row interface.
 --

 Key: HBASE-5670
 URL: https://issues.apache.org/jira/browse/HBASE-5670
 Project: HBase
  Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Trivial
 Fix For: 0.94.1

 Attachments: 5670-0.94.txt, 5670-trunk.txt


 In HBASE-4347 I factored some code from Put/Delete/Append in Mutation.
 In a discussion with a co-worker I noticed that Put/Delete/Append still 
 implement the Row interface, but Mutation does not.
 In a trivial change I would like to move that interface up to Mutation, along 
 with changing HTable.batch(ListRow) to HTable.batch(List? extends Row) 
 (HConnection.processBatch takes List? extends Row already anyway), so that 
 HTable.batch can be used with a list of Mutations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5610) Add GA to hbase.apache.org


[ 
https://issues.apache.org/jira/browse/HBASE-5610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240888#comment-13240888
 ] 

Hudson commented on HBASE-5610:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5610 Add GA to hbase.apache.org (Revision 1305970)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/site/site.vm


 Add GA to hbase.apache.org
 --

 Key: HBASE-5610
 URL: https://issues.apache.org/jira/browse/HBASE-5610
 Project: HBase
  Issue Type: Task
Reporter: stack
Assignee: stack
 Fix For: 0.96.0

 Attachments: ga.txt


 Lets add the bit of script necessary tracking hbase.apache.org in google 
 analytics.  I was going to get it going first then open it to the PMC for 
 viewing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5190) Limit the IPC queue size based on calls' payload size


[ 
https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240892#comment-13240892
 ] 

Hudson commented on HBASE-5190:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5190 Limit the IPC queue size based on calls' payload size
   (Ted's addendum) (Revision 1305468)

 Result = FAILURE
jdcryans : 
Files : 
* 
/hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureServer.java


 Limit the IPC queue size based on calls' payload size
 -

 Key: HBASE-5190
 URL: https://issues.apache.org/jira/browse/HBASE-5190
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
 Fix For: 0.94.0, 0.96.0

 Attachments: 5190.addendum, HBASE-5190-v2.patch, HBASE-5190-v3.patch, 
 HBASE-5190.patch


 Currently we limit the number of calls in the IPC queue only on their count. 
 It used to be really high and was dropped down recently to num_handlers * 10 
 (so 100 by default) because it was easy to OOME yourself when huge calls were 
 being queued. It's still possible to hit this problem if you use really big 
 values and/or a lot of handlers, so the idea is that we should take into 
 account the payload size. I can see 3 solutions:
  - Do the accounting outside of the queue itself for all calls coming in and 
 out and when a call doesn't fit, throw a retryable exception.
  - Same accounting but instead block the call when it comes in until space is 
 made available.
  - Add a new parameter for the maximum size (in bytes) of a Call and then set 
 the size the IPC queue (in terms of the number of items) so that it could 
 only contain as many items as some predefined maximum size (in bytes) for the 
 whole queue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5660) [book] creating Case Studies chapter


[ 
https://issues.apache.org/jira/browse/HBASE-5660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240890#comment-13240890
 ] 

Hudson commented on HBASE-5660:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
hbase-5660.  Creating Case Studies chapter in RefGuide. (Revision 1306015)

 Result = FAILURE

 [book] creating Case Studies chapter
 

 Key: HBASE-5660
 URL: https://issues.apache.org/jira/browse/HBASE-5660
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: docbkx_hbase_5660.patch


 There is currently a single case study in Troubleshooting I added a few weeks 
 ago, and now there are several other candidates worthy of adding.  Case 
 Studies needs to be a chapter of it's own.
 * Moving the existing case study currently in Troubleshooting to this new 
 chapter.
 * Adding 3 other sections for links that have come up recently on the 
 dist-list.
 * Changed existing links in the RefGuide to point to the new chapter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5623) Race condition when rolling the HLog and hlogFlush


[ 
https://issues.apache.org/jira/browse/HBASE-5623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240889#comment-13240889
 ] 

Hudson commented on HBASE-5623:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5623 Race condition when rolling the HLog and hlogFlush (Enis 
Soztutar and LarsH) (Revision 1305556)

 Result = FAILURE
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogWriter.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRollingNoCluster.java


 Race condition when rolling the HLog and hlogFlush
 --

 Key: HBASE-5623
 URL: https://issues.apache.org/jira/browse/HBASE-5623
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 0.94.0
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Priority: Critical
 Fix For: 0.94.0

 Attachments: 5623-suggestion.txt, 5623-v7.txt, 5623-v8.txt, 5623.txt, 
 5623v2.txt, HBASE-5623_v0.patch, HBASE-5623_v4.patch, HBASE-5623_v5.patch, 
 HBASE-5623_v6-alt.patch, HBASE-5623_v6-alt.patch


 When doing a ycsb test with a large number of handlers 
 (regionserver.handler.count=60), I get the following exceptions:
 {code}
 Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: 
 java.lang.NullPointerException
   at 
 org.apache.hadoop.io.SequenceFile$Writer.getLength(SequenceFile.java:1099)
   at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.getLength(SequenceFileLogWriter.java:314)
   at org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1291)
   at org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1388)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchPut(HRegion.java:2192)
   at org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1985)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3400)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1351)
   at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:920)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:152)
   at $Proxy1.multi(Unknown Source)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1691)
   at 
 org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1689)
   at 
 org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:214)
 {code}
 and 
 {code}
   java.lang.NullPointerException
   at 
 org.apache.hadoop.io.SequenceFile$Writer.checkAndWriteSync(SequenceFile.java:1026)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1068)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1035)
   at 
 org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.append(SequenceFileLogWriter.java:279)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLog$LogSyncer.hlogFlush(HLog.java:1237)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLog.syncer(HLog.java:1271)
   at 
 org.apache.hadoop.hbase.regionserver.wal.HLog.sync(HLog.java:1391)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchPut(HRegion.java:2192)
   at 
 org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1985)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3400)
   at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at 
 org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:366)
   at 
 org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1351)
 {code}
 It seems the root cause of the issue is that we open a new log writer and 
 close the old one at HLog#rollWriter() holding the updateLock, but the other 
 threads doing syncer() calls
 {code}

[jira] [Commented] (HBASE-5639) The logic used in waiting for region servers during startup is broken


[ 
https://issues.apache.org/jira/browse/HBASE-5639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240891#comment-13240891
 ] 

Hudson commented on HBASE-5639:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5639 The logic used in waiting for region servers during startup is 
broken (J-D and NKeyval) (Revision 1306012)

 Result = FAILURE
larsh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java


 The logic used in waiting for region servers during startup is broken
 -

 Key: HBASE-5639
 URL: https://issues.apache.org/jira/browse/HBASE-5639
 Project: HBase
  Issue Type: Bug
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
 Fix For: 0.94.0

 Attachments: HBASE-5639.patch


 See the tail of HBASE-4993, which I'll report here:
 Me:
 {quote}
 I think a bug was introduced here. Here's the new waiting logic in 
 waitForRegionServers:
 the 'hbase.master.wait.on.regionservers.mintostart' is reached AND
there have been no new region server in for
   'hbase.master.wait.on.regionservers.interval' time
 And the code that verifies that:
 !(lastCountChange+interval  now  count = minToStart)
 {quote}
 Nic:
 {quote}
 It seems that changing the code to
 (count  minToStart ||
 lastCountChange+interval  now)
 would make the code works as documented.
 If you have 0 region servers that checked in and you are under the interval, 
 you wait: (true or true) = true.
 If you have 0 region servers but you are above the interval, you wait: (true 
 or false) = true.
 If you have 1 or more region servers that checked in and you are under the 
 interval, you wait: (false or true) = true.
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5533) Add more metrics to HBase


[ 
https://issues.apache.org/jira/browse/HBASE-5533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240893#comment-13240893
 ] 

Hudson commented on HBASE-5533:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5533 Add more metrics to HBase (Revision 1305499)

 Result = FAILURE
stack : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/ExactCounterMetric.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/ExponentiallyDecayingSample.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/MetricsHistogram.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Sample.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/Snapshot.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/metrics/histogram/UniformSample.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Threads.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/metrics/TestExactCounterMetric.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/metrics/TestExponentiallyDecayingSample.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/metrics/TestMetricsHistogram.java


 Add more metrics to HBase
 -

 Key: HBASE-5533
 URL: https://issues.apache.org/jira/browse/HBASE-5533
 Project: HBase
  Issue Type: Improvement
Affects Versions: 0.92.2, 0.94.0
Reporter: Shaneal Manek
Assignee: Shaneal Manek
Priority: Minor
 Fix For: 0.94.0, 0.96.0

 Attachments: BlockingQueueContention.java, HBASE-5533-0.92-v4.patch, 
 HBASE-5533-TRUNK-v6.patch, HBASE-5533-TRUNK-v6.patch, TimingOverhead.java, 
 hbase-5533-0.92.patch, hbase5533-0.92-v2.patch, hbase5533-0.92-v3.patch, 
 hbase5533-0.92-v5.patch, histogram_web_ui.png


 To debug/monitor production clusters, there are some more metrics I wish I 
 had available.
 In particular:
 - Although the average FS latencies are useful, a 'histogram' of recent 
 latencies (90% of reads completed in under 100ms, 99% in under 200ms, etc) 
 would be more useful
 - Similar histograms of latencies on common operations (GET, PUT, DELETE) 
 would be useful
 - Counting the number of accesses to each region to detect hotspotting
 - Exposing the current number of HLog files

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5544) Add metrics to HRegion.processRow()


[ 
https://issues.apache.org/jira/browse/HBASE-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240894#comment-13240894
 ] 

Hudson commented on HBASE-5544:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5544 Add metrics to HRegion.processRow() (Scott Chen) (Revision 
1306648)

 Result = FAILURE
tedyu : 
Files : 
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/BaseRowProcessor.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RowProcessor.java
* 
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRowProcessorEndpoint.java


 Add metrics to HRegion.processRow()
 ---

 Key: HBASE-5544
 URL: https://issues.apache.org/jira/browse/HBASE-5544
 Project: HBase
  Issue Type: New Feature
Reporter: Scott Chen
Assignee: Scott Chen
 Fix For: 0.96.0

 Attachments: HBASE-5544.D2457.1.patch, HBASE-5544.D2457.2.patch, 
 HBASE-5544.D2457.2.patch


 Add metrics of
 1. time for waiting for the lock
 2. processing time (scan time)
 3. time spent while holding the lock
 4. total call time
 5. number of failures / calls

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5661) [book] add section for hbase history in appendix


[ 
https://issues.apache.org/jira/browse/HBASE-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240896#comment-13240896
 ] 

Hudson commented on HBASE-5661:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
hbase-5661.  adding hbase history section in appendix (Revision 1306032)

 Result = FAILURE

 [book] add section for hbase history in appendix
 

 Key: HBASE-5661
 URL: https://issues.apache.org/jira/browse/HBASE-5661
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
 Attachments: book_hbase_5661.xml.patch


 Adding a section for HBase History in the appendix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5596) Few minor bugs from HBASE-5209


[ 
https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240899#comment-13240899
 ] 

Hudson commented on HBASE-5596:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5596 Few minor bugs from HBASE-5209 (David S. Wang) (Revision 1305661)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ServerName.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 Few minor bugs from HBASE-5209
 --

 Key: HBASE-5596
 URL: https://issues.apache.org/jira/browse/HBASE-5596
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 0.92.1, 0.94.0
Reporter: David S. Wang
Assignee: David S. Wang
Priority: Minor
 Fix For: 0.92.2, 0.94.0, 0.96.0

 Attachments: HBASE-5596.patch, hbase-5596-0.94.patch


 A few leftover bugs from HBASE-5209.  Comments are documented here:
 https://reviews.apache.org/r/3892/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5658) pseduo-distributed.xml - first link pointing to wrong site


[ 
https://issues.apache.org/jira/browse/HBASE-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240895#comment-13240895
 ] 

Hudson commented on HBASE-5658:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
hbase-5658.  Update to pseudo-distributed.xml's first link on page to point 
to RefGuide (Revision 1305990)

 Result = FAILURE

 pseduo-distributed.xml - first link pointing to wrong site
 --

 Key: HBASE-5658
 URL: https://issues.apache.org/jira/browse/HBASE-5658
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: pseudo-distributed_hbase_5658.xml.patch


 In the web-page 'pseudo-distributed.xml' the first link Distributed 
 Operation: Pseudo- and Fully-distributed modes is linking to the Javadoc 
 when it should be linking to the Reference Guide in the config section.
 Thanks to Dave Wang for pointing this out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5209) HConnection/HMasterInterface should allow for way to get hostname of currently active master in multi-master HBase setup


[ 
https://issues.apache.org/jira/browse/HBASE-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240901#comment-13240901
 ] 

Hudson commented on HBASE-5209:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
HBASE-5596 Few minor bugs from HBASE-5209 (David S. Wang) (Revision 1305661)

 Result = FAILURE
jmhsieh : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ServerName.java
* 
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 HConnection/HMasterInterface should allow for way to get hostname of 
 currently active master in multi-master HBase setup
 

 Key: HBASE-5209
 URL: https://issues.apache.org/jira/browse/HBASE-5209
 Project: HBase
  Issue Type: Improvement
  Components: master
Affects Versions: 0.90.5, 0.92.0, 0.94.0
Reporter: Aditya Acharya
Assignee: David S. Wang
 Fix For: 0.92.1, 0.94.0

 Attachments: 5209.addendum, HBASE_5209_v5.diff


 I have a multi-master HBase set up, and I'm trying to programmatically 
 determine which of the masters is currently active. But the API does not 
 allow me to do this. There is a getMaster() method in the HConnection class, 
 but it returns an HMasterInterface, whose methods do not allow me to find out 
 which master won the last race. The API should have a 
 getActiveMasterHostname() or something to that effect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5657) [book] updating development chapter for 100 chars/line (from 80)


[ 
https://issues.apache.org/jira/browse/HBASE-5657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240898#comment-13240898
 ] 

Hudson commented on HBASE-5657:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
hbase-5657.  developer.xml, updating line-width from 80 to 100 in book (and 
example) (Revision 1305979)

 Result = FAILURE

 [book] updating development chapter for 100 chars/line (from 80)
 

 Key: HBASE-5657
 URL: https://issues.apache.org/jira/browse/HBASE-5657
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: developer_hbase_5657.xml.patch


 Per a recent discussion on the dist-list, updating the max-chars from 80 to 
 100.
 Also updating the long lines example.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5662) [book] add section in Ops chapter for HLogPrettyPrinter


[ 
https://issues.apache.org/jira/browse/HBASE-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13240897#comment-13240897
 ] 

Hudson commented on HBASE-5662:
---

Integrated in HBase-TRUNK #2697 (See 
[https://builds.apache.org/job/HBase-TRUNK/2697/])
hbase-5662.  ops_mgt.xml, adding entry for HLogPrettyPrinter in tools 
section. (Revision 1306037)

 Result = FAILURE

 [book] add section in Ops chapter for HLogPrettyPrinter
 ---

 Key: HBASE-5662
 URL: https://issues.apache.org/jira/browse/HBASE-5662
 Project: HBase
  Issue Type: Improvement
Reporter: Doug Meil
Assignee: Doug Meil
Priority: Minor
 Attachments: ops_mgt_hbase_5662.xml.patch


 ops_mgt.xml
 * added section under HLog (in tools) for HLogPrettyPrinter.
 * The description is a bit terse, but at least there is an entry in the book 
 for it.  The need for this came up recently on the dist-list.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings