date:20150301

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342587#comment-14342587
]

stack commented on HBASE-13082:
---

A combined patch, this and HBASE-13071, is slower instead of being a compound
of the benefits this patch adds and the benefits HBASE-13071 adds. It is like
we go over a perf hill and on the otherside of the hill, we slow. Let me poke
some more. Will post more pictures.

Coarsen StoreScanner locks to RegionScanner
---

Key: HBASE-13082
URL: https://issues.apache.org/jira/browse/HBASE-13082
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 13082-test.txt, 13082.txt, 13082.txt, gc.png, next.png

Continuing where HBASE-10015 left of.
We can avoid locking (and memory fencing) inside StoreScanner by deferring to
the lock already held by the RegionScanner.
In tests this shows quite a scan improvement and reduced CPU (the fences make
the cores wait for memory fetches).
There are some drawbacks too:
* All calls to RegionScanner need to be remain synchronized
* Implementors of coprocessors need to be diligent in following the locking
contract. For example Phoenix does not lock RegionScanner.nextRaw() and
required in the documentation (not picking on Phoenix, this one is my fault
as I told them it's OK)
* possible starving of flushes and compaction with heavy read load.
RegionScanner operations would keep getting the locks and the
flushes/compactions would not be able finalize the set of files.
I'll have a patch soon.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning


[ 
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342597#comment-14342597
 ] 

Lars Hofhansl commented on HBASE-13109:
---

All the test failures have the same cause (using KeyOnlyKeyValue), I removed 
that part of the code (was just to avoid some code duplication).

 Make better SEEK vs SKIP decisions during scanning
 --

 Key: HBASE-13109
 URL: https://issues.apache.org/jira/browse/HBASE-13109
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
 Attachments: 13109-trunk.txt


 I'm re-purposing this issue to add a heuristic as to when to SEEK and when to 
 SKIP Cells. This has come up in various issues, and I think I have a way to 
 finally fix this now. HBASE-9778, HBASE-12311, and friends are related.
 --- Old description ---
 This is a continuation of HBASE-9778.
 We've seen a scenario of a very slow scan over a region using a timerange 
 that happens to fall after the ts of any Cell in the region.
 Turns out we spend a lot of time seeking.
 Tested with a 5 column table, and the scan is 5x faster when the timerange 
 falls before all Cells' ts.
 We can use the lookahead hint introduced in HBASE-9778 to do opportunistic 
 SKIPing before we actually seek.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13133) NPE when running TestSplitLogManager

zhangduo created HBASE-13133:


 Summary: NPE when running TestSplitLogManager
 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo


https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/

{noformat}
2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
TestOrphanTaskAcquisition
2015-03-01 01:34:58,904 DEBUG [Thread-23] 
coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
2015-03-01 01:34:58,907 INFO  [Thread-23] 
coordination.ZKSplitLogManagerCoordination(594): found orphan task 
orphan%2Ftest%2Fslash
2015-03-01 01:34:58,913 INFO  [Thread-23] 
coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
rescan nodes
2015-03-01 01:34:58,913 ERROR [main-EventThread] 
zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
at 
org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
at 
org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
at 
org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
at 
org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
{noformat}

I got this NPE almost every time when running TestSplitLogManager locally. I am 
not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342591#comment-14342591
 ] 

Ted Yu commented on HBASE-13133:


+1

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12468) AUTHORIZATIONS should be part of Visibility Label Docs


 [ 
https://issues.apache.org/jira/browse/HBASE-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-12468:

Status: Open  (was: Patch Available)

 AUTHORIZATIONS should be part of Visibility Label Docs
 --

 Key: HBASE-12468
 URL: https://issues.apache.org/jira/browse/HBASE-12468
 Project: HBase
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.98.6.1
Reporter: Kevin Odell
Assignee: Misty Stanley-Jones
 Attachments: HBASE-12468-v1.patch, HBASE-12468.patch


 Per https://issues.apache.org/jira/browse/HBASE-12346 you need to use 
 AUTHORIZATIONS or setAuthorizations to see your labels. We may want to update 
 http://hbase.apache.org/book/ch08s03.html with that information



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342576#comment-14342576
 ] 

Andrey Stepachev commented on HBASE-13133:
--

not sure about root of the problem, but it seems can happen that async read 
callback was set
in ZKSplitLogManagerCoordination#getDataSetWatch and called before 
method SplitLogManagerCoordination#setDetails was called and details were set.
it seems that patch can be very straitforward, need to change order of 
initialization.

but other possibility for that is not set coordination engine, in that case 
details would not
be initialized, but chore still can be called.


 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo

 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13052) Explain each region split policy


 [ 
https://issues.apache.org/jira/browse/HBASE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13052:

Status: Patch Available  (was: Open)

 Explain each region split policy
 

 Key: HBASE-13052
 URL: https://issues.apache.org/jira/browse/HBASE-13052
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13052-v1.patch, HBASE-13052.patch


 {quote}
 there are five region split policies today so that let's add a section which 
 explains:
 1. How each policies work. We can start from current java doc:
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html
 2. How users can choose a good policy per their scenario basis
 3. Pros and cons over each policies
 {quote}
 from [~daisuke.kobayashi]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13052) Explain each region split policy


 [ 
https://issues.apache.org/jira/browse/HBASE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13052:

Attachment: HBASE-13052-v1.patch

[~apurtell] please have a look and see if I have answered the questions with 
the right answers. I wasn't really sure on a few of them. Maybe there are some 
other questions and answers that would be more interesting.

 Explain each region split policy
 

 Key: HBASE-13052
 URL: https://issues.apache.org/jira/browse/HBASE-13052
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13052-v1.patch, HBASE-13052.patch


 {quote}
 there are five region split policies today so that let's add a section which 
 explains:
 1. How each policies work. We can start from current java doc:
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html
 2. How users can choose a good policy per their scenario basis
 3. Pros and cons over each policies
 {quote}
 from [~daisuke.kobayashi]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13052) Explain each region split policy


 [ 
https://issues.apache.org/jira/browse/HBASE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13052:

Status: Open  (was: Patch Available)

 Explain each region split policy
 

 Key: HBASE-13052
 URL: https://issues.apache.org/jira/browse/HBASE-13052
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13052-v1.patch, HBASE-13052.patch


 {quote}
 there are five region split policies today so that let's add a section which 
 explains:
 1. How each policies work. We can start from current java doc:
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html
 http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html
 2. How users can choose a good policy per their scenario basis
 3. Pros and cons over each policies
 {quote}
 from [~daisuke.kobayashi]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-7126) Update website with info on how to report security bugs


 [ 
https://issues.apache.org/jira/browse/HBASE-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-7126:
---
Status: Patch Available  (was: Open)

 Update website with info on how to report security bugs 
 

 Key: HBASE-7126
 URL: https://issues.apache.org/jira/browse/HBASE-7126
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Eli Collins
Priority: Critical
  Labels: website
 Attachments: HBASE-7126.patch


 The HBase website should be updated with information on how to report 
 potential security vulnerabilities. In Hadoop land we have a private security 
 list that anyone case post to that we point to on our list page: Hadoop 
 example http://hadoop.apache.org/general_lists.html#Security.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-7126) Update website with info on how to report security bugs


 [ 
https://issues.apache.org/jira/browse/HBASE-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones reassigned HBASE-7126:
--

Assignee: Misty Stanley-Jones

 Update website with info on how to report security bugs 
 

 Key: HBASE-7126
 URL: https://issues.apache.org/jira/browse/HBASE-7126
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Eli Collins
Assignee: Misty Stanley-Jones
Priority: Critical
  Labels: website
 Attachments: HBASE-7126.patch


 The HBase website should be updated with information on how to report 
 potential security vulnerabilities. In Hadoop land we have a private security 
 list that anyone case post to that we point to on our list page: Hadoop 
 example http://hadoop.apache.org/general_lists.html#Security.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-7126) Update website with info on how to report security bugs


 [ 
https://issues.apache.org/jira/browse/HBASE-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-7126:
---
Attachment: HBASE-7126.patch

Please review.

 Update website with info on how to report security bugs 
 

 Key: HBASE-7126
 URL: https://issues.apache.org/jira/browse/HBASE-7126
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Eli Collins
Priority: Critical
  Labels: website
 Attachments: HBASE-7126.patch


 The HBase website should be updated with information on how to report 
 potential security vulnerabilities. In Hadoop land we have a private security 
 list that anyone case post to that we point to on our list page: Hadoop 
 example http://hadoop.apache.org/general_lists.html#Security.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12468) AUTHORIZATIONS should be part of Visibility Label Docs


 [ 
https://issues.apache.org/jira/browse/HBASE-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-12468:

Attachment: HBASE-12468-v1.patch

Incorporated both bits of feedback.

 AUTHORIZATIONS should be part of Visibility Label Docs
 --

 Key: HBASE-12468
 URL: https://issues.apache.org/jira/browse/HBASE-12468
 Project: HBase
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.98.6.1
Reporter: Kevin Odell
Assignee: Misty Stanley-Jones
 Attachments: HBASE-12468-v1.patch, HBASE-12468.patch


 Per https://issues.apache.org/jira/browse/HBASE-12346 you need to use 
 AUTHORIZATIONS or setAuthorizations to see your labels. We may want to update 
 http://hbase.apache.org/book/ch08s03.html with that information



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12468) AUTHORIZATIONS should be part of Visibility Label Docs


 [ 
https://issues.apache.org/jira/browse/HBASE-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-12468:

Status: Patch Available  (was: Open)

 AUTHORIZATIONS should be part of Visibility Label Docs
 --

 Key: HBASE-12468
 URL: https://issues.apache.org/jira/browse/HBASE-12468
 Project: HBase
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.98.6.1
Reporter: Kevin Odell
Assignee: Misty Stanley-Jones
 Attachments: HBASE-12468-v1.patch, HBASE-12468.patch


 Per https://issues.apache.org/jira/browse/HBASE-12346 you need to use 
 AUTHORIZATIONS or setAuthorizations to see your labels. We may want to update 
 http://hbase.apache.org/book/ch08s03.html with that information



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13132) Improve RemoveColumn action debug message

2015-03-01 Thread Sean Busbey (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342471#comment-14342471
 ] 

Sean Busbey commented on HBASE-13132:
-

+1 lgtm

 Improve RemoveColumn action debug message 
 --

 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: hbase-13132.patch


 Trivial fix for this unsightly log message:
 {code}
 2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] 
 actions.Action: Performing action: Removing [B@64275127 from 
 TestAcidGuarantees
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-13133) NPE when running TestSplitLogManager


 [ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Stepachev reassigned HBASE-13133:


Assignee: Andrey Stepachev

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13133) NPE when running TestSplitLogManager


 [ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Stepachev updated HBASE-13133:
-
Attachment: HBASE-13133.patch

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342577#comment-14342577
 ] 

Andrey Stepachev commented on HBASE-13133:
--

[~Apache9], can you check that this patch can help. thanks.

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342578#comment-14342578
]

stack commented on HBASE-13082:
---

Other notes. I tried the HBASE-13071 patch. It had a profile like this one,
almost as good (scan next count and lowered GC profile) so a postive there. I
then tried compounding this patch and HBASE-13071 but its like we are up
against a wall. I doubled the scanner count and next counts actually went down
(GC went up). Thread dumping gives little clue. Let me add profiler to figure
more on why.

Coarsen StoreScanner locks to RegionScanner
---

Key: HBASE-13082
URL: https://issues.apache.org/jira/browse/HBASE-13082
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 13082-test.txt, 13082.txt, 13082.txt, gc.png, next.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342043#comment-14342043
 ] 

Hudson commented on HBASE-13131:


SUCCESS: Integrated in HBase-1.0 #779 (See 
[https://builds.apache.org/job/HBase-1.0/779/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev 9769fb6d94d2ba3e41b1ff288d15a9a6d32ea9df)
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342059#comment-14342059
 ] 

Hudson commented on HBASE-13131:


FAILURE: Integrated in HBase-0.94-JDK7 #229 (See 
[https://builds.apache.org/job/HBase-0.94-JDK7/229/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev ec8cfe14f1a558b5da1d1987d53cccf3ce0bd5f4)
* src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342062#comment-14342062
 ] 

Hudson commented on HBASE-13131:


FAILURE: Integrated in HBase-TRUNK #6188 (See 
[https://builds.apache.org/job/HBase-TRUNK/6188/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev dad2474f08d201d09989e36f5cf1c25d3fa4acee)
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342061#comment-14342061
 ] 

Hudson commented on HBASE-13131:


FAILURE: Integrated in HBase-0.94 #1461 (See 
[https://builds.apache.org/job/HBase-0.94/1461/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev ec8cfe14f1a558b5da1d1987d53cccf3ce0bd5f4)
* src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342187#comment-14342187
 ] 

Hudson commented on HBASE-13131:


SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #833 (See 
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/833/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev 28032b3daa720caac034c01187d7096b0d9bc390)
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eshcar Hillel updated HBASE-13071:
--
Attachment: (was: HBASE-13071-v1.patch)

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBaseStreamingScanDesign.pdf,
HbaseStreamingScanEvaluation.pdf

A scan operation iterates over all rows of a table or a subrange of the
table. The synchronous nature in which the data is served at the client side
hinders the speed the application traverses the data: it increases the
overall processing time, and may cause a great variance in the times the
application waits for the next piece of data.
The scanner next() method at the client side invokes an RPC to the
regionserver and then stores the results in a cache. The application can
specify how many rows will be transmitted per RPC; by default this is set to
100 rows.
The cache can be considered as a producer-consumer queue, where the hbase
client pushes the data to the queue and the application consumes it.
Currently this queue is synchronous, i.e., blocking. More specifically, when
the application consumed all the data from the cache --- so the cache is
empty --- the hbase client retrieves additional data from the server and
re-fills the cache with new data. During this time the application is blocked.
Under the assumption that the application processing time can be balanced by
the time it takes to retrieve the data, an asynchronous approach can reduce
the time the application is waiting for data.
We attach a design document.
We also have a patch that is based on a private branch, and some evaluation
results of this code.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eshcar Hillel updated HBASE-13071:
--
Attachment: (was: HBASE-13071-v2.patch)

Hbase Streaming Scan Feature

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eshcar Hillel updated HBASE-13071:
--
Attachment: HBASE-13071_98_1.patch

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBASE-13071_98_1.patch, HBaseStreamingScanDesign.pdf,
HbaseStreamingScanEvaluation.pdf

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eshcar Hillel updated HBASE-13071:
--
Attachment: HBASE-13071_trunk_1.patch

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch,
HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature


[ 
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342129#comment-14342129
 ] 

Hadoop QA commented on HBASE-13071:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12701690/HBASE-13071_trunk_1.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701690

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:red}-1 checkstyle{color}.  The applied patch generated 
1946 checkstyle errors (more than the master's current 1937 errors).

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+ClusterConnection connection, 
RpcRetryingCallerFactory rpcCallerFactory,
+
super(configuration,scan,name,connection,rpcCallerFactory,rpcControllerFactory,pool,replicaCallTimeoutMicroSecondScan);
+  public ClientSimpleScanner(Configuration configuration, Scan scan, TableName 
name, ClusterConnection connection,
+ RpcRetryingCallerFactory rpcCallerFactory, 
RpcControllerFactory rpcControllerFactory,
+ ExecutorService pool, int 
replicaCallTimeoutMicroSecondScan) throws IOException {
+
super(configuration,scan,name,connection,rpcCallerFactory,rpcControllerFactory,pool,replicaCallTimeoutMicroSecondScan);
+  public static final String HBASE_CLIENT_SCANNER_ASYNC_PREFETCH = 
hbase.client.scanner.async.prefetch;

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations
  org.apache.hadoop.hbase.client.TestOperation

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//artifact/patchprocess/checkstyle-aggregate.html

Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13021//console

This message is automatically generated.

 Hbase Streaming Scan Feature
 

 Key: HBASE-13071
 URL: https://issues.apache.org/jira/browse/HBASE-13071
 Project: HBase
  Issue Type: New Feature
Affects Versions: 0.98.11
Reporter:

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342052#comment-14342052
 ] 

Hudson commented on HBASE-13131:


SUCCESS: Integrated in HBase-0.94-security #574 (See 
[https://builds.apache.org/job/HBase-0.94-security/574/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev ec8cfe14f1a558b5da1d1987d53cccf3ce0bd5f4)
* src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor


[ 
https://issues.apache.org/jira/browse/HBASE-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342110#comment-14342110
 ] 

Hudson commented on HBASE-13131:


FAILURE: Integrated in HBase-0.98 #875 (See 
[https://builds.apache.org/job/HBase-0.98/875/])
HBASE-13131 ReplicationAdmin must clean up connections if constructor fails. 
(busbey: rev 28032b3daa720caac034c01187d7096b0d9bc390)
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java


 ReplicationAdmin leaks connections if there's an error in the constructor
 -

 Key: HBASE-13131
 URL: https://issues.apache.org/jira/browse/HBASE-13131
 Project: HBase
  Issue Type: Bug
  Components: Replication
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
 Fix For: 2.0.0, 1.0.1, 1.1.0, 0.94.27, 0.98.11

 Attachments: HBASE-13131-0.94.1.patch.txt, HBASE-13131.1.patch.txt


 We do some work in the ReplicationAdmin constructor after creating 
 connections to ZK and the HBase cluster. If any part of that work throws an 
 exception that we propagate out of the constructor (like initializing peers), 
 then we need to clean up before we do so.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342109#comment-14342109
]

Eshcar Hillel commented on HBASE-13071:
---

New patches for 0.98 and trunk are available.
Link to review board.

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch,
HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342581#comment-14342581
]

Lars Hofhansl commented on HBASE-13082:
---

Sorry, you saying after HBASE-13071 this patch makes things slower? How'd you
test? Same way you tested yesterday?

Coarsen StoreScanner locks to RegionScanner
---

Key: HBASE-13082
URL: https://issues.apache.org/jira/browse/HBASE-13082
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 13082-test.txt, 13082.txt, 13082.txt, gc.png, next.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12466) Document visibility scan label generator usage and behavior


 [ 
https://issues.apache.org/jira/browse/HBASE-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-12466:

Attachment: HBASE-12466.patch

I needed to merge the work from HBASE-12468 into this patch, because I needed 
to create an anchor and link to it for this one.

 Document visibility scan label generator usage and behavior
 ---

 Key: HBASE-12466
 URL: https://issues.apache.org/jira/browse/HBASE-12466
 Project: HBase
  Issue Type: Task
  Components: documentation, security
Reporter: Jerry He
Assignee: Misty Stanley-Jones
 Attachments: HBASE-12466.patch


 Document the SLGs we provide and their behavior.
 Document the out-of-box default behavior.
 Document how to configure SLG if non-default behavior is desired. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12466) Document visibility scan label generator usage and behavior


 [ 
https://issues.apache.org/jira/browse/HBASE-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-12466:

Status: Patch Available  (was: Open)

 Document visibility scan label generator usage and behavior
 ---

 Key: HBASE-12466
 URL: https://issues.apache.org/jira/browse/HBASE-12466
 Project: HBase
  Issue Type: Task
  Components: documentation, security
Reporter: Jerry He
Assignee: Misty Stanley-Jones
 Attachments: HBASE-12466.patch


 Document the SLGs we provide and their behavior.
 Document the out-of-box default behavior.
 Document how to configure SLG if non-default behavior is desired. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces

2015-03-01 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-12975:
---
Status: Open  (was: Patch Available)

 Supportable SplitTransaction and RegionMergeTransaction interfaces
 --

 Key: HBASE-12975
 URL: https://issues.apache.org/jira/browse/HBASE-12975
 Project: HBase
  Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Andrew Purtell
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-12975.patch, HBASE-12975.patch


 Making SplitTransaction, RegionMergeTransaction limited private is required 
 to support local indexing feature in Phoenix to ensure regions colocation. 
 We can ensure region split, regions merge in the coprocessors in few method 
 calls without touching internals like creating zk's, file layout changes or 
 assignments.
 1) stepsBeforePONR, stepsAfterPONR we can ensure split.
 2) meta entries can pass through coprocessors to atomically update with the 
 normal split/merge.
 3) rollback on failure.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (HBASE-12975) Supportable SplitTransaction and RegionMergeTransaction interfaces

2015-03-01 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell updated HBASE-12975:
---
Comment: was deleted

(was: {color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701528/HBASE-12975.patch
  against master branch at commit bec2b0d320554b0af8c891fddc147a953f35765f.
  ATTACHMENT ID: 12701528

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 36 new 
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:red}-1 checkstyle{color}.  The applied patch generated 
1941 checkstyle errors (more than the master's current 1938 errors).

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+  public static final String MERGE_TRANSACTION_IMPL_KEY = 
hbase.regionserver.merge.transaction.impl;
+  public RegionMergeTransactionImpl create(final HRegion a, final HRegion b, 
final boolean forcible) {
+  public static final String SPLIT_TRANSACTION_IMPL_KEY = 
hbase.regionserver.split.transaction.impl;
+expectedReferenceFileCount != 
FSUtils.getRegionReferenceFileCount(this.parent.getFilesystem(), dir)) {
+  private PairPath, Path splitStoreFile(final byte[] family, final StoreFile 
sf) throws IOException {
+  private SplitTransactionImpl prepareGOOD_SPLIT_ROW(final HRegion 
parentRegion) throws IOException {

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.regionserver.TestSplitTransaction
  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
  
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/checkstyle-aggregate.html

Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13009//console

This message is automatically generated.)

 Supportable SplitTransaction and RegionMergeTransaction interfaces
 --

 Key: HBASE-12975
 URL: https://issues.apache.org/jira/browse/HBASE-12975

[jira] [Commented] (HBASE-7126) Update website with info on how to report security bugs

[
https://issues.apache.org/jira/browse/HBASE-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342637#comment-14342637
]

Hadoop QA commented on HBASE-7126:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701748/HBASE-7126.patch
against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
ATTACHMENT ID: 12701748

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+Please use link:https://issues.apache.org/jira/browse/hbase[JIRA] to
report non-security-related bugs.
+To protect existing HBase installations from new vulnerabilities, please *do
not* use JIRA to report security-related bugs. Instead, send your report to the
mailing list priv...@apache.org, which allows anyone to send messages, but
restricts who can read them. Someone on that list will contact you to follow up
on your report.
+NOTE: To protect existing HBase installations from new vulnerabilities, please
*do not* use JIRA to report security-related bugs. Instead, send your report to
the mailing list priv...@apache.org, which allows anyone to send messages, but
restricts who can read them. Someone on that list will contact you to follow up
on your report.

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//artifact/patchprocess/checkstyle-aggregate.html

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13023//console

This message is automatically generated.

Update website with info on how to report security bugs

Key: HBASE-7126
URL: https://issues.apache.org/jira/browse/HBASE-7126
Project: HBase
Issue Type: Task
Components: documentation
Reporter: Eli Collins
Assignee: Misty Stanley-Jones
Priority: Critical
Labels: website
Attachments: HBASE-7126.patch

The

[jira] [Updated] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


 [ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-13134:
---
Fix Version/s: 1.1.0
   2.0.0

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342644#comment-14342644
 ] 

zhangduo commented on HBASE-13133:
--

This patch fix the NullPointer, and testOrphanTaskAcquisition does not fail any 
more.
But testGetPreviousRecoveryMode is still failing with 

{noformat}
java.lang.AssertionError: Mode4=LOG_SPLITTING
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.hadoop.hbase.master.TestSplitLogManager.testGetPreviousRecoveryMode(TestSplitLogManager.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

{noformat}

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide


 [ 
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13135:

Status: Patch Available  (was: Open)

 Move replication ops mgmt stuff from Javadoc to Ref Guide
 -

 Key: HBASE-13135
 URL: https://issues.apache.org/jira/browse/HBASE-13135
 Project: HBase
  Issue Type: Bug
  Components: documentation, Replication
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13135.patch


 As per discussion with [~jmhsieh] and [~saint@gmail.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide


 [ 
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13135:

Attachment: HBASE-13135.patch

Please review. cc [~jdcryans]

 Move replication ops mgmt stuff from Javadoc to Ref Guide
 -

 Key: HBASE-13135
 URL: https://issues.apache.org/jira/browse/HBASE-13135
 Project: HBase
  Issue Type: Bug
  Components: documentation, Replication
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13135.patch


 As per discussion with [~jmhsieh] and [~saint@gmail.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13052) Explain each region split policy


[ 
https://issues.apache.org/jira/browse/HBASE-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342681#comment-14342681
 ] 

Hadoop QA commented on HBASE-13052:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701760/HBASE-13052-v1.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701760

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+HBase includes five different region split policies. In addition, you can 
create your own split policy. See region.split.policies.custom. To 
configure a split policy globally or for a given table, see 
region.split.policy.configure.
+* 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html[IncreasingToUpperBoundRegionSplitPolicy]
 -- the default split policy since HBase 0.94. Split regions based upon the 
size of the store files, but split more aggressively depending on the number of 
regions of the same table hosted on a given RegionServer.
+* 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.html[KeyPrefixRegionSplitPolicy]
 -- Extends 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html[IncreasingToUpperBoundRegionSplitPolicy].
 You can determine a given prefix length, and rows that share the same prefix 
up to that length are always kept together.
+* 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.html[DelimitedKeyPrefixRegionSplitPolicy]
 -- Extends 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html[IncreasingToUpperBoundRegionSplitPolicy].
 If your row-keys are delimited, for instance, with underscores as in 
`userid_eventtype_eventid`, this split policy ensures that all rows starting 
with the same `userid` are grouped together during splits.
+* 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.html[ConstantSizeRegionSplitPolicy]
 -- the default split policy in HBase 0.94 and earlier. Split regions based 
only upon the size of the store file.
+* 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.html[DisabledRegionSplitPolicy]
 -- disable splitting for this table. Not recommended!
+* Are your row keys chunked by common prefixes that are useful when 
scanning? Consider 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.html[KeyPrefixRegionSplitPolicy].
+* Are your row keys delimited by specific patterns that are useful when 
scanning? Consider 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.html[DelimitedKeyPrefixRegionSplitPolicy].
+* For a given table, do different columns hold cells of radically different 
sizes? Consider 
ink:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.html[IncreasingToUpperBoundRegionSplitPolicy].
+* Do your needs fall outside the scope of any of the existing region split 
policies? In this case, consider implementing your own 
region.split.policies.custom,custom split policy.

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13025//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13025//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html

[jira] [Updated] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning


 [ 
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-13109:
--
Attachment: 13109-trunk-v2.txt

New patch.
# fixes the test failures
# avoids the array creation for comparison (compares the indexed key with the 
next key in place)

 Make better SEEK vs SKIP decisions during scanning
 --

 Key: HBASE-13109
 URL: https://issues.apache.org/jira/browse/HBASE-13109
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
 Attachments: 13109-trunk-v2.txt, 13109-trunk.txt


 I'm re-purposing this issue to add a heuristic as to when to SEEK and when to 
 SKIP Cells. This has come up in various issues, and I think I have a way to 
 finally fix this now. HBASE-9778, HBASE-12311, and friends are related.
 --- Old description ---
 This is a continuation of HBASE-9778.
 We've seen a scenario of a very slow scan over a region using a timerange 
 that happens to fall after the ts of any Cell in the region.
 Turns out we spend a lot of time seeking.
 Tested with a 5 column table, and the scan is 5x faster when the timerange 
 falls before all Cells' ts.
 We can use the lookahead hint introduced in HBASE-9778 to do opportunistic 
 SKIPing before we actually seek.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342726#comment-14342726
 ] 

Hadoop QA commented on HBASE-13134:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701767/HBASE-13134.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701767

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 8 new 
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at 
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor.testRegionMerge(TestNamespaceAuditor.java:308)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13027//console

This message is automatically generated.

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342731#comment-14342731
 ] 

Ted Yu commented on HBASE-13134:


Integrated to branch-1 and master branch.

Thanks for the patch, Francis.

Ping [~apurtell] and [~enis]

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13136) TestSplitLogManager.testGetPreviousRecoveryMode is flakey

zhangduo created HBASE-13136:


 Summary: TestSplitLogManager.testGetPreviousRecoveryMode is flakey
 Key: HBASE-13136
 URL: https://issues.apache.org/jira/browse/HBASE-13136
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo


Add test code to run it 100 times then we can make it fail always.
{code:title=TestSplitLogManager.java}
  @Test
  public void test() throws Exception {
for (int i = 0; i  100; i++) {
  setup();
  testGetPreviousRecoveryMode();
  teardown();
}
  }
{code}

Add then add some ugly debug logs(Yeah I usually debug in this way...)
{code:title=ZKSplitLogManagerCoordination.java}
  @Override
  public void setRecoveryMode(boolean isForInitialization) throws IOException {
synchronized(this) {
  if (this.isDrainingDone) {
// when there is no outstanding splitlogtask after master start up, we 
already have up to 
// date recovery mode
return;
  }
}
if (this.watcher == null) {
  // when watcher is null(testing code) and recovery mode can only be 
LOG_SPLITTING
  synchronized(this) {
this.isDrainingDone = true;
this.recoveryMode = RecoveryMode.LOG_SPLITTING;
  }
  return;
}
boolean hasSplitLogTask = false;
boolean hasRecoveringRegions = false;
RecoveryMode previousRecoveryMode = RecoveryMode.UNKNOWN;
RecoveryMode recoveryModeInConfig =
(isDistributedLogReplay(conf)) ? RecoveryMode.LOG_REPLAY : 
RecoveryMode.LOG_SPLITTING;

// Firstly check if there are outstanding recovering regions
try {
  ListString regions = ZKUtil.listChildrenNoWatch(watcher, 
watcher.recoveringRegionsZNode);
  LOG.debug(=== + regions);
  if (regions != null  !regions.isEmpty()) {
hasRecoveringRegions = true;
previousRecoveryMode = RecoveryMode.LOG_REPLAY;
  }
  if (previousRecoveryMode == RecoveryMode.UNKNOWN) {
// Secondly check if there are outstanding split log task
ListString tasks = ZKUtil.listChildrenNoWatch(watcher, 
watcher.splitLogZNode);
LOG.debug(=== + tasks);
if (tasks != null  !tasks.isEmpty()) {
  hasSplitLogTask = true;
  if (isForInitialization) {
// during initialization, try to get recovery mode from splitlogtask
int listSize = tasks.size();
for (int i = 0; i  listSize; i++) {
  String task = tasks.get(i);
  try {
byte[] data =
ZKUtil.getData(this.watcher, 
ZKUtil.joinZNode(watcher.splitLogZNode, task));
if (data == null) continue;
SplitLogTask slt = SplitLogTask.parseFrom(data);
previousRecoveryMode = slt.getMode();
if (previousRecoveryMode == RecoveryMode.UNKNOWN) {
  // created by old code base where we don't set recovery mode 
in splitlogtask
  // we can safely set to LOG_SPLITTING because we're in master 
initialization code
  // before SSH is enabled  there is no outstanding recovering 
regions
  previousRecoveryMode = RecoveryMode.LOG_SPLITTING;
}
break;
  } catch (DeserializationException e) {
LOG.warn(Failed parse data for znode  + task, e);
  } catch (InterruptedException e) {
throw new InterruptedIOException();
  }
}
  }
}
  }
} catch (KeeperException e) {
  throw new IOException(e);
}

synchronized (this) {
  if (this.isDrainingDone) {
return;
  }
  if (!hasSplitLogTask  !hasRecoveringRegions) {
this.isDrainingDone = true;
LOG.debug(set to  + recoveryModeInConfig);
this.recoveryMode = recoveryModeInConfig;
return;
  } else if (!isForInitialization) {
// splitlogtask hasn't drained yet, keep existing recovery mode
return;
  }

  if (previousRecoveryMode != RecoveryMode.UNKNOWN) {
LOG.debug(set to  + previousRecoveryMode);
this.isDrainingDone = (previousRecoveryMode == recoveryModeInConfig);
this.recoveryMode = previousRecoveryMode;
  } else {
LOG.debug(set to  + recoveryModeInConfig);
this.recoveryMode = recoveryModeInConfig;
  }
}
  }
{code}

When failing, I got this
{noformat}
2015-03-02 12:26:12,555 INFO  [main] master.TestSplitLogManager(654): 
testGetPreviousRecoveryMode
2015-03-02 12:26:12,600 DEBUG [main-EventThread] 
zookeeper.ZooKeeperWatcher(388): 
split-log-manager-tests465da653-db4c-475b-bf73-91ac0ca13323-0x14bd8bbf5d9, 
quorum=localhost:53589, baseZNode=/hbase Received ZooKeeper Event, type=None, 
state=Disconnected, path=null
2015-03-02 12:26:12,600 DEBUG [main-EventThread] 
zookeeper.ZooKeeperWatcher(477):

[jira] [Commented] (HBASE-13136) TestSplitLogManager.testGetPreviousRecoveryMode is flakey


[ 
https://issues.apache.org/jira/browse/HBASE-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342739#comment-14342739
 ] 

zhangduo commented on HBASE-13136:
--

Experts needed. [~stack] [~jeffreyz]

 TestSplitLogManager.testGetPreviousRecoveryMode is flakey
 -

 Key: HBASE-13136
 URL: https://issues.apache.org/jira/browse/HBASE-13136
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo

 Add test code to run it 100 times then we can make it fail always.
 {code:title=TestSplitLogManager.java}
   @Test
   public void test() throws Exception {
 for (int i = 0; i  100; i++) {
   setup();
   testGetPreviousRecoveryMode();
   teardown();
 }
   }
 {code}
 Add then add some ugly debug logs(Yeah I usually debug in this way...)
 {code:title=ZKSplitLogManagerCoordination.java}
   @Override
   public void setRecoveryMode(boolean isForInitialization) throws IOException 
 {
 synchronized(this) {
   if (this.isDrainingDone) {
 // when there is no outstanding splitlogtask after master start up, 
 we already have up to 
 // date recovery mode
 return;
   }
 }
 if (this.watcher == null) {
   // when watcher is null(testing code) and recovery mode can only be 
 LOG_SPLITTING
   synchronized(this) {
 this.isDrainingDone = true;
 this.recoveryMode = RecoveryMode.LOG_SPLITTING;
   }
   return;
 }
 boolean hasSplitLogTask = false;
 boolean hasRecoveringRegions = false;
 RecoveryMode previousRecoveryMode = RecoveryMode.UNKNOWN;
 RecoveryMode recoveryModeInConfig =
 (isDistributedLogReplay(conf)) ? RecoveryMode.LOG_REPLAY : 
 RecoveryMode.LOG_SPLITTING;
 // Firstly check if there are outstanding recovering regions
 try {
   ListString regions = ZKUtil.listChildrenNoWatch(watcher, 
 watcher.recoveringRegionsZNode);
   LOG.debug(=== + regions);
   if (regions != null  !regions.isEmpty()) {
 hasRecoveringRegions = true;
 previousRecoveryMode = RecoveryMode.LOG_REPLAY;
   }
   if (previousRecoveryMode == RecoveryMode.UNKNOWN) {
 // Secondly check if there are outstanding split log task
 ListString tasks = ZKUtil.listChildrenNoWatch(watcher, 
 watcher.splitLogZNode);
 LOG.debug(=== + tasks);
 if (tasks != null  !tasks.isEmpty()) {
   hasSplitLogTask = true;
   if (isForInitialization) {
 // during initialization, try to get recovery mode from 
 splitlogtask
 int listSize = tasks.size();
 for (int i = 0; i  listSize; i++) {
   String task = tasks.get(i);
   try {
 byte[] data =
 ZKUtil.getData(this.watcher, 
 ZKUtil.joinZNode(watcher.splitLogZNode, task));
 if (data == null) continue;
 SplitLogTask slt = SplitLogTask.parseFrom(data);
 previousRecoveryMode = slt.getMode();
 if (previousRecoveryMode == RecoveryMode.UNKNOWN) {
   // created by old code base where we don't set recovery 
 mode in splitlogtask
   // we can safely set to LOG_SPLITTING because we're in 
 master initialization code
   // before SSH is enabled  there is no outstanding 
 recovering regions
   previousRecoveryMode = RecoveryMode.LOG_SPLITTING;
 }
 break;
   } catch (DeserializationException e) {
 LOG.warn(Failed parse data for znode  + task, e);
   } catch (InterruptedException e) {
 throw new InterruptedIOException();
   }
 }
   }
 }
   }
 } catch (KeeperException e) {
   throw new IOException(e);
 }
 synchronized (this) {
   if (this.isDrainingDone) {
 return;
   }
   if (!hasSplitLogTask  !hasRecoveringRegions) {
 this.isDrainingDone = true;
 LOG.debug(set to  + recoveryModeInConfig);
 this.recoveryMode = recoveryModeInConfig;
 return;
   } else if (!isForInitialization) {
 // splitlogtask hasn't drained yet, keep existing recovery mode
 return;
   }
   if (previousRecoveryMode != RecoveryMode.UNKNOWN) {
 LOG.debug(set to  + previousRecoveryMode);
 this.isDrainingDone = (previousRecoveryMode == recoveryModeInConfig);
 this.recoveryMode = previousRecoveryMode;
   } else {
 LOG.debug(set to  + recoveryModeInConfig);
 this.recoveryMode = recoveryModeInConfig;
   }
 }
   }
 {code}
 When failing, I got this
 {noformat}
 2015-03-02 12:26:12,555 INFO  [main] master.TestSplitLogManager(654): 
 testGetPreviousRecoveryMode
 2015-03-02

[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner


[ 
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342740#comment-14342740
 ] 

Lars Hofhansl commented on HBASE-13082:
---

Not if it makes things slower :)
Not sure how not taking a bunch of locks and deferring to an already existing 
lock could ever make things slower - *unless* the lock is contended and we 
increased the contention this way (which we're not).


 Coarsen StoreScanner locks to RegionScanner
 ---

 Key: HBASE-13082
 URL: https://issues.apache.org/jira/browse/HBASE-13082
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
 Attachments: 13082-test.txt, 13082.txt, 13082.txt, gc.png, gc.png, 
 next.png, next.png


 Continuing where HBASE-10015 left of.
 We can avoid locking (and memory fencing) inside StoreScanner by deferring to 
 the lock already held by the RegionScanner.
 In tests this shows quite a scan improvement and reduced CPU (the fences make 
 the cores wait for memory fetches).
 There are some drawbacks too:
 * All calls to RegionScanner need to be remain synchronized
 * Implementors of coprocessors need to be diligent in following the locking 
 contract. For example Phoenix does not lock RegionScanner.nextRaw() and 
 required in the documentation (not picking on Phoenix, this one is my fault 
 as I told them it's OK)
 * possible starving of flushes and compaction with heavy read load. 
 RegionScanner operations would keep getting the locks and the 
 flushes/compactions would not be able finalize the set of files.
 I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342755#comment-14342755
]

stack commented on HBASE-13082:
---

Argh. It makes things faster. So does HBASE-13071. Its when I put the two
together that I see slowdown. I'll look more into it.

Coarsen StoreScanner locks to RegionScanner
---

Key: HBASE-13082
URL: https://issues.apache.org/jira/browse/HBASE-13082
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 13082-test.txt, 13082.txt, 13082.txt, gc.png, gc.png,
next.png, next.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning


[ 
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342754#comment-14342754
 ] 

Lars Hofhansl commented on HBASE-13109:
---

Numbers with new patch (avoiding the array creation helps):
With patch:
||Wildcard||Col 2+4||
|3.9|4.4|

The ExplicitColumnTracker is now only 10% slower than the wildcard column 
tracker (was almost 2x before).


 Make better SEEK vs SKIP decisions during scanning
 --

 Key: HBASE-13109
 URL: https://issues.apache.org/jira/browse/HBASE-13109
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
 Attachments: 13109-trunk-v2.txt, 13109-trunk.txt


 I'm re-purposing this issue to add a heuristic as to when to SEEK and when to 
 SKIP Cells. This has come up in various issues, and I think I have a way to 
 finally fix this now. HBASE-9778, HBASE-12311, and friends are related.
 --- Old description ---
 This is a continuation of HBASE-9778.
 We've seen a scenario of a very slow scan over a region using a timerange 
 that happens to fall after the ts of any Cell in the region.
 Turns out we spend a lot of time seeking.
 Tested with a 5 column table, and the scan is 5x faster when the timerange 
 falls before all Cells' ts.
 We can use the lookahead hint introduced in HBASE-9778 to do opportunistic 
 SKIPing before we actually seek.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342765#comment-14342765
]

Lars Hofhansl commented on HBASE-13082:
---

Oh I get that :)
What I meant to say is that under no circumstances should fewer locks be slower
than more unless there is contention. Lemme lock at HBASE-13071 to see if
something sticks out.

Also check out HBASE-13109. Not entirely related, but it finally solves the
SKIP vs. SEEK performance issues that have been plaguing us (SEEK is
inefficient unless it can at least seek into the next block).

With these three we should see some nice perf gains for scanning.

Coarsen StoreScanner locks to RegionScanner
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13014) Java Tool For Region Moving


[ 
https://issues.apache.org/jira/browse/HBASE-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342767#comment-14342767
 ] 

Lars Hofhansl commented on HBASE-13014:
---

I'll take a look tomorrow.

 Java Tool For Region Moving 
 

 Key: HBASE-13014
 URL: https://issues.apache.org/jira/browse/HBASE-13014
 Project: HBase
  Issue Type: Improvement
Reporter: Abhishek Singh Chouhan
Assignee: Abhishek Singh Chouhan
 Attachments: HBASE-13014-v2.patch, HBASE-13014.patch


 As per discussion on HBASE-12989 we should move the functionality of 
 region_mover.rb into a Java tool and use region_mover.rb only only as a 
 wrapper around it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13095) Document how to retrieve replication stats from HBase Shell


[ 
https://issues.apache.org/jira/browse/HBASE-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342772#comment-14342772
 ] 

Hadoop QA commented on HBASE-13095:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701773/HBASE-13095.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701773

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+You can use the HBase Shell command `status 'replication'` to monitor the 
replication status on your cluster. The  command has two variations:
+* `status 'replication', 'source'` -- prints the status for each replication 
source, sorted by hostname.

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

 {color:red}-1 core zombie tests{color}.  There are 5 zombie test(s):   
at 
org.apache.hadoop.hbase.util.TestIdLock.testMultipleClients(TestIdLock.java:102)
at 
org.apache.hadoop.hbase.TestAcidGuarantees.testScanAtomicity(TestAcidGuarantees.java:348)
at 
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2.testMRIncrementalLoad(TestHFileOutputFormat2.java:373)
at 
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScanBase.testScan(TestTableInputFormatScanBase.java:238)
at 
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1.testScanEmptyToEmpty(TestTableInputFormatScan1.java:44)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13028//console

This message is automatically generated.

 Document how to retrieve replication stats from HBase Shell
 ---

 Key: HBASE-13095
 URL: https://issues.apache.org/jira/browse/HBASE-13095
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, Replication
Affects Versions: 1.0.0

[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide

[
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342776#comment-14342776
]

Hadoop QA commented on HBASE-13135:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701775/HBASE-13135.patch
against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
ATTACHMENT ID: 12701775

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+pCluster replication documentation has been moved to the
link:http://hbase.apache.org/book.html#_cluster_replication[Cluster
Replication] section of the link:http://hbase.apache.org/book.html[Apache HBase
Reference Guide]./p
+NOTE: This material was previously available in the
link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/replication/package-summary.html#requirements[Replication
API documentation].
+Several HBase Shell commands are available to assist you in managing
replication relationships between clusters (also called peers).
+ * CLUSTER_KEY: composed using the following template, with appropriate
place-holders:
`hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent`
+ Disable a replication relationship. HBase will no longer send edits to that
peer cluster, but it still keeps track of all the new WALs that it will need to
replicate if and when it is re-enabled.
+ Disable and remove a replication relationship. HBase will no longer send
edits to that peer cluster or keep track of WALs.
+The `VerifyReplication` MapReduce job, which is included in HBase, performs a
systematic comparison of replicated data between two different clusters. Run
the VerifyReplication job on the master cluster, supplying it with the peer ID
and table name to use for validation. You can limit the verification further by
specifying a time range or specific families. The job's short name is
`verifyrep`. To run the job, use a command like the following:

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13029//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle

[jira] [Commented] (HBASE-13095) Document how to retrieve replication stats from HBase Shell

2015-03-01 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342775#comment-14342775
 ] 

Ashish Singhi commented on HBASE-13095:
---

We need to also mention about *status 'replication', 'sink'* under the 
variations.
Thanks.

 Document how to retrieve replication stats from HBase Shell
 ---

 Key: HBASE-13095
 URL: https://issues.apache.org/jira/browse/HBASE-13095
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, Replication
Affects Versions: 1.0.0
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 1.0.0, 2.0.0, 1.1.0, 0.98.11

 Attachments: HBASE-13095.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342795#comment-14342795
 ] 

Hudson commented on HBASE-13134:


FAILURE: Integrated in HBase-1.1 #233 (See 
[https://builds.apache.org/job/HBase-1.1/233/])
HBASE-13134 mutateRow and checkAndMutate apis don't throw region level 
exceptions (Francis Liu) (tedyu: rev c64686e2e172f70a25ecf3cb771b68e0a3eee03c)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCheckAndMutate.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java


 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning

[
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342803#comment-14342803
]

Hadoop QA commented on HBASE-13109:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701779/13109-trunk-v2.txt
against master branch at commit 70ecf18817ef219389a9e024ff21ffb99b6615d9.
ATTACHMENT ID: 12701779

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.

{color:red}-1 checkstyle{color}. The applied patch generated
1938 checkstyle errors (more than the master's current 1937 errors).

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestBlocksRead

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/checkstyle-aggregate.html

Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13030//console

This message is automatically generated.

Make better SEEK vs SKIP decisions during scanning
--

Key: HBASE-13109
URL: https://issues.apache.org/jira/browse/HBASE-13109
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
Attachments: 13109-trunk-v2.txt, 13109-trunk.txt

I'm re-purposing this issue to add a heuristic as to when to SEEK and when to
SKIP Cells. This has come up in various issues, and I think I have a way to
finally fix this now. HBASE-9778, HBASE-12311, and friends are related.
--- Old description ---
This is a continuation of HBASE-9778.
We've seen a scenario of a very slow scan over a region using a timerange
that happens to fall after the ts of any Cell in the region.
Turns out we spend a lot of time seeking.
Tested with a 5 column table, and the scan is 5x faster when the timerange
falls before all Cells' ts.
We can use the lookahead hint introduced in HBASE-9778 to do opportunistic

[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide


[ 
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342813#comment-14342813
 ] 

stack commented on HBASE-13135:
---

+1 Looks great. Thanks [~misty]

 Move replication ops mgmt stuff from Javadoc to Ref Guide
 -

 Key: HBASE-13135
 URL: https://issues.apache.org/jira/browse/HBASE-13135
 Project: HBase
  Issue Type: Bug
  Components: documentation, Replication
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13135.patch


 As per discussion with [~jmhsieh] and [~saint@gmail.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13123) Minor bug in ROW bloom filter

2015-03-01 Thread ramkrishna.s.vasudevan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-13123:
---
   Resolution: Fixed
Fix Version/s: 1.1.0
   1.0.1
   2.0.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

 Minor bug in ROW bloom filter
 -

 Key: HBASE-13123
 URL: https://issues.apache.org/jira/browse/HBASE-13123
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: HBASE-13163.patch


 While checking the code for Bloom filter found that while checking if a key 
 passes the ROW bloom check we try to create a bloom key. The bloom key should 
 be constructed only with the row part of the key. But try to form the bloom 
 key including the meta data part of the key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13123) Minor bug in ROW bloom filter

2015-03-01 Thread ramkrishna.s.vasudevan (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342820#comment-14342820
 ] 

ramkrishna.s.vasudevan commented on HBASE-13123:


Pushed to master, branch-1 and branch-1.0.  Thanks for the review Ted.

 Minor bug in ROW bloom filter
 -

 Key: HBASE-13123
 URL: https://issues.apache.org/jira/browse/HBASE-13123
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: HBASE-13163.patch


 While checking the code for Bloom filter found that while checking if a key 
 passes the ROW bloom check we try to create a bloom key. The bloom key should 
 be constructed only with the row part of the key. But try to form the bloom 
 key including the meta data part of the key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide

2015-03-01 Thread Ashish Singhi (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342831#comment-14342831
 ] 

Ashish Singhi commented on HBASE-13135:
---

bq. +$ hadoop jar /usr/lib/hbase/hbase.jar verifyrep --starttime=timestamp
Can we use like this ?
+$ HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` 
${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-server-VERSION.jar verifyrep 
--starttime=timestamp
Thanks

 Move replication ops mgmt stuff from Javadoc to Ref Guide
 -

 Key: HBASE-13135
 URL: https://issues.apache.org/jira/browse/HBASE-13135
 Project: HBase
  Issue Type: Bug
  Components: documentation, Replication
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Attachments: HBASE-13135.patch


 As per discussion with [~jmhsieh] and [~saint@gmail.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342836#comment-14342836
 ] 

Hudson commented on HBASE-13134:


FAILURE: Integrated in HBase-TRUNK #6190 (See 
[https://builds.apache.org/job/HBase-TRUNK/6190/])
HBASE-13134 mutateRow and checkAndMutate apis don't throw region level 
exceptions (Francis Liu) (tedyu: rev 4980bfe642ef256eb9c31ff8927ab6f77d110d95)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestCheckAndMutate.java


 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning

[
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342839#comment-14342839
]

stack commented on HBASE-13109:
---

Should Scan.LOOK_AHEAD be deprecated/become a noop in case someone using it?

We need to add more compare to KV? There ain't enough going on in there already
(smile)?

getNextIndexedKey makes sense but should we be returning byte [] ? Why not
Cell? byte [] presumes a certain format?

getKeyForNextRow is commented out. Remove?

I like the way you add in this optimize method and it works or it doesn't.

When will optimize be optimal? When will it not add value ( you say selecting
2 and 4 in above is worse case but generally?) Sorry for dumb questions. I
don't know this stuff well.

Make better SEEK vs SKIP decisions during scanning
--

Key: HBASE-13109
URL: https://issues.apache.org/jira/browse/HBASE-13109
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
Attachments: 13109-trunk-v2.txt, 13109-trunk.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13125) Purge hbase.bucketcache.sizes from the refguide; no such config (should be hbase.bucketcache.size?)


[ 
https://issues.apache.org/jira/browse/HBASE-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342841#comment-14342841
 ] 

stack commented on HBASE-13125:
---

There is more to clean up around bucket cache doc going by subsequent question 
on mailing list.  See BucketCache Configuration Problem

 Purge hbase.bucketcache.sizes from the refguide; no such config (should be 
 hbase.bucketcache.size?)
 ---

 Key: HBASE-13125
 URL: https://issues.apache.org/jira/browse/HBASE-13125
 Project: HBase
  Issue Type: Bug
  Components: documentation
Reporter: stack
Priority: Trivial

 Came up on mailing list today... a gentleman saw hbase.bucketcache.sizes and 
 didn't know what it was about.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342842#comment-14342842
]

stack commented on HBASE-13071:
---

I tried this and saw improvement. Let me come back with some pretty pictures.

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch,
HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)


[ 
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342847#comment-14342847
 ] 

stack commented on HBASE-11165:
---

Some notes on having a scalable meta here: 
https://docs.google.com/document/d/1eCuqf7i2dkWHL0PxcE1HE1nLRQ_tCyXI4JsOB6TAk60/edit?usp=sharing
  Let me attach a PDF version.

 Scaling so cluster can host 1M regions and beyond (50M regions?)
 

 Key: HBASE-11165
 URL: https://issues.apache.org/jira/browse/HBASE-11165
 Project: HBase
  Issue Type: Brainstorming
Reporter: stack
 Attachments: HBASE-11165.zip, Region Scalability test.pdf, 
 zk_less_assignment_comparison_2.pdf


 This discussion issue comes out of Co-locate Meta And Master HBASE-10569 
 and comments on the doc posted there.
 A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M 
 regions maybe even 50M later.  This issue is about discussing how we will do 
 that (or if not 50M on a cluster, how otherwise we can attain same end).
 More detail to follow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11165) Scaling so cluster can host 1M regions and beyond (50M regions?)


 [ 
https://issues.apache.org/jira/browse/HBASE-11165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack updated HBASE-11165:
--
Attachment: ScalableMeta.pdf

 Scaling so cluster can host 1M regions and beyond (50M regions?)
 

 Key: HBASE-11165
 URL: https://issues.apache.org/jira/browse/HBASE-11165
 Project: HBase
  Issue Type: Brainstorming
Reporter: stack
 Attachments: HBASE-11165.zip, Region Scalability test.pdf, 
 ScalableMeta.pdf, zk_less_assignment_comparison_2.pdf


 This discussion issue comes out of Co-locate Meta And Master HBASE-10569 
 and comments on the doc posted there.
 A user -- our Francis Liu -- needs to be able to scale a cluster to do 1M 
 regions maybe even 50M later.  This issue is about discussing how we will do 
 that (or if not 50M on a cluster, how otherwise we can attain same end).
 More detail to follow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning

[
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342854#comment-14342854
]

Lars Hofhansl commented on HBASE-13109:
---

Should deprecate Scan.LOOK_AHEAD in 1.0.1, so that we can remove it in 1.1.
(per our policy that is possible)

The indexed key comes out of the HFile as a key - and yes it presumes a
KeyValue-key all over the place. :(
Translating this into a Cell would be measurably slower, could try to record it
as Cell in the first place.

The compare in KV is needed unfortunately to avoid materializing the seek key
just for this check. I did not like to write that part.

Yeah need to remove commented stuff.

Optimize is optimizing heuristically.
* many versions of KVs are spread all over the HFiles. The heuristic of
checking the top scanner might not be optimal in that case. But then too, we'd
need to seek into many files for the reset, so compared the cost should be low.
* SQM says SEEK, and optimize does not change this. In that case we wasted a
compare, that's OK, seek is *way* more expensive.
* It *is* a heuristic. In some one off cases we might be doing some SKIP before
we end up seeking.

I'd not be afraid to deploy for us in production (I am most worried that I got
the new compare method wrong... Any chance eyeballing that [~stack]?)

New patch coming to fix the test. The test is weird, setting the block size to
1 (yes, 1 byte), and then it counts the blocks loaded for Bloom filters - of
course this throws this off. I will disable this optimization for Gets anyway,
there's no point.

Make better SEEK vs SKIP decisions during scanning
--

Key: HBASE-13109
URL: https://issues.apache.org/jira/browse/HBASE-13109
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
Attachments: 13109-trunk-v2.txt, 13109-trunk.txt

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13109) Make better SEEK vs SKIP decisions during scanning


 [ 
https://issues.apache.org/jira/browse/HBASE-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated HBASE-13109:
--
Attachment: 13109-trunk-v3.txt

* passes test
* disabled optimization for Gets
* removed commented code
* minor cleanups



 Make better SEEK vs SKIP decisions during scanning
 --

 Key: HBASE-13109
 URL: https://issues.apache.org/jira/browse/HBASE-13109
 Project: HBase
  Issue Type: Bug
Reporter: Lars Hofhansl
Priority: Minor
 Attachments: 13109-trunk-v2.txt, 13109-trunk-v3.txt, 13109-trunk.txt


 I'm re-purposing this issue to add a heuristic as to when to SEEK and when to 
 SKIP Cells. This has come up in various issues, and I think I have a way to 
 finally fix this now. HBASE-9778, HBASE-12311, and friends are related.
 --- Old description ---
 This is a continuation of HBASE-9778.
 We've seen a scenario of a very slow scan over a region using a timerange 
 that happens to fall after the ts of any Cell in the region.
 Turns out we spend a lot of time seeking.
 Tested with a 5 column table, and the scan is 5x faster when the timerange 
 falls before all Cells' ts.
 We can use the lookahead hint introduced in HBASE-9778 to do opportunistic 
 SKIPing before we actually seek.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-13026) Wrong error message in case incorrect snapshot name OR Incorrect table name

2015-03-01 Thread Bhupendra Kumar Jain (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhupendra Kumar Jain reassigned HBASE-13026:


Assignee: Bhupendra Kumar Jain

 Wrong error message in case incorrect snapshot name OR Incorrect table name
 ---

 Key: HBASE-13026
 URL: https://issues.apache.org/jira/browse/HBASE-13026
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Bhupendra Kumar Jain
Assignee: Bhupendra Kumar Jain
Priority: Minor
 Fix For: 1.0.0, 2.0.0, 0.98.11

 Attachments: 
 0001-HBASE-13026-Wrong-error-message-in-case-incorrect-sn.patch


 hbase(main):009:0 snapshot 't1', '.t1Snapshot'
 ERROR: Illegal first character 46 at 0. {color:red}*Namespaces*{color} can 
 only start with alphanumeric characters': i.e. [a-zA-Z_0-9]: t1Snapshot
 hbase(main):008:0 create '-test' , 'cf1'
 ERROR: Illegal first character 45 at 0.{color:red}*Namespaces*{color} can 
 only start with alphanumeric characters': i.e. [a-zA-Z_0-9]: -test
  As per message Namespaces is wrong. But in this scenario, snapshot / 
  table name start character is wrong.
 Its because in the code the message is as below
 {code}
 if (qualifierName[start] == '.' || qualifierName[start] == '-') {
   throw new IllegalArgumentException(Illegal first character  + 
 qualifierName[0] +
   at 0. Namespaces can only start 
 with alphanumeric  +
  characters': i.e. [a-zA-Z_0-9]:  +
  Bytes.toString(qualifierName));
 {code}
 The correct code should be as below
 {code}
 if (qualifierName[start] == '.' || qualifierName[start] == '-') {
   throw new IllegalArgumentException(Illegal first character  + 
 qualifierName[start] +
   at 0.  + (isSnapshot ? 
 Snapshot : User-space table) +
   qualifiers can only start with 
 'alphanumeric  +
  characters': i.e. [a-zA-Z_0-9]:  +
  Bytes.toString(qualifierName, start, 
 end));
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions

2015-03-01 Thread Francis Liu (JIRA)

Francis Liu created HBASE-13134:
---

 Summary: mutateRow and checkAndMutate apis don't throw region 
level exceptions
 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu


The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
after getting a response it does not go through returned region action result 
list to check for exceptions. 

Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342645#comment-14342645
 ] 

Ted Yu commented on HBASE-13134:


+1

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13135) Move replication ops mgmt stuff from Javadoc to Ref Guide

Misty Stanley-Jones created HBASE-13135:
---

 Summary: Move replication ops mgmt stuff from Javadoc to Ref Guide
 Key: HBASE-13135
 URL: https://issues.apache.org/jira/browse/HBASE-13135
 Project: HBase
  Issue Type: Bug
  Components: documentation, Replication
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones


As per discussion with [~jmhsieh] and [~saint@gmail.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13095) Document how to retrieve replication stats from HBase Shell


 [ 
https://issues.apache.org/jira/browse/HBASE-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13095:

Status: Patch Available  (was: Open)

 Document how to retrieve replication stats from HBase Shell
 ---

 Key: HBASE-13095
 URL: https://issues.apache.org/jira/browse/HBASE-13095
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, Replication
Affects Versions: 1.0.0
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 2.0.0, 1.1.0, 0.98.11, 1.0.0

 Attachments: HBASE-13095.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13095) Document how to retrieve replication stats from HBase Shell


 [ 
https://issues.apache.org/jira/browse/HBASE-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones updated HBASE-13095:

Attachment: HBASE-13095.patch

Please review.

 Document how to retrieve replication stats from HBase Shell
 ---

 Key: HBASE-13095
 URL: https://issues.apache.org/jira/browse/HBASE-13095
 Project: HBase
  Issue Type: Sub-task
  Components: documentation, Replication
Affects Versions: 1.0.0
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 1.0.0, 2.0.0, 1.1.0, 0.98.11

 Attachments: HBASE-13095.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13014) Java Tool For Region Moving

2015-03-01 Thread Hari Krishna Dara (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342677#comment-14342677
 ] 

Hari Krishna Dara commented on HBASE-13014:
---

+1 for using a proper review tool, it is too painful to have to copy paste the 
code snippets here.

A quick look at the 2nd patch shows more {{close()}} calls outside finally 
blocks. I see {{stripServer()}} is still the same, I recommend simplifying it 
for the sake of better understanding and maintainability. There are others that 
haven't changed like {{new String()}}. It is getting difficult to track which 
review comments have been incorporated and which haven't been.

 Java Tool For Region Moving 
 

 Key: HBASE-13014
 URL: https://issues.apache.org/jira/browse/HBASE-13014
 Project: HBase
  Issue Type: Improvement
Reporter: Abhishek Singh Chouhan
Assignee: Abhishek Singh Chouhan
 Attachments: HBASE-13014-v2.patch, HBASE-13014.patch


 As per discussion on HBASE-12989 we should move the functionality of 
 region_mover.rb into a Java tool and use region_mover.rb only only as a 
 wrapper around it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342673#comment-14342673
 ] 

zhangduo commented on HBASE-13133:
--

+1 on commit this patch first.

I will open another issue to address testGetPreviousRecoveryMode if I find 
something.

Thanks.

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12468) AUTHORIZATIONS should be part of Visibility Label Docs

[
https://issues.apache.org/jira/browse/HBASE-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342676#comment-14342676
]

Hadoop QA commented on HBASE-12468:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12701759/HBASE-12468-v1.patch
against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
ATTACHMENT ID: 12701759

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.

{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors

{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+When you issue a Scan or Get, HBase uses your default set of
authorizations to filter out cells that you do not have access to. A superuser
can see the default set of authorizations for a given user by using the
`set_auths` HBase Shell command or the
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/security/visibility/VisibilityClient.htmlsgetAuths%28org.apache.hadoop.conf.Configuration,%20java.lang.String%29[setAuths()]
method.
+You can specify a different authorization during the Scan or Get, by passing
the AUTHORIZATIONS option in HBase Shell, or the
link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setAuthorizations%28org.apache.hadoop.hbase.security.visibility.Authorizations%29[setAuthorizations()]
method if you use the API. This authorization will be combined with your
default set as an additional filter. It will further filter your results,
rather than giving you additional authorization.

{color:green}+1 site{color}. The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}. The patch passed unit tests in .

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//artifact/patchprocess/checkstyle-aggregate.html

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13024//console

This message is automatically generated.

AUTHORIZATIONS should be part of Visibility Label Docs
--

Key: HBASE-12468
URL: https://issues.apache.org/jira/browse/HBASE-12468
Project: HBase
Issue Type: Bug
Components: documentation

[jira] [Commented] (HBASE-12466) Document visibility scan label generator usage and behavior


[ 
https://issues.apache.org/jira/browse/HBASE-12466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342685#comment-14342685
 ] 

Hadoop QA commented on HBASE-12466:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701761/HBASE-12466.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701761

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+When you issue a Scan or Get, HBase uses your default set of 
authorizations to filter out cells that you do not have access to. A superuser 
can see the default set of authorizations for a given user by using the 
`set_auths` HBase Shell command or the 
link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/security/visibility/VisibilityClient.htmlsgetAuths%28org.apache.hadoop.conf.Configuration,%20java.lang.String%29[setAuths()]
 method. 
+You can specify a different authorization during the Scan or Get, by passing 
the AUTHORIZATIONS option in HBase Shell, or the 
link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#setAuthorizations%28org.apache.hadoop.hbase.security.visibility.Authorizations%29[setAuthorizations()]
 method if you use the API. This authorization will be combined with your 
default set as an additional filter. It will further filter your results, 
rather than giving you additional authorization.
+The default implementation class is 
`org.apache.hadoop.hbase.security.visibility.DefaultScanLabelGenerator`, which 
was implemented in 
link:https://issues.apache.org/jira/browse/HBASE-12466[HBASE-12468]. See 
reading_cells_with_labels.
+You can also configure a set, or stack, of `ScanLabelGenerator`s to be used 
by the system, as a comma-separated list. The output for the first 
`ScanLabelGenerator` will be the input for the next one, until the end of the 
list.

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at 
org.apache.hadoop.hbase.TestHColumnDescriptorDefaultVersions.testCreateTableWithDefaultFromConf(TestHColumnDescriptorDefaultVersions.java:104)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13026//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:

[jira] [Updated] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions

2015-03-01 Thread Francis Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francis Liu updated HBASE-13134:

Status: Patch Available  (was: Open)

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13134) mutateRow and checkAndMutate apis don't throw region level exceptions

2015-03-01 Thread Francis Liu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Francis Liu updated HBASE-13134:

Attachment: HBASE-13134.patch

 mutateRow and checkAndMutate apis don't throw region level exceptions
 -

 Key: HBASE-13134
 URL: https://issues.apache.org/jira/browse/HBASE-13134
 Project: HBase
  Issue Type: Bug
Reporter: Francis Liu
Assignee: Francis Liu
 Attachments: HBASE-13134.patch


 The HTable.mutateRow() and checkAndMutate api internally uses multi(). Tho 
 after getting a response it does not go through returned region action result 
 list to check for exceptions. 
 Adding a patch for trunk tho we've observed this issue in 0.98 as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-13133) NPE when running TestSplitLogManager


 [ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-13133.
---
   Resolution: Fixed
Fix Version/s: 1.1.0
   2.0.0
 Hadoop Flags: Reviewed

Pushed to branch-1+

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner

[
https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-13082:
--
Attachment: next.png
gc.png

The fourth spike is the combined patch; more gc and less nexts'. Will look more
tomorrow.

I think we should commit this patch to 2.0 and 1.1.

Coarsen StoreScanner locks to RegionScanner
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13123) Minor bug in ROW bloom filter


[ 
https://issues.apache.org/jira/browse/HBASE-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342888#comment-14342888
 ] 

Hudson commented on HBASE-13123:


FAILURE: Integrated in HBase-1.0 #780 (See 
[https://builds.apache.org/job/HBase-1.0/780/])
HBASE-13123 - Minor bug in ROW bloom filter (Ram) (ramkrishna: rev 
bd8544f662a26d0ec78c53fe62c07a5591600809)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java


 Minor bug in ROW bloom filter
 -

 Key: HBASE-13123
 URL: https://issues.apache.org/jira/browse/HBASE-13123
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: HBASE-13163.patch


 While checking the code for Bloom filter found that while checking if a key 
 passes the ROW bloom check we try to create a bloom key. The bloom key should 
 be constructed only with the row part of the key. But try to form the bloom 
 key including the meta data part of the key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13133) NPE when running TestSplitLogManager


[ 
https://issues.apache.org/jira/browse/HBASE-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342887#comment-14342887
 ] 

Andrey Stepachev commented on HBASE-13133:
--

thats good. thanks [~Apache9] for confirming.
thanks Michael, Ted for reviewing.

 NPE when running TestSplitLogManager
 

 Key: HBASE-13133
 URL: https://issues.apache.org/jira/browse/HBASE-13133
 Project: HBase
  Issue Type: Bug
Reporter: zhangduo
Assignee: Andrey Stepachev
 Fix For: 2.0.0, 1.1.0

 Attachments: HBASE-13133.patch


 https://builds.apache.org/job/HBase-TRUNK/6187/testReport/junit/org.apache.hadoop.hbase.master/TestSplitLogManager/testOrphanTaskAcquisition/
 {noformat}
 2015-03-01 01:34:58,902 INFO  [Thread-23] master.TestSplitLogManager(298): 
 TestOrphanTaskAcquisition
 2015-03-01 01:34:58,904 DEBUG [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(870): Distributed log replay=true
 2015-03-01 01:34:58,907 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(594): found orphan task 
 orphan%2Ftest%2Fslash
 2015-03-01 01:34:58,913 INFO  [Thread-23] 
 coordination.ZKSplitLogManagerCoordination(598): Found 1 orphan tasks and 0 
 rescan nodes
 2015-03-01 01:34:58,913 ERROR [main-EventThread] 
 zookeeper.ClientCnxn$EventThread(613): Caught unexpected throwable
 java.lang.NullPointerException
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.findOrCreateOrphanTask(ZKSplitLogManagerCoordination.java:546)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.heartbeat(ZKSplitLogManagerCoordination.java:556)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.getDataSetWatchSuccess(ZKSplitLogManagerCoordination.java:467)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination.access$700(ZKSplitLogManagerCoordination.java:74)
   at 
 org.apache.hadoop.hbase.coordination.ZKSplitLogManagerCoordination$GetDataAsyncCallback.processResult(ZKSplitLogManagerCoordination.java:1020)
   at 
 org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:561)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
 {noformat}
 I got this NPE almost every time when running TestSplitLogManager locally. I 
 am not sure whether it is the root cause of test failing, but seems related.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13123) Minor bug in ROW bloom filter


[ 
https://issues.apache.org/jira/browse/HBASE-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342889#comment-14342889
 ] 

Hudson commented on HBASE-13123:


SUCCESS: Integrated in HBase-1.1 #234 (See 
[https://builds.apache.org/job/HBase-1.1/234/])
HBASE-13123 - Minor bug in ROW bloom filter (Ram):w (ramkrishna: rev 
9c3cbdcd059054d555f4d32c2d700ead0ca6fae0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java


 Minor bug in ROW bloom filter
 -

 Key: HBASE-13123
 URL: https://issues.apache.org/jira/browse/HBASE-13123
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: HBASE-13163.patch


 While checking the code for Bloom filter found that while checking if a key 
 passes the ROW bloom check we try to create a bloom key. The bloom key should 
 be constructed only with the row part of the key. But try to form the bloom 
 key including the meta data part of the key.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13053) Add support of Visibility Labels in PerformanceEvaluation

2015-03-01 Thread Jerry He (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342408#comment-14342408
 ] 

Jerry He commented on HBASE-13053:
--

Looked into LoadTestTool. 
The LoadTestTool has been covered pretty well with the data generators, with 
LoadTestDataGeneratorWithTags, LoadTestDataGeneratorWithVisibilityLabels, etc.
I will try to do the same way in PE.

 Add support of Visibility Labels in PerformanceEvaluation
 -

 Key: HBASE-13053
 URL: https://issues.apache.org/jira/browse/HBASE-13053
 Project: HBase
  Issue Type: Improvement
Affects Versions: 1.0.0, 0.98.10.1
Reporter: Jerry He
Assignee: Jerry He
Priority: Minor
 Fix For: 2.0.0, 1.0.1, 0.98.11

 Attachments: HBASE-13053-0.98.patch, HBASE-13053-master.patch


 Add support of Visibility Labels in PerformanceEvaluation:
 During write operations, support adding a visibility expression to KVs.
 During read/scan operations, support using visibility authorization.
 Here is the usage:
 {noformat}
 Options:
 ...
 visibilityExp   Writes the visibility expression along with KVs. Use for 
 write commands. Visiblity labels need to pre-exist.
 visibilityAuth  Specify the visibility auths (comma separated labels) used in 
 read or scan. Visiblity labels need to pre-exist.
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13132) Improve RemoveColumn action debug message

2015-03-01 Thread Jonathan Hsieh (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-13132:
---
Status: Patch Available  (was: Open)

 Improve RemoveColumn action debug message 
 --

 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: hbase-13132.patch


 Trivial fix for this unsightly log message:
 {code}
 2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] 
 actions.Action: Performing action: Removing [B@64275127 from 
 TestAcidGuarantees
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13132) Improve RemoveColumn action debug message

2015-03-01 Thread Jonathan Hsieh (JIRA)

Jonathan Hsieh created HBASE-13132:
--

 Summary: Improve RemoveColumn action debug message 
 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, 1.0.1, 1.1.0


Trivial fix for this unsightly log message:

{code}
2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] actions.Action: 
Performing action: Removing [B@64275127 from TestAcidGuarantees
{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature

[
https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342280#comment-14342280
]

Ted Yu commented on HBASE-13071:

ClientAsyncPrefetchScanner.java and ClientSimpleScanner.java need audience
annotation.

Hbase Streaming Scan Feature

Key: HBASE-13071
URL: https://issues.apache.org/jira/browse/HBASE-13071
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.11
Reporter: Eshcar Hillel
Attachments: HBASE-13071_98_1.patch, HBASE-13071_trunk_1.patch,
HBaseStreamingScanDesign.pdf, HbaseStreamingScanEvaluation.pdf

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13132) Improve RemoveColumn action debug message

2015-03-01 Thread Jonathan Hsieh (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-13132:
---
Attachment: hbase-13132.patch

 Improve RemoveColumn action debug message 
 --

 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: hbase-13132.patch


 Trivial fix for this unsightly log message:
 {code}
 2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] 
 actions.Action: Performing action: Removing [B@64275127 from 
 TestAcidGuarantees
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13132) Improve RemoveColumn action debug message


[ 
https://issues.apache.org/jira/browse/HBASE-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342354#comment-14342354
 ] 

Hadoop QA commented on HBASE-13132:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12701703/hbase-13132.patch
  against master branch at commit dad2474f08d201d09989e36f5cf1c25d3fa4acee.
  ATTACHMENT ID: 12701703

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.1 2.5.2 2.6.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/13022//console

This message is automatically generated.

 Improve RemoveColumn action debug message 
 --

 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, 1.0.1, 1.1.0

 Attachments: hbase-13132.patch


 Trivial fix for this unsightly log message:
 {code}
 2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] 
 actions.Action: Performing action: Removing [B@64275127 from 
 TestAcidGuarantees
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13087) branch-1 isn't rolling upgradable from 0.98

2015-03-01 Thread Nick Dimiduk (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14342378#comment-14342378
 ] 

Nick Dimiduk commented on HBASE-13087:
--

Can we back out the breaking change from branch-1 and try again? Requiring 
gymnastics is not consistent with our semver commitment.

 branch-1 isn't rolling upgradable from 0.98
 ---

 Key: HBASE-13087
 URL: https://issues.apache.org/jira/browse/HBASE-13087
 Project: HBase
  Issue Type: Bug
Reporter: Elliott Clark
Assignee: Rajesh Nishtala
Priority: Blocker
 Fix For: 2.0.0, 1.1.0


 {code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: 
 Failed 1 action: 
 org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column 
 family table does not exist in region hbase:meta,,1.1588230740 in table 
 'hbase:meta', {TABLE_ATTRIBUTES = {IS_META = 'true', coprocessor$1 = 
 '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, 
 {NAME = 'info', BLOOMFILTER = 'NONE', VERSIONS = '10', IN_MEMORY = 
 'true', KEEP_DELETED_CELLS = 'FALSE', DATA_BLOCK_ENCODING = 'NONE', TTL = 
 'FOREVER', COMPRESSION = 'NONE', MIN_VERSIONS = '0', BLOCKCACHE = 'true', 
 BLOCKSIZE = '8192', REPLICATION_SCOPE = '0'}
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
   at 
 org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
   at 
 org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
   at 
 org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
   at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
   at java.lang.Thread.run(Thread.java:745)
 : 1 time, 
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
   at 
 org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
   at 
 org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
   at 
 org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
   at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
   at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
   at 
 org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
   at 
 org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
   at 
 org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
   at 
 org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
   at 
 org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
   at 
 org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
   at 
 org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
   at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
   at 
 org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
   at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
   at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
   at java.lang.Thread.run(Thread.java:745)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13131) ReplicationAdmin leaks connections if there's an error in the constructor