date:20150814

[
https://issues.apache.org/jira/browse/HBASE-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696917#comment-14696917
]

Hadoop QA commented on HBASE-13708:
---

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12750496/PE_mapred_output.txt
against master branch at commit f2a9dab30eb339c86222db47430f18f7abf405c2.
ATTACHMENT ID: 12750496

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15100//console

This message is automatically generated.

PE - Add option to specify the range

Key: HBASE-13708
URL: https://issues.apache.org/jira/browse/HBASE-13708
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.1
Reporter: Jean-Marc Spaggiari
Assignee: Matt Warhaftig
Priority: Minor
Labels: beginner
Fix For: 2.0.0, 1.3.0

Attachments: HBASE-13708-master_v1.patch, PE_mapred_output.txt,
PE_nomapred_output.txt, hbase-13708-v2.patch, hbase-13708-v2.patch

When specifying --rows=YYY for randomReads in PE, it will read YYY rows but
only betweem 0 and YYY.
We might want to read YYY rows randomly between 0 and ZZZ.
YYY should only be the number of rows we want to ready. Not the high range.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13966) Limit column width in table.jsp


[ 
https://issues.apache.org/jira/browse/HBASE-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696531#comment-14696531
 ] 

Hadoop QA commented on HBASE-13966:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12742681/hbase-13966-v1.patch
  against master branch at commit 4dd30ab019cfbf3691fd08f7941d33d8bbc37f05.
  ATTACHMENT ID: 12742681

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+tableHeader = h2Table Regions/h2table class=\table 
table-striped\ style=\table-layout: fixed; word-wrap: break-word;\trth 
style=\width:22%\Name/ththRegion Server/thth 
style=\width:22%\Start Key/thth style=\width:22%\End 
Key/ththLocality/ththRequests/ththReplicaID/th/tr;
+tableHeader = h2Table Regions/h2table class=\table table-striped\ 
style=\table-layout: fixed; word-wrap: break-word;\trth 
style=\width:22%\Name/ththRegion Server/thth 
style=\width:22%\Start Key/thth style=\width:22%\End 
Key/ththLocality/ththRequests/th/tr;

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15094//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15094//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15094//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15094//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15094//console

This message is automatically generated.

 Limit column width in table.jsp
 ---

 Key: HBASE-13966
 URL: https://issues.apache.org/jira/browse/HBASE-13966
 Project: HBase
  Issue Type: Bug
  Components: UI
Affects Versions: 1.1.0.1
Reporter: Jean-Marc Spaggiari
Assignee: Matt Warhaftig
Priority: Minor
  Labels: beginner
 Attachments: HBASE-13966_post.tiff, HBASE-13966_pre.tiff, 
 hbase-13966-v1.patch


 In table.jsp, for tables with very wide keys like URLs, the page can be very 
 wide. On my own cluster, this page is 8 screens wide, almost un-usable.
 Might be good to have a way to resize the columns, or wrap the long keys or 
 truncate them, or anything else. When we want to look at the last colums, 
 this is a big difficult.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells


 [ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-14144:
---
Status: Open  (was: Patch Available)

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14222) Improve DrainBarrier


 [ 
https://issues.apache.org/jira/browse/HBASE-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroshi Ikeda updated HBASE-14222:
--
Status: Patch Available  (was: Open)

 Improve DrainBarrier
 

 Key: HBASE-14222
 URL: https://issues.apache.org/jira/browse/HBASE-14222
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor
 Attachments: HBASE-14222.patch


 1. {{DrainBarrier.stopAndDrainOps}} may wait forever if 
 {{DrainBarrier.endOp}} changes its state and calls {{Object.notifyAll}} just 
 before {{stopAndDrainOps}} enters the synchronized block to call 
 {{Object.wait}}. Moreover, {{Object.wait}} may wake up false-positively, and 
 {{stopAndDrainOps}} may break the block before outstanding operations are 
 complete.
 2. Some tests for {{DrainBarrier}} catch and ignore {{AssertionError}} 
 explicitly thrown JUnit's {{fail}} method.
 The implementation of {{DrainBarrier}} is a little complex, and I'll fix and 
 refactor it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells


 [ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-14144:
---
Status: Patch Available  (was: Open)

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch, HBASE-14144_6.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells


 [ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-14144:
---
Attachment: HBASE-14144_6.patch

Renames the FakeCell and FakeBBCell to EmptyCell and EmptyByteBufferedCell. 
This is what I will commit.
[~anoopsamjohn]
You fine with this?

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch, HBASE-14144_6.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14211) Add more rigorous integration tests of splits


[ 
https://issues.apache.org/jira/browse/HBASE-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696534#comment-14696534
 ] 

Hadoop QA commented on HBASE-14211:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12750416/HBASE-14211-v3.patch
  against master branch at commit 4dd30ab019cfbf3691fd08f7941d33d8bbc37f05.
  ATTACHMENT ID: 12750416

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 25 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15093//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15093//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15093//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15093//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15093//console

This message is automatically generated.

 Add more rigorous integration tests of splits
 -

 Key: HBASE-14211
 URL: https://issues.apache.org/jira/browse/HBASE-14211
 Project: HBase
  Issue Type: Bug
  Components: integration tests, test
Affects Versions: 2.0.0, 1.2.0
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 2.0.0, 1.2.0

 Attachments: HBASE-14211-v1.patch, HBASE-14211-v2.patch, 
 HBASE-14211-v3.patch, HBASE-14211.patch


 Add a chaos action that will turn down region size.
 * Eventually this will cause regions to split a lot.
 * It will need to have a min region size.
 Add a chaos monkey action that will change split policy
 * Change between Uniform and SplittingUpTo and back
 Add chaos monkey action that will request splits of every region.
 * When regions all reach the size a the exact same time the compactions add a 
 lot of work.
 * This simulates a very well distributed write pattern reaching the region 
 size.
 Add the ability to start with fewer regions than normal to ITBLL



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14222) Improve DrainBarrier

Hiroshi Ikeda created HBASE-14222:
-

 Summary: Improve DrainBarrier
 Key: HBASE-14222
 URL: https://issues.apache.org/jira/browse/HBASE-14222
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor


1. {{DrainBarrier.stopAndDrainOps}} may wait forever if {{DrainBarrier.endOp}} 
changes its state and calls {{Object.notifyAll}} just before 
{{stopAndDrainOps}} enters the synchronized block to call {{Object.wait}}. 
Moreover, {{Object.wait}} may wake up false-positively, and {{stopAndDrainOps}} 
may break the block before outstanding operations are complete.

2. Some tests for {{DrainBarrier}} catch and ignore {{AssertionError}} 
explicitly thrown JUnit's {{fail}} method.

The implementation of {{DrainBarrier}} is a little complex, and I'll fix and 
refactor it.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14222) Improve DrainBarrier


 [ 
https://issues.apache.org/jira/browse/HBASE-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hiroshi Ikeda updated HBASE-14222:
--
Attachment: HBASE-14222.patch

Added a patch.

 Improve DrainBarrier
 

 Key: HBASE-14222
 URL: https://issues.apache.org/jira/browse/HBASE-14222
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor
 Attachments: HBASE-14222.patch


 1. {{DrainBarrier.stopAndDrainOps}} may wait forever if 
 {{DrainBarrier.endOp}} changes its state and calls {{Object.notifyAll}} just 
 before {{stopAndDrainOps}} enters the synchronized block to call 
 {{Object.wait}}. Moreover, {{Object.wait}} may wake up false-positively, and 
 {{stopAndDrainOps}} may break the block before outstanding operations are 
 complete.
 2. Some tests for {{DrainBarrier}} catch and ignore {{AssertionError}} 
 explicitly thrown JUnit's {{fail}} method.
 The implementation of {{DrainBarrier}} is a little complex, and I'll fix and 
 refactor it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14181) Add Spark DataFrame DataSource to HBase-Spark Module


[ 
https://issues.apache.org/jira/browse/HBASE-14181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696644#comment-14696644
 ] 

Hadoop QA commented on HBASE-14181:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12750443/HBASE-14181.4.patch
  against master branch at commit 4dd30ab019cfbf3691fd08f7941d33d8bbc37f05.
  ATTACHMENT ID: 12750443

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:red}-1 checkstyle{color}.  The applied patch generated 
1858 checkstyle errors (more than the master's current 1856 errors).

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+ val schemaMappingDefinition:java.util.HashMap[String, 
SchemaQualifierDefinition],
+requiredQualifierDefinitionArray.foreach( d = 
get.addColumn(d.columnFamilyBytes, d.qualifierBytes))
+requiredQualifierDefinitionArray.foreach( d = 
scan.addColumn(d.columnFamilyBytes, d.qualifierBytes))
+  requiredQualifierDefinitionArray.foreach( d = 
scan.addColumn(d.columnFamilyBytes, d.qualifierBytes))
+   requiredQualifierDefinitionArray: 
mutable.MutableList[SchemaQualifierDefinition]):Unit = {
+  Map(hbase.columns.mapping - KEY_FIELD STRING :key, A_FIELD STRING 
c:a, B_FIELD STRING c:b,,

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   
org.apache.hadoop.hbase.master.procedure.TestWALProcedureStoreOnHDFS

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15095//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15095//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15095//artifact/patchprocess/checkstyle-aggregate.html

Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15095//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15095//console

This message is automatically generated.

 Add Spark DataFrame DataSource to HBase-Spark Module
 

 Key: HBASE-14181
 URL: https://issues.apache.org/jira/browse/HBASE-14181
 Project: HBase
  Issue Type: New Feature
  Components: spark
Reporter: Ted Malaska
Assignee: Ted Malaska
Priority: Minor
 Attachments: HBASE-14181.1.patch, HBASE-14181.2.patch, 
 HBASE-14181.3.patch, HBASE-14181.4.patch


 Build a RelationProvider for HBase-Spark Module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14144) Bloomfilter path to work with Byte buffered cells

2015-08-14 Thread Anoop Sam John (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696539#comment-14696539
 ] 

Anoop Sam John commented on HBASE-14144:


+1..  Just add some comments to these classes that they are not real and used 
for seek/reseek..  Just in case when some one sees this code later, they should 
get to know what these cells mean for..  Add on commit pls.

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch, HBASE-14144_6.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells


 [ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ramkrishna.s.vasudevan updated HBASE-14144:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master. Thanks for the reviews Stack and Anoop.
Added the comment as suggested on commit.

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch, HBASE-14144_6.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14082) Add replica id to JMX metrics names

2015-08-14 Thread Lei Chen (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lei Chen updated HBASE-14082:
-
Attachment: HBASE-14082-v3.patch

Updates:
1. moved replica id from metrics name to a separate metric.
2. updated related test

Add replica id to JMX metrics names
---

Key: HBASE-14082
URL: https://issues.apache.org/jira/browse/HBASE-14082
Project: HBase
Issue Type: Improvement
Components: metrics
Reporter: Lei Chen
Assignee: Lei Chen
Attachments: HBASE-14082-v1.patch, HBASE-14082-v2.patch,
HBASE-14082-v3.patch

Today, via JMX, one cannot distinguish a primary region from a replica. A
possible solution is to add replica id to JMX metrics names. The benefits may
include, for example:
# Knowing the latency of a read request on a replica region means the first
attempt to the primary region has timeout.
# Write requests on replicas are due to the replication process, while the
ones on primary are from clients.
# In case of looking for hot spots of read operations, replicas should be
excluded since TIMELINE reads are sent to all replicas.
To implement, we can change the format of metrics names found at
{code}Hadoop-HBase-RegionServer-Regions-Attributes{code}
from
{code}namespace_namespace_table_tablename_region_regionname_metric_metricname{code}
to
{code}namespace_namespace_table_tablename_region_regionname_replicaid_replicaid_metric_metricname{code}

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14082) Add replica id to JMX metrics names

2015-08-14 Thread Lei Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei Chen updated HBASE-14082:
-
Status: Patch Available  (was: Open)

 Add replica id to JMX metrics names
 ---

 Key: HBASE-14082
 URL: https://issues.apache.org/jira/browse/HBASE-14082
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Reporter: Lei Chen
Assignee: Lei Chen
 Attachments: HBASE-14082-v1.patch, HBASE-14082-v2.patch, 
 HBASE-14082-v3.patch


 Today, via JMX, one cannot distinguish a primary region from a replica. A 
 possible solution is to add replica id to JMX metrics names. The benefits may 
 include, for example:
 # Knowing the latency of a read request on a replica region means the first 
 attempt to the primary region has timeout.
 # Write requests on replicas are due to the replication process, while the 
 ones on primary are from clients.
 # In case of looking for hot spots of read operations, replicas should be 
 excluded since TIMELINE reads are sent to all replicas.
 To implement, we can change the format of metrics names found at 
 {code}Hadoop-HBase-RegionServer-Regions-Attributes{code}
 from 
 {code}namespace_namespace_table_tablename_region_regionname_metric_metricname{code}
 to
 {code}namespace_namespace_table_tablename_region_regionname_replicaid_replicaid_metric_metricname{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11902) RegionServer was blocked while aborting


[ 
https://issues.apache.org/jira/browse/HBASE-11902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696562#comment-14696562
 ] 

Hiroshi Ikeda commented on HBASE-11902:
---

This HBASE-11902 seems duplicate with HBASE-13592, which has been resolved. 
I have created a new issue HBASE-14222 about DrainBarrier.


 RegionServer was blocked while aborting
 ---

 Key: HBASE-11902
 URL: https://issues.apache.org/jira/browse/HBASE-11902
 Project: HBase
  Issue Type: Bug
  Components: regionserver, wal
Affects Versions: 0.98.4
 Environment: hbase-0.98.4, hadoop-2.3.0-cdh5.1, jdk1.7
Reporter: Victor Xu
Assignee: Qiang Tian
 Attachments: hbase-hadoop-regionserver-hadoop461.cm6.log, 
 hbase11902-master.patch, hbase11902-master_v2.patch, 
 hbase11902-master_v3.patch, jstack_hadoop461.cm6.log


 Generally, regionserver automatically aborts when isHealth() returns false. 
 But it sometimes got blocked while aborting. I saved the jstack and logs, and 
 found out that it was caused by datanodes failures. The regionserver60020 
 thread was blocked while closing WAL. 
 This issue doesn't happen so frequently, but if it happens, it always leads 
 to huge amount of requests failure. The only way to do is KILL -9.
 I think it's a bug, but I haven't found a decent solution. Does anyone have 
 the same problem?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14210) Create test for cell level ACLs involving user group


 [ 
https://issues.apache.org/jira/browse/HBASE-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HBASE-14210:
--
Status: Patch Available  (was: Open)

 Create test for cell level ACLs involving user group
 

 Key: HBASE-14210
 URL: https://issues.apache.org/jira/browse/HBASE-14210
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Assignee: Ashish Singhi
 Attachments: HBASE-14210.patch


 Currently we have TestCellACLs and TestCellACLWithMultipleVersions which 
 exercise cell level ACLs for users.
 However, test for cell level ACLs involving user group is missing.
 This issue is to add such test(s)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14210) Create test for cell level ACLs involving user group


 [ 
https://issues.apache.org/jira/browse/HBASE-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HBASE-14210:
--
Attachment: HBASE-14210.patch

 Create test for cell level ACLs involving user group
 

 Key: HBASE-14210
 URL: https://issues.apache.org/jira/browse/HBASE-14210
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Assignee: Ashish Singhi
 Attachments: HBASE-14210.patch


 Currently we have TestCellACLs and TestCellACLWithMultipleVersions which 
 exercise cell level ACLs for users.
 However, test for cell level ACLs involving user group is missing.
 This issue is to add such test(s)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14210) Create test for cell level ACLs involving user group


[ 
https://issues.apache.org/jira/browse/HBASE-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696718#comment-14696718
 ] 

Ashish Singhi commented on HBASE-14210:
---

Attached patch for adding test coverage for cell level ACLs involving user 
group.
Please review.

Planning to add test coverage for ACLs with default value(true) for 
{{hbase.security.access.early_out}}, in a separate jira if it sounds good.

 Create test for cell level ACLs involving user group
 

 Key: HBASE-14210
 URL: https://issues.apache.org/jira/browse/HBASE-14210
 Project: HBase
  Issue Type: Test
Reporter: Ted Yu
Assignee: Ashish Singhi
 Attachments: HBASE-14210.patch


 Currently we have TestCellACLs and TestCellACLWithMultipleVersions which 
 exercise cell level ACLs for users.
 However, test for cell level ACLs involving user group is missing.
 This issue is to add such test(s)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11368) Multi-column family BulkLoad fails if compactions go on too long

2015-08-14 Thread Enis Soztutar (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696724#comment-14696724
]

Enis Soztutar commented on HBASE-11368:
---

bq. How? Would the following case be true without the bulk load getting the
region write lock?
We do not do this now, but in theory it can be done similarly to regular
writes.
- obtain new seqId as a write transaction
- bulk load all files across CFs with the seqId.
- advance mvcc read point only when all bulk loads are complete.
This way the scanners are guaranteed to atomically observe the bulk loaded data
atomically without the region-write-lock.
bq. In the 0.98 code line, we don't have seqid, and the atomicity is still
guaranteed there.
Yes. Not worth changing 0.98 line.
bq. I think it is being propagated properly to the scanner. Think about the
same notifyChangedReadersObservers is being used at the end of compaction and
flushes as well. The reset of the readers should work.
I am not sure about that. Agreed that the cells at the store level will
actually get re-ordered, but the heap at the region level is never re-ordered.
So, after a bulk load, the ordering of store scanners at the region level might
change, but the scanner will miss it if I understand this correctly.
bq. Atomicity may be a false blanket considering HBASE-4652 is still unresolved.
Very good point. We need a transactional commit for the BL files.

Multi-column family BulkLoad fails if compactions go on too long

Key: HBASE-11368
URL: https://issues.apache.org/jira/browse/HBASE-11368
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Qiang Tian
Attachments: hbase-11368-0.98.5.patch, hbase11368-master.patch,
key_stacktrace_hbase10882.TXT, performance_improvement_verification_98.5.patch

Compactions take a read lock. If a multi-column family region, before bulk
loading, we want to take a write lock on the region. If the compaction takes
too long, the bulk load fails.
Various recipes include:
+ Making smaller regions (lame)
+ [~victorunique] suggests major compacting just before bulk loading over in
HBASE-10882 as a work around.
Does the compaction need a read lock for that long? Does the bulk load need
a full write lock when multiple column families? Can we fail more gracefully
at least?

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14222) Improve DrainBarrier


[ 
https://issues.apache.org/jira/browse/HBASE-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696727#comment-14696727
 ] 

Hadoop QA commented on HBASE-14222:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12750463/HBASE-14222.patch
  against master branch at commit 4dd30ab019cfbf3691fd08f7941d33d8bbc37f05.
  ATTACHMENT ID: 12750463

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 3 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestIOFencing
  org.apache.hadoop.hbase.master.TestDistributedLogSplitting

 {color:red}-1 core zombie tests{color}.  There are 4 zombie test(s):   
at 
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient.testCloneLinksAfterDelete(TestCloneSnapshotFromClient.java:223)
at 
org.apache.hadoop.hbase.client.TestReplicasClient.testScanWithReplicas(TestReplicasClient.java:600)
at 
org.apache.hadoop.hbase.client.TestFromClientSide.testJira6912(TestFromClientSide.java:5344)
at 
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient.testCloneSnapshot(TestMobCloneSnapshotFromClient.java:175)
at 
org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient.testCloneSnapshotCrossNamespace(TestMobCloneSnapshotFromClient.java:190)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15097//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15097//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15097//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15097//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15097//console

This message is automatically generated.

 Improve DrainBarrier
 

 Key: HBASE-14222
 URL: https://issues.apache.org/jira/browse/HBASE-14222
 Project: HBase
  Issue Type: Bug
  Components: util
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor
 Attachments: HBASE-14222.patch


 1. {{DrainBarrier.stopAndDrainOps}} may wait forever if 
 {{DrainBarrier.endOp}} changes its state and calls {{Object.notifyAll}} just 
 before {{stopAndDrainOps}} enters the synchronized block to call 
 {{Object.wait}}. Moreover, {{Object.wait}} may wake up false-positively, and 
 {{stopAndDrainOps}} may break the block before outstanding operations are 
 complete.
 2. Some tests for {{DrainBarrier}} catch and ignore {{AssertionError}} 
 explicitly thrown JUnit's {{fail}} method.
 The implementation of {{DrainBarrier}} is a little complex, and I'll fix and 
 refactor it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14144) Bloomfilter path to work with Byte buffered cells


[ 
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696726#comment-14696726
 ] 

Hadoop QA commented on HBASE-14144:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12750461/HBASE-14144_6.patch
  against master branch at commit 4dd30ab019cfbf3691fd08f7941d33d8bbc37f05.
  ATTACHMENT ID: 12750461

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 1 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at 
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat2.testMRIncrementalLoadWithLocality(TestHFileOutputFormat2.java:400)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15096//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15096//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15096//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15096//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/15096//console

This message is automatically generated.

 Bloomfilter path to work with Byte buffered cells
 -

 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0

 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, 
 HBASE-14144_2.patch, HBASE-14144_3.patch, HBASE-14144_4.patch, 
 HBASE-14144_5.patch, HBASE-14144_6.patch


 This JIRA is to check if there will be a need to make the bloom filters to 
 work with ByteBuffer cells. During POC this path created lot of duplicated 
 code but considering other refactorings done in this path  may lead to less 
 duplication. This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14210) Create test for cell level ACLs involving user group