date:20150722


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637136#comment-14637136
 ] 

Hudson commented on HBASE-14115:


FAILURE: Integrated in HBase-1.2 #79 (See 
[https://builds.apache.org/job/HBase-1.2/79/])
HBASE-14115 Fix resource leak in HMasterCommandLine (Yuhao Bi) (tedyu: rev 
f0042a4038486cfb17ba807134ea448617c8b94d)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java


 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.2.0, 1.3.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637131#comment-14637131
 ] 

Hudson commented on HBASE-14115:


SUCCESS: Integrated in HBase-1.3 #69 (See 
[https://builds.apache.org/job/HBase-1.3/69/])
HBASE-14115 Fix resource leak in HMasterCommandLine (Yuhao Bi) (tedyu: rev 
45d52129dd53207efb0a6408cf6869e0d5b23df3)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java


 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637101#comment-14637101
 ] 

Hadoop QA commented on HBASE-14115:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12746533/HBASE-14115-branch-1-v1.patch
  against branch-1 branch at commit 5ec5552be0534dbf4b07ef6607741ae6f9ab0495.
  ATTACHMENT ID: 12746533

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

 {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s):   
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerNodeType(RepositoryUtil.java:148)
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerSlingNodeTypes(RepositoryUtil.java:158)
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerNodeType(RepositoryUtil.java:148)
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerSlingNodeTypes(RepositoryUtil.java:160)
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerNodeType(RepositoryUtil.java:148)
at 
org.apache.sling.commons.testing.jcr.RepositoryUtil.registerSlingNodeTypes(RepositoryUtil.java:160)
at 
org.apache.sling.discovery.impl.cluster.ClusterTest.testDuplicateInstanceIn2Clusters4139(ClusterTest.java:267)

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14860//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14860//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14860//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14860//console

This message is automatically generated.

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14115) Fix resource leak in HMasterCommandLine


 [ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuhao Bi updated HBASE-14115:
-
Affects Version/s: 1.3.0
   1.2.0

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.2.0, 1.3.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer

[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637314#comment-14637314
]

Elliott Clark commented on HBASE-12751:
---

Targeting 1.3 but we can always re-schedule if this takes longer than
anticipated.

Allow RowLock to be reader writer
-

Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 1.3.0

Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch, HBASE-12751.patch

Right now every write operation grabs a row lock. This is to prevent values
from changing during a read modify write operation (increment or check and
put). However it limits parallelism in several different scenarios.
If there are several puts to the same row but different columns or stores
then this is very limiting.
If there are puts to the same column then mvcc number should ensure a
consistent ordering. So locking is not needed.
However locking for check and put or increment is still needed.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14058) Stabilizing default heap memory tuner

2015-07-22 Thread Abhilash (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-14058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Abhilash updated HBASE-14058:
-
Attachment: 0001-Stabilizing-default-heap-memory-tuner.patch

Stabilizing default heap memory tuner
-

Key: HBASE-14058
URL: https://issues.apache.org/jira/browse/HBASE-14058
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 2.0.0, 1.2.0, 1.3.0
Reporter: Abhilash
Assignee: Abhilash
Attachments: 0001-Stabilizing-default-heap-memory-tuner.patch,
HBASE-14058-v1.patch, HBASE-14058.patch, after_modifications.png,
before_modifications.png

The memory tuner works well in general cases but when we have a work load
that is both read heavy as well as write heavy the tuner does too many
tuning. We should try to control the number of tuner operation and stabilize
it. The main problem was that the tuner thinks it is in steady state even if
it sees just one neutral tuner period thus does too many tuning operations
and too many reverts that too with large step sizes(step size was set to
maximum even after one neutral period). So to stop this I have thought of
these steps:
1) The division created by μ + δ/2 and μ - δ/2 is too small. Statistically
~62% periods will lie outside this range, which means 62% of the data points
are considered either high or low which is too much. Use μ + δ*0.8 and μ -
δ*0.8 instead. On expectations it will decrease number of tuner operations
per 100 periods from 19 to just 10. If we use δ/2 then 31% of data values
will be considered to be high and 31% will be considered to be low (2*0.31 *
0.31 = 0.19), on the other hand if we use δ*0.8 then 22% will be low and 22%
will be high(2*0.22*0.22 ~ 0.10).
2) Defining proper steady state by looking at past few periods(it is equal to
hbase.regionserver.heapmemory.autotuner.lookup.periods) rather than just last
tuner operation. We say tuner is in steady state when last few tuner periods
were NEUTRAL. We keep decreasing step size unless it is extremely low. Then
leave system in that state for some time.
3) Rather then decreasing step size only while reverting, decrease the
magnitude of step size whenever we are trying to revert tuning done in last
few periods(sum the changes of last few periods and compare to current step)
rather than just looking at last period. When its magnitude gets too low then
make tuner steps NEUTRAL(no operation). This will cause step size to
continuously decrease unless we reach steady state. After that tuning process
will restart (tuner step size rests again when we reach steady state).
4) The tuning done in last few periods will be decaying sum of past tuner
steps with sign. This parameter will be positive for increase in memstore and
negative for increase in block cache. Rather than using arithmetic mean we
use this to give more priority to recent tuner steps.
Please see the attachments. One represents the size of memstore(green) and
size of block cache(blue) adjusted by tuner without these modification and
other with the above modifications. The x-axis is time axis and y-axis is the
fraction of heap memory available to memstore and block cache at that time(it
always sums up to 80%). I configured min/max ranges for both components to
0.1 and 0.7 respectively(so in the plots the y-axis min and max is 0.1 and
0.7). In both cases the tuner tries to distribute memory by giving ~15% to
memstore and ~65% to block cache. But the modified one does it much more
smoothly.
I got these results from YCSB test. The test was doing approximately 5000
inserts and 500 reads per second (for one region server). The results can be
further fine tuned and number of tuner operation can be reduced with these
changes in configuration.
For more fine tuning:
a) lower max step size (suggested = 4%)
b) lower min step size ( default if also fine )
To further decrease frequency of tuning operations:
c) increase the number of lookup periods ( in the tests it was just 10,
default is 60 )
d) increase tuner period ( in the tests it was just 20 secs, default is
60secs)
I used smaller tuner period/ number of look up periods to get more data
points.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14129) If any regionserver gets shutdown uncleanly during full cluster restart, locality looks to be lost

2015-07-22 Thread churro morales (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

churro morales updated HBASE-14129:
---
Affects Version/s: 0.98.14
2.0.0

If any regionserver gets shutdown uncleanly during full cluster restart,
locality looks to be lost
--

Key: HBASE-14129
URL: https://issues.apache.org/jira/browse/HBASE-14129
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.14
Reporter: churro morales
Attachments: HBASE-14129.patch

We were doing a cluster restart the other day. Some regionservers did not
shut down cleanly. Upon restart our locality went from 99% to 5%. Upon
looking at the AssignmentManager.joinCluster() code it calls
AssignmentManager.processDeadServersAndRegionsInTransition().
If the failover flag gets set for any reason it seems we don't call
assignAllUserRegions(). Then it looks like the balancer does the work in
assigning those regions, we don't use a locality aware balancer and we lost
our region locality.
I don't have a solid grasp on the reasoning for these checks but there could
be some potential workarounds here.
1. After shutting down your cluster, move your WALs aside (replay later).
2. Clean up your zNodes
That seems to work, but requires a lot of manual labor. Another solution
which I prefer would be to have a flag for ./start-hbase.sh --clean
If we start master with that flag then we do a check in
AssignmentManager.processDeadServersAndRegionsInTransition() thus if this
flag is set we call: assignAllUserRegions() regardless of the failover state.
I have a patch for the later solution, that is if I am understanding the
logic correctly.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12751) Allow RowLock to be reader writer


 [ 
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-12751:
--
Attachment: HBASE-12751-v14.patch

 Allow RowLock to be reader writer
 -

 Key: HBASE-12751
 URL: https://issues.apache.org/jira/browse/HBASE-12751
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, 
 HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, 
 HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch, 
 HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, 
 HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, 
 HBASE-12751-v9.patch, HBASE-12751.patch


 Right now every write operation grabs a row lock. This is to prevent values 
 from changing during a read modify write operation (increment or check and 
 put). However it limits parallelism in several different scenarios.
 If there are several puts to the same row but different columns or stores 
 then this is very limiting.
 If there are puts to the same column then mvcc number should ensure a 
 consistent ordering. So locking is not needed.
 However locking for check and put or increment is still needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12751) Allow RowLock to be reader writer


 [ 
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-12751:
--
Affects Version/s: 1.3.0
   2.0.0
Fix Version/s: 1.3.0
   2.0.0

 Allow RowLock to be reader writer
 -

 Key: HBASE-12751
 URL: https://issues.apache.org/jira/browse/HBASE-12751
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, 
 HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, 
 HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch, 
 HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, 
 HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, 
 HBASE-12751-v9.patch, HBASE-12751.patch


 Right now every write operation grabs a row lock. This is to prevent values 
 from changing during a read modify write operation (increment or check and 
 put). However it limits parallelism in several different scenarios.
 If there are several puts to the same row but different columns or stores 
 then this is very limiting.
 If there are puts to the same column then mvcc number should ensure a 
 consistent ordering. So locking is not needed.
 However locking for check and put or increment is still needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer

[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637309#comment-14637309
]

Elliott Clark commented on HBASE-12751:
---

So here's a new patch with a cleaner row locking. It uses the java provided
re-entrant lock for reader writer.
It passes the few unit tests that I've been using for spot checking. Lets see
how this looks on hadoop qa.

Allow RowLock to be reader writer
-

Key: HBASE-12751
URL: https://issues.apache.org/jira/browse/HBASE-12751
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Elliott Clark
Assignee: Elliott Clark
Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v2.patch,
HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch,
HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch,
HBASE-12751-v9.patch, HBASE-12751.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14129) If any regionserver gets shutdown uncleanly during full cluster restart, locality looks to be lost

2015-07-22 Thread churro morales (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

churro morales updated HBASE-14129:
---
Fix Version/s: 0.98.14
2.0.0

If any regionserver gets shutdown uncleanly during full cluster restart,
locality looks to be lost
--

Key: HBASE-14129
URL: https://issues.apache.org/jira/browse/HBASE-14129
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.14
Reporter: churro morales
Fix For: 2.0.0, 0.98.14

Attachments: HBASE-14129.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11339) HBase MOB

2015-07-22 Thread Jonathan Hsieh (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-11339:
---
Release Note: 
The Moderate Object Storage (MOB) feature (HBASE-11339[1]) is modified I/O and 
compaction path that allows individual moderately sized values (100KB-10MB) to 
be stored in a way that write amplification is reduced when compared to the 
normal I/O path. MOB is defined in the column family and it is almost isolated 
with other components, the features and performance cannot be effected in 
normal columns.

For more details on how to use the feature please consult the HBase Reference 
Guide

 HBase MOB
 -

 Key: HBASE-11339
 URL: https://issues.apache.org/jira/browse/HBASE-11339
 Project: HBase
  Issue Type: Umbrella
  Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: 2.0.0, hbase-11339

 Attachments: 11339-master-v10.patch, 11339-master-v3.txt, 
 11339-master-v4.txt, 11339-master-v5.txt, 11339-master-v6.txt, 
 11339-master-v7.txt, 11339-master-v8.patch, 11339-master-v9.patch, HBase MOB 
 Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB 
 Design-v5.pdf, HBase MOB Design.pdf, MOB user guide.docx, MOB user 
 guide_v2.docx, MOB user guide_v3.docx, MOB user guide_v4.docx, MOB user 
 guide_v5.docx, hbase-11339-150519.patch, hbase-11339-in-dev.patch, 
 hbase-11339.150417.patch, merge-150212.patch, merge.150212b.patch, 
 merge.150212c.patch, merge.150710.patch


   It's quite useful to save the medium binary data like images, documents 
 into Apache HBase. Unfortunately directly saving the binary MOB(medium 
 object) to HBase leads to a worse performance since the frequent split and 
 compaction.
   In this design, the MOB data are stored in an more efficient way, which 
 keeps a high write/read performance and guarantees the data consistency in 
 Apache HBase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14129) If any regionserver gets shutdown uncleanly during full cluster restart, locality looks to be lost

2015-07-22 Thread churro morales (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-14129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

churro morales updated HBASE-14129:
---
Affects Version/s: (was: 0.98.14)
(was: 2.0.0)

If any regionserver gets shutdown uncleanly during full cluster restart,
locality looks to be lost
--

Key: HBASE-14129
URL: https://issues.apache.org/jira/browse/HBASE-14129
Project: HBase
Issue Type: Bug
Reporter: churro morales
Fix For: 2.0.0, 0.98.14

Attachments: HBASE-14129.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13992) Integrate SparkOnHBase into HBase


 [ 
https://issues.apache.org/jira/browse/HBASE-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Malaska updated HBASE-13992:

Attachment: HBASE-13992.8.patch

Added the change lars found about HBaseRDD.collect

Also added more tests around that change

 Integrate SparkOnHBase into HBase
 -

 Key: HBASE-13992
 URL: https://issues.apache.org/jira/browse/HBASE-13992
 Project: HBase
  Issue Type: New Feature
  Components: spark
Reporter: Ted Malaska
Assignee: Ted Malaska
 Fix For: 2.0.0

 Attachments: HBASE-13992.5.patch, HBASE-13992.6.patch, 
 HBASE-13992.7.patch, HBASE-13992.8.patch, HBASE-13992.patch, 
 HBASE-13992.patch.3, HBASE-13992.patch.4, HBASE-13992.patch.5


 This Jira is to ask if SparkOnHBase can find a home in side HBase core.
 Here is the github: 
 https://github.com/cloudera-labs/SparkOnHBase
 I am the core author of this project and the license is Apache 2.0
 A blog explaining this project is here
 http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
 A spark Streaming example is here
 http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
 A real customer using this in produce is blogged here
 http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/
 Please debate and let me know what I can do to make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13992) Integrate SparkOnHBase into HBase


[ 
https://issues.apache.org/jira/browse/HBASE-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637463#comment-14637463
 ] 

Ted Malaska commented on HBASE-13992:
-

Hold on that last patch.  I have a couple more changes left.  give me 10 minutes

 Integrate SparkOnHBase into HBase
 -

 Key: HBASE-13992
 URL: https://issues.apache.org/jira/browse/HBASE-13992
 Project: HBase
  Issue Type: New Feature
  Components: spark
Reporter: Ted Malaska
Assignee: Ted Malaska
 Fix For: 2.0.0

 Attachments: HBASE-13992.5.patch, HBASE-13992.6.patch, 
 HBASE-13992.7.patch, HBASE-13992.8.patch, HBASE-13992.patch, 
 HBASE-13992.patch.3, HBASE-13992.patch.4, HBASE-13992.patch.5


 This Jira is to ask if SparkOnHBase can find a home in side HBase core.
 Here is the github: 
 https://github.com/cloudera-labs/SparkOnHBase
 I am the core author of this project and the license is Apache 2.0
 A blog explaining this project is here
 http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
 A spark Streaming example is here
 http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
 A real customer using this in produce is blogged here
 http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/
 Please debate and let me know what I can do to make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-12751) Allow RowLock to be reader writer


 [ 
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-12751:
--
Attachment: HBASE-12751-v15.patch

 Allow RowLock to be reader writer
 -

 Key: HBASE-12751
 URL: https://issues.apache.org/jira/browse/HBASE-12751
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 2.0.0, 1.3.0
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 2.0.0, 1.3.0

 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, 
 HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, 
 HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, 
 HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, 
 HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, 
 HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch


 Right now every write operation grabs a row lock. This is to prevent values 
 from changing during a read modify write operation (increment or check and 
 put). However it limits parallelism in several different scenarios.
 If there are several puts to the same row but different columns or stores 
 then this is very limiting.
 If there are puts to the same column then mvcc number should ensure a 
 consistent ordering. So locking is not needed.
 However locking for check and put or increment is still needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-11339) HBase MOB

2015-07-22 Thread Jonathan Hsieh (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Hsieh updated HBASE-11339:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
   Status: Resolved  (was: Patch Available)

I've merged the branch in to master now.  Thanks Jingcheng for all the work, 
along with ram, anoop, and ted for the reviews.  Also thanks to the folks who 
participated on the pre-merge discussion thread and votes.

I committed before I could get a full run of all the unit tests in to avoid 
commit race (i was about do and then there was another commit that landed) -- 
we'll tackle issues if any arise due to this as we'd handle with normal patches.

 HBase MOB
 -

 Key: HBASE-11339
 URL: https://issues.apache.org/jira/browse/HBASE-11339
 Project: HBase
  Issue Type: Umbrella
  Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: 2.0.0, hbase-11339

 Attachments: 11339-master-v10.patch, 11339-master-v3.txt, 
 11339-master-v4.txt, 11339-master-v5.txt, 11339-master-v6.txt, 
 11339-master-v7.txt, 11339-master-v8.patch, 11339-master-v9.patch, HBase MOB 
 Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB 
 Design-v5.pdf, HBase MOB Design.pdf, MOB user guide.docx, MOB user 
 guide_v2.docx, MOB user guide_v3.docx, MOB user guide_v4.docx, MOB user 
 guide_v5.docx, hbase-11339-150519.patch, hbase-11339-in-dev.patch, 
 hbase-11339.150417.patch, merge-150212.patch, merge.150212b.patch, 
 merge.150212c.patch, merge.150710.patch


   It's quite useful to save the medium binary data like images, documents 
 into Apache HBase. Unfortunately directly saving the binary MOB(medium 
 object) to HBase leads to a worse performance since the frequent split and 
 compaction.
   In this design, the MOB data are stored in an more efficient way, which 
 keeps a high write/read performance and guarantees the data consistency in 
 Apache HBase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637172#comment-14637172
 ] 

Hudson commented on HBASE-14115:


SUCCESS: Integrated in HBase-1.2-IT #62 (See 
[https://builds.apache.org/job/HBase-1.2-IT/62/])
HBASE-14115 Fix resource leak in HMasterCommandLine (Yuhao Bi) (tedyu: rev 
f0042a4038486cfb17ba807134ea448617c8b94d)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java


 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.2.0, 1.3.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637202#comment-14637202
 ] 

Hudson commented on HBASE-14115:


SUCCESS: Integrated in HBase-1.3-IT #54 (See 
[https://builds.apache.org/job/HBase-1.3-IT/54/])
HBASE-14115 Fix resource leak in HMasterCommandLine (Yuhao Bi) (tedyu: rev 
45d52129dd53207efb0a6408cf6869e0d5b23df3)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java


 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.2.0, 1.3.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14058) Stabilizing default heap memory tuner

[
https://issues.apache.org/jira/browse/HBASE-14058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Elliott Clark updated HBASE-14058:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
2.0.0
Status: Resolved (was: Patch Available)

Pushed to branch-1 and master. Thanks for the patch.

Stabilizing default heap memory tuner
-

Attachments: 0001-Stabilizing-default-heap-memory-tuner.patch,
HBASE-14058-v1.patch, HBASE-14058.patch, after_modifications.png,
before_modifications.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14145) Allow the Canary in regionserver mode to try all regions on the server, not just one

Elliott Clark created HBASE-14145:
-

 Summary: Allow the Canary in regionserver mode to try all regions 
on the server, not just one
 Key: HBASE-14145
 URL: https://issues.apache.org/jira/browse/HBASE-14145
 Project: HBase
  Issue Type: Bug
  Components: canary, util
Affects Versions: 1.1.0.1, 2.0.0
Reporter: Elliott Clark
 Fix For: 2.0.0, 1.3.0


We want a pretty in-depth canary that will try every region on a cluster. When 
doing that for the whole cluster one machine is too slow, so we wanted to split 
it up and have each regionserver run a canary. That works however the canary 
does less work as it just tries one random region.

Lets add a flag that will allow the canary to try all regions on a regionserver.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer

[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637558#comment-14637558
]

Elliott Clark commented on HBASE-12751:
---

Do we want to make the locking fair? or should normal puts be able to starve
check and mutates?
Fairness will increase the size of the read write lock. However it will keep
things from starving.

Allow RowLock to be reader writer
-

Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch,
HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch,
HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch,
HBASE-12751-v2.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch,
HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch,
HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636717#comment-14636717
 ] 

Yuhao Bi commented on HBASE-14115:
--

You are right, adm.shutdown() is not in the finally block.
Here comes the patch.

Thanks.

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0

 Attachments: HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14115) Fix resource leak in HMasterCommandLine


 [ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuhao Bi updated HBASE-14115:
-
Attachment: HBASE-14115-branch-1.patch

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0

 Attachments: HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine

2015-07-22 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636636#comment-14636636
 ] 

Ted Yu commented on HBASE-14115:


In branch-1, we have:
{code}
Admin adm = null;
{code}
The adm is never explicitly closed.

Can you attach a patch to fix that ?

Thanks

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0

 Attachments: HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14115) Fix resource leak in HMasterCommandLine

2015-07-22 Thread Ted Yu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-14115:
---
   Resolution: Fixed
Fix Version/s: 1.3.0
   1.2.0
   Status: Resolved  (was: Patch Available)

Thanks for the patch, Yuhao.

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636927#comment-14636927
 ] 

Hadoop QA commented on HBASE-14115:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12746526/HBASE-14115-branch-1.patch
  against branch-1 branch at commit 5ec5552be0534dbf4b07ef6607741ae6f9ab0495.
  ATTACHMENT ID: 12746526

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14859//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14859//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14859//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14859//console

This message is automatically generated.

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0, 1.2.0, 1.3.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13992) Integrate SparkOnHBase into HBase


[ 
https://issues.apache.org/jira/browse/HBASE-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636943#comment-14636943
 ] 

Ted Malaska commented on HBASE-13992:
-

So today from 3 to 6 I will have time to work on this.  So if this jira is 
closed by 3 I will make a new one for the HBaseRDD.collect other wise I will 
add an additional patch to this jira.

But after that next patch please help me close this out.  I do have a number of 
jiras I would like to start to add additional functionality to this original 
patch

 Integrate SparkOnHBase into HBase
 -

 Key: HBASE-13992
 URL: https://issues.apache.org/jira/browse/HBASE-13992
 Project: HBase
  Issue Type: New Feature
  Components: spark
Reporter: Ted Malaska
Assignee: Ted Malaska
 Fix For: 2.0.0

 Attachments: HBASE-13992.5.patch, HBASE-13992.6.patch, 
 HBASE-13992.7.patch, HBASE-13992.patch, HBASE-13992.patch.3, 
 HBASE-13992.patch.4, HBASE-13992.patch.5


 This Jira is to ask if SparkOnHBase can find a home in side HBase core.
 Here is the github: 
 https://github.com/cloudera-labs/SparkOnHBase
 I am the core author of this project and the license is Apache 2.0
 A blog explaining this project is here
 http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
 A spark Streaming example is here
 http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
 A real customer using this in produce is blogged here
 http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/
 Please debate and let me know what I can do to make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14144) Bloomfilter path to work with Byte buffered cells

2015-07-22 Thread ramkrishna.s.vasudevan (JIRA)

ramkrishna.s.vasudevan created HBASE-14144:
--

 Summary: Bloomfilter path to work with Byte buffered cells
 Key: HBASE-14144
 URL: https://issues.apache.org/jira/browse/HBASE-14144
 Project: HBase
  Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 2.0.0


This JIRA is to check if there will be a need to make the bloom filters to work 
with ByteBuffer cells. During POC this path created lot of duplicated code but 
considering other refactorings done in this path  may lead to less duplication. 
This JIRA is a placeholder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14115) Fix resource leak in HMasterCommandLine

2015-07-22 Thread Ted Yu (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636769#comment-14636769
 ] 

Ted Yu commented on HBASE-14115:


shutdown() call can be in try block.
What I meant was that the close of adm should be in finally block.

What do you think ?

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-14115) Fix resource leak in HMasterCommandLine


 [ 
https://issues.apache.org/jira/browse/HBASE-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuhao Bi updated HBASE-14115:
-
Attachment: HBASE-14115-branch-1-v1.patch

patch v2 for branch-1, should work.

 Fix resource leak in HMasterCommandLine
 ---

 Key: HBASE-14115
 URL: https://issues.apache.org/jira/browse/HBASE-14115
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yuhao Bi
Assignee: Yuhao Bi
 Fix For: 2.0.0

 Attachments: HBASE-14115-branch-1-v1.patch, 
 HBASE-14115-branch-1.patch, HBASE-14115.patch


 In HMasterCommandLine#stopMaster(), admin is not closed.
 {code:title=HMasterCommandLine.java|borderStyle=solid}
 try (Connection connection = ConnectionFactory.createConnection(conf)) {
   try (Admin admin = connection.getAdmin()) {
 connection.getAdmin().shutdown();
   } catch (Throwable t) {
 LOG.error(Failed to stop master, t);
 return 1;
   }
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14058) Stabilizing default heap memory tuner

[
https://issues.apache.org/jira/browse/HBASE-14058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637726#comment-14637726
]

Hudson commented on HBASE-14058:

SUCCESS: Integrated in HBase-1.3 #70 (See
[https://builds.apache.org/job/HBase-1.3/70/])
HBASE-14058 Stabilizing default heap memory tuner (eclark: rev
89ecb93c95b63159cca427100b6003802870f1d8)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultHeapMemoryTuner.java

Stabilizing default heap memory tuner
-

Attachments: 0001-Stabilizing-default-heap-memory-tuner.patch,
HBASE-14058-v1.patch, HBASE-14058.patch, after_modifications.png,
before_modifications.png

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13739) Remove KeyValueUtil.ensureKeyValue(cell) from MOB code.


[ 
https://issues.apache.org/jira/browse/HBASE-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637832#comment-14637832
 ] 

Hudson commented on HBASE-13739:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13739 Remove KeyValueUtil.ensureKeyValue(cell) from MOB code.(Jingcheng) 
(anoopsamjohn: rev 132f65ea1f7b4681809dd5e545b3b3b802f0c469)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MemStoreWrapper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java


 Remove KeyValueUtil.ensureKeyValue(cell) from MOB code.
 ---

 Key: HBASE-13739
 URL: https://issues.apache.org/jira/browse/HBASE-13739
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13739.diff


 Now KeyValueUtil.ensureKeyValue(cell) is used in a few places of MOB, we need 
 to remove them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13141) IntegrationTestAcidGuarantees returns incorrect values for getColumnFamilies


[ 
https://issues.apache.org/jira/browse/HBASE-13141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637802#comment-14637802
 ] 

Hudson commented on HBASE-13141:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13141 IntegrationTestAcidGuarantees returns incorrect values for 
getColumnFamilies (jmhsieh: rev cf4f4fcbb37fd80572aa11f01b9255a5a3a613c2)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java


 IntegrationTestAcidGuarantees returns incorrect values for getColumnFamilies
 

 Key: HBASE-13141
 URL: https://issues.apache.org/jira/browse/HBASE-13141
 Project: HBase
  Issue Type: Bug
  Components: integration tests
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: 2.0.0, hbase-11339, 1.0.1, 1.1.0

 Attachments: hbase-13141.patch


 IntegrationTestAcidGuarantees with monkeys on fails occasionally after a 
 RemoveColumnAction because the getColumnFamilies values are incorrect.   We 
 actually were naming the cf's [B@12ef34ab instead of A, B, and C.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11683) Metrics for MOB


[ 
https://issues.apache.org/jira/browse/HBASE-11683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637855#comment-14637855
 ] 

Hudson commented on HBASE-11683:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11683 Metrics for MOB (Jingcheng Du) (jmhsieh: rev 
e5a1b86dbad650f9740eb24a7f544c4a51f05654)
* 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedMobStoreScanner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* 
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobCacheConfig.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* 
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MobStoreScanner.java


 Metrics for MOB
 ---

 Key: HBASE-11683
 URL: https://issues.apache.org/jira/browse/HBASE-11683
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11683-V2.diff, HBASE-11683-V3.diff, 
 HBASE-11683-V4.diff, HBASE-11683-V5.diff, HBASE-11683-V6.diff, 
 HBASE-11683-V7.diff, HBASE-11683-V8.diff, HBASE-11683.diff


 We need to make sure to capture metrics about mobs.
 Some basic ones include:
 # of mob writes
 # of mob reads
 # avg size of mob (?)
 # mob files
 # of mob compactions / sweeps



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13763) Handle the rename, annotation and typo stuff in MOB


[ 
https://issues.apache.org/jira/browse/HBASE-13763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637862#comment-14637862
 ] 

Hudson commented on HBASE-13763:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13763 Handle the rename, annotation and typo stuff in MOB. (Jingcheng) 
(anoopsamjohn: rev b31a6acf4c24d55ac74eaf4d669a33078e27ef89)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ExpiredMobFileCleaner.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobStoreEngine.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
* hbase-shell/src/main/ruby/hbase/admin.rb
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactionRequest.java
* 
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/MobCompactionRequest.java
* hbase-common/src/main/resources/hbase-default.xml
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/MobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestPartitionedMobFileCompactor.java
* 
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactionRequest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/CompactMobAction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobFileCompactionChore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestPartitionedMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/MobFileCompactionRequest.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMobCompactionThread.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestMobFileCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MemStoreWrapper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCache.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMobFileCompactionThread.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactionRequest.java
* 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobCompactionChore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/HFileLink.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperStub.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestPartitionedMobFileCompactionRequest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileName.java
*

[jira] [Commented] (HBASE-13151) IllegalArgumentException in compaction when table has a namespace


[ 
https://issues.apache.org/jira/browse/HBASE-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637813#comment-14637813
 ] 

Hudson commented on HBASE-13151:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13151 IllegalArgumentException in compaction when table has a namespace 
(Jingcheng Du) (jmhsieh: rev 8b4671c10e64f7b6fdedcbac7f6f9de3942f79ba)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestMobFileCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java


 IllegalArgumentException in compaction when table has a namespace
 -

 Key: HBASE-13151
 URL: https://issues.apache.org/jira/browse/HBASE-13151
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13151-V2.diff, HBASE-13151-V3.diff, 
 HBASE-13151-V4.diff, HBASE-13151-V5.diff, HBASE-13151.diff


 In the PartitionedMobFileCompactor we have a bulk load path, currently it 
 directly use table.getNameAsString as part of the path, if the table has a 
 namespace, the path contains character : which is regarded as a illegal 
 character in file path. As fixing, we need to divide the namespace and 
 qualifier into directories like the way in table path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13720) Mob files are not encrypting in mob compaction and Sweeper


[ 
https://issues.apache.org/jira/browse/HBASE-13720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637809#comment-14637809
 ] 

Hudson commented on HBASE-13720:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13720 - Mob files are not encrypting in mob compaction and Sweeper 
(ramkrishna: rev 5428c9fdd3517889e80b80486010d6b6ceb698b4)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestMobFileCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MemStoreWrapper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 Mob files are not encrypting in mob compaction and Sweeper
 --

 Key: HBASE-13720
 URL: https://issues.apache.org/jira/browse/HBASE-13720
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13720-V2.diff, HBASE-13720-V3.diff, 
 HBASE-13720.diff


 The mob files are not encrypted. Part of the issue was fixed in HBASE-13693. 
 Still we have more places need the encryption too, for example the writer 
 used in mob file compaction and Sweeper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13302) fix new javadoc warns introduced by mob.


[ 
https://issues.apache.org/jira/browse/HBASE-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637797#comment-14637797
 ] 

Hudson commented on HBASE-13302:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13302 fix new javadoc introduced by mob (jmhsieh: rev 
fe389d1f194c47742fba91e5e3424bb2c0eb0fce)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java


 fix new  javadoc warns introduced by mob.
 -

 Key: HBASE-13302
 URL: https://issues.apache.org/jira/browse/HBASE-13302
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-13302.hbase-11339.patch


 now that we have the qabot able to work on branches, we've detected a few 
 javadoc issues. this will fix it.
 {code}
 [WARNING] Javadoc Warnings
 [WARNING] 
 /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java:397:
  warning - @param argument path is not a parameter name.
 [WARNING] 
 /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:578:
  warning - @param argument @param is not a parameter name.
 [WARNING] 
 /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java:578:
  warning - @param argument path is not a parameter name.
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12591) Ignore the count of mob compaction metrics when there is issue


[ 
https://issues.apache.org/jira/browse/HBASE-12591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637843#comment-14637843
 ] 

Hudson commented on HBASE-12591:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12591 Ignore the count of mob compaction metrics when there is 
issue.(Jiajia Li) (anoopsamjohn: rev 33fc1918dea9a73e5bc4300db927d419016542f9)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java


 Ignore the count of mob compaction metrics when there is issue
 --

 Key: HBASE-12591
 URL: https://issues.apache.org/jira/browse/HBASE-12591
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Affects Versions: hbase-11339
Reporter: Jiajia Li
Assignee: Jiajia Li
Priority: Minor
 Fix For: hbase-11339

 Attachments: HBASE-12591-V2.patch, HBASE-12591.patch


 In mob compaction, the metrics of mobCompactedFromMobCellsCount and 
 mobCompactedFromMobCellsSize should not be count when there is issue when 
 retrieve the mob cell from the mob file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11647) MOB integration testing


[ 
https://issues.apache.org/jira/browse/HBASE-11647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637840#comment-14637840
 ] 

Hudson commented on HBASE-11647:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11647 MOB integration testing. (Jingcheng Du) (anoopsamjohn: rev 
3e563c5cc72152cec0742956b7ec61b1841e78ac)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestDataGeneratorWithMOB.java
HBASE-11647 MOB integration testing. - addendum(Jingcheng Du) (anoopsamjohn: 
rev 0c86d83e1f966b5e0c72e53664f7e9ff5a71e488)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java
HBASE-11647 addendum to fix compile issues. (anoopsamjohn: rev 
e5d3850776174a63ddc2e0b5ead58409ca7c8706)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java


 MOB integration testing
 ---

 Key: HBASE-11647
 URL: https://issues.apache.org/jira/browse/HBASE-11647
 Project: HBase
  Issue Type: Sub-task
  Components: Performance, test
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11647-without-sweep-tool-V2.diff, 
 HBASE-11647-without-sweep-tool-V3.diff, 
 HBASE-11647-without-sweep-tool-addendum.diff, 
 HBASE-11647-without-sweep-tool.diff, HBASE-11647.diff, 
 HBASE-11647_addendum2.patch


 The integration testings include the integration function testing and 
 performance testing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13277) add mob_threshold option to load test tool


[ 
https://issues.apache.org/jira/browse/HBASE-13277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637868#comment-14637868
 ] 

Hudson commented on HBASE-13277:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13277 add mob_threshold option to load test tool (jmhsieh: rev 
753ceb6e7641d2b5b2e32d51742bc5246aa48b90)
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java


 add mob_threshold option to load test tool
 --

 Key: HBASE-13277
 URL: https://issues.apache.org/jira/browse/HBASE-13277
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: HBASE-13277.hbase-11339.patch, 
 hbase-13277.v2-hbase-11339.patch


 This adds '-mob_threshold value' option to the load test tool to simplify 
 mob load testing 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12391) Correct a typo in the mob metrics


[ 
https://issues.apache.org/jira/browse/HBASE-12391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637824#comment-14637824
 ] 

Hudson commented on HBASE-12391:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12391 Correct a typo in the mob metrics (Jingcheng Du) (ramkrishna: rev 
3876bb764d6ac2bd8b4d1f5f7b8e73cd36d32e59)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java


 Correct a typo in the mob metrics
 -

 Key: HBASE-12391
 URL: https://issues.apache.org/jira/browse/HBASE-12391
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Priority: Minor
 Fix For: hbase-11339

 Attachments: HBASE-12391.diff


 There's a typo in the temp variable in the region server metrics for mob. 
 It's now testMobCompactedFromMobCellsSize, and should be changed to 
 tempMobCompactedFromMobCellsSize



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12820) Use table lock instead of MobZookeeper


[ 
https://issues.apache.org/jira/browse/HBASE-12820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637853#comment-14637853
 ] 

Hudson commented on HBASE-12820:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12820 Use table lock instead of MobZookeeper.(Jingcheng Du) 
(anoopsamjohn: rev fbbb3249d9ef6aa5bf12ca2abbf67f4a89c86c18)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepMapper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJobNodeTracker.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepReducer.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobZookeeper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepMapper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java


 Use table lock instead of MobZookeeper
 --

 Key: HBASE-12820
 URL: https://issues.apache.org/jira/browse/HBASE-12820
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12820-V2.diff, HBASE-12820-V3.diff, 
 HBASE-12820-V4.diff, HBASE-12820-V5.diff, HBASE-12820-V6.diff, 
 HBASE-12820-V7.diff, HBASE-12820.diff


 We had a lock to synchronize the major compaction, and sweep tool. Now we 
 will have MR-less mob compaction in the HBase, and need the lock as well. And 
 the table lock is a better choice. In this JIRA, clean the MobZookeeper code 
 and use TableLockManager instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13804) Revert the changes in pom.xml


[ 
https://issues.apache.org/jira/browse/HBASE-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637849#comment-14637849
 ] 

Hudson commented on HBASE-13804:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13804 Revert the changes in pom.xml (Jingcheng Du) (jmhsieh: rev 
efbef296d60462580b2551ec4899728dff358c42)
* pom.xml


 Revert the changes in pom.xml
 -

 Key: HBASE-13804
 URL: https://issues.apache.org/jira/browse/HBASE-13804
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13804.diff


 Some code were delete in pom.xml.
 {noformat}
 systemPropertyVariables
jacoco-agent.destfiletarget/jacoco.exec/jacoco-agent.destfile
 /systemPropertyVariables
 {noformat}
 We can revert the changes if this change is not necessary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13922) Do not reset mvcc in compactions for mob-enabled column


[ 
https://issues.apache.org/jira/browse/HBASE-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637847#comment-14637847
 ] 

Hudson commented on HBASE-13922:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13922 Do not reset mvcc in compactions for mob-enabled column.(Jingcheng 
Du) (anoopsamjohn: rev 3f062ee23668020c15f9d06a966a0978ca9373f6)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java


 Do not reset mvcc in compactions for mob-enabled column
 ---

 Key: HBASE-13922
 URL: https://issues.apache.org/jira/browse/HBASE-13922
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13922.patch


 In major compaction, the mvcc of cells ( whose mvcc=readPt) are set to 0.
 In some cases, this brings issues, for example the following scenario:
 # We have mob enabled cf, the threshold is 5 bytes.
 # Add a cell (r0,ts0,seqId=5,mobValue0), and flush it to a mob file.
 # Add another cell (r0,ts0,seqId=10,new), and flush the memstore, this is 
 not a mob cell since it's value is smaller than 5 bytes.
 # Add the third cell (r1:ts1:seqId =15, mobValue1), and flush it to a mob 
 file. Now we have two mob files.
 # Now run a major compaction in hfiles, we got two cells left 
 (r0:ts0:seqId=0,new) and (r1:ts1:seqId=0,'mobValue1).
 # Now run a mob major compaction, two mob files are merged into one. The 
 update ref cell is bulk loaded back to hbase, they are 
 (r0,ts0,seqId=5,mobValue0) and (r1:ts1:seqId=0,mobValue1).
 # Now open a scanner, the value of r0 is mobValue0 whereas the correct value 
 new.
 This issue is caused by the mvcc reset in compactions. We should disable it 
 in compactions for mob-enabled columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11644) External MOB compaction tools


[ 
https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637844#comment-14637844
 ] 

Hudson commented on HBASE-11644:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11644 External MOB compaction tools (Jingcheng Du) (jmhsieh: rev 
84e957c875ae971578a5b147775445368ea26188)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepJob.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepMapper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MemStoreWrapper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ExpiredMobFileCleaner.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MobReferenceOnlyFilter.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJobNodeTracker.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobZookeeper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestExpiredMobFileCleaner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepMapper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MobFilePathHashPartitioner.java
* hbase-common/src/main/resources/hbase-default.xml
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepReducer.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java


 External MOB compaction tools
 -

 Key: HBASE-11644
 URL: https://issues.apache.org/jira/browse/HBASE-11644
 Project: HBase
  Issue Type: Sub-task
  Components: Compaction, master
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11644-Sep-15.diff, HBASE-11644-Sep-16.diff, 
 HBASE-11644-Sep-16.diff, HBASE-11644-Sep-18.diff, 
 HBASE-11644-Sep-19-V2.patch, HBASE-11644-Sep-19.diff, HBASE-11644.diff, 
 HBASE-11646-0918-bad.patch


 From the design doc,  mob files are not involved in the normal HBase 
 compaction process.  This means deleted mobs would still take up space and 
 that we never really merge mob files that accrue over time.   Currently, MOBs 
 depend on two external tools:
 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded 
 minVersions.
 2) A 'sweep tool' cleaner that remove mobs that have had their references 
 deleted and merges small files into larger ones.  
 Today the tools are triggered by admins.  The longer term goal would be to 
 integrate them into hbase such that by default mobs are cleaned.  The tools 
 will be preserved however so that advanced admins can disable automatic 
 cleanups and manually trigger these compaction like operaitons.  #1 would 
 likely be a chore in the master while #2 requires some design work to 
 integrate into hbase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11643) Read and write MOB in HBase


[ 
https://issues.apache.org/jira/browse/HBASE-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637842#comment-14637842
 ] 

Hudson commented on HBASE-11643:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11643 Read and write MOB in HBase (Jingcheng Du) (jmhsieh: rev 
5c14f749b3196fe5d9e1efe6dd5d6d7356cc72d0)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFile.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobDataBlockEncoding.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFileCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobStoreEngine.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileName.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobCacheConfig.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFileName.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/CachedMobFile.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestDefaultMobStoreFlusher.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestCachedMobFile.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MobStoreScanner.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/MobTestUtil.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDeleteMobTable.java
* hbase-common/src/main/resources/hbase-default.xml
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedMobStoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreEngine.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobFileCache.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHMobStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/TagType.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFile.java


 Read and write MOB in HBase
 ---

 Key: HBASE-11643
 URL: https://issues.apache.org/jira/browse/HBASE-11643
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11643-V10.diff, HBASE-11643-V11.diff, 
 HBASE-11643-V12.diff, HBASE-11643-V13.diff, HBASE-11643-V14.diff, 
 HBASE-11643-V15.diff, HBASE-11643-V16.diff, HBASE-11643-V17.diff, 
 HBASE-11643-V18.diff, HBASE-11643-V19.diff, HBASE-11643-V2.diff, 
 HBASE-11643-V20.diff, HBASE-11643-V3.diff, HBASE-11643-V4.diff, 
 HBASE-11643-V5.diff, HBASE-11643-V6.diff, HBASE-11643-V7.diff, 
 HBASE-11643-V8.diff, HBASE-11643-V9.diff, HBase-11643.diff


 The read/write MOB in HBase are implemented in this JIRA. Normally, the Cells 
 are saved in the MemStore, and flushed to the HFiles when necessary. For MOB, 
 the Cells are saved in the MemStore as well, but they're flushed to elsewhere 
 out of HBase in the format of HFiles.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13132) Improve RemoveColumn action debug message


[ 
https://issues.apache.org/jira/browse/HBASE-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637856#comment-14637856
 ] 

Hudson commented on HBASE-13132:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13132 Improve RemoveColumn action debug message (jmhsieh: rev 
d66f88134cfaf93eab50c5d5584f1e36f519c98c)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/RemoveColumnAction.java


 Improve RemoveColumn action debug message 
 --

 Key: HBASE-13132
 URL: https://issues.apache.org/jira/browse/HBASE-13132
 Project: HBase
  Issue Type: Improvement
  Components: integration tests
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Trivial
 Fix For: 2.0.0, hbase-11339, 1.0.1, 1.1.0, 0.98.13

 Attachments: hbase-13132.patch


 Trivial fix for this unsightly log message:
 {code}
 2015-02-22 14:04:46,357 DEBUG [TwoConcurrentAction-1-thread-2] 
 actions.Action: Performing action: Removing [B@64275127 from 
 TestAcidGuarantees
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12201) Close the writers in the MOB sweep tool


[ 
https://issues.apache.org/jira/browse/HBASE-12201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637846#comment-14637846
 ] 

Hudson commented on HBASE-12201:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12201 Close the writers in the MOB sweep tool (Jingcheng Du) (ramkrishna: 
rev aa523164e825567d93eb0b2a191955ca195ea242)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java


 Close the writers in the MOB sweep tool
 ---

 Key: HBASE-12201
 URL: https://issues.apache.org/jira/browse/HBASE-12201
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Priority: Minor
 Fix For: hbase-11339

 Attachments: HBASE-12201-V2.diff, HBASE-12201-V3.diff, 
 HBASE-12201.diff


  When running the sweep tool, we encountered such an exception.
 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
  No lease on 
 /hbase/mobdir/.tmp/mobcompaction/SweepJob-SweepMapper-SweepReducer-testSweepToolExpiredNoMinVersion-data/working/names/all
  (inode 46500): File does not exist. Holder 
 DFSClient_NONMAPREDUCE_-1863270027_1 does not have any open files.
 at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3319)
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3407)
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3377)
   at 
 org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:673)
   at 
 org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.complete(AuthorizationProviderProxyClientProtocol.java:219)
   at 
 org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:520)
   at 
 org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
   at 
 org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:415)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
   at org.apache.hadoop.ipc.Client.call(Client.java:1411)
   at org.apache.hadoop.ipc.Client.call(Client.java:1364)
   at 
 org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
   at com.sun.proxy.$Proxy15.complete(Unknown Source)
   at 
 org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:435)
   at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at 
 org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy16.complete(Unknown Source)
at 
 org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2180)
at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2164)
at 
 org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:908)
   at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:925)
   at 
 org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:861)
   at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2687)
   at 
 org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2704)
   at 
 org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
 This is because we have several writers opened by fs.create(path, true) are 
 not closed properly.
 Meanwhile, in the current implementation, we save the temp files under 
 sweepJobDir/working/..., and when we remove the directory of the sweep job 
 only the working is deleted. We should remove the whole sweepJobDir instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12000) isMob and mobThreshold do not follow column descriptor property naming conventions


[ 
https://issues.apache.org/jira/browse/HBASE-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637857#comment-14637857
 ] 

Hudson commented on HBASE-12000:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12000 isMob and mobThreshold do not adhere to naming conventions 
(mstanleyjones: rev d147eced575ef9a47c4abdcb87931dd11b4f6c37)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java


 isMob and mobThreshold do not follow column descriptor property naming 
 conventions
 --

 Key: HBASE-12000
 URL: https://issues.apache.org/jira/browse/HBASE-12000
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: hbase-11339

 Attachments: HBASE-12000-v1.patch, HBASE-12000.patch


 This patch changes isMob to IS_MOB and mobThreshold to MOB_THRESHOLD to go 
 with conventions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13230) [mob] reads hang when trying to read rows with large mobs (10MB)


[ 
https://issues.apache.org/jira/browse/HBASE-13230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637861#comment-14637861
 ] 

Hudson commented on HBASE-13230:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13230 [mob] reads hang when trying to read rows with large mobs (10MB) 
(jmhsieh: rev aedd0ebe9b8d1968f7d91772b369829e00606087)
* hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
* 
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/AsyncServerResponseHandler.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java


 [mob] reads hang when trying to read rows with large mobs (10MB)
 -

 Key: HBASE-13230
 URL: https://issues.apache.org/jira/browse/HBASE-13230
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: HBASE-13230.hbase-11339.patch


 Using load tests tool to read and write out 5MB, 10MB, 20MB objects works 
 fine, but problems are encountered when trying to read values 20MB.  This is 
 due to the default protobuf size limit of 64MB.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13152) NPE in ExpiredMobFileCleanerChore


[ 
https://issues.apache.org/jira/browse/HBASE-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637818#comment-14637818
 ] 

Hudson commented on HBASE-13152:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13152 NPE in ExpiredMobFileCleanerChore (Jingcheng Du) (jmhsieh: rev 
d55405a7cee5747af294c0784e02dc41ef379521)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java


 NPE in ExpiredMobFileCleanerChore
 -

 Key: HBASE-13152
 URL: https://issues.apache.org/jira/browse/HBASE-13152
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13152.diff


 NullPointerException in ExpiredMobFileCleanerChore. In the 
 ExpiredMobFileCleanerChore, we invoked ExpiredMobFileCleaner to clean the 
 expired mob files. But we didn't pass the conf in which led to a NPE when 
 invoking methods of conf in cleaner. This patch will fix the NPE issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13805) Use LimitInputStream in hbase-common instead of ProtobufUtil.LimitedInputStream


[ 
https://issues.apache.org/jira/browse/HBASE-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637851#comment-14637851
 ] 

Hudson commented on HBASE-13805:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13805 Use LimitInputStream in hbase-common instead of 
ProtobufUtil.LimitedInputStream. (Jingcheng) (anoopsamjohn: rev 
b5641d8edf7c01c73cae006790e4482f8112dd4a)
* hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java


 Use LimitInputStream in hbase-common instead of 
 ProtobufUtil.LimitedInputStream
 ---

 Key: HBASE-13805
 URL: https://issues.apache.org/jira/browse/HBASE-13805
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13805.diff


 Now we have a LimitedInputStream defined in ProtobufUtil.java. We have 
 similar code in LimitInputStream of hbase-common, we can use it instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11901) Improve the value size of the reference cell in mob column


[ 
https://issues.apache.org/jira/browse/HBASE-11901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637863#comment-14637863
 ] 

Hudson commented on HBASE-11901:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11901 Improve the value size of the reference cell in mob 
column.(Jingcheng Du) (anoopsamjohn: rev 
e8938c40d6d26ede097f7e2a5397fe45f38e6486)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHMobStore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDeleteMobTable.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 Improve the value size of the reference cell in mob column
 --

 Key: HBASE-11901
 URL: https://issues.apache.org/jira/browse/HBASE-11901
 Project: HBase
  Issue Type: Sub-task
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11901.diff


  Now in the value of a reference cell in a mob column, it's 
 realMobValueLength(Use a long type currently) + mobFileName.
  Actually the value length in cell is a int type, it's not necessary to use a 
 long here. In the improvement, we'll use the int type instead long in a 
 reference mob value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13012) Add shell commands to trigger the mob file compactor


[ 
https://issues.apache.org/jira/browse/HBASE-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637812#comment-14637812
 ] 

Hudson commented on HBASE-13012:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13012 Add shell commands to trigger the mob file compactor (Jingcheng Du 
and Jiajia Li) (jmhsieh: rev 47ed5cd7ed0da2976ab6bf10dec3816fa724bd55)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobFileCompactionChore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/MobFileCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestMobFileCompactor.java
* hbase-shell/src/main/ruby/shell/commands/major_compact_mob.rb
* hbase-shell/src/main/ruby/hbase/admin.rb
* hbase-shell/src/main/ruby/shell/commands/compact_mob.rb
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestPartitionedMobFileCompactor.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* hbase-shell/src/main/ruby/shell.rb
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterMobFileCompactionThread.java


 Add shell commands to trigger the mob file compactor 
 -

 Key: HBASE-13012
 URL: https://issues.apache.org/jira/browse/HBASE-13012
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13012-V5.diff, HBASE-13012-V6.diff, 
 HBASE-13012-V7.diff, HBASE-13012-V9.diff, HBASE-13012.v8.patch, 
 hbase-13012-V1.patch, hbase-13012-V2.diff, hbase-13012-V3.diff, 
 hbase-13012-V4.diff


 Currently the MobFileCompactor is run by HMaster periodically, we need to add 
 a shell to trigger the compactor by commands.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12030) Wrong compaction report and assert when MOB compaction switches to minor


[ 
https://issues.apache.org/jira/browse/HBASE-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637838#comment-14637838
 ] 

Hudson commented on HBASE-12030:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12030 Wrong compaction report and assert when MOB compaction switches to 
minor. (anoopsamjohn: rev a0d9e18ccf3842395a410a6c966f108ab2f55473)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 Wrong compaction report and assert when MOB compaction switches to minor
 

 Key: HBASE-12030
 URL: https://issues.apache.org/jira/browse/HBASE-12030
 Project: HBase
  Issue Type: Bug
  Components: Compaction, regionserver
Affects Versions: hbase-11339
Reporter: Matteo Bertozzi
Assignee: Anoop Sam John
Priority: Critical
 Fix For: hbase-11339

 Attachments: HBASE-12030-V2.diff, HBASE-12030.patch, 
 HBASE-12030_V3.patch


 when zookeeper is down during a major compaction or a sweep tool run in 
 progress, we switch to a minor.
 {code}
 try {
   zk = MobZookeeper.newInstance(this.conf, compactionName);
 } catch (KeeperException e) {
   LOG.error(Cannot connect to the zookeeper, ready to perform the minor 
 compaction instead, e);
   // change the major compaction into a minor one
   compaction.getRequest().setIsMajor(false);
   return super.compact(compaction);
 }
 {code}
 but the request start (HRegion.reportCompactionRequestStart) is major and 
 increments the major-compactions counter
 while the request end (HRegion.reportCompactionRequestEnd) is minor and 
 decrements the minor-compactions counter
 triggering the assert newValue = 0, since we are decrementing the wrong 
 counter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13836) Do not reset the mvcc for bulk loaded mob reference cells in reading


[ 
https://issues.apache.org/jira/browse/HBASE-13836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637833#comment-14637833
 ] 

Hudson commented on HBASE-13836:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13836 Do not reset the mvcc for bulk loaded mob reference cells in 
reading. (Jingcheng) (anoopsamjohn: rev 
26893aa451215ef0395b7df16f129414b7b86c86)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java


 Do not reset the mvcc for bulk loaded mob reference cells in reading
 

 Key: HBASE-13836
 URL: https://issues.apache.org/jira/browse/HBASE-13836
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13836-V2.diff, HBASE-13836-V3.diff, 
 HBASE-13836.diff


 Now in scanner, the cells mvcc of the bulk loaded files are reset by the 
 seqId parsed from the file name. We need to skip this if the hfiles are 
 bulkloaded in mob compactions.
 In mob compaction, the bulk loaded ref cell might not be the latest cell 
 among the ones that have the same row key. In reading, the mvcc is reset by 
 the largest one, it will cover the real latest ref cell. We have to skip the 
 resetting in this case.
 The solution is we add a new field to fileinfo, when this field is set as 
 true, we skip the resetting.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13762) Use the same HFileContext with store files in mob files


[ 
https://issues.apache.org/jira/browse/HBASE-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637865#comment-14637865
 ] 

Hudson commented on HBASE-13762:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13762 Use the same HFileContext with store files in mob files. 
(Jingcheng) (anoopsamjohn: rev 6388b3baf68f399f2c83c99da9687fd1cf4dcf66)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java


 Use the same HFileContext with store files in mob files
 ---

 Key: HBASE-13762
 URL: https://issues.apache.org/jira/browse/HBASE-13762
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13762.diff


 Now we use the default settings of HFileContext in mob files, we can use the 
 same ones with store files in mob files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12540) TestRegionServerMetrics#testMobMetrics test failure


[ 
https://issues.apache.org/jira/browse/HBASE-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637828#comment-14637828
 ] 

Hudson commented on HBASE-12540:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12540 TestRegionServerMetrics#testMobMetrics test failure (Jingcheng Du 
and Jiajia Li) (jmhsieh: rev bd5b3e92de37a8ff77d12e05131ba43eeb187a30)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java


 TestRegionServerMetrics#testMobMetrics test failure
 ---

 Key: HBASE-12540
 URL: https://issues.apache.org/jira/browse/HBASE-12540
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: hbase-11339
Reporter: stack
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12540-V2.patch, HBASE-12540.diff, 
 hbase-12540-v3.patch, log.txt


 Got this on an internal rig run.  Maybe you want to take a looksee 
 [~jingchengdu]?
 {code}
 Error Message
 Metrics Counters should be equal expected:5 but was:2
 Stacktrace
 java.lang.AssertionError: Metrics Counters should be equal expected:5 but 
 was:2
   at org.junit.Assert.fail(Assert.java:88)
   at org.junit.Assert.failNotEquals(Assert.java:743)
   at org.junit.Assert.assertEquals(Assert.java:118)
   at org.junit.Assert.assertEquals(Assert.java:555)
   at 
 org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.assertCounter(MetricsAssertHelperImpl.java:185)
   at 
 org.apache.hadoop.hbase.regionserver.TestRegionServerMetrics.testMobMetrics(TestRegionServerMetrics.java:448)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
 org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at 
 org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at 
 org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at 
 org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at 
 org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at 
 org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
   at 
 org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13107) Refactor MOB Snapshot logic to reduce code duplication.


[ 
https://issues.apache.org/jira/browse/HBASE-13107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637819#comment-14637819
 ] 

Hudson commented on HBASE-13107:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13107 Refactor MOB Snapshot logic to reduce code duplication (Jingcheng 
Du) (jmhsieh: rev dae8f236dcb20ed6e411777f69c362aef72ec390)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java


 Refactor MOB Snapshot logic to reduce code duplication.
 ---

 Key: HBASE-13107
 URL: https://issues.apache.org/jira/browse/HBASE-13107
 Project: HBase
  Issue Type: Sub-task
  Components: mob, snapshots
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13107.diff


 The MOB Snapshot code contains a lot of code duplication with the normal 
 snapshot code path. We should do some refactoring to clean this up before 
 merging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12489) Incorrect 'mobFileCacheMissCount' calculated in the mob metrics


[ 
https://issues.apache.org/jira/browse/HBASE-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637823#comment-14637823
 ] 

Hudson commented on HBASE-12489:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12489 Incorrect 'mobFileCacheMissCount' calculated in the mob 
(ramkrishna: rev 7aea38118e7af7d2c5e1ce54ca135b8c0261a2df)
* 
hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSourceImpl.java


  Incorrect 'mobFileCacheMissCount' calculated in the mob metrics
 

 Key: HBASE-12489
 URL: https://issues.apache.org/jira/browse/HBASE-12489
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jiajia Li
Assignee: Jiajia Li
 Fix For: hbase-11339

 Attachments: HBASE-12489-V1.patch


  It's now get the mobFileCacheMissCount from getMobFileCacheAccessCount(), 
 and should be changed to getMobFileCacheMissCount()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13157) Add mob compaction actions and monkeys to Chaos Monkey


[ 
https://issues.apache.org/jira/browse/HBASE-13157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637816#comment-14637816
 ] 

Hudson commented on HBASE-13157:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13157 Add Mob compaction actions and monkeys to Chaos Monkey (jmhsieh: 
rev 0d6cac9b1eaa311ee34c84ce00f1a24d4f98d823)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/MobSlowDeterministicMonkeyFactory.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/actions/CompactMobAction.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/MonkeyFactory.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/MobNoKillMonkeyFactory.java


 Add mob compaction actions and monkeys to Chaos Monkey
 --

 Key: HBASE-13157
 URL: https://issues.apache.org/jira/browse/HBASE-13157
 Project: HBase
  Issue Type: Sub-task
  Components: integration tests, mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-13157.patch


 After HBASE-13012 (mob compaction shell and admin api) we can now trigger mob 
 compaction.  This adds an action to chaos monkey so we can randomly trigger 
 these compactions during integration test runs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12698) Add mob cell count to the metadata of each mob file


[ 
https://issues.apache.org/jira/browse/HBASE-12698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637859#comment-14637859
 ] 

Hudson commented on HBASE-12698:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12698 Add mob cell count to the metadata of each mob file (Jingcheng Du) 
(jmhsieh: rev 917adbf0e53903e986ed2270a6b270495428b0fb)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/MemStoreWrapper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java


 Add mob cell count to the metadata of each mob file
 ---

 Key: HBASE-12698
 URL: https://issues.apache.org/jira/browse/HBASE-12698
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12698.diff


   Currently we don't know the cell count according to the existing file 
 information in the mob file. This value is very useful in the mob compaction, 
 we should have it in the metadata of each mob file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11986) Document MOB in Ref Guide


[ 
https://issues.apache.org/jira/browse/HBASE-11986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637869#comment-14637869
 ] 

Hudson commented on HBASE-11986:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11986 Document MOB in Reference Guide (mstanleyjones: rev 
7cd71d1db678fe8ab6efc2191310228d9bb27849)
* src/main/docbkx/ops_mgt.xml
* src/main/docbkx/book.xml
* src/main/docbkx/schema_design.xml
HBASE-11986 Addendum - Added hbase_mob.xml (mstanleyjones: rev 
ac65b1fb756cb3d8cb1df117ed4a7d056b122afe)
* src/main/docbkx/hbase_mob.xml
HBASE-11986 Fixed comma mistake (mstanleyjones: rev 
9cf46dcbe37c84a8468555c7ba7cb645bcaf1d22)
* src/main/docbkx/hbase_mob.xml


 Document MOB in Ref Guide
 -

 Key: HBASE-11986
 URL: https://issues.apache.org/jira/browse/HBASE-11986
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: hbase-11339

 Attachments: HBASE-11986-addendum.patch, HBASE-11986-v1.patch, 
 HBASE-11986-v3.patch, HBASE-11986.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13886) Return empty value when the mob file is corrupt instead of throwing exceptions


[ 
https://issues.apache.org/jira/browse/HBASE-13886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637866#comment-14637866
 ] 

Hudson commented on HBASE-13886:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13886 Return empty value when the mob file is corrupt instead of throwing 
exceptions. (Jingcheng) (anoopsamjohn: rev 
b889339006eff73428ba93f39c7dbe4ab483ff9a)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedMobStoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MobStoreScanner.java


 Return empty value when the mob file is corrupt instead of throwing exceptions
 --

 Key: HBASE-13886
 URL: https://issues.apache.org/jira/browse/HBASE-13886
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13886-V2.diff, HBASE-13886-V3.patch, 
 HBASE-13886-V4.patch, HBASE-13886-V5.patch, HBASE-13886.diff


 Now in reading, CorruptHFileException is thrown if the target mob file is 
 corrupt. We can return empty value for that mob cell instead of throwing 
 exceptions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12486) Move the mob table name tag to the 2nd one


[ 
https://issues.apache.org/jira/browse/HBASE-12486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637835#comment-14637835
 ] 

Hudson commented on HBASE-12486:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12486 - Move the mob table name tag to the 2nd one (Jingcheng Du) 
(ramkrishna: rev 0166ed2676c65d0e4c2ed94f60f093cdf9cb8f8a)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java


 Move the mob table name tag to the 2nd one
 --

 Key: HBASE-12486
 URL: https://issues.apache.org/jira/browse/HBASE-12486
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12486.diff


 Currently the tag for the mob table name is in the last position in tags.
 In order to accelerate the reading in both mob and cloned mob data, it's 
 better to move this tag to the 2nd position just after the mob ref tag.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12543) Incorrect log info in the store compaction of mob


[ 
https://issues.apache.org/jira/browse/HBASE-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637826#comment-14637826
 ] 

Hudson commented on HBASE-12543:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12543 Incorrect log info in the store compaction of mob (Jiajia Li) 
(ramkrishna: rev 10e4ef7402a7b7d0155b74311f61b87cd67bb692)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 Incorrect log info in the store compaction of mob
 -

 Key: HBASE-12543
 URL: https://issues.apache.org/jira/browse/HBASE-12543
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver
Affects Versions: hbase-11339
Reporter: Jiajia Li
Assignee: Jiajia Li
Priority: Minor
 Fix For: hbase-11339

 Attachments: HBASE-12543.diff


 Incorrect log info in the store compaction of mob



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12332) [mob] improve how we resolve mobfiles in reads


[ 
https://issues.apache.org/jira/browse/HBASE-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637858#comment-14637858
 ] 

Hudson commented on HBASE-12332:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12332 [mob] improve how we resolve mobfiles in reads (Jingcheng Du and 
Jiajia Li) (jmhsieh: rev 45711ebaafe938dc10c97b167c007f64ea492d03)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHMobStore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 [mob] improve how we resolve mobfiles in reads
 --

 Key: HBASE-12332
 URL: https://issues.apache.org/jira/browse/HBASE-12332
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jiajia Li
 Fix For: hbase-11339

 Attachments: HBASE-12332-V1.diff, HBASE-12332-V2.patch, 
 HBASE-12332-V3.patch, HBASE-12332-V5.patch, HBASE-12332-V6.diff, 
 hbase-12332.link.v4.patch, hbase-12332.patch


 in the snapshot code, hmobstore was modified to traverse an hfile link to a 
 mob.   Ideally this should use the transparent filelink code to read the data.
 Also there will likely be some issues with the mob file cache with these 
 links.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11861) Native MOB Compaction mechanisms.


[ 
https://issues.apache.org/jira/browse/HBASE-11861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637810#comment-14637810
 ] 

Hudson commented on HBASE-11861:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11861 Native MOB Compaction mechanisms (Jingcheng Du) (jmhsieh: rev 
2c4934eda68e8ed1290c2e3fb50604c2d77bdf64)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactionRequest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/MobFileCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobFileCompactionChore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestPartitionedMobFileCompactionRequest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestPartitionedMobFileCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/MobFileCompactionRequest.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/filecompactions/TestMobFileCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* hbase-common/src/main/resources/hbase-default.xml
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/filecompactions/PartitionedMobFileCompactor.java


 Native MOB Compaction mechanisms.
 -

 Key: HBASE-11861
 URL: https://issues.apache.org/jira/browse/HBASE-11861
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Jonathan Hsieh
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: 141030-mob-compaction.pdf, HBASE-11861-V1.diff, 
 HBASE-11861-V2.diff, HBASE-11861-V3.diff, HBASE-11861-V4.diff, 
 HBASE-11861-V5.diff, HBASE-11861-V6.diff, HBASE-11861.diff, mob 
 compaction-out-of-region.pdf, mob compaction.pdf


 Currently, the first cut of mob will have external processes to age off old 
 mob data (the ttl cleaner), and to compact away deleted or over written data 
 (the sweep tool).  
 From an operational point of view, having two external tools, especially one 
 that relies on MapReduce is undesirable.  In this issue we'll tackle 
 integrating these into hbase without requiring external processes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14076) ResultSerialization and MutationSerialization can throw InvalidProtocolBufferException when serializing a cell larger than 64MB


[ 
https://issues.apache.org/jira/browse/HBASE-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637852#comment-14637852
 ] 

Hudson commented on HBASE-14076:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-14076 ResultSerialization and MutationSerialization can throw 
InvalidProtocolBufferException when serializing a cell larger than 64MB 
(esteban: rev 7ddae3939e38ee4a910eef63c051c9d470d32629)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/ResultSerialization.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSerialization.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/MutationSerialization.java


 ResultSerialization and MutationSerialization can throw 
 InvalidProtocolBufferException when serializing a cell larger than 64MB
 ---

 Key: HBASE-14076
 URL: https://issues.apache.org/jira/browse/HBASE-14076
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, hbase-11339, 1.2.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
 Fix For: hbase-11339

 Attachments: HBASE-14076.hbase-11339.patch, 
 HBASE-14076.hbase-11339.patch, HBASE-14076.v2.hbase-11339.patch


 This was reported in CRUNCH-534 but is a problem how we handle 
 deserialization of large Cells ( 64MB) in ResultSerialization and 
 MutationSerialization.
 The fix is just re-using what it was done in HBASE-13230.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12670) Add unit tests that exercise the added hfilelink link mob paths


[ 
https://issues.apache.org/jira/browse/HBASE-12670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637808#comment-14637808
 ] 

Hudson commented on HBASE-12670:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12670 Add unit tests that exercise the added hfilelink link mob paths 
(Jingcheng Du) (jmhsieh: rev a8d0dfe71c9e508969ee88e4999d585e704b375d)
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFileLink.java


 Add unit tests that exercise the added hfilelink link mob paths
 ---

 Key: HBASE-12670
 URL: https://issues.apache.org/jira/browse/HBASE-12670
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12670.diff


 HBASe-12646 introduced the mob path to HFileLink -- we didn't add unit tests 
 for it however.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12085) mob status should print human readable numbers.


[ 
https://issues.apache.org/jira/browse/HBASE-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637870#comment-14637870
 ] 

Hudson commented on HBASE-12085:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12085 mob status should print human readable numbers.(Jingcheng Du) 
(anoopsamjohn: rev 629042f4cefb833ce65ae87a883cb92e2700b7b9)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobConstants.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/PrettyPrinter.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweepReducer.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobDataBlockEncoding.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestDefaultMobStoreFlusher.java
* src/main/docbkx/hbase_mob.xml
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobStoreScanner.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDeleteMobTable.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestMobFileCache.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestHColumnDescriptor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/ExpiredMobFileCleaner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHMobStore.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/TestExpiredMobFileCleaner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java


 mob status should print human readable numbers.
 ---

 Key: HBASE-12085
 URL: https://issues.apache.org/jira/browse/HBASE-12085
 Project: HBase
  Issue Type: Sub-task
  Components: mob, UI
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12085-V2.diff, HBASE-12085-V3.diff, 
 HBASE-12085-V4.diff, HBASE-12085-V5.diff, HBASE-12085.diff


 Currently, mob cf configuration stuff shows up as ugly byte strings instead 
 of meaningful numeric values.
 {code}
 IntegrationTestIngestWithMOB  20  'IntegrationTestIngestWithMOB', {NAME 
 = 'test_cf', METADATA = {'mobThreshold' = 
 '\x00\x00\x00\x00\x00\x01\x90\x00', 'isMob' = '\xFF'}}
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13806) Check the mob files when there are mob-enabled columns in HFileCorruptionChecker


[ 
https://issues.apache.org/jira/browse/HBASE-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637864#comment-14637864
 ] 

Hudson commented on HBASE-13806:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13806 Check the mob files when there are mob-enabled columns in 
HFileCorruptionChecker. (Jingcheng) (anoopsamjohn: rev 
13fe542bccea2ac70f052017ac71b99b10dbcda1)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/util/hbck/HFileCorruptionChecker.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java


 Check the mob files when there are mob-enabled columns in 
 HFileCorruptionChecker
 

 Key: HBASE-13806
 URL: https://issues.apache.org/jira/browse/HBASE-13806
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13806-V2.diff, HBASE-13806-V3.diff, 
 HBASE-13806-V4.diff, HBASE-13806-V5.diff, HBASE-13806-V6.diff, 
 HBASE-13806.diff


 Now in HFileCorruptionChecker, it only checks the files in regions. We need 
 check the mob files too if there are mob-enabled columns in that table.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12015) Not cleaning Mob data when Mob CF is removed from table


[ 
https://issues.apache.org/jira/browse/HBASE-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637796#comment-14637796
 ] 

Hudson commented on HBASE-12015:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12015 Not cleaning Mob data when Mob CF is removed from table.(Pankaj 
Kumar) (anoopsamjohn: rev d1f7bcbff7fbea1e253d3f9fb45cb98ae759916b)
* hbase-server/src/main/java/org/apache/hadoop/hbase/util/HFileArchiveUtil.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/MasterDDLOperationHelper.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/AddColumnFamilyProcedure.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteColumnFamilyProcedure.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDeleteMobTable.java


 Not cleaning Mob data when Mob CF is removed from table
 ---

 Key: HBASE-12015
 URL: https://issues.apache.org/jira/browse/HBASE-12015
 Project: HBase
  Issue Type: Sub-task
Affects Versions: hbase-11339
Reporter: Anoop Sam John
Assignee: Pankaj Kumar
 Fix For: hbase-11339

 Attachments: 12015-hbase-11339.patch, 
 HBASE-12015-hbase-11339-V2.patch, HBASE-12015-hbase-11339-V3.patch, 
 HBASE-12015-hbase-11339-V4.patch, HBASE-12015-hbase-11339-V5.patch, 
 HBASE-12015-hbase-11339.patch, HBASE-12015.patch


 During modifyTable, if a MOB CF is removed from a table, the corresponding 
 mob data also should get removed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13790) Remove the DeleteTableHandler


[ 
https://issues.apache.org/jira/browse/HBASE-13790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637803#comment-14637803
 ] 

Hudson commented on HBASE-13790:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13790 Remove the DeleteTableHandler. (Jingcheng) (anoopsamjohn: rev 
a84e829e127865a5fc3d54db9d02f091fc89f97b)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DeleteTableProcedure.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java


 Remove the DeleteTableHandler
 -

 Key: HBASE-13790
 URL: https://issues.apache.org/jira/browse/HBASE-13790
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13790.diff


 Now the DeleteTableHandler had been removed from the trunk, we need to remove 
 it in the branch too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13154) Add support for mob in TestAcidGuarantees and IntegrationTestAcidGuarantees


[ 
https://issues.apache.org/jira/browse/HBASE-13154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637817#comment-14637817
 ] 

Hudson commented on HBASE-13154:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13154 add support for mob in TestAcidGuarantees and 
IntegrationTestAcidGuarantees (jmhsieh: rev 
2124d3df7070045fcd43bffd7b6036dc91a11b40)
* hbase-server/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestAcidGuarantees.java


 Add support for mob in TestAcidGuarantees and IntegrationTestAcidGuarantees
 ---

 Key: HBASE-13154
 URL: https://issues.apache.org/jira/browse/HBASE-13154
 Project: HBase
  Issue Type: Sub-task
  Components: integration tests, mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-13154.patch


 Adding support to run acid guarantees tests on data that is primarily in the 
 mob path.
 We essentially set the mob size to be very small (4) and run the same test.  
 This is activated using the -DuseMob=true tool style command line arg.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12066) Avoid major compaction in TestMobSweeper


[ 
https://issues.apache.org/jira/browse/HBASE-12066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637821#comment-14637821
 ] 

Hudson commented on HBASE-12066:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12066 Avoid major compaction in TestMobSweeper (jmhsieh: rev 
836021df2817a625eb7166ea194c5da26d6d6a47)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/mob/mapreduce/TestMobSweeper.java


 Avoid major compaction in TestMobSweeper
 

 Key: HBASE-12066
 URL: https://issues.apache.org/jira/browse/HBASE-12066
 Project: HBase
  Issue Type: Sub-task
  Components: Compaction, mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-12066.patch


 TestMobSweeper is flaky on some machines.  This is due to major compactions 
 happening and blocking the unit test's sweep job from running.  
 This patch bumps up the min hfiles before compaction to properly exercise the 
 sweeper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13737) [HBase MOB] MOBTable cloned from a snapshot leads to data loss, when that actual snapshot and main table is deleted.

[
https://issues.apache.org/jira/browse/HBASE-13737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637830#comment-14637830
]

Hudson commented on HBASE-13737:

FAILURE: Integrated in HBase-TRUNK #6672 (See
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13737 [HBase MOB] MOBTable cloned from a snapshot leads to data loss,
when that actual snapshot and main table is deleted. (Ashutosh Jindal)
(anoopsamjohn: rev 2e2742183fac824dcf19c556fdc8ae1da326ba7c)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileLinkCleaner.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobSnapshotCloneIndependence.java

[HBase MOB] MOBTable cloned from a snapshot leads to data loss, when that
actual snapshot and main table is deleted.

Key: HBASE-13737
URL: https://issues.apache.org/jira/browse/HBASE-13737
Project: HBase
Issue Type: Bug
Components: mob
Affects Versions: hbase-11339
Reporter: Y. SREENIVASULU REDDY
Assignee: Ashutosh Jindal
Priority: Critical
Fix For: hbase-11339

Attachments: HBASE-13737-V2.diff, HBASE-13737-hbase-11339.patch

clone snapshot on mob feature leads to data loss for mob data.
steps to reproduce:
===
1. created MOB table with two column families like mobcf (mob is enabled)
and norcf
2. insert mob data and normal data at a time into the table.
3. scan the MOB table.
4. take snapshot by specifying the table name.
5. clone the snapshot by specifying the new table name.
6. check the new table is created or not and try scan for the new table which
have both mob data and normal data should give back to the client.
7. delete the snapshot which done in step 4.
8. delete the main table which done in step 1.
9. Now scan the new table again.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11646) Handle the MOB in compaction


[ 
https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637837#comment-14637837
 ] 

Hudson commented on HBASE-11646:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11646 Handle the MOB in compaction (Jingcheng Du) (jmhsieh: rev 
7dbd828a90e36c3b94fcadf659ce047a1ddbce0e)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMobCompaction.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/StripeCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobCompactor.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobStoreEngine.java


 Handle the MOB in compaction
 

 Key: HBASE-11646
 URL: https://issues.apache.org/jira/browse/HBASE-11646
 Project: HBase
  Issue Type: Sub-task
  Components: Compaction
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11646-V2.diff, HBASE-11646-V3.diff, 
 HBASE-11646-V4.diff, HBASE-11646-V5.diff, HBASE-11646-V6.diff, 
 HBASE-11646-V7.diff, HBASE-11646-V9.diff, HBASE-11646.diff, 
 hbase-11646-v10.patch, hbase-11646-v8.patch


 In the updated MOB design however, admins can set CF level thresholds that 
 would force cell values  the threshold to use the MOB write path instead of 
 the traditional path.  There are two cases where mobs need to interact with 
 this threshold
 1) How do we handle the case when the threshold size is changed?
 2) Today, you can bulkload hfiles that contain MOBs.  These cells will work 
 as normal inside hbase.  Unfortunately the cells with MOBs in them will never 
 benefit form the MOB write path.
 The proposal here is to modify compaction in mob enabled cf's such that the 
 threshold value is honored with compactions.  This handles case #1 -- 
 elements that should be moved out of the normal hfiles get 'compacted' into 
 refs and mob hfiles, and values that should be pulled into the cf get derefed 
 and written out wholy in the compaction.  For case #2, we can maintain the 
 same behavior and compaction would move data into the mob writepath/lifecycle.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11645) Snapshot for MOB


[ 
https://issues.apache.org/jira/browse/HBASE-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637845#comment-14637845
 ] 

Hudson commented on HBASE-11645:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11645 Snapshot for MOB (Jingcheng Du) (jmhsieh: rev 
7df56b20392c563d5b8d65340921b3041843cf96)
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestMobRestoreSnapshotHelper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/SnapshotTestingUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/RestoreSnapshotHelper.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobSnapshotFromClientWithRegionReplicas.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/MobUtils.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotReferenceUtil.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClientWithRegionReplicas.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobSnapshotCloneIndependence.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestMobRestoreFlushSnapshotFromClient.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestMobSecureExportSnapshot.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/DisabledTableSnapshotHandler.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestMobExportSnapshot.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestMobFlushSnapshotFromClient.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/ExportSnapshot.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobRestoreSnapshotFromClientWithRegionReplicas.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobSnapshotFromClient.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotManifest.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobCloneSnapshotFromClient.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/MobSnapshotTestingUtils.java
* 
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMobRestoreSnapshotFromClient.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/MasterSnapshotVerifier.java


 Snapshot for MOB
 

 Key: HBASE-11645
 URL: https://issues.apache.org/jira/browse/HBASE-11645
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-11645-V2.diff, HBASE-11645-V3.diff, 
 HBASE-11645-V4.diff, HBASE-11645.diff


  Add snapshot support for MOB.  In the initial implementation, taking a table 
 snapshot does not preserve the mob data.  This issue will make sure that when 
 a snapshot is taken, mob data is properly preserved and is restorable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-11339) HBase MOB


[ 
https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637806#comment-14637806
 ] 

Hudson commented on HBASE-11339:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-11339 integrated updates made to the MOB Handbook DOCX file 
(mstanleyjones: rev b72eb7f92eac483e90b460d536166445f84b1de4)
* src/main/docbkx/hbase_mob.xml
HBASE-11339 Converted hbase_mob.xml to Asciidoc and added it to the Asciidoc 
TOC (mstanleyjones: rev a1e9ce3d877035a6e21aab6df8eccd8e959e92dc)
* src/main/docbkx/hbase_mob.xml
* src/main/asciidoc/book.adoc
* src/main/asciidoc/_chapters/hbase_mob.adoc
HBASE-11339 Addendum: Put back the sweeper tool docs for now (mstanleyjones: 
rev 33a6a819a467e09ce80e7d42362c774e62d35809)
* src/main/asciidoc/_chapters/hbase_mob.adoc


 HBase MOB
 -

 Key: HBASE-11339
 URL: https://issues.apache.org/jira/browse/HBASE-11339
 Project: HBase
  Issue Type: Umbrella
  Components: regionserver, Scanners
Affects Versions: 2.0.0
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: 2.0.0, hbase-11339

 Attachments: 11339-master-v10.patch, 11339-master-v3.txt, 
 11339-master-v4.txt, 11339-master-v5.txt, 11339-master-v6.txt, 
 11339-master-v7.txt, 11339-master-v8.patch, 11339-master-v9.patch, HBase MOB 
 Design-v2.pdf, HBase MOB Design-v3.pdf, HBase MOB Design-v4.pdf, HBase MOB 
 Design-v5.pdf, HBase MOB Design.pdf, MOB user guide.docx, MOB user 
 guide_v2.docx, MOB user guide_v3.docx, MOB user guide_v4.docx, MOB user 
 guide_v5.docx, hbase-11339-150519.patch, hbase-11339-in-dev.patch, 
 hbase-11339.150417.patch, merge-150212.patch, merge.150212b.patch, 
 merge.150212c.patch, merge.150710.patch


   It's quite useful to save the medium binary data like images, documents 
 into Apache HBase. Unfortunately directly saving the binary MOB(medium 
 object) to HBase leads to a worse performance since the frequent split and 
 compaction.
   In this design, the MOB data are stored in an more efficient way, which 
 keeps a high write/read performance and guarantees the data consistency in 
 Apache HBase.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12691) sweep job needs to exit non-zero if job fails for any reason.


[ 
https://issues.apache.org/jira/browse/HBASE-12691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637841#comment-14637841
 ] 

Hudson commented on HBASE-12691:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12691 Mob Sweeper job needs to exit with non-zero error value if job 
fails for any reason (jmhsieh: rev c1eacd6221218aaedd82c560a8130d119c511ff2)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/Sweeper.java


 sweep job needs to exit non-zero if job fails for any reason.
 -

 Key: HBASE-12691
 URL: https://issues.apache.org/jira/browse/HBASE-12691
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-12691.patch


 When building up automated testing,  I noticed that the sweepjob would not 
 fail because it exited 0 even if the job failed.  This add the proper exit 
 hygiene adding non-zero exit codes on failure events so that we can rely upon 
 the job in automation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12080) Shorten the run time of integration test by default when using mvn failsafe:integration-test


[ 
https://issues.apache.org/jira/browse/HBASE-12080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637867#comment-14637867
 ] 

Hudson commented on HBASE-12080:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12080 Shorten the run time of integration test by default when using mvn 
failsafe:integration-test (Jingcheng Du) (jmhsieh: rev 
a7f8035ec5ffaa204a47e157cee4ba826eb4afdf)
* 
hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestIngestWithMOB.java


 Shorten the run time of integration test by default when using mvn 
 failsafe:integration-test
 

 Key: HBASE-12080
 URL: https://issues.apache.org/jira/browse/HBASE-12080
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 2.0.0
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12080.diff


 By default, using mvn failsafe:integration-test to execute the integration 
 test with MOB will run more than 10 minutes. In this JIRA, we'll shorten this 
 run time by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13855) Race in multi threaded PartitionedMobCompactor causes NPE


[ 
https://issues.apache.org/jira/browse/HBASE-13855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637801#comment-14637801
 ] 

Hudson commented on HBASE-13855:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13855 Race in multi threaded PartitionedMobCompactor causes NPE. 
(Jingcheng) (anoopsamjohn: rev faefb9073f388663df91d0ef2db24b00d6512519)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/compactions/PartitionedMobCompactor.java


 Race in multi threaded PartitionedMobCompactor causes NPE
 -

 Key: HBASE-13855
 URL: https://issues.apache.org/jira/browse/HBASE-13855
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Priority: Critical
 Fix For: hbase-11339

 Attachments: HBASE-13855.diff


 In PartitionedMobCompactor, mob files are split into partitions, the 
 compactions of partitions run in parallel.
 The partitions share  the same set of del files. There might be race 
 conditions when open readers of del store files in each partition which can 
 cause NPE.
 In this patch, we will pre-create the reader for each del store file to avoid 
 this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13177) Document HBASE-13012


[ 
https://issues.apache.org/jira/browse/HBASE-13177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637850#comment-14637850
 ] 

Hudson commented on HBASE-13177:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13177 Document compact and major_compact HBase shell commands and Admin 
APIs (mstanleyjones: rev 8c1edeb2b8d8da6058e78df2ec907ec4df986012)
* src/main/asciidoc/_chapters/hbase_mob.adoc


 Document HBASE-13012
 

 Key: HBASE-13177
 URL: https://issues.apache.org/jira/browse/HBASE-13177
 Project: HBase
  Issue Type: Task
  Components: documentation, mob
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: hbase-11339






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12392) Incorrect implementation of CompactionRequest.isRetainDeleteMarkers


[ 
https://issues.apache.org/jira/browse/HBASE-12392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637822#comment-14637822
 ] 

Hudson commented on HBASE-12392:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12392 Incorrect implementation of (ramkrishna: rev 
8a8b7de760a33c8195e287975d5b4a337d71b1fa)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java


 Incorrect implementation of CompactionRequest.isRetainDeleteMarkers
 ---

 Key: HBASE-12392
 URL: https://issues.apache.org/jira/browse/HBASE-12392
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Priority: Critical
 Fix For: hbase-11339

 Attachments: HBASE-12392.diff


 Now in the implementation of the isRetainDeleteMarkers method, the code look 
 like,
 {code}
 return (this.retainDeleteMarkers != null) ? 
 this.retainDeleteMarkers.booleanValue()
 : isAllFiles();
 {code}
 It means for a major compaction in a normal store, this method returns true. 
 Consequently the delete marks could not be deleted in the major compaction, 
 which leads the unit test TestKeepDeletes fails.
 The correct implementation should be,
 {code}
 return (this.retainDeleteMarkers != null) ? 
 this.retainDeleteMarkers.booleanValue()
 : !isAllFiles();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12758) treat mob region as any other region when generating rs manifest.


[ 
https://issues.apache.org/jira/browse/HBASE-12758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637807#comment-14637807
 ] 

Hudson commented on HBASE-12758:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12758 treat mob region as any other region when generating rs manifest 
(jmhsieh: rev 9f1f8c3bc6b09ede3dd9285e3a989337a135505e)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java


 treat mob region as any other region when generating rs manifest.
 -

 Key: HBASE-12758
 URL: https://issues.apache.org/jira/browse/HBASE-12758
 Project: HBase
  Issue Type: Sub-task
  Components: mob, snapshots
Affects Versions: hbase-11339
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
 Fix For: hbase-11339

 Attachments: hbase-12758.patch


 Matteo on inspection noticed that we created two separate manifest files on 
 the region server that adds the mob region when we snapshot.  This patch 
 makes it so that we only create one on that region server. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12487) Explicitly flush the file name in sweep job


[ 
https://issues.apache.org/jira/browse/HBASE-12487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637836#comment-14637836
 ] 

Hudson commented on HBASE-12487:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12487 Explicitly flush the file name in sweep job (Jingcheng Du) 
(ramkrishna: rev 170eb1e88d51776b093f7b546e4b9b0dc6e2cf52)
* hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepJob.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/mob/mapreduce/SweepReducer.java


 Explicitly flush the file name in sweep job
 ---

 Key: HBASE-12487
 URL: https://issues.apache.org/jira/browse/HBASE-12487
 Project: HBase
  Issue Type: Sub-task
  Components: regionserver, Scanners
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12487.diff


 Currently in the implementation of sweeper, we need to remove the mob files 
 that are not referenced by hbase any more.
 # List the names of all the existing files and write them to a seq file.
 # In each reducer, write the visited mob file names to a seq file.
 # After the mapreduce is done, remove the files that are existent in step1 
 but not in step2 (those are the unused/unreferenced files).
 Currently the flush of the writer depends on the IOUtils.closeStream(writer), 
 if this close operation fails silently, the file names won't be written to 
 seq files, some files that are still referenced by hbase will be archived 
 after the mapreduce is finished.
 We should explicitly invoke write.hflush() to flush out the user buffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13932) Add mob integrity check in HFilePrettyPrinter


[ 
https://issues.apache.org/jira/browse/HBASE-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637825#comment-14637825
 ] 

Hudson commented on HBASE-13932:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13932 - Add mob integrity check in HFilePrettyPrinter (Jingcheng du) 
(ramkrishna: rev ba4ba32b0dd5166b1cc2862e55e5c1c6eacfdf43)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFilePrettyPrinter.java


 Add mob integrity check in HFilePrettyPrinter
 -

 Key: HBASE-13932
 URL: https://issues.apache.org/jira/browse/HBASE-13932
 Project: HBase
  Issue Type: Sub-task
  Components: mob
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13932-V2.patch, HBASE-13932-V3.patch, 
 HBASE-13932-V4.patch, HBASE-13932.patch


 We need to know whether a reference cell is dangling in mob. We can add this 
 to HFilePrettyPrinter.
 We can add a new option to the command, to check the integrity of mob cells 
 either per region or per file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13670) [HBase MOB] ExpiredMobFileCleaner tool deletes mob files later for one more day after they are expired


[ 
https://issues.apache.org/jira/browse/HBASE-13670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637829#comment-14637829
 ] 

Hudson commented on HBASE-13670:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13670 [HBase MOB] ExpiredMobFileCleaner tool deletes mob files later for 
one more day after they are expired. (Jingcheng) (anoopsamjohn: rev 
a1060d1390a82708d2547b1659325a4735a91761)
* hbase-common/src/main/resources/hbase-default.xml


 [HBase MOB] ExpiredMobFileCleaner tool deletes mob files later for one more 
 day after they are expired
 --

 Key: HBASE-13670
 URL: https://issues.apache.org/jira/browse/HBASE-13670
 Project: HBase
  Issue Type: Improvement
  Components: documentation, mob
Affects Versions: hbase-11339
Reporter: Y. SREENIVASULU REDDY
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13670-03.patch, HBASE-13670-04.patch, 
 HBASE-13670.patch, HBASE-13670_01.patch, HBASE-13670_02.patch


 Currently the ExpiredMobFileCleaner cleans the expired mob file according to 
 the date in the mob file name. The minimum unit of the date is day. So 
 ExpiredMobFileCleaner only cleans the expired mob files later for one more 
 day after they are expired. We need to document this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-13013) Add read lock to ExpiredMobFileCleanerChore


[ 
https://issues.apache.org/jira/browse/HBASE-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637814#comment-14637814
 ] 

Hudson commented on HBASE-13013:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-13013 - Add read lock to ExpiredMobFileCleanerChore (Jingcheng Du) 
(ramkrishna: rev 8f5dae471a9da95b144ecc15010028ba1ce7120c)
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/ExpiredMobFileCleanerChore.java
* 
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MobFileCompactionChore.java


 Add read lock to ExpiredMobFileCleanerChore
 ---

 Key: HBASE-13013
 URL: https://issues.apache.org/jira/browse/HBASE-13013
 Project: HBase
  Issue Type: Sub-task
  Components: master
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-13013-V2.diff, HBASE-13013.diff


 Now we have two chores for mob, one is the cleaner of expired mob files, the 
 other is for the mob file compactor. They might handle the same file set 
 concurrently, and might impact each other when doing that. We need to 
 synchronize them, now we have a write lock for the mob file compaction chore, 
 we need to add a read lock for the chore of expired mob file cleaner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12027) The ZooKeeperWatcher in HMobStore only uses the default conf


[ 
https://issues.apache.org/jira/browse/HBASE-12027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637834#comment-14637834
 ] 

Hudson commented on HBASE-12027:


FAILURE: Integrated in HBase-TRUNK #6672 (See 
[https://builds.apache.org/job/HBase-TRUNK/6672/])
HBASE-12027 The ZooKeeperWatcher in HMobStore only uses the default conf 
(Jingcheng Du) (jmhsieh: rev 6769ea9e206cbbd555769f4cc9af1dfa4e5aa09d)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HMobStore.java


 The ZooKeeperWatcher in HMobStore only uses the default conf
 

 Key: HBASE-12027
 URL: https://issues.apache.org/jira/browse/HBASE-12027
 Project: HBase
  Issue Type: Bug
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
 Fix For: hbase-11339

 Attachments: HBASE-12027.diff


 In HMobStore we need to create a ZooKeeperWatcher for a major compaction.
 There the conf of store is directly passed into the ZookeeperWatcher, this 
 conf is a CompoundConfiguration.
 In ZookeeperWatcher, this conf is converted into a Configuration by new 
 Configuration(compoundConfiguration), and the zk-related conf are lost, only 
 default settings are used by the zookeeper.
 In the HMobStore, we should pass the region.getBaseConf() which is a 
 Configuration to the ZookeeperWatcher.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13992) Integrate SparkOnHBase into HBase


 [ 
https://issues.apache.org/jira/browse/HBASE-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Malaska updated HBASE-13992:

Attachment: HBASE-13992.9.patch

Fixed the examples along with the HBaseRDD.collect change.

Also fixed the line length issues I just added in patch 8

 Integrate SparkOnHBase into HBase
 -

 Key: HBASE-13992
 URL: https://issues.apache.org/jira/browse/HBASE-13992
 Project: HBase
  Issue Type: New Feature
  Components: spark
Reporter: Ted Malaska
Assignee: Ted Malaska
 Fix For: 2.0.0

 Attachments: HBASE-13992.5.patch, HBASE-13992.6.patch, 
 HBASE-13992.7.patch, HBASE-13992.8.patch, HBASE-13992.9.patch, 
 HBASE-13992.patch, HBASE-13992.patch.3, HBASE-13992.patch.4, 
 HBASE-13992.patch.5


 This Jira is to ask if SparkOnHBase can find a home in side HBase core.
 Here is the github: 
 https://github.com/cloudera-labs/SparkOnHBase
 I am the core author of this project and the license is Apache 2.0
 A blog explaining this project is here
 http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
 A spark Streaming example is here
 http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
 A real customer using this in produce is blogged here
 http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/
 Please debate and let me know what I can do to make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14148) Web UI Framable Page


[ 
https://issues.apache.org/jira/browse/HBASE-14148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638025#comment-14638025
 ] 

Hadoop QA commented on HBASE-14148:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12746673/HBASE-14148-master.patch
  against master branch at commit 493f36c899bc990de0400fbf777a2bf29c5c60e3.
  ATTACHMENT ID: 12746673

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 4 new 
or modified tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:red}-1 release audit{color}.  The applied patch generated 1 release 
audit warnings (more than the master's current 0 warnings).

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
   org.apache.hadoop.hbase.TestCheckTestClasses

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14867//testReport/
Release audit warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14867//artifact/patchprocess/patchReleaseAuditWarnings.txt
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14867//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14867//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14867//console

This message is automatically generated.

 Web UI Framable Page
 

 Key: HBASE-14148
 URL: https://issues.apache.org/jira/browse/HBASE-14148
 Project: HBase
  Issue Type: Bug
Reporter: Apekshit Sharma
Assignee: Apekshit Sharma
 Attachments: HBASE-14148-master.patch


 The web UIs do not include the X-Frame-Options header to prevent the pages 
 from being framed from another site.  
 Reference:
 https://www.owasp.org/index.php/Clickjacking
 https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet
 https://developer.mozilla.org/en-US/docs/Web/HTTP/X-Frame-Options



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HBASE-13758) Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0

2015-07-22 Thread Gabor Liptak (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Liptak reassigned HBASE-13758:


Assignee: Gabor Liptak

 Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0
 

 Key: HBASE-13758
 URL: https://issues.apache.org/jira/browse/HBASE-13758
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
 Attachments: HBASE-13758.1.patch


 Running
 $ mvn idea:idea
 shows following warning:
 [WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
 no dependency information available
 [WARNING] Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
 org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not 
 be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in 
 http://repository.apache.org/snapshots/ was cached in the local repository, 
 resolution will not be reattempted until the update interval of 
 apache.snapshots has elapsed or updates are forced
 Looking at the URL:
 https://repository.apache.org/content/groups/snapshots/org/
 the eclipse directory is not present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13758) Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0

2015-07-22 Thread Gabor Liptak (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Liptak updated HBASE-13758:
-
Release Note: HBASE-13758 Correct warning on 
org.eclipse.m2e:lifecycle-mapping:1.0.0
  Status: Patch Available  (was: Open)

 Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0
 

 Key: HBASE-13758
 URL: https://issues.apache.org/jira/browse/HBASE-13758
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
 Attachments: HBASE-13758.1.patch


 Running
 $ mvn idea:idea
 shows following warning:
 [WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
 no dependency information available
 [WARNING] Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
 org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not 
 be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in 
 http://repository.apache.org/snapshots/ was cached in the local repository, 
 resolution will not be reattempted until the update interval of 
 apache.snapshots has elapsed or updates are forced
 Looking at the URL:
 https://repository.apache.org/content/groups/snapshots/org/
 the eclipse directory is not present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HBASE-13758) Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0

2015-07-22 Thread Gabor Liptak (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-13758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gabor Liptak updated HBASE-13758:
-
Attachment: HBASE-13758.1.patch

 Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0
 

 Key: HBASE-13758
 URL: https://issues.apache.org/jira/browse/HBASE-13758
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Gabor Liptak
Priority: Minor
 Attachments: HBASE-13758.1.patch


 Running
 $ mvn idea:idea
 shows following warning:
 [WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
 no dependency information available
 [WARNING] Failed to retrieve plugin descriptor for 
 org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
 org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not 
 be resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in 
 http://repository.apache.org/snapshots/ was cached in the local repository, 
 resolution will not be reattempted until the update interval of 
 apache.snapshots has elapsed or updates are forced
 Looking at the URL:
 https://repository.apache.org/content/groups/snapshots/org/
 the eclipse directory is not present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-14143) remove obsolete maven repositories


[ 
https://issues.apache.org/jira/browse/HBASE-14143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638084#comment-14638084
 ] 

Hadoop QA commented on HBASE-14143:
---

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12746674/HBASE-14143.branch-1.2.patch
  against branch-1.2 branch at commit 493f36c899bc990de0400fbf777a2bf29c5c60e3.
  ATTACHMENT ID: 12746674

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation, build,
or dev-support patch that doesn't require tests.

{color:green}+1 hadoop versions{color}. The patch compiles with all 
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 protoc{color}.  The applied patch does not increase the 
total number of protoc compiler warnings.

{color:green}+1 javadoc{color}.  The javadoc tool did not generate any 
warning messages.

{color:green}+1 checkstyle{color}.  The applied patch does not increase the 
total number of checkstyle errors

{color:green}+1 findbugs{color}.  The patch does not introduce any  new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14868//testReport/
Release Findbugs (version 2.0.3)warnings: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14868//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14868//artifact/patchprocess/checkstyle-aggregate.html

  Console output: 
https://builds.apache.org/job/PreCommit-HBASE-Build/14868//console

This message is automatically generated.

 remove obsolete maven repositories
 --

 Key: HBASE-14143
 URL: https://issues.apache.org/jira/browse/HBASE-14143
 Project: HBase
  Issue Type: Bug
Reporter: Gabor Liptak
Assignee: Gabor Liptak
Priority: Minor
 Attachments: HBASE-14143.base-1.1.patch, HBASE-14143.branch-1.2.patch


 The root pom.xml contains repositories no longer available and/or updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer