[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
Release Note: 
The methods which change to use TableDescriptor/ColumnFamilyDescriptor are 
shown below.
+ preCreateTable( ObserverContext,TableDescriptor, HRegionInfo[])
+ postCreateTable(ObserverContext ,TableDescriptor, HRegionInfo[])
+ preCreateTableAction(ObserverContext, TableDescriptor,HRegionInfo[])
+ postCompletedCreateTableAction(ObserverContext,TableDescriptor,HRegionInfo[])
+ preModifyTable(ObserverContext,TableName, TableDescriptor)
+ postModifyTable(ObserverContext,TableName, TableDescriptor)
+ preModifyTableAction( ObserverContext,TableName,TableDescriptor)
+ postCompletedModifyTableAction( ObserverContext,TableName,TableDescriptor)
+ preAddColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ postAddColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ preAddColumnFamilyAction(ObserverContext,TableName,ColumnFamilyDescriptor)
+ postCompletedAddColumnFamilyAction(ObserverContext,TableName, 
ColumnFamilyDescriptor)
+ preModifyColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ 
preModifyColumnFamilyAction(ObserverContext Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116074#comment-16116074
 ] 

Jerry He commented on HBASE-18522:
--

Hi, Ted

Thanks for the review. 
bq. Can we not use Integer ? All legitimate indices should be >= 0. How about 
using -1 to represent the null case ?
Could you elaborate?  The HashMap needs 

> Add RowMutations support to Batch
> -
>
> Key: HBASE-18522
> URL: https://issues.apache.org/jira/browse/HBASE-18522
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 1.2.6
>Reporter: Jerry He
>Assignee: Jerry He
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18522-branch-1.patch
>
>
> RowMutations is multiple Puts and/or Deletes atomically on a single row. 
> Current Batch call does not support RowMutations as part of the batch.
> We should add this missing part. We should be able to batch RowMutations.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17125) Inconsistent result when use filter to read data

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116073#comment-16116073
 ] 

Hadoop QA commented on HBASE-17125:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m  8s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
18s{color} | {color:red} hbase-client generated 5 new + 0 unchanged - 0 fixed = 
5 total (was 0) {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
40s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 17s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
35s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 25s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | 
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure |
|   | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
|   | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
|   | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
|   | org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics |
|   | org.apache.hadoop.hbase.master.TestGetInfoPort |
|   | org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence |
|   | org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster |
|   | 
org.apache.hadoop.hbase.master.normalizer.TestSimpleRegionNormalizerOnCluster |
|   | org.apache.hadoop.hbase.master.TestTableStateManager 

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116072#comment-16116072
 ] 

Anoop Sam John commented on HBASE-18502:


Thanks...  Ya me and Stack discussed abt this last week. 
So we have changed the CP methods for 2.0 now.  Should be deprecate them with 
addition of new methods for branch-1?  How we were doing this in other jiras?

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount

[Jerry He (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116071#comment-16116071
 ] 

Jerry He commented on HBASE-18469:
--

Since it is not 'RegionAction' level, and it is really 'RowAction',  maybe 
'totalRowActionCount" or 'requestRowActionCount'?

Will it work at the same place where the read request is updated in 
RSRpcServices.scan()?
{code}
region.updateReadRequestsCount(numOfResults);
{code}

> Correct  RegionServer metric of  totalRequestCount
> --
>
> Key: HBASE-18469
> URL: https://issues.apache.org/jira/browse/HBASE-18469
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics, regionserver
>Affects Versions: 1.2.0
>Reporter: Shibin Zhang
>Assignee: Yu Li
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18469.patch
>
>
> when i get the metric ,i found  this three metric may be have some error  as 
> follow :
> "totalRequestCount" : 17541,
> "readRequestCount" : 17483,
> "writeRequestCount" : 1633,



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose

[Anoop Sam John (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116068#comment-16116068
 ] 

Anoop Sam John commented on HBASE-18298:


After our F2F discussion, I hope its clear for u now.  Why we need an 
interface.  CP needs certain RS services. Those are here in old RSS interface.  
Also down the line, Region , Store etc needs some other services too.  We added 
all of them together in single interface and exposed that to CP. For our UTs to 
work with mocking we need a single interface with all the services also.  So 
what I did is maintain the old RSS interface with CP needed methods alone and 
extended that to make an IRSS with extra methods.  Any better name for the 
interface?  This new interface is RSS (To be exposed) + some things extra. This 
is the interface what Server modules internally need.   Ditto will come for 
Region, Store etc.  We need a better name I guess. My bad naming may be. Pls 
suggest .

> RegionServerServices Interface cleanup for CP expose
> 
>
> Key: HBASE-18298
> URL: https://issues.apache.org/jira/browse/HBASE-18298
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, 
> HBASE-18298_V3.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116049#comment-16116049
 ] 

Chia-Ping Tsai commented on HBASE-18528:


ping [~jamestaylor]

> Support to modify TableDescriptor/ColumnFamilyDescriptor through 
> MasterObserver; Or disable that.
> -
>
> Key: HBASE-18528
> URL: https://issues.apache.org/jira/browse/HBASE-18528
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
>
> We are replacing the HTableDescriptor by TableDescriptor from code base. The 
> TableDescriptor is designed to be a read-only object so user can't modifiy it 
> through MasterObserver. HBASE-18502 change many methods of MasterObserver to 
> use TableDescriptor but some deprecated methods still accept the 
> HTableDescriptor. User may be confused by why some methods can't modify the 
> table descriptor.
> In short, Should we allow user to modify the passed table descriptor?
> # if yes, we should introduce a mechanism that user can return a modified 
> table descripror
> # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove 
> all methods accepting the HTableDescriptor
> Ditto for HColumnDescriptor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18528:
---
Component/s: master
 Coprocessors

> Support to modify TableDescriptor/ColumnFamilyDescriptor through 
> MasterObserver; Or disable that.
> -
>
> Key: HBASE-18528
> URL: https://issues.apache.org/jira/browse/HBASE-18528
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
>
> We are replacing the HTableDescriptor by TableDescriptor from code base. The 
> TableDescriptor is designed to be a read-only object so user can't modifiy it 
> through MasterObserver. HBASE-18502 change many methods of MasterObserver to 
> use TableDescriptor but some deprecated methods still accept the 
> HTableDescriptor. User may be confused by why some methods can't modify the 
> table descriptor.
> In short, Should we allow user to modify the passed table descriptor?
> # if yes, we should introduce a mechanism that user can return a modified 
> table descripror
> # if no, we should pass ImmutableHTableDescriptor to user. Or we just remove 
> all methods accepting the HTableDescriptor
> Ditto for HColumnDescriptor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
Component/s: master
 Coprocessors

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors, master
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18528) Support to modify TableDescriptor/ColumnFamilyDescriptor through MasterObserver; Or disable that.

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-18528:
--

 Summary: Support to modify TableDescriptor/ColumnFamilyDescriptor 
through MasterObserver; Or disable that.
 Key: HBASE-18528
 URL: https://issues.apache.org/jira/browse/HBASE-18528
 Project: HBase
  Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Critical
 Fix For: 3.0.0, 2.0.0-alpha-2


We are replacing the HTableDescriptor by TableDescriptor from code base. The 
TableDescriptor is designed to be a read-only object so user can't modifiy it 
through MasterObserver. HBASE-18502 change many methods of MasterObserver to 
use TableDescriptor but some deprecated methods still accept the 
HTableDescriptor. User may be confused by why some methods can't modify the 
table descriptor.

In short, Should we allow user to modify the passed table descriptor?
# if yes, we should introduce a mechanism that user can return a modified table 
descripror
# if no, we should pass ImmutableHTableDescriptor to user. Or we just remove 
all methods accepting the HTableDescriptor

Ditto for HColumnDescriptor.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
  Resolution: Fixed
Hadoop Flags: Incompatible change,Reviewed
Release Note: 
MasterObserver class
+ preCreateTable( ObserverContext,TableDescriptor, HRegionInfo[])
+ postCreateTable(ObserverContext ,TableDescriptor, HRegionInfo[])
+ preCreateTableAction(ObserverContext, TableDescriptor,HRegionInfo[])
+ postCompletedCreateTableAction(ObserverContext,TableDescriptor,HRegionInfo[])
+ preModifyTable(ObserverContext,TableName, TableDescriptor)
+ postModifyTable(ObserverContext,TableName, TableDescriptor)
+ preModifyTableAction( ObserverContext,TableName,TableDescriptor)
+ postCompletedModifyTableAction( ObserverContext,TableName,TableDescriptor)
+ preAddColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ postAddColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ preAddColumnFamilyAction(ObserverContext,TableName,ColumnFamilyDescriptor)
+ postCompletedAddColumnFamilyAction(ObserverContext,TableName, 
ColumnFamilyDescriptor)
+ preModifyColumnFamily(ObserverContext,TableName, ColumnFamilyDescriptor)
+ 
preModifyColumnFamilyAction(ObserverContext Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116032#comment-16116032
 ] 

Umesh Agashe commented on HBASE-18525:
--

Thanks [~stack]! yes, I would like to consider this together with other related 
JIRAs linked to HBASE-18366. Let discuss this further.

Thanks [~tedyu] for the V2 patch! I'm looking at the patch. It looks better. 
You have mentioned that the test is hanging with this patch. Let me check the 
logs and see why its hanging.


> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18515:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

[~yifan.stan] Thanks for the patch.

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18469) Correct RegionServer metric of totalRequestCount

[Yu Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116026#comment-16116026
 ] 

Yu Li commented on HBASE-18469:
---

bq. For multi actions, the row actions can be on the same row.
True, so maybe "totalRegionActionRequestCount" and "requestRegionActionCount" 
are better names?

bq. Also, it is possible to add/increment the request row counter somehow at 
the higher level in batch, not one row a time?
For multi, we're already doing this:
{code}
-  this.requestCount.add(regionAction.getActionCount());
+  this.requestRowsCount.add(regionAction.getActionCount());
{code}
For scan, it's hard to foresee which limit will be reached first, so we better 
increase the counter after each successful scan.next call, right?

> Correct  RegionServer metric of  totalRequestCount
> --
>
> Key: HBASE-18469
> URL: https://issues.apache.org/jira/browse/HBASE-18469
> Project: HBase
>  Issue Type: Improvement
>  Components: metrics, regionserver
>Affects Versions: 1.2.0
>Reporter: Shibin Zhang
>Assignee: Yu Li
>Priority: Critical
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-18469.patch
>
>
> when i get the metric ,i found  this three metric may be have some error  as 
> follow :
> "totalRequestCount" : 17541,
> "readRequestCount" : 17483,
> "writeRequestCount" : 1633,



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-17125) Inconsistent result when use filter to read data

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-17125:
---
Attachment: HBASE-17125.master.020.patch

> Inconsistent result when use filter to read data
> 
>
> Key: HBASE-17125
> URL: https://issues.apache.org/jira/browse/HBASE-17125
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: 17125-slack-13.txt, example.diff, 
> HBASE-17125.master.001.patch, HBASE-17125.master.002.patch, 
> HBASE-17125.master.002.patch, HBASE-17125.master.003.patch, 
> HBASE-17125.master.004.patch, HBASE-17125.master.005.patch, 
> HBASE-17125.master.006.patch, HBASE-17125.master.007.patch, 
> HBASE-17125.master.008.patch, HBASE-17125.master.009.patch, 
> HBASE-17125.master.009.patch, HBASE-17125.master.010.patch, 
> HBASE-17125.master.011.patch, HBASE-17125.master.011.patch, 
> HBASE-17125.master.012.patch, HBASE-17125.master.013.patch, 
> HBASE-17125.master.014.patch, HBASE-17125.master.015.patch, 
> HBASE-17125.master.016.patch, HBASE-17125.master.017.patch, 
> HBASE-17125.master.018.patch, HBASE-17125.master.019.patch, 
> HBASE-17125.master.020.patch, HBASE-17125.master.020.patch, 
> HBASE-17125.master.checkReturnedVersions.patch, 
> HBASE-17125.master.no-specified-filter.patch
>
>
> Assume a cloumn's max versions is 3, then we write 4 versions of this column. 
> The oldest version doesn't remove immediately. But from the user view, the 
> oldest version has gone. When user use a filter to query, if the filter skip 
> a new version, then the oldest version will be seen again. But after compact 
> the region, then the oldest version will never been seen. So it is weird for 
> user. The query will get inconsistent result before and after region 
> compaction.
> The reason is matchColumn method of UserScanQueryMatcher. It first check the 
> cell by filter, then check the number of versions needed. So if the filter 
> skip the new version, then the oldest version will be seen again when it is 
> not removed.
> Have a discussion offline with [~Apache9] and [~fenghh], now we have two 
> solution for this problem. The first idea is check the number of versions 
> first, then check the cell by filter. As the comment of setFilter, the filter 
> is called after all tests for ttl, column match, deletes and max versions 
> have been run.
> {code}
>   /**
>* Apply the specified server-side filter when performing the Query.
>* Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
>* for ttl, column match, deletes and max versions have been run.
>* @param filter filter to run on the server
>* @return this for invocation chaining
>*/
>   public Query setFilter(Filter filter) {
> this.filter = filter;
> return this;
>   }
> {code}
> But this idea has another problem, if a column's max version is 5 and the 
> user query only need 3 versions. It first check the version's number, then 
> check the cell by filter. So the cells number of the result may less than 3. 
> But there are 2 versions which don't read anymore.
> So the second idea has three steps.
> 1. check by the max versions of this column
> 2. check the kv by filter
> 3. check the versions which user need.
> But this will lead the ScanQueryMatcher more complicated. And this will break 
> the javadoc of Query.setFilter.
> Now we don't have a final solution for this problem. Suggestions are welcomed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-18485:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Pushed to master and branch-2. Thanks [~tedyu] and [~chia7712] for review.

> Performance issue: ClientAsyncPrefetchScanner is slower than 
> ClientSimpleScanner
> 
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch, 
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch, 
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch, 
> HBASE-18485-v6.patch, HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18527) update nightly builds to compensate for jenkins plugin upgrades

[Sean Busbey (JIRA)]

Sean Busbey created HBASE-18527:
---

 Summary: update nightly builds to compensate for jenkins plugin 
upgrades
 Key: HBASE-18527
 URL: https://issues.apache.org/jira/browse/HBASE-18527
 Project: HBase
  Issue Type: Task
  Components: community, test
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Blocker


Last night as a part of some infra work a bunch of plugins updated. one of them 
was the git-branch-source thingy. Its upgrade stopped reusing stuff from the 
general plugin for checking out git stuff, so our checking out into a directory 
stopped working.

Tracked upstream as 
[JENKINS-46013|https://issues.jenkins-ci.org/browse/JENKINS-46013]. Until we 
get a fix we need a workaround.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18405) Track scope for HBase-Spark module

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115969#comment-16115969
 ] 

stack commented on HBASE-18405:
---

Read update1. Excellent writeup [~busbey]

Non-important thoughts:

+ Composite row key is spark being able to grok a preexisting hbase row key; 
i.e. teaching spark how to extract subparts. How would this be done in the 
future do you think?
+ No typing then in the first pass. Ain't that going to be ugly perf-wise sir? 
Downsides? How you see the project that does the smooth mapping of spark to 
hbase-type (whether phoenix or hbase data type).
+ What is this one about... "... multiple secure HBase deployments"? A spark 
job spanning hbase clusters?

Avro? Because? You can do schema apart from data?
Java Native is Bytes.toLong, etc.?

You looked at providing a 'catalog'? Would it be tough? Could we ship a 
hard-coded table with type info in it with a facade that implements catalog to 
satisfy spark? Or would folks be needing to extend with their own compound 
types, etc.

bq. Where practical, we will avoid duplication of implementation source code..

Is this like our current hadoop-compatible for metrics, etc., with hadoop 2 and 
hadoop1 implementations?

On unit tests, will we have to spin up clusters? Can we get away with things 
like the RegionAsTable dohickey (puts a Table Interface on a Region...). You 
know what I'm worried about ... You start the build when you leave on your 
two-week vacation and you hope it is done when you get back.

Spark in a new project like hbase-thirdparty?

Great writeup.




> Track scope for HBase-Spark module
> --
>
> Key: HBASE-18405
> URL: https://issues.apache.org/jira/browse/HBASE-18405
> Project: HBase
>  Issue Type: Task
>  Components: spark
>Reporter: Sean Busbey
>Assignee: Sean Busbey
> Fix For: 1.4.0, 2.0.0-beta-1
>
> Attachments: Apache HBase - Apache Spark Integration Scope.pdf, 
> Apache HBase - Apache Spark Integration Scope - update 1.pdf
>
>
> Start with [\[DISCUSS\]  status of and plans for our hbase-spark integration 
> |https://lists.apache.org/thread.html/fd74ef9b9da77abf794664f06ea19c839fb3d543647fb29115081683@%3Cdev.hbase.apache.org%3E]
>  and formalize into a scope document for bringing this feature into a release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov reassigned HBASE-18526:
-

Assignee: Vladimir Rodionov

> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
>   message = "Default TTL is not supported for FIFO compaction";
>   throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
>   message = "MIN_VERSION > 0 is not supported for FIFO compaction";
>   throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
>   blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
>   message =
>   "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
>   + " is below recommended minimum of 1000";
>   throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at 
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at 
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at 
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at 
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at 
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at 
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115960#comment-16115960
 ] 

Ted Yu commented on HBASE-17056:


Attempted the following:
{code}
diff --git a/hbase-rsgroup/pom.xml b/hbase-rsgroup/pom.xml
index 0bbabe9..b531086 100644
--- a/hbase-rsgroup/pom.xml
+++ b/hbase-rsgroup/pom.xml
@@ -60,6 +60,11 @@
 
   compile
 
+ 
+  
+
${basedir}/../hbase-protocol/src/main/protobuf
+  
+ 
   
 
   
{code}
And got the following:
{code}
[ERROR] 
/mnt/disk2/a/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java/org/apache/hadoop/hbase/shaded/protobuf/generated/RegionServerStatusProtos.java:[446,20]
 no suitable method found for parseFrom(java.nio.ByteBuffer)
method 
org.apache.hadoop.hbase.shaded.com.google.protobuf.Parser.parseFrom(org.apache.hadoop.hbase.shaded.com.google.protobuf.CodedInputStream)
 is not applicable
  (argument mismatch; java.nio.ByteBuffer cannot be converted to 
org.apache.hadoop.hbase.shaded.com.google.protobuf.CodedInputStream)
method 
org.apache.hadoop.hbase.shaded.com.google.protobuf.Parser.parseFrom(org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteString)
 is not applicable
  (argument mismatch; java.nio.ByteBuffer cannot be converted to 
org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteString)
method 
org.apache.hadoop.hbase.shaded.com.google.protobuf.Parser.parseFrom(byte[]) is 
not applicable
  (argument mismatch; java.nio.ByteBuffer cannot be converted to byte[])
method 
org.apache.hadoop.hbase.shaded.com.google.protobuf.Parser.parseFrom(java.io.InputStream)
 is not applicable
  (argument mismatch; java.nio.ByteBuffer cannot be converted to 
java.io.InputStream)
{code}

> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch, HBASE-17056.master.004.patch, 
> HBASE-17056.master.005.patch, HBASE-17056.master.006.patch, 
> HBASE-17056.master.006.patch, HBASE-17056.master.007.patch, 
> HBASE-17056.master.007.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-17056) Remove checked in PB generated files

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115953#comment-16115953
 ] 

Ted Yu commented on HBASE-17056:


It seems the site generation is broken after this went in.
>From 
>https://builds.apache.org/job/hbase_generate_website/1072/artifact/hbase-install-log-82d554e3783372cc6b05489452c815b57c06f6cd.txt
> :
{code}
[ERROR] 
/home/jenkins/jenkins-slave/workspace/hbase_generate_website/hbase/hbase-rsgroup/src/main/protobuf/RSGroup.proto
 [0:0]: HBase.proto: File not found.
RSGroup.proto: Import "HBase.proto" was not found or had errors.
RSGroup.proto:31:12: "ServerName" is not defined.
RSGroup.proto:32:12: "TableName" is not defined.
{code}


> Remove checked in PB generated files 
> -
>
> Key: HBASE-17056
> URL: https://issues.apache.org/jira/browse/HBASE-17056
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Enis Soztutar
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 
> 0002-HBASE-17056-Remove-checked-in-PB-generated-files.patch, 
> HBASE-17056.master.001.patch, HBASE-17056.master.002.patch, 
> HBASE-17056.master.003.patch, HBASE-17056.master.004.patch, 
> HBASE-17056.master.005.patch, HBASE-17056.master.006.patch, 
> HBASE-17056.master.006.patch, HBASE-17056.master.007.patch, 
> HBASE-17056.master.007.patch
>
>
> Now that we have the new PB maven plugin, there is no need to have the PB 
> files checked in to the repo. The reason we did that was to ease up developer 
> env setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18519) Use zero-copy cell to optimize CellUtil.createCell

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115946#comment-16115946
 ] 

Ted Yu commented on HBASE-18519:


{code}
29public static CellBuilder newBuilder() {
30  return ExtendedCellBuilder.newBuilder();
31}
{code}
It would be better if the above method is moved out of CellBuilder since 
CellBuilder is an interface which should not be tied with implementation.

> Use zero-copy cell to optimize CellUtil.createCell
> --
>
> Key: HBASE-18519
> URL: https://issues.apache.org/jira/browse/HBASE-18519
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18519.v0.patch
>
>
> The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the 
> input arguments without copying.  We can substitute IndividualBytesFieldCell 
> for KeyValue used by CellUtil.createCell.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115942#comment-16115942
 ] 

Ted Yu commented on HBASE-16179:


I am currently outside US.
Sean:
It is fine if you have bandwidth to continue.

Please outline your plan in finishing this.

> Fix compilation errors when building hbase-spark against Spark 2.0
> --
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
>  Labels: build
> Fix For: 2.0.0
>
> Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 
> 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 
> 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, 
> 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 
> 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, 
> 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the 
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer 
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18519) Use zero-copy cell to optimize CellUtil.createCell

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115924#comment-16115924
 ] 

Hadoop QA commented on HBASE-18519:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m  6s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
13s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}110m 
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
31s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18519 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880573/HBASE-18519.v0.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 7dc96c271440 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 2a71745 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7955/testReport/ |
| modules | C: hbase-common hbase-server U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7955/console |
| Powered by | Apache Yetus 

[jira] [Commented] (HBASE-17766) Generate Javadoc for hbase-spark module

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115906#comment-16115906
 ] 

Sean Busbey commented on HBASE-17766:
-

I think we should wait for HBASE-16179 to land so we can make sure the 
multiple-scala handling goes okay. My gut reaction based on HBASE-18405 is that 
we should pick an approach that avoids a top-level scala version.

> Generate Javadoc for hbase-spark module 
> 
>
> Key: HBASE-17766
> URL: https://issues.apache.org/jira/browse/HBASE-17766
> Project: HBase
>  Issue Type: Bug
>  Components: documentation, spark
>Reporter: Yi Liang
>Assignee: Yi Liang
> Fix For: 2.0.0
>
> Attachments: HBASE-17766-V1.patch, scaladoc.patch, spark-api.jpg, 
> user-api.jpg
>
>
>  Scala classes in hbase-spark module aren't showing up in our API docs nor 
> our internal API docs. see https://hbase.apache.org/apidocs/ 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115905#comment-16115905
 ] 

Sean Busbey commented on HBASE-16179:
-

Now that we have scope details in HBASE-18405, [~tedyu], you still interested 
in finishing this up? If not I can start from the last patch and update it.

> Fix compilation errors when building hbase-spark against Spark 2.0
> --
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
>  Issue Type: Bug
>  Components: spark
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Critical
>  Labels: build
> Fix For: 2.0.0
>
> Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 
> 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 
> 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, 
> 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 
> 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, 
> 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the 
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer 
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18519) Use zero-copy cell to optimize CellUtil.createCell

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18519:
---
Attachment: HBASE-18519.v0.patch

> Use zero-copy cell to optimize CellUtil.createCell
> --
>
> Key: HBASE-18519
> URL: https://issues.apache.org/jira/browse/HBASE-18519
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18519.v0.patch
>
>
> The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the 
> input arguments without copying.  We can substitute IndividualBytesFieldCell 
> for KeyValue used by CellUtil.createCell.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18519) Use zero-copy cell to optimize CellUtil.createCell

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18519:
---
Status: Patch Available  (was: Open)

> Use zero-copy cell to optimize CellUtil.createCell
> --
>
> Key: HBASE-18519
> URL: https://issues.apache.org/jira/browse/HBASE-18519
> Project: HBase
>  Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18519.v0.patch
>
>
> The IndividualBytesFieldCell, which is introduced by HBASE-14882, carries the 
> input arguments without copying.  We can substitute IndividualBytesFieldCell 
> for KeyValue used by CellUtil.createCell.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18467) nightly job needs to comment on jira

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115863#comment-16115863
 ] 

Sean Busbey commented on HBASE-18467:
-

my attempts to use jenkins plugins for this have failed. going to post via 
stuff already present in yetus instead.

> nightly job needs to comment on jira
> 
>
> Key: HBASE-18467
> URL: https://issues.apache.org/jira/browse/HBASE-18467
> Project: HBase
>  Issue Type: Improvement
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
>
> follow on from HBASE-18147, need a post action that pings all newly-committed 
> jiras with result of the branch build



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18426) nightly job should use independent stages to check supported jdks

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-18426:

   Resolution: Fixed
Fix Version/s: 1.1.12
   2.0.0-alpha-2
   1.2.7
   1.5.0
   1.3.2
   1.4.0
   3.0.0
   Status: Resolved  (was: Patch Available)

> nightly job should use independent stages to check supported jdks
> -
>
> Key: HBASE-18426
> URL: https://issues.apache.org/jira/browse/HBASE-18426
> Project: HBase
>  Issue Type: Improvement
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-18426.0.patch
>
>
> follow on from HBASE-18147 to handle the lack of multijdk support for unit 
> tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-14220) nightly tests should verify src tgz generates and builds correctly

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14220:

Summary: nightly tests should verify src tgz generates and builds correctly 
 (was: test-patch.sh should verify src tgz generates and builds correctly)

> nightly tests should verify src tgz generates and builds correctly
> --
>
> Key: HBASE-14220
> URL: https://issues.apache.org/jira/browse/HBASE-14220
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-14220.0.patch
>
>
> Twice at release time I've bumped into broken src tgz packaging. This is 
> somewhat expected as builds tend to have dark, hidden corners. Let's add a 
> check to our build bot so we find these issues earlier.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-14220) test-patch.sh should verify src tgz generates and builds correctly

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14220:

Issue Type: Improvement  (was: Task)

> test-patch.sh should verify src tgz generates and builds correctly
> --
>
> Key: HBASE-14220
> URL: https://issues.apache.org/jira/browse/HBASE-14220
> Project: HBase
>  Issue Type: Improvement
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-14220.0.patch
>
>
> Twice at release time I've bumped into broken src tgz packaging. This is 
> somewhat expected as builds tend to have dark, hidden corners. Let's add a 
> check to our build bot so we find these issues earlier.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-14220) test-patch.sh should verify src tgz generates and builds correctly

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-14220:

   Resolution: Fixed
Fix Version/s: (was: 2.0.0)
   1.1.12
   2.0.0-alpha-2
   1.2.7
   1.5.0
   1.3.2
   1.4.0
   3.0.0
   Status: Resolved  (was: Patch Available)

> test-patch.sh should verify src tgz generates and builds correctly
> --
>
> Key: HBASE-14220
> URL: https://issues.apache.org/jira/browse/HBASE-14220
> Project: HBase
>  Issue Type: Task
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Minor
>  Labels: beginner
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2, 1.1.12
>
> Attachments: HBASE-14220.0.patch
>
>
> Twice at release time I've bumped into broken src tgz packaging. This is 
> somewhat expected as builds tend to have dark, hidden corners. Let's add a 
> check to our build bot so we find these issues earlier.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115853#comment-16115853
 ] 

Hadoop QA commented on HBASE-15511:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m 12s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}  
1m 14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  7m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
26s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
18s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
35s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}108m 
54s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  1m 
 0s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}173m 37s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-15511 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880565/HBASE-15511.master.009.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  cc  hbaseprotoc  |
| uname | Linux 222d929b208b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| 

[jira] [Commented] (HBASE-18426) nightly job should use independent stages to check supported jdks

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115851#comment-16115851
 ] 

Sean Busbey commented on HBASE-18426:
-

yes, I think it does. I'll fix that on commit. Thanks Ted!

> nightly job should use independent stages to check supported jdks
> -
>
> Key: HBASE-18426
> URL: https://issues.apache.org/jira/browse/HBASE-18426
> Project: HBase
>  Issue Type: Improvement
>  Components: community, test
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Critical
> Attachments: HBASE-18426.0.patch
>
>
> follow on from HBASE-18147 to handle the lack of multijdk support for unit 
> tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18315) Eliminate the findbugs warnings for hbase-rest

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115841#comment-16115841
 ] 

Chia-Ping Tsai commented on HBASE-18315:


Ping for reviews~Let us get rid of the findbugs warnings.

> Eliminate the findbugs warnings for hbase-rest
> --
>
> Key: HBASE-18315
> URL: https://issues.apache.org/jira/browse/HBASE-18315
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-alpha-2
>
> Attachments: HBASE-18315.branch-1.2.v0.patch, 
> HBASE-18315.branch-1.3.v0.patch, HBASE-18315.branch-1.v0.patch, 
> HBASE-18315.v0.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18518) Remove jersey1* dependencies from project and jersey1* jars from lib dir

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115836#comment-16115836
 ] 

Hadoop QA commented on HBASE-18518:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: 
hbase-assembly {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
33s{color} | {color:red} hbase-rest in master has 3 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m  9s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: 
hbase-assembly {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
16s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
47s{color} | {color:green} hbase-rest in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
14s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
19s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 17s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18518 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880566/HBASE-18518-master-02.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  xml  findbugs  
hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux c7d70a78c883 3.13.0-119-generic #166-Ubuntu SMP 

[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115835#comment-16115835
 ] 

Chia-Ping Tsai commented on HBASE-18485:


+1

> Performance issue: ClientAsyncPrefetchScanner is slower than 
> ClientSimpleScanner
> 
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch, 
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch, 
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch, 
> HBASE-18485-v6.patch, HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18518) Remove jersey1* dependencies from project and jersey1* jars from lib dir

[Samir Ahmic (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samir Ahmic updated HBASE-18518:

Attachment: HBASE-18518-master-02.patch

Here is new patch addressing test failure.

> Remove jersey1* dependencies from project and jersey1* jars from lib dir
> 
>
> Key: HBASE-18518
> URL: https://issues.apache.org/jira/browse/HBASE-18518
> Project: HBase
>  Issue Type: Task
>  Components: dependencies, pom, REST
>Affects Versions: 3.0.0, 2.0.0-alpha-1
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>  Labels: cleanup
> Fix For: 3.0.0
>
> Attachments: HBASE-18518-master-01.patch, HBASE-18518-master-02.patch
>
>
> Recently i have opened https://issues.apache.org/jira/browse/HBASE-18506 and 
> it is clear that is caused by mixing jersey1 and jersey2 jars in classpath. 
> With https://issues.apache.org/jira/browse/HBASE-12894 we have introduced 
> jersey2 to project,  and we also  have bunch of transitive dependencies 
> (mainly from hadoop) on jersey1 which is not happiest situation since jersey1 
> and jersey2 under same classpath can case runtime issues as it was case with 
> rest.
> This task will have following steps
> * Clean code and replace jersey1 constructs with jersey2 versions(there 
> should not be much of this)
> * Add exclusions for transitive jersey1 dependencies in pom.xml
> * Add exclusions  in hadoop-two-compat.xml to prevent jersey1 jars in lib dir



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115816#comment-16115816
 ] 

Hadoop QA commented on HBASE-18485:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m  2s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
42s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}111m 
59s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
28s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}164m 25s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18485 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880560/HBASE-18485-v6.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 8514bee0aa0c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 637f7ab |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7952/testReport/ |
| modules | C: hbase-client hbase-server U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7952/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Performance issue: ClientAsyncPrefetchScanner 

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-15511:
--
Attachment: HBASE-15511.master.009.patch

Fix unit test errors.

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>Assignee: Reid Chan
> Attachments: HBASE-15511.master.001.patch, 
> HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, 
> HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, 
> HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, 
> HBASE-15511.master.008.patch, HBASE-15511.master.009.patch
>
>
> The current ClusterStatus response returns too much information about the 
> load per region and replication cluster wide. Sometimes that response can be 
> quite large (10s or 100s of MBs) and methods like getServerSize() or 
> getRegionsCount() don't really need the full response. One possibility is to 
> provide a scope (or filter) for the ClusterStatus requests to limit the 
> response back to the client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-15511:
--
Status: Patch Available  (was: Open)

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>Assignee: Reid Chan
> Attachments: HBASE-15511.master.001.patch, 
> HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, 
> HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, 
> HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, 
> HBASE-15511.master.008.patch, HBASE-15511.master.009.patch
>
>
> The current ClusterStatus response returns too much information about the 
> load per region and replication cluster wide. Sometimes that response can be 
> quite large (10s or 100s of MBs) and methods like getServerSize() or 
> getRegionsCount() don't really need the full response. One possibility is to 
> provide a scope (or filter) for the ClusterStatus requests to limit the 
> response back to the client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope

[Reid Chan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reid Chan updated HBASE-15511:
--
Status: Open  (was: Patch Available)

> ClusterStatus should be able to return responses by scope
> -
>
> Key: HBASE-15511
> URL: https://issues.apache.org/jira/browse/HBASE-15511
> Project: HBase
>  Issue Type: Improvement
>Reporter: Esteban Gutierrez
>Assignee: Reid Chan
> Attachments: HBASE-15511.master.001.patch, 
> HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, 
> HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, 
> HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, 
> HBASE-15511.master.008.patch
>
>
> The current ClusterStatus response returns too much information about the 
> load per region and replication cluster wide. Sometimes that response can be 
> quite large (10s or 100s of MBs) and methods like getServerSize() or 
> getRegionsCount() don't really need the full response. One possibility is to 
> provide a scope (or filter) for the ClusterStatus requests to limit the 
> response back to the client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115812#comment-16115812
 ] 

Chia-Ping Tsai edited comment on HBASE-18502 at 8/6/17 2:04 PM:


Will commit it tomorrow unless any disapprobation is raised


was (Author: chia7712):
Will commit it tomorrow unless any disapprobations are raised

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115812#comment-16115812
 ] 

Chia-Ping Tsai commented on HBASE-18502:


Will commit it tomorrow unless any disapprobations are raised

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115802#comment-16115802
 ] 

Ted Yu commented on HBASE-18502:


lgtm

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115793#comment-16115793
 ] 

Chia-Ping Tsai commented on HBASE-18515:


Will commit it tomorrow if no objections.

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115792#comment-16115792
 ] 

Chia-Ping Tsai commented on HBASE-18502:


[~tedyu] Could you take a look at v2? Thanks

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115790#comment-16115790
 ] 

Hadoop QA commented on HBASE-18502:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 12 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
 9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
37m 42s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}125m 
44s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
35s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
54s{color} | {color:green} hbase-examples in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
51s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}196m 19s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18502 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880554/HBASE-18502.v2.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux da9d9f7cb1bf 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 637f7ab |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7950/testReport/ |
| modules | C: hbase-server hbase-rsgroup hbase-examples U: . |
| Console output | 

[jira] [Commented] (HBASE-18504) Add documentation for WAL compression

[Peter Somogyi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115783#comment-16115783
 ] 

Peter Somogyi commented on HBASE-18504:
---

Could someone review this?

> Add documentation for WAL compression
> -
>
> Key: HBASE-18504
> URL: https://issues.apache.org/jira/browse/HBASE-18504
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Minor
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18504.master.001.patch, 
> HBASE-18504.master.001.patch
>
>
> The Reference Guide does not have any documentation about WAL compression.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18485) Performance issue: ClientAsyncPrefetchScanner is slower than ClientSimpleScanner

[Guanghao Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang updated HBASE-18485:
---
Attachment: HBASE-18485-v6.patch

Retry for hadoop QA.

> Performance issue: ClientAsyncPrefetchScanner is slower than 
> ClientSimpleScanner
> 
>
> Key: HBASE-18485
> URL: https://issues.apache.org/jira/browse/HBASE-18485
> Project: HBase
>  Issue Type: Improvement
>Affects Versions: 3.0.0, 2.0.0-alpha-2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18485-v1.patch, HBASE-18485-v2.patch, 
> HBASE-18485-v3.patch, HBASE-18485-v4.patch, HBASE-18485-v4.patch, 
> HBASE-18485-v4.patch, HBASE-18485-v5.patch, HBASE-18485-v5.patch, 
> HBASE-18485-v6.patch, HBASE-18485-v6.patch
>
>
> Copied the test result from HBASE-17994.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred scan 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred --asyncPrefetch=True scan 1
> {code}
> Mean latency.
> || ||Test1|| Test2 || Test3 || Test4|| Test5||
> |scan| 12.21 | 14.32 | 13.25 | 13.07 | 11.83 |
> |scan with prefetch=True | 37.36 | 37.88 | 37.56 | 37.66 | 38.28 |



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115765#comment-16115765
 ] 

Hadoop QA commented on HBASE-18515:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
30m 33s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  2m 
44s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}110m 
12s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  1m 
58s{color} | {color:green} hbase-endpoint in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
43s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}170m 20s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18515 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880550/HBASE-18515.master.v4.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  
hbaseanti  checkstyle  compile  |
| uname | Linux 6af19fa0db3c 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 637f7ab |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC3 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7949/testReport/ |
| modules | C: hbase-client hbase-server hbase-endpoint U: . |
| Console output | 

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115743#comment-16115743
 ] 

stack commented on HBASE-18525:
---

Thanks [~yuzhih...@gmail.com].  V2 has some goodness might steal.

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115742#comment-16115742
 ] 

Hadoop QA commented on HBASE-18142:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} rubocop {color} | {color:red}  0m 
14s{color} | {color:red} The patch generated 9 new + 441 unchanged - 24 fixed = 
450 total (was 465) {color} |
| {color:red}-1{color} | {color:red} ruby-lint {color} | {color:red}  0m 
13s{color} | {color:red} The patch generated 3 new + 506 unchanged - 3 fixed = 
509 total (was 509) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
38m  1s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m 
20s{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
11s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 12s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18142 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880555/HBASE-18142.master.v7.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  rubocop  ruby_lint  |
| uname | Linux 8002f5ccbbd0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 637f7ab |
| Default Java | 1.8.0_131 |
| rubocop | v0.49.1 |
| rubocop | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7951/artifact/patchprocess/diff-patch-rubocop.txt
 |
| ruby-lint | v2.3.1 |
| ruby-lint | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7951/artifact/patchprocess/diff-patch-ruby-lint.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7951/testReport/ |
| modules | C: hbase-shell U: hbase-shell |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7951/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> Deletion of a cell deletes the previous versions too
> 
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
>  Issue Type: Bug
>  Components: API, shell
>Affects Versions: 3.0.0
>Reporter: Karthick
>Assignee: ChunHao
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18142.master.v0.patch, 
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, 
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, 
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, 
> HBASE-18142.master.v7.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the 
> previous versions of the same cell also got deleted. But when I tried the 
> same using the Java API, then the previous versions 

[jira] [Commented] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115736#comment-16115736
 ] 

Jui-Yu Hsieh commented on HBASE-18439:
--

Thank you :)[#Sahil Aggarwal]

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Jui-Yu Hsieh
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Sahil Aggarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sahil Aggarwal reassigned HBASE-18439:
--

Assignee: Jui-Yu Hsieh  (was: Sahil Aggarwal)

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Jui-Yu Hsieh
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-7129) Need documentation for REST atomic operations (HBASE-4720)

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115734#comment-16115734
 ] 

Jui-Yu Hsieh commented on HBASE-7129:
-

Hi , Sean Busbey
I want to solve this issue.
Could you assign this issue to me ? thanks.

> Need documentation for REST atomic operations (HBASE-4720)
> --
>
> Key: HBASE-7129
> URL: https://issues.apache.org/jira/browse/HBASE-7129
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation, REST
>Reporter: Joe Pallas
>Priority: Minor
>  Labels: beginner
>
> HBASE-4720 added checkAndPut/checkAndDelete capability to the REST interface, 
> but the REST documentation (in the package summary) needs to be updated so 
> people know that this feature exists and how to use it.
> http://wiki.apache.org/hadoop/Hbase/Stargate
> http://hbase.apache.org/book/rest.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-7129) Need documentation for REST atomic operations (HBASE-4720)

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai reassigned HBASE-7129:
-

Assignee: Jui-Yu Hsieh

> Need documentation for REST atomic operations (HBASE-4720)
> --
>
> Key: HBASE-7129
> URL: https://issues.apache.org/jira/browse/HBASE-7129
> Project: HBase
>  Issue Type: Improvement
>  Components: documentation, REST
>Reporter: Joe Pallas
>Assignee: Jui-Yu Hsieh
>Priority: Minor
>  Labels: beginner
>
> HBASE-4720 added checkAndPut/checkAndDelete capability to the REST interface, 
> but the REST documentation (in the package summary) needs to be updated so 
> people know that this feature exists and how to use it.
> http://wiki.apache.org/hadoop/Hbase/Stargate
> http://hbase.apache.org/book/rest.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115733#comment-16115733
 ] 

Chia-Ping Tsai commented on HBASE-18439:


I have added [~johnfly2003] to contributors.

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Sahil Aggarwal (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115731#comment-16115731
 ] 

Sahil Aggarwal commented on HBASE-18439:


[~johnfly2003] Sure, i haven't started on it yet. I can't find you to assign it 
to you, probably because you are not added to contributers.

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Sahil Aggarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sahil Aggarwal reassigned HBASE-18439:
--

Assignee: Sahil Aggarwal

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Sahil Aggarwal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sahil Aggarwal reassigned HBASE-18439:
--

Assignee: (was: Sahil Aggarwal)

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115728#comment-16115728
 ] 

Jui-Yu Hsieh edited comment on HBASE-18439 at 8/6/17 9:25 AM:
--

ｓｏｒｒｙ～
I made a mistake about the of create time


was (Author: johnfly2003):
ｓｏｒｒｙ～
I made a mistake about the time of create

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115728#comment-16115728
 ] 

Jui-Yu Hsieh commented on HBASE-18439:
--

ｓｏｒｒｙ～
I made a mistake about the time 

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115728#comment-16115728
 ] 

Jui-Yu Hsieh edited comment on HBASE-18439 at 8/6/17 9:25 AM:
--

ｓｏｒｒｙ～
I made a mistake about the time of create


was (Author: johnfly2003):
ｓｏｒｒｙ～
I made a mistake about the time 

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18142) Deletion of a cell deletes the previous versions too

[ChunHao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ChunHao updated HBASE-18142:

Status: Open  (was: Patch Available)

> Deletion of a cell deletes the previous versions too
> 
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
>  Issue Type: Bug
>  Components: API, shell
>Affects Versions: 3.0.0
>Reporter: Karthick
>Assignee: ChunHao
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18142.master.v0.patch, 
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, 
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, 
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, 
> HBASE-18142.master.v7.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the 
> previous versions of the same cell also got deleted. But when I tried the 
> same using the Java API, then the previous versions are not deleted and I can 
> retrive the previous values.
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
> see this file to fix the issue. This method (public Delete addColumns(final 
> byte [] family, final byte [] qualifier, final long timestamp)) only deletes 
> the current version of the cell. The previous versions are not deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18142) Deletion of a cell deletes the previous versions too

[ChunHao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ChunHao updated HBASE-18142:

Status: Patch Available  (was: Open)

> Deletion of a cell deletes the previous versions too
> 
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
>  Issue Type: Bug
>  Components: API, shell
>Affects Versions: 3.0.0
>Reporter: Karthick
>Assignee: ChunHao
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18142.master.v0.patch, 
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, 
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, 
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, 
> HBASE-18142.master.v7.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the 
> previous versions of the same cell also got deleted. But when I tried the 
> same using the Java API, then the previous versions are not deleted and I can 
> retrive the previous values.
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
> see this file to fix the issue. This method (public Delete addColumns(final 
> byte [] family, final byte [] qualifier, final long timestamp)) only deletes 
> the current version of the cell. The previous versions are not deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18142) Deletion of a cell deletes the previous versions too

[ChunHao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ChunHao updated HBASE-18142:

Attachment: HBASE-18142.master.v7.patch

> Deletion of a cell deletes the previous versions too
> 
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
>  Issue Type: Bug
>  Components: API, shell
>Affects Versions: 3.0.0
>Reporter: Karthick
>Assignee: ChunHao
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18142.master.v0.patch, 
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, 
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, 
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch, 
> HBASE-18142.master.v7.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the 
> previous versions of the same cell also got deleted. But when I tried the 
> same using the Java API, then the previous versions are not deleted and I can 
> retrive the previous values.
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
> see this file to fix the issue. This method (public Delete addColumns(final 
> byte [] family, final byte [] qualifier, final long timestamp)) only deletes 
> the current version of the cell. The previous versions are not deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115715#comment-16115715
 ] 

Ted Yu commented on HBASE-18525:


Stack:
I am fine with you or Umesh finishing this work.

Just curious whether patch v2 is closer (there may be intricacies as you 
mentioned).

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18439) Subclasses of o.a.h.h.chaos.actions.Action all use the same logger

[Jui-Yu Hsieh (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115714#comment-16115714
 ] 

Jui-Yu Hsieh commented on HBASE-18439:
--

Hi , Sahil Aggarwal ,
I want to solve this issue.
Could you assign this issue to me ? thanks.

> Subclasses of o.a.h.h.chaos.actions.Action all use the same logger
> --
>
> Key: HBASE-18439
> URL: https://issues.apache.org/jira/browse/HBASE-18439
> Project: HBase
>  Issue Type: Bug
>  Components: integration tests
>Reporter: Mike Drob
>Assignee: Sahil Aggarwal
>Priority: Minor
>  Labels: beginner
>
> A bunch of the actions all use the same logger inherited from the super 
> class. We should have them declare distinct loggers, either each one in class 
> or perhaps we can do something dynamically like 
> {{LogFactory.getLogger(MethodHandles.lookup().lookupClass()}} and drop the 
> static modifier on the log field.
> I'm not sure that exact incantation would actually work, but the 
> MethodHandles approach in general is how logger resolution happens in Solr 
> and it actually works out pretty well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115713#comment-16115713
 ] 

stack commented on HBASE-18525:
---

Also, the setting of UnassignProcedure explicit fail was intentional over in 
HBASE-18491. Thought was that there might be scenarios where this would cause 
issue. You seem to have identified one [~te...@apache.org]. There are few 
developments going on in assign at moment (see tail of HBASE-18366). Patches 
from left field as yours is, we would like to take under consideration and not 
hurry in to make sure it fits broader plan. Thanks.

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
Attachment: HBASE-18502.v2.patch

v2 patch
# update the example in 
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java
# rebase patch owing to HBASE-18520

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
Status: Patch Available  (was: Open)

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch, HBASE-18502.v2.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai updated HBASE-18502:
---
Status: Open  (was: Patch Available)

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115711#comment-16115711
 ] 

Chia-Ping Tsai commented on HBASE-18515:


+1 if the QA doesn't complain

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18142) Deletion of a cell deletes the previous versions too

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115710#comment-16115710
 ] 

Chia-Ping Tsai commented on HBASE-18142:


[~chunhao] Could you address the comment from [~mdrob] ?
{quote}
Not a fan of the rubocop:disable directives. We can leave those errors present 
and work through them incrementally in future issues and then disable the 
checks once we decide that there really is nothing to be done.
{quote}

BTW, please check the new warnings.
{code}
/testptch/hbase/hbase-shell/src/main/ruby/hbase/table.rb:165:1: C: Tab detected.
/testptch/hbase/hbase-shell/src/main/ruby/hbase/table.rb:166:1: C: Tab detected.
{code}

> Deletion of a cell deletes the previous versions too
> 
>
> Key: HBASE-18142
> URL: https://issues.apache.org/jira/browse/HBASE-18142
> Project: HBase
>  Issue Type: Bug
>  Components: API, shell
>Affects Versions: 3.0.0
>Reporter: Karthick
>Assignee: ChunHao
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18142.master.v0.patch, 
> HBASE-18142.master.v1.patch, HBASE-18142.master.v2.patch, 
> HBASE-18142.master.v3.patch, HBASE-18142.master.v4.patch, 
> HBASE-18142.master.v5.patch, HBASE-18142.master.v6.patch
>
>
> When I tried to delete a cell using it's timestamp in the Hbase Shell, the 
> previous versions of the same cell also got deleted. But when I tried the 
> same using the Java API, then the previous versions are not deleted and I can 
> retrive the previous values.
> https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Delete.java
> see this file to fix the issue. This method (public Delete addColumns(final 
> byte [] family, final byte [] qualifier, final long timestamp)) only deletes 
> the current version of the cell. The previous versions are not deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope

[Lars George (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115709#comment-16115709
 ] 

Lars George commented on HBASE-18526:
-

Pinging [~vrodionov] as per the mailing list...

> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
>  Issue Type: Bug
>  Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
>   message = "Default TTL is not supported for FIFO compaction";
>   throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
>   message = "MIN_VERSION > 0 is not supported for FIFO compaction";
>   throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
>   blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
>   message =
>   "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
>   + " is below recommended minimum of 1000";
>   throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at 
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at 
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at 
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at 
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at 
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at 
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope

[Lars George (JIRA)]

Lars George created HBASE-18526:
---

 Summary: FIFOCompactionPolicy pre-check uses wrong scope
 Key: HBASE-18526
 URL: https://issues.apache.org/jira/browse/HBASE-18526
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 1.3.1
Reporter: Lars George


See https://issues.apache.org/jira/browse/HBASE-14468

It adds this check to {{HMaster.checkCompactionPolicy()}}:

{code}
// 1. Check TTL
if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
  message = "Default TTL is not supported for FIFO compaction";
  throw new IOException(message);
}

// 2. Check min versions
if (hcd.getMinVersions() > 0) {
  message = "MIN_VERSION > 0 is not supported for FIFO compaction";
  throw new IOException(message);
}

// 3. blocking file count
String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
if (sbfc != null) {
  blockingFileCount = Integer.parseInt(sbfc);
}
if (blockingFileCount < 1000) {
  message =
  "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
+ blockingFileCount
  + " is below recommended minimum of 1000";
  throw new IOException(message);
}
{code}

Why does it only check the blocking file count on the HTD level, while
others are check on the HCD level? Doing this for example fails
because of it:

{noformat}
hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
=> 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
'hbase.hstore.blockingStoreFiles' => 2000 } }

ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
minimum of 1000 Set hbase.table.sanity.checks to false at conf or
table descriptor if you want to bypass sanity checks
at 
org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
at 
org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
at 
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
at 
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
Caused by: java.io.IOException: blocking file count
'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
1000
at 
org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
at 
org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
... 7 more
{noformat}

The check should be performed on the column family level instead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[stack (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115708#comment-16115708
 ] 

stack commented on HBASE-18525:
---

[~ted_yu] Thanks for pointing out the issue. Please let [~uagashe] and or 
myself deal with it. Assign and Unassign are not symmetrical. Retrying 
unassigns introduces fuzzyness around whether region is unassigned or not, 
something we'd rather do w/o if possible. With the new assignment manager w/ 
want to be able to be crisp about what state a region is in. Thanks.

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method

[Chia-Ping Tsai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115706#comment-16115706
 ] 

Chia-Ping Tsai commented on HBASE-18500:


bq. How you mean
It is better to make the behavior of Table#put consistent with Table#delete. 
What that means is we should remove the succeed PUTs from list. The broken 
compatibility is caused by that we did not modify the input list before. 
Personally, i prefer removed the succeed PUTs because that make Table#put more 
easier to exercise. We had good discussion in HBASE-13271.

> Performance issue: Don't use BufferedMutator for HTable's put method
> 
>
> Key: HBASE-18500
> URL: https://issues.apache.org/jira/browse/HBASE-18500
> Project: HBase
>  Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test 
> with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred --autoFlush=True randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=10 
> --nomapred --autoFlush=True asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the 
> almost same latency with the blocking write. But for master branch, the 
> result shows the async write has better latency than the blocking client.  
> Take a look about the code, I thought the difference is the BufferedMutator. 
> For master branch, HTable don't have a write buffer and all write request 
> will be flushed directly. And user can use BufferedMutator when user want to 
> perform client-side buffering of writes. For the performance issue 
> (autoFlush=True), I thought we can use rpc caller directly in HTable's put 
> method. Thanks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-18525:
---
Status: Open  (was: Patch Available)

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated HBASE-18525:
---
Attachment: 18525.v2.txt

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt, 18525.v2.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Xie YiFan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xie YiFan updated HBASE-18515:
--
Status: Open  (was: Patch Available)

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Xie YiFan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xie YiFan updated HBASE-18515:
--
Status: Patch Available  (was: Open)

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (HBASE-18515) Introduce Delete.add as a replacement for Delete#addDeleteMarker

[Xie YiFan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xie YiFan updated HBASE-18515:
--
Attachment: HBASE-18515.master.v4.patch

>  Introduce Delete.add as a replacement for Delete#addDeleteMarker
> -
>
> Key: HBASE-18515
> URL: https://issues.apache.org/jira/browse/HBASE-18515
> Project: HBase
>  Issue Type: Task
>  Components: Client
>Reporter: Chia-Ping Tsai
>Assignee: Xie YiFan
>  Labels: beginner
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18515.master.v0.patch, 
> HBASE-18515.master.v1.patch, HBASE-18515.master.v2.patch, 
> HBASE-18515.master.v3.patch, HBASE-18515.master.v4.patch
>
>
> {quote}
>   public Delete addDeleteMarker(Cell kv) throws IOException {
> // TODO: Deprecate and rename 'add' so it matches how we add KVs to Puts.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115677#comment-16115677
 ] 

Umesh Agashe commented on HBASE-18525:
--

bq. However, that wouldn't affect the condition in UnassignProcedure which is 
pertinent to test failure:
{code}
if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
{code}

The condition would be true only after max attempts are exhausted, right?

> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115675#comment-16115675
 ] 

Ted Yu commented on HBASE-18525:


If max attempts for unassign operation is added, where would the check for 
number of attempts be added ?
In AssignProcedure, I see:
{code}
if (aborted.get() && regionNode.isInState(State.CLOSED, State.OFFLINE)) {
  if (incrementAndCheckMaxAttempts(env, regionNode)) {
{code}
However, that wouldn't affect the condition in UnassignProcedure which is 
pertinent to test failure:
{code}
if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
{code}


> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18517) limit max log message width in log4j

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115671#comment-16115671
 ] 

Hadoop QA commented on HBASE-18517:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 
33m 20s{color} | {color:green} Patch does not cause any errors with Hadoop 
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m  0s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18517 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12880548/HBASE-18517.master.001.patch
 |
| Optional Tests |  asflicense  |
| uname | Linux 056d807a4a39 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
 |
| git revision | master / 637f7ab |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/7948/console |
| Powered by | Apache Yetus 0.4.0   http://yetus.apache.org |


This message was automatically generated.



> limit max log message width in log4j
> 
>
> Key: HBASE-18517
> URL: https://issues.apache.org/jira/browse/HBASE-18517
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 3.0.0, 1.5.0
>Reporter: Vikas Vishwakarma
>Assignee: Vikas Vishwakarma
> Fix For: 3.0.0, 1.5.0
>
> Attachments: HBASE-18517.branch-1.001.patch, 
> HBASE-18517.master.001.patch
>
>
> We had two cases now in our prod / pilot setups which is leading to humongous 
> log lines in RegionServer logs. 
> In first case, one of the phoenix user had constructed a query with a really 
> large list of Id filters (61 MB) that translated into HBase scan that was 
> running slow which lead to responseTooSlow messages in the logs with the 
> entire filter list being printed in the logs, example
> ipc.RpcServer - (responseTooSlow): 
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)","starttimems":1501457864417,"responsesize":11,"method":"Scan","param":"region
>  { type: REGION_NAME value:  . 
> org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter\\022\351\\200\\036\\n(org.apache.phoenix.filter.SkipScanFilter
>  ...  ... 
> There was another case where a use case had created a table with really large 
> key/region names. This was causing humongous log lines for flush and 
> compaction on these regions filling up the RS logs
> These large logs usually cause issues with disk I/O load, loading the splunk 
> servers, even machine perf degradations. With 61 MB log lines basic log 
> processing commands like vim, scrolling the logs, wc -l , etc were getting 
> stuck. High GC activity was also noted on this cluster although not 100% sure 
> if it was related to above issue. 
> We should consider limiting the message size in logs which can be easily done 
> by adding a maximum width format modifier on the message conversion character 
> in log4j.properties
> log4j.appender.console.layout.ConversionPattern=...: %m%n
> to 
> log4j.appender.console.layout.ConversionPattern=...: %.1m%n



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18525) TestAssignmentManager#testSocketTimeout fails in master branch

[Umesh Agashe (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115670#comment-16115670
 ] 

Umesh Agashe commented on HBASE-18525:
--

Hi [~tedyu],

I think with current code UnassignProcedure is expected to fail and in short 
term the test can be changed accordingly.

The test instantiates SocketTimeoutRsExecutor with maxServerRetries of 3. In 
case of AssignProcedure retries are determined by 
AssignmentManager.getAssignMaxAttempts() but in case of UnassignProcedure on 
first failure procedure is determined to be failed. Long term fix is to have 
max attempts for unassign operation like assign.

Pre HBASE-18491, we just assumed that when communication fails with RS, 
ServerCrashProcedure will take care of reassigning region and unaasign region 
operation can be considered successful. The assumption is a cause of the 
failure of other test re-enabled in HBASE-18491.

Let me know your thoughts.

Thanks, Umesh


> TestAssignmentManager#testSocketTimeout fails in master branch
> --
>
> Key: HBASE-18525
> URL: https://issues.apache.org/jira/browse/HBASE-18525
> Project: HBase
>  Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 18525.v1.txt
>
>
> Toward the end of the test output, I saw:
> {code}
> 2017-08-05 03:30:16,591 INFO  [Time-limited test] 
> assignment.TestAssignmentManager(446): ExecutionException
> java.util.concurrent.ExecutionException: 
> org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:104)
>   at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait$ProcedureFuture.get(ProcedureSyncWait.java:62)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.waitOnFuture(TestAssignmentManager.java:444)
>   at 
> org.apache.hadoop.hbase.master.assignment.TestAssignmentManager.testSocketTimeout(TestAssignmentManager.java:255)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>   at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>   at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
>   at 
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.master.procedure.ServerCrashException: 
> ServerCrashProcedure pid=3, server=localhost,103,1
>   at 
> org.apache.hadoop.hbase.master.assignment.UnassignProcedure.updateTransition(UnassignProcedure.java:169)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:274)
>   at 
> org.apache.hadoop.hbase.master.assignment.RegionTransitionProcedure.execute(RegionTransitionProcedure.java:57)
>   at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:847)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1440)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1209)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:79)
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1719)
> {code}
> This test failure seems to happen after HBASE-18491 was checked in.
> Looking at the change in UnassignProcedure, it seems we should handle the two 
> conditions differently:
> {code}
>  if (serverCrashed.get() || !isServerOnline(env, regionNode)) {
> {code}
> With attached patch, TestAssignmentManager#testSocketTimeout and 
> TestServerCrashProcedure#testRecoveryAndDoubleExecutionOnRsWithMeta pass.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115663#comment-16115663
 ] 

Ted Yu edited comment on HBASE-18502 at 8/6/17 6:37 AM:


lgtm

Looked at 
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java
 which doesn't need to be modified


was (Author: yuzhih...@gmail.com):
lgtm

Please cover 
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java
 as well

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18502) Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115663#comment-16115663
 ] 

Ted Yu commented on HBASE-18502:


lgtm

Please cover 
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java
 as well

> Change MasterObserver to use TableDescriptor and ColumnFamilyDescriptor
> ---
>
> Key: HBASE-18502
> URL: https://issues.apache.org/jira/browse/HBASE-18502
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-2
>
> Attachments: HBASE-18502.v0.patch, HBASE-18502.v1.patch, 
> HBASE-18502.v1.patch
>
>
> MasterObserver is IA.COPROC so we can make some Incompatible change for 3.0 
> and 2.0



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HBASE-18296) NPE in ProcedureExecutor.removeChore

[Ping Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HBASE-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115661#comment-16115661
 ] 

Ping Liu commented on HBASE-18296:
--

[~mdrob]
Do you remember which test it is where you found this error?  There are many 
places where HRegionServer starts up.  It'll be great if the specific test can 
be identified.

> NPE in ProcedureExecutor.removeChore
> 
>
> Key: HBASE-18296
> URL: https://issues.apache.org/jira/browse/HBASE-18296
> Project: HBase
>  Issue Type: Bug
>  Components: proc-v2
>Affects Versions: 2.0.0-alpha-1
>Reporter: Mike Drob
>
> Saw this in logs while debugging other failing tests:
> {noformat}
> 2017-06-29 13:54:15,995 ERROR [M:0;172.18.16.40:55484] 
> server.NIOServerCnxnFactory$1(44): Thread 
> Thread[M:0;172.18.16.40:55484,5,FailOnTimeoutGroup] died
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.removeChore(ProcedureExecutor.java:656)
>   at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.stop(AssignmentManager.java:233)
>   at 
> org.apache.hadoop.hbase.master.HMaster.stopServiceThreads(HMaster.java:1154)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1189)
>   at java.lang.Thread.run(Thread.java:748)
> {noformat}
> I think it was related to an initialization failure, but we should code more 
> defensively anyway



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)