[GitHub] [hbase] pankaj72981 commented on issue #1223: HBASE-23739 BoundedRecoveredHFilesOutputSink should read the table de…

2020-02-29 Thread GitBox
pankaj72981 commented on issue #1223: HBASE-23739 
BoundedRecoveredHFilesOutputSink should read the table de…
URL: https://github.com/apache/hbase/pull/1223#issuecomment-593061669
 
 
   Reading table descriptor always from FS wont be good actually, it will cause 
many RPC to Namenode for same operation from all RS. (Please correct me if 
wrong) 
   Currently HMaster already load the table decriptors during startup and RS 
retrieve it. IMO RS should read table descriptor from FS directly only when it 
fails to retrieve from HM due to incomplete initialization. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Commented] (HBASE-22514) Reimplement rsgroup feature and move it into core of HBase

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048469#comment-17048469
 ] 

Hudson commented on HBASE-22514:


Results for branch HBASE-22514
[build #289 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/289/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/289//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/289//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/289//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Reimplement rsgroup feature and move it into core of HBase
> --
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
>  Issue Type: Umbrella
>  Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch, 
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public 
> we need to use java api  RSGroupAdminClient  to manager RSG 
> so  RSGroupAdminClient should be public
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23795) Enable all tests to be run in parallel on reused JVMs.

2020-02-29 Thread Mark Robert Miller (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048462#comment-17048462
 ] 

Mark Robert Miller commented on HBASE-23795:


So, after getting to know the tests more individually, they did not match up 
with experience of trying to run the test suite. So I jumped ahead a bit.

I reduced resources and improvement. But the test suite still did not make 
sense.

I started shutting down resources and resolving deadlock or deadlock like 
situations on shutdown.

I removed System.exit type calls that not appropriate for Junit tests and kill 
JVM’s on us.

Again, I saw improvements but the test runs still didn’t make sense.

So I started hacking and providing higher limits to find out why nothing made 
sense.

In the end, lots of connections are fired up, few are closed, unless you run 
the tests fairly slowly, you will bed DOS attacked by them.

The worst of the leaks appears to be within the rpc client.

These tests are much faster, the rest of the flakies become conquerable, 
running tests with more parallel JVMs and even in the same JVM is now not that 
difficult, though some of the code does make it a little exercise. Good news 
all around.


> Enable all tests to be run in parallel on reused JVMs.
> --
>
> Key: HBASE-23795
> URL: https://issues.apache.org/jira/browse/HBASE-23795
> Project: HBase
>  Issue Type: Wish
>Reporter: Mark Robert Miller
>Priority: Major
>
> I'd like to be able to run HBase tests in under 30-40 minutes on good 
> parallel hardware.
> It will require some small changes / fixes for that wish to come true.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (HBASE-23895) STUCK Region-In-Transition when failed to insert procedure to procedure store

2020-02-29 Thread Michael Stack (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048439#comment-17048439
 ] 

Michael Stack edited comment on HBASE-23895 at 3/1/20 1:54 AM:
---

Nice find [~zghao] .

On #1, the Region used by ProcedureStore is internal to the Master only and 
should be shielded from externals. I'd think this special Region should not be 
making use of this RpcServer.getCall mechanism. How about putting the 
RpcServer.getCall call behind a boolean set on Region open? The boolean would 
be whether the Region is for in-process access or accessed via rpc?

Or, rather than a boolean, instead, we'd add a getDeadline() method to the 
RegionServerServices private Interface that would host the RpcServer.getCall 
for RegionServers but for this Master in-process Region would return 
Long.MAX_VALUE?

On #2, is this an easy fix? Removing the Procedure on exception on submit?

 


was (Author: stack):
Nice find [~zghao] .

On #1, the Region used by ProcedureStore is internal to the Master only and 
should be shielded from externals. I'd think this special Region should not be 
making use of this RpcServer.getCall mechanism. How about putting the 
RpcServer.getCall call behind a boolean set on Region open? The boolean would 
be whether the Region is for in-process access or accessed via rpc?

On #2, is this an easy fix? Removing the Procedure on exception on submit?

 

> STUCK Region-In-Transition when failed to insert procedure to procedure store
> -
>
> Key: HBASE-23895
> URL: https://issues.apache.org/jira/browse/HBASE-23895
> Project: HBase
>  Issue Type: Bug
>  Components: proc-v2, RegionProcedureStore
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> When move an region, it will generate a TRSP first and set the procedure to 
> the region state node. But if the submit TRSP failed, the procedure cannot be 
> unset now and the region will stuck in RIT.
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
> {code:java}
> public Future moveAsync(RegionPlan regionPlan) throws 
> HBaseIOException {
> TransitRegionStateProcedure proc =
>   createMoveRegionProcedure(regionPlan.getRegionInfo(), 
> regionPlan.getDestination());
> return 
> ProcedureSyncWait.submitProcedure(master.getMasterProcedureExecutor(), proc);
>   }
>   public TransitRegionStateProcedure createMoveRegionProcedure(RegionInfo 
> regionInfo,
>   ServerName targetServer) throws HBaseIOException {
> RegionStateNode regionNode = 
> this.regionStates.getRegionStateNode(regionInfo);
> if (regionNode == null) {
>   throw new UnknownRegionException("No RegionStateNode found for " +
>   regionInfo.getEncodedName() + "(Closed/Deleted?)");
> }
> TransitRegionStateProcedure proc;
> regionNode.lock();
> try {
>   preTransitCheck(regionNode, STATES_EXPECTED_ON_UNASSIGN_OR_MOVE);
>   regionNode.checkOnline();
>   proc = TransitRegionStateProcedure.move(getProcedureEnvironment(), 
> regionInfo, targetServer);
>   regionNode.setProcedure(proc);
> } finally {
>   regionNode.unlock();
> }
> return proc;
>   }
> {code}
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
> {code:java}
>   public void setProcedure(TransitRegionStateProcedure proc) {
> assert this.procedure == null;
> this.procedure = proc;
> ritMap.put(regionInfo, this);
>   }
>   public void unsetProcedure(TransitRegionStateProcedure proc) {
> assert this.procedure == proc;
> this.procedure = null;
> ritMap.remove(regionInfo, this);
>   } 
> {code}
> {code:java}
> 2020-02-26,13:45:21,344 ERROR 
> [RpcServer.default.RWQ.Fifo.read.handler=437,queue=5,port=21500] 
> org.apache.hadoop.hbase.ipc.RpcServer: Unexpected throwable object
> java.io.UncheckedIOException: 
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out waiting for 
> lock for row: \x00\x00\x00\x00\x00\x0B\xAB\xD2 in region 
> 9731aea823e7f83264b14713ae486fb7
> at 
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.update(RegionProcedureStore.java:588)
> at 
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.insert(RegionProcedureStore.java:545)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:1042)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:860)
> at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:123)
> at 
> 

[jira] [Commented] (HBASE-23895) STUCK Region-In-Transition when failed to insert procedure to procedure store

2020-02-29 Thread Michael Stack (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048439#comment-17048439
 ] 

Michael Stack commented on HBASE-23895:
---

Nice find [~zghao] .

On #1, the Region used by ProcedureStore is internal to the Master only and 
should be shielded from externals. I'd think this special Region should not be 
making use of this RpcServer.getCall mechanism. How about putting the 
RpcServer.getCall call behind a boolean set on Region open? The boolean would 
be whether the Region is for in-process access or accessed via rpc?

On #2, is this an easy fix? Removing the Procedure on exception on submit?

 

> STUCK Region-In-Transition when failed to insert procedure to procedure store
> -
>
> Key: HBASE-23895
> URL: https://issues.apache.org/jira/browse/HBASE-23895
> Project: HBase
>  Issue Type: Bug
>  Components: proc-v2, RegionProcedureStore
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> When move an region, it will generate a TRSP first and set the procedure to 
> the region state node. But if the submit TRSP failed, the procedure cannot be 
> unset now and the region will stuck in RIT.
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
> {code:java}
> public Future moveAsync(RegionPlan regionPlan) throws 
> HBaseIOException {
> TransitRegionStateProcedure proc =
>   createMoveRegionProcedure(regionPlan.getRegionInfo(), 
> regionPlan.getDestination());
> return 
> ProcedureSyncWait.submitProcedure(master.getMasterProcedureExecutor(), proc);
>   }
>   public TransitRegionStateProcedure createMoveRegionProcedure(RegionInfo 
> regionInfo,
>   ServerName targetServer) throws HBaseIOException {
> RegionStateNode regionNode = 
> this.regionStates.getRegionStateNode(regionInfo);
> if (regionNode == null) {
>   throw new UnknownRegionException("No RegionStateNode found for " +
>   regionInfo.getEncodedName() + "(Closed/Deleted?)");
> }
> TransitRegionStateProcedure proc;
> regionNode.lock();
> try {
>   preTransitCheck(regionNode, STATES_EXPECTED_ON_UNASSIGN_OR_MOVE);
>   regionNode.checkOnline();
>   proc = TransitRegionStateProcedure.move(getProcedureEnvironment(), 
> regionInfo, targetServer);
>   regionNode.setProcedure(proc);
> } finally {
>   regionNode.unlock();
> }
> return proc;
>   }
> {code}
> hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateNode.java
> {code:java}
>   public void setProcedure(TransitRegionStateProcedure proc) {
> assert this.procedure == null;
> this.procedure = proc;
> ritMap.put(regionInfo, this);
>   }
>   public void unsetProcedure(TransitRegionStateProcedure proc) {
> assert this.procedure == proc;
> this.procedure = null;
> ritMap.remove(regionInfo, this);
>   } 
> {code}
> {code:java}
> 2020-02-26,13:45:21,344 ERROR 
> [RpcServer.default.RWQ.Fifo.read.handler=437,queue=5,port=21500] 
> org.apache.hadoop.hbase.ipc.RpcServer: Unexpected throwable object
> java.io.UncheckedIOException: 
> org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timed out waiting for 
> lock for row: \x00\x00\x00\x00\x00\x0B\xAB\xD2 in region 
> 9731aea823e7f83264b14713ae486fb7
> at 
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.update(RegionProcedureStore.java:588)
> at 
> org.apache.hadoop.hbase.procedure2.store.region.RegionProcedureStore.insert(RegionProcedureStore.java:545)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:1042)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:860)
> at 
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitProcedure(ProcedureSyncWait.java:123)
> at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.moveAsync(AssignmentManager.java:657)
> at 
> org.apache.hadoop.hbase.master.HMaster.executeRegionPlansWithThrottling(HMaster.java:1793)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1761)
> at 
> org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:654)
> at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:135)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:352)
> at 
> 

[jira] [Commented] (HBASE-23892) SecureTestCluster should allow its subclasses to pass their Class reference on HBaseKerberosUtils.setSSLConfiguration

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048418#comment-17048418
 ] 

Hudson commented on HBASE-23892:


Results for branch branch-2.2
[build #801 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/801/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/801//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/801//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/801//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> SecureTestCluster should allow its subclasses to pass their Class reference 
> on HBaseKerberosUtils.setSSLConfiguration
> -
>
> Key: HBASE-23892
> URL: https://issues.apache.org/jira/browse/HBASE-23892
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0, 2.2.3
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4
>
>
> Was working on a third party project that relies on hbase-server as a 
> dependency and defines a UT class that extends *SecureTestCluster*. In this 
> project, hbase-server jar is added on the test classpath, and it relies on 
> *KeyStoreTestUtil* to decide where to place related ssl files. Current 
> *KeyStoreTestUtil* code assumes related class files would be under an 
> existing local FS path, but when those are loaded from a jar, related class 
> URI path returns null and causes an NPE that errors out the test execution. 
>  
> The test code in this case extends *SecureTestCluster*, so when 
> *SecureTestCluster.setup* runs, it calls 
> *HBaseKerberosUtils.setSSLConfiguration* passing its own class reference 
> [here|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/SecureTestCluster.java#L72].
> **
> *HBaseKerberosUtils.setSSLConfiguration*, in turn, calls 
> *KeyStoreTestUtil.getClasspathDir* 
> [here|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/security/HBaseKerberosUtils.java#L173].
>  *KeyStoreTestUtil.getClasspathDir* uses the passed class reference name to 
> look for its related resource in the class loader. [This 
> line|https://github.com/apache/hbase/blob/master/hbase-http/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java#L58]
>  returns an URL value as follows, when the passed class reference resource 
> resides on jar file in the class loader (instead of being in a valid local FS 
> path):
> {{jar:file:/Users/wellingtonchevreuil/.m2/repository/org/apache/hbase/hbase-server/2.2.3./hbase-server-2.2.3-tests.jar!/org/apache/hadoop/hbase/security/token/SecureTestCluster.class}}
> This causes [subsequent 
> line|https://github.com/apache/hbase/blob/master/hbase-http/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java#L59]
>  calling {{url.toURI().getPath()}} to return null, then we get the NPE thrown 
> in the following line, and the test errors out:
> {{java.lang.NullPointerException at 
> org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.getClasspathDir(KeyStoreTestUtil.java:59)
>  at 
> org.apache.hadoop.hbase.security.HBaseKerberosUtils.setSSLConfiguration(HBaseKerberosUtils.java:173)
>  at 
> org.apache.hadoop.hbase.security.token.SecureTestCluster.setUp(SecureTestCluster.java:74)
>  at 
> com.cloudera.hbase.security.token.TestCldrManualDelegationToken.setUp(TestCldrManualDelegationToken.java:168)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) 
> at 
> 

[jira] [Commented] (HBASE-22978) Online slow response log

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048417#comment-17048417
 ] 

Hudson commented on HBASE-22978:


Results for branch branch-2
[build #2520 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2520/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2520//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2520//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2520//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Online slow response log
> 
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0, 2.3.0, 1.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.7.0
>
> Attachments: Screen Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot 
> 2019-10-19 at 2.32.54 AM.png, Screen Shot 2019-10-19 at 2.34.11 AM.png, 
> Screen Shot 2019-10-19 at 2.36.14 AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a 
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer - 
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value: 
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above 
> example. We do this because the human readable representation is verbose, the 
> rate of too slow warnings may be high, and the combination of these things 
> can overwhelm the log capture system. The truncation is unfortunate because 
> it eliminates much of the utility of the warnings. For example, the region 
> name, the start and end keys, and the filter hierarchy are all important 
> clues for debugging performance problems caused by moderate to low 
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be 
> too slow in addition to the responseTooSlow logging. The in-memory 
> representation can be complete and compressed. A new admin API and shell 
> command can provide access to the ring buffer for online performance 
> debugging. A modest sizing of the ring buffer will prevent excessive memory 
> utilization for a minor performance debugging feature by limiting the total 
> number of retained records. There is some chance a high rate of requests will 
> cause information on other interesting requests to be overwritten before it 
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted 
> in the request. We don't need to retain all key-values in the mutation, which 
> may be too large to comfortably retain. We only need a unique set of row 
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to 
> apply fast compression to ring buffer entries (if codec support is 
> available), something like snappy or zstandard, and decompress on the fly 
> when servicing the retrieval API request. This will minimize the impact of 
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same 
> information provided by responseTooSlow warnings. Total size of response 
> serialization, possibly also total cell or row counts, should be sufficient 
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more 
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls 
> behind and we are unable to persist an entry before it is 

[jira] [Commented] (HBASE-23876) Add JDK11 compilation and unit test support to nightly job

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048413#comment-17048413
 ] 

Hudson commented on HBASE-23876:


Results for branch HBASE-23876/jdk11-nightly-master
[build #20 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/20/]:
 (x) *{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/20//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/20//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/20//JDK8_Nightly_Build_Report_(Hadoop3)/]


(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23876%252Fjdk11-nightly-master/20//JDK11_Nightly_Build_Report/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Add JDK11 compilation and unit test support to nightly job
> --
>
> Key: HBASE-23876
> URL: https://issues.apache.org/jira/browse/HBASE-23876
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> We already test against multiple JDK versions in a handful of places. Let's 
> get JDK11 added to the mix. Applies to nightly job.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23743) Release 1.4.13

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048395#comment-17048395
 ] 

Hudson commented on HBASE-23743:


Results for branch branch-1.4
[build #1194 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1194/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1194//General_Nightly_Build_Report/]


(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1194//JDK7_Nightly_Build_Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1194//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Release 1.4.13
> --
>
> Key: HBASE-23743
> URL: https://issues.apache.org/jira/browse/HBASE-23743
> Project: HBase
>  Issue Type: Task
>  Components: build, community
>Affects Versions: 1.4.13
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.4.13
>
>
> over due for 1.4.13. I'll try to use the release manager scripts this time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [hbase] busbey commented on a change in pull request #1226: HBASE-23743 Add HBase 1.4.13 to the downloads page

2020-02-29 Thread GitBox
busbey commented on a change in pull request #1226: HBASE-23743 Add HBase 
1.4.13 to the downloads page
URL: https://github.com/apache/hbase/pull/1226#discussion_r386054314
 
 

 ##
 File path: src/site/xdoc/downloads.xml
 ##
 @@ -113,23 +113,23 @@ under the License.
 
 
   
-1.4.12
+1.4.13
   
   
-2019/11/29
+2020/02/29
   
   
-https://apache.org/dist/hbase/hbase-1.4.12/compat-check-report.html;>1.4.11
 vs 1.4.12
+https://apache.org/dist/hbase/1.4.13/compat-check-report.html;>1.4.12 vs 
1.4.13
 
 Review comment:
   did a release script choose this directory name? I want to get us 
consistently naming the main repo releases as "hbase-X.Y.Z".


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] virajjasani commented on issue #1222: HBASE-23868 : Replace usages of HColumnDescriptor(byte [] familyName)…

2020-02-29 Thread GitBox
virajjasani commented on issue #1222: HBASE-23868 : Replace usages of 
HColumnDescriptor(byte [] familyName)…
URL: https://github.com/apache/hbase/pull/1222#issuecomment-592984021
 
 
   @HorizonNet I have addressed comments. Could you please take a look?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] virajjasani merged pull request #1228: HBASE-22978 : Online slow response log

2020-02-29 Thread GitBox
virajjasani merged pull request #1228: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/1228
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] Apache-HBase commented on issue #1228: HBASE-22978 : Online slow response log

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1228: HBASE-22978 : Online slow response log
URL: https://github.com/apache/hbase/pull/1228#issuecomment-592973964
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 23s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +0 :ok: |  prototool  |   0m  1s |  prototool was not available.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
6 new or modified test files.  |
   ||| _ branch-2 Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 58s |  branch-2 passed  |
   | +1 :green_heart: |  compile  |   3m 21s |  branch-2 passed  |
   | +1 :green_heart: |  checkstyle  |   5m 19s |  branch-2 passed  |
   | +0 :ok: |  refguide  |  11m 48s |  branch has no errors when building the 
reference guide. See footer for rendered docs, which you should manually 
inspect.  |
   | +1 :green_heart: |  shadedjars  |   4m 43s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   5m 14s |  branch-2 passed  |
   | +0 :ok: |  spotbugs  |   1m 25s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |  23m  3s |  branch-2 passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |   3m 20s |  the patch passed  |
   | +1 :green_heart: |  cc  |   3m 20s |  the patch passed  |
   | +1 :green_heart: |  javac  |   3m 20s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 10s |  The patch passed checkstyle 
in hbase-protocol-shaded  |
   | +1 :green_heart: |  checkstyle  |   0m 24s |  The patch passed checkstyle 
in hbase-common  |
   | +1 :green_heart: |  checkstyle  |   0m 39s |  The patch passed checkstyle 
in hbase-client  |
   | +1 :green_heart: |  checkstyle  |   1m 17s |  hbase-server: The patch 
generated 0 new + 139 unchanged - 1 fixed = 139 total (was 140)  |
   | +1 :green_heart: |  checkstyle  |   0m 39s |  The patch passed checkstyle 
in hbase-thrift  |
   | +1 :green_heart: |  checkstyle  |   0m 10s |  The patch passed checkstyle 
in hbase-shell  |
   | +1 :green_heart: |  checkstyle  |   2m  1s |  root: The patch generated 0 
new + 569 unchanged - 1 fixed = 569 total (was 570)  |
   | +1 :green_heart: |  rubocop  |   0m 11s |  There were no new rubocop 
issues.  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  4s |  The patch has no ill-formed XML 
file.  |
   | +0 :ok: |  refguide  |   7m  7s |  patch has no errors when building the 
reference guide. See footer for rendered docs, which you should manually 
inspect.  |
   | +1 :green_heart: |  shadedjars  |   4m 42s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 31s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  hbaseprotoc  |   8m 47s |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   5m 25s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |  24m 25s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  | 147m 29s |  root in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   3m 26s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 303m 14s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1228/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1228 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc 
prototool rubocop |
   | uname | Linux cef71b6732a1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1228/out/precommit/personality/provided.sh
 |
   | git revision | branch-2 / 836e1a1caf |
   | Default Java | 1.8.0_181 |
   | refguide | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1228/1/artifact/out/branch-site/book.html
 |
   | refguide | 

[GitHub] [hbase] Apache-HBase commented on issue #1225: HBASE-23912 Resolve the TODO of FSTableDescriptor's construct method

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1225: HBASE-23912 Resolve the TODO of 
FSTableDescriptor's construct method
URL: https://github.com/apache/hbase/pull/1225#issuecomment-592967663
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 32s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
8 new or modified test files.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 37s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 19s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 24s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 30s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 43s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 57s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 22s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   5m  6s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m  4s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 24s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 24s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 12s |  hbase-server: The patch 
generated 0 new + 210 unchanged - 5 fixed = 210 total (was 215)  |
   | +1 :green_heart: |  checkstyle  |   0m 15s |  The patch passed checkstyle 
in hbase-backup  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 44s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  15m 55s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   5m 25s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |  64m 28s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  unit  |   7m 48s |  hbase-backup in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   1m  8s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 135m 29s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hbase.master.procedure.TestSCPWithMeta |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/4/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1225 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 4f79d3ef0ab4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1225/out/precommit/personality/provided.sh
 |
   | git revision | master / a420f0482e |
   | Default Java | 1.8.0_181 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/4/artifact/out/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/4/testReport/
 |
   | Max. process+thread count | 9620 (vs. ulimit of 1) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/4/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] Apache-HBase commented on issue #1227: HBASE-23914 : Set hbase.client.retries.number for TestThriftHBaseServ…

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1227: HBASE-23914 : Set 
hbase.client.retries.number for TestThriftHBaseServ…
URL: https://github.com/apache/hbase/pull/1227#issuecomment-592944418
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 31s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
1 new or modified test files.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 29s |  master passed  |
   | +1 :green_heart: |  compile  |   0m 35s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   0m 42s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 37s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  master passed  |
   | +0 :ok: |  spotbugs  |   1m 35s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   1m 34s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m  1s |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 34s |  the patch passed  |
   | +1 :green_heart: |  javac  |   0m 34s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 40s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 41s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 19s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   1m 55s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   2m 11s |  hbase-thrift in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 15s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  54m 26s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1227/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1227 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 8377718aa2ff 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 
16:55:30 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1227/out/precommit/personality/provided.sh
 |
   | git revision | master / a420f0482e |
   | Default Java | 1.8.0_181 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1227/1/testReport/
 |
   | Max. process+thread count | 3143 (vs. ulimit of 1) |
   | modules | C: hbase-thrift U: hbase-thrift |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1227/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Updated] (HBASE-23914) TestThriftHBaseServiceHandler.testMetricsWithException failing

2020-02-29 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-23914:
-
Status: Patch Available  (was: In Progress)

> TestThriftHBaseServiceHandler.testMetricsWithException failing
> --
>
> Key: HBASE-23914
> URL: https://issues.apache.org/jira/browse/HBASE-23914
> Project: HBase
>  Issue Type: Test
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> TestThriftHBaseServiceHandler.testMetricsWithException() is failing as test 
> due to timeout. We need to set hbase.client.retries.number



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work started] (HBASE-23914) TestThriftHBaseServiceHandler.testMetricsWithException failing

2020-02-29 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HBASE-23914 started by Viraj Jasani.

> TestThriftHBaseServiceHandler.testMetricsWithException failing
> --
>
> Key: HBASE-23914
> URL: https://issues.apache.org/jira/browse/HBASE-23914
> Project: HBase
>  Issue Type: Test
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> TestThriftHBaseServiceHandler.testMetricsWithException() is failing as test 
> due to timeout. We need to set hbase.client.retries.number



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [hbase] virajjasani opened a new pull request #1228: HBASE-22978 : Online slow response log

2020-02-29 Thread GitBox
virajjasani opened a new pull request #1228: HBASE-22978 : Online slow response 
log
URL: https://github.com/apache/hbase/pull/1228
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] virajjasani opened a new pull request #1227: HBASE-23914 : Set hbase.client.retries.number for TestThriftHBaseServ…

2020-02-29 Thread GitBox
virajjasani opened a new pull request #1227: HBASE-23914 : Set 
hbase.client.retries.number for TestThriftHBaseServ…
URL: https://github.com/apache/hbase/pull/1227
 
 
   …iceHandler


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Updated] (HBASE-23914) TestThriftHBaseServiceHandler.testMetricsWithException failing

2020-02-29 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-23914:
-
Fix Version/s: 2.3.0
   3.0.0

> TestThriftHBaseServiceHandler.testMetricsWithException failing
> --
>
> Key: HBASE-23914
> URL: https://issues.apache.org/jira/browse/HBASE-23914
> Project: HBase
>  Issue Type: Test
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> TestThriftHBaseServiceHandler.testMetricsWithException() is failing as test 
> due to timeout. We need to set hbase.client.retries.number



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HBASE-23914) TestThriftHBaseServiceHandler.testMetricsWithException failing

2020-02-29 Thread Viraj Jasani (Jira)
Viraj Jasani created HBASE-23914:


 Summary: TestThriftHBaseServiceHandler.testMetricsWithException 
failing
 Key: HBASE-23914
 URL: https://issues.apache.org/jira/browse/HBASE-23914
 Project: HBase
  Issue Type: Test
Reporter: Viraj Jasani
Assignee: Viraj Jasani


TestThriftHBaseServiceHandler.testMetricsWithException() is failing as test due 
to timeout. We need to set hbase.client.retries.number



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23899) [Flakey Test] Stabilizations and Debug

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048283#comment-17048283
 ] 

Hudson commented on HBASE-23899:


Results for branch master
[build #1647 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1647/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> [Flakey Test] Stabilizations and Debug
> --
>
> Key: HBASE-23899
> URL: https://issues.apache.org/jira/browse/HBASE-23899
> Project: HBase
>  Issue Type: Bug
>  Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Bunch of test stabilization and extra debug. This set of changes make it so 
> local 'test' runs pass about 20% of the time (where before they didn't) when 
> run on a linux vm and on a mac.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23904) Procedure updating meta and Master shutdown are incompatible: CODE-BUG

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048284#comment-17048284
 ] 

Hudson commented on HBASE-23904:


Results for branch master
[build #1647 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1647/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Procedure updating meta and Master shutdown are incompatible: CODE-BUG
> --
>
> Key: HBASE-23904
> URL: https://issues.apache.org/jira/browse/HBASE-23904
> Project: HBase
>  Issue Type: Bug
>  Components: amv2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Chasing flakies, studying TestMasterAbortWhileMergingTable, I noticed a 
> failure because
> {code:java}
> 2020-02-27 00:57:51,702 ERROR [PEWorker-6] 
> procedure2.ProcedureExecutor(1688): CODE-BUG: Uncaught runtime exception: 
> pid=14, state=RUNNABLE:MERGE_TABLE_REGIONS_UPDATE_META, locked=true; 
> MergeTableRegionsProcedure table=test, 
> regions=[48c9be922fa4356bfc7fc61b5b0785f3, ef196d5377c5c1d143e9a2a2ea056a9c], 
> force=false
> java.util.concurrent.RejectedExecutionException: Task 
> java.util.concurrent.FutureTask@28b956c7 rejected from 
> java.util.concurrent.ThreadPoolExecutor@639f20e5[Terminated, pool size = 0, 
> active threads = 0, queued tasks = 0, completed tasks = 5]
> at 
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
> at 
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at 
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at 
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
> at 
> org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:974)
> at 
> org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:953)
> at 
> org.apache.hadoop.hbase.MetaTableAccessor.multiMutate(MetaTableAccessor.java:1771)
> at 
> org.apache.hadoop.hbase.MetaTableAccessor.mergeRegions(MetaTableAccessor.java:1637)
> at 
> org.apache.hadoop.hbase.master.assignment.RegionStateStore.mergeRegions(RegionStateStore.java:268)
> at 
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.markRegionAsMerged(AssignmentManager.java:1854)
> at 
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.updateMetaForMergedRegions(MergeTableRegionsProcedure.java:687)
> at 
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:229)
> at 
> org.apache.hadoop.hbase.master.assignment.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:77)
> at 
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:194)
> at 
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1669)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1416)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:79)
> at 
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1986)
>  {code}
> A few seconds above, as part of the test, we'd stopped Master
> {code:java}
> 2020-02-27 00:57:51,620 INFO  [Time-limited test] 
> regionserver.HRegionServer(2212): * STOPPING region server 
> 'rn-hbased-lapp01.rno.exampl.com,36587,1582765058324' *
> 2020-02-27 00:57:51,620 INFO  [Time-limited test] 
> regionserver.HRegionServer(2226): STOPPED: Stopping master 0 {code}
> The rejected execution damages the merge procedure. It shows as an unhandled 
> CODE-BUG.
> Why we let a runtime exception out when trying to update meta is mildly 
> interesting. We 

[jira] [Commented] (HBASE-23892) SecureTestCluster should allow its subclasses to pass their Class reference on HBaseKerberosUtils.setSSLConfiguration

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048282#comment-17048282
 ] 

Hudson commented on HBASE-23892:


Results for branch master
[build #1647 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1647/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> SecureTestCluster should allow its subclasses to pass their Class reference 
> on HBaseKerberosUtils.setSSLConfiguration
> -
>
> Key: HBASE-23892
> URL: https://issues.apache.org/jira/browse/HBASE-23892
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 3.0.0, 2.3.0, 2.2.3
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4
>
>
> Was working on a third party project that relies on hbase-server as a 
> dependency and defines a UT class that extends *SecureTestCluster*. In this 
> project, hbase-server jar is added on the test classpath, and it relies on 
> *KeyStoreTestUtil* to decide where to place related ssl files. Current 
> *KeyStoreTestUtil* code assumes related class files would be under an 
> existing local FS path, but when those are loaded from a jar, related class 
> URI path returns null and causes an NPE that errors out the test execution. 
>  
> The test code in this case extends *SecureTestCluster*, so when 
> *SecureTestCluster.setup* runs, it calls 
> *HBaseKerberosUtils.setSSLConfiguration* passing its own class reference 
> [here|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/SecureTestCluster.java#L72].
> **
> *HBaseKerberosUtils.setSSLConfiguration*, in turn, calls 
> *KeyStoreTestUtil.getClasspathDir* 
> [here|https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/security/HBaseKerberosUtils.java#L173].
>  *KeyStoreTestUtil.getClasspathDir* uses the passed class reference name to 
> look for its related resource in the class loader. [This 
> line|https://github.com/apache/hbase/blob/master/hbase-http/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java#L58]
>  returns an URL value as follows, when the passed class reference resource 
> resides on jar file in the class loader (instead of being in a valid local FS 
> path):
> {{jar:file:/Users/wellingtonchevreuil/.m2/repository/org/apache/hbase/hbase-server/2.2.3./hbase-server-2.2.3-tests.jar!/org/apache/hadoop/hbase/security/token/SecureTestCluster.class}}
> This causes [subsequent 
> line|https://github.com/apache/hbase/blob/master/hbase-http/src/test/java/org/apache/hadoop/hbase/http/ssl/KeyStoreTestUtil.java#L59]
>  calling {{url.toURI().getPath()}} to return null, then we get the NPE thrown 
> in the following line, and the test errors out:
> {{java.lang.NullPointerException at 
> org.apache.hadoop.hbase.http.ssl.KeyStoreTestUtil.getClasspathDir(KeyStoreTestUtil.java:59)
>  at 
> org.apache.hadoop.hbase.security.HBaseKerberosUtils.setSSLConfiguration(HBaseKerberosUtils.java:173)
>  at 
> org.apache.hadoop.hbase.security.token.SecureTestCluster.setUp(SecureTestCluster.java:74)
>  at 
> com.cloudera.hbase.security.token.TestCldrManualDelegationToken.setUp(TestCldrManualDelegationToken.java:168)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) 
> at 
> 

[jira] [Commented] (HBASE-22978) Online slow response log

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048281#comment-17048281
 ] 

Hudson commented on HBASE-22978:


Results for branch master
[build #1647 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1647/]: (x) 
*{color:red}-1 overall{color}*

details (if available):

(x) {color:red}-1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//General_Nightly_Build_Report/]




(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) 
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1647//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(/) {color:green}+1 client integration test{color}


> Online slow response log
> 
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0, 2.3.0, 1.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.7.0
>
> Attachments: Screen Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot 
> 2019-10-19 at 2.32.54 AM.png, Screen Shot 2019-10-19 at 2.34.11 AM.png, 
> Screen Shot 2019-10-19 at 2.36.14 AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a 
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer - 
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value: 
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above 
> example. We do this because the human readable representation is verbose, the 
> rate of too slow warnings may be high, and the combination of these things 
> can overwhelm the log capture system. The truncation is unfortunate because 
> it eliminates much of the utility of the warnings. For example, the region 
> name, the start and end keys, and the filter hierarchy are all important 
> clues for debugging performance problems caused by moderate to low 
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be 
> too slow in addition to the responseTooSlow logging. The in-memory 
> representation can be complete and compressed. A new admin API and shell 
> command can provide access to the ring buffer for online performance 
> debugging. A modest sizing of the ring buffer will prevent excessive memory 
> utilization for a minor performance debugging feature by limiting the total 
> number of retained records. There is some chance a high rate of requests will 
> cause information on other interesting requests to be overwritten before it 
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted 
> in the request. We don't need to retain all key-values in the mutation, which 
> may be too large to comfortably retain. We only need a unique set of row 
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to 
> apply fast compression to ring buffer entries (if codec support is 
> available), something like snappy or zstandard, and decompress on the fly 
> when servicing the retrieval API request. This will minimize the impact of 
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same 
> information provided by responseTooSlow warnings. Total size of response 
> serialization, possibly also total cell or row counts, should be sufficient 
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more 
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls 
> behind and we are unable to persist an entry before it is 

[jira] [Commented] (HBASE-23066) Create a config that forces to cache blocks on compaction

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048274#comment-17048274
 ] 

Hudson commented on HBASE-23066:


Results for branch branch-1
[build #1248 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK7_Nightly_Build_Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Create a config that forces to cache blocks on compaction
> -
>
> Key: HBASE-23066
> URL: https://issues.apache.org/jira/browse/HBASE-23066
> Project: HBase
>  Issue Type: Improvement
>  Components: Compaction, regionserver
>Affects Versions: 1.4.10
>Reporter: Jacob LeBlanc
>Assignee: Jacob LeBlanc
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23066.patch, performance_results.png, 
> prefetchCompactedBlocksOnWrite.patch
>
>
> In cases where users care a lot about read performance for tables that are 
> small enough to fit into a cache (or the cache is large enough), 
> prefetchOnOpen can be enabled to make the entire table available in cache 
> after the initial region opening is completed. Any new data can also be 
> guaranteed to be in cache with the cacheBlocksOnWrite setting.
> However, the missing piece is when all blocks are evicted after a compaction. 
> We found very poor performance after compactions for tables under heavy read 
> load and a slower backing filesystem (S3). After a compaction the prefetching 
> threads need to compete with threads servicing read requests and get 
> constantly blocked as a result. 
> This is a proposal to introduce a new cache configuration option that would 
> cache blocks on write during compaction for any column family that has 
> prefetch enabled. This would virtually guarantee all blocks are kept in cache 
> after the initial prefetch on open is completed allowing for guaranteed 
> steady read performance despite a slow backing file system.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23631) Backport HBASE-23066, HBASE-23588, HBASE-23350 to branch-1

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048275#comment-17048275
 ] 

Hudson commented on HBASE-23631:


Results for branch branch-1
[build #1248 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK7_Nightly_Build_Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Backport HBASE-23066, HBASE-23588, HBASE-23350 to branch-1
> --
>
> Key: HBASE-23631
> URL: https://issues.apache.org/jira/browse/HBASE-23631
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.6.0
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.6.1
>
> Attachments: HBASE-23631.branch-1.000.patch
>
>
> Backport HBASE-23066 and all relevant subtasks to branch-1:
> HBASE-23066: Create a config that forces to cache blocks on compaction 
> HBASE-23588: Cache index blocks and bloom blocks on write if 
> CacheCompactedBlocksOnWrite is enabled
> HBASE-23350: Make compaction files cacheonWrite configurable based on 
> threshold



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HBASE-23588) Cache index blocks and bloom blocks on write if CacheCompactedBlocksOnWrite is enabled

2020-02-29 Thread Hudson (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048276#comment-17048276
 ] 

Hudson commented on HBASE-23588:


Results for branch branch-1
[build #1248 on 
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248/]: 
(x) *{color:red}-1 overall{color}*

details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//General_Nightly_Build_Report/]


(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK7_Nightly_Build_Report/]


(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) 
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/1248//JDK8_Nightly_Build_Report_(Hadoop2)/]




(x) {color:red}-1 source release artifact{color}
-- See build output for details.


> Cache index blocks and bloom blocks on write if CacheCompactedBlocksOnWrite 
> is enabled
> --
>
> Key: HBASE-23588
> URL: https://issues.apache.org/jira/browse/HBASE-23588
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-23588.master.addendum000.patch
>
>
> The existing behaviour even even cacheOnWrite is enabled is that we don cache 
> the index or bloom blocks. Now with HBASE-23066 in place we also write blocks 
> on compaction. So it may be better to cache the index/bloom blocks also if 
> cacheOnWrite is enabled?
> FYI [~javaman_chen]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [hbase] virajjasani commented on issue #1222: HBASE-23868 : Replace usages of HColumnDescriptor(byte [] familyName)…

2020-02-29 Thread GitBox
virajjasani commented on issue #1222: HBASE-23868 : Replace usages of 
HColumnDescriptor(byte [] familyName)…
URL: https://github.com/apache/hbase/pull/1222#issuecomment-592932803
 
 
   Test failures in the latest QA results are not relevant


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] Apache-HBase commented on issue #1222: HBASE-23868 : Replace usages of HColumnDescriptor(byte [] familyName)…

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1222: HBASE-23868 : Replace usages of 
HColumnDescriptor(byte [] familyName)…
URL: https://github.com/apache/hbase/pull/1222#issuecomment-592931171
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   0m 34s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  3s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
134 new or modified test files.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 42s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 22s |  master passed  |
   | +1 :green_heart: |  compile  |   4m 44s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   4m 34s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   4m 42s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   3m 20s |  master passed  |
   | +0 :ok: |  spotbugs  |   1m 30s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |  10m 37s |  master passed  |
   | -0 :warning: |  patch  |   2m 42s |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 16s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m  8s |  the patch passed  |
   | +1 :green_heart: |  compile  |   4m 53s |  the patch passed  |
   | +1 :green_heart: |  javac  |   4m 53s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   0m 32s |  hbase-client: The patch 
generated 0 new + 94 unchanged - 4 fixed = 94 total (was 98)  |
   | +1 :green_heart: |  checkstyle  |   1m 33s |  hbase-server: The patch 
generated 0 new + 929 unchanged - 44 fixed = 929 total (was 973)  |
   | +1 :green_heart: |  checkstyle  |   0m 22s |  hbase-mapreduce: The patch 
generated 0 new + 65 unchanged - 1 fixed = 65 total (was 66)  |
   | +1 :green_heart: |  checkstyle  |   0m 44s |  The patch passed checkstyle 
in hbase-thrift  |
   | +1 :green_heart: |  checkstyle  |   0m 15s |  The patch passed checkstyle 
in hbase-endpoint  |
   | +1 :green_heart: |  checkstyle  |   0m 16s |  The patch passed checkstyle 
in hbase-backup  |
   | +1 :green_heart: |  checkstyle  |   0m 22s |  hbase-it: The patch 
generated 0 new + 128 unchanged - 1 fixed = 128 total (was 129)  |
   | +1 :green_heart: |  checkstyle  |   0m 20s |  The patch passed checkstyle 
in hbase-rest  |
   | +1 :green_heart: |  checkstyle  |   0m 13s |  The patch passed checkstyle 
in hbase-shaded-client-project  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   4m 45s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  15m 57s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   3m 18s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |  11m 47s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   0m 56s |  hbase-client in the patch passed.  
|
   | -1 :x: |  unit  |  62m 12s |  hbase-server in the patch failed.  |
   | -1 :x: |  unit  |   6m 50s |  hbase-mapreduce in the patch failed.  |
   | -1 :x: |  unit  |  14m  1s |  hbase-thrift in the patch failed.  |
   | +1 :green_heart: |  unit  |   3m 26s |  hbase-endpoint in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   7m 34s |  hbase-backup in the patch passed.  
|
   | +1 :green_heart: |  unit  |   1m 12s |  hbase-it in the patch passed.  |
   | +1 :green_heart: |  unit  |   2m 46s |  hbase-rest in the patch passed.  |
   | +1 :green_heart: |  unit  |   1m  6s |  hbase-shaded-client-project in the 
patch passed.  |
   | +1 :green_heart: |  asflicense  |   5m  3s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 199m 48s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hbase.wal.TestWALSplit |
   |   | hadoop.hbase.mapreduce.TestTableInputFormatScanEmptyToEmpty |
   |   | hadoop.hbase.thrift2.TestThriftHBaseServiceHandler |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1222/4/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1222 |
   | Optional Tests | dupname asflicense javac javadoc 

[GitHub] [hbase] Apache-HBase commented on issue #1226: HBASE-23743 Add HBase 1.4.13 to the downloads page

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1226: HBASE-23743 Add HBase 1.4.13 to the 
downloads page
URL: https://github.com/apache/hbase/pull/1226#issuecomment-592930395
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 14s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   ||| _ master Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   6m 10s |  master passed  |
   | +1 :green_heart: |  mvnsite  |  17m 46s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   5m 37s |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |  17m 28s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  2s |  The patch has no ill-formed XML 
file.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 20s |  The patch does not generate 
ASF License warnings.  |
   |  |   |  49m 50s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1226/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1226 |
   | Optional Tests | dupname asflicense mvnsite xml |
   | uname | Linux 7940c28e99be 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1226/out/precommit/personality/provided.sh
 |
   | git revision | master / a420f0482e |
   | Max. process+thread count | 86 (vs. ulimit of 1) |
   | modules | C: . U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1226/1/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Commented] (HBASE-23743) Release 1.4.13

2020-02-29 Thread Sakthi (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048259#comment-17048259
 ] 

Sakthi commented on HBASE-23743:


Pending:
 * Update the Downloads page with link and date of the release 
([https://github.com/apache/hbase/pull/1226])
 * Announcement Email - Once the above pr goes in. 

Any reviews on the above pr, please?

> Release 1.4.13
> --
>
> Key: HBASE-23743
> URL: https://issues.apache.org/jira/browse/HBASE-23743
> Project: HBase
>  Issue Type: Task
>  Components: build, community
>Affects Versions: 1.4.13
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.4.13
>
>
> over due for 1.4.13. I'll try to use the release manager scripts this time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [hbase] jatsakthi opened a new pull request #1226: HBASE-23743 Add HBase 1.4.13 to the downloads page

2020-02-29 Thread GitBox
jatsakthi opened a new pull request #1226: HBASE-23743 Add HBase 1.4.13 to the 
downloads page
URL: https://github.com/apache/hbase/pull/1226
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [hbase] Apache-HBase commented on issue #1225: HBASE-23912 Resolve the TODO of FSTableDescriptor's construct method

2020-02-29 Thread GitBox
Apache-HBase commented on issue #1225: HBASE-23912 Resolve the TODO of 
FSTableDescriptor's construct method
URL: https://github.com/apache/hbase/pull/1225#issuecomment-592922194
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m  5s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any 
anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  The patch appears to include 
6 new or modified test files.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 35s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 54s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 26s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 35s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m  8s |  branch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   0m 54s |  master passed  |
   | +0 :ok: |  spotbugs  |   4m 53s |  Used deprecated FindBugs config; 
considering switching to SpotBugs.  |
   | +1 :green_heart: |  findbugs  |   5m 39s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 34s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 26s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 26s |  the patch passed  |
   | +1 :green_heart: |  checkstyle  |   1m 16s |  hbase-server: The patch 
generated 0 new + 189 unchanged - 5 fixed = 189 total (was 194)  |
   | +1 :green_heart: |  checkstyle  |   0m 14s |  The patch passed checkstyle 
in hbase-backup  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  shadedjars  |   5m  7s |  patch has no errors when 
building our shaded downstream artifacts.  |
   | +1 :green_heart: |  hadoopcheck  |  17m 32s |  Patch does not cause any 
errors with Hadoop 2.8.5 2.9.2 or 3.1.2.  |
   | +1 :green_heart: |  javadoc  |   0m 51s |  the patch passed  |
   | +1 :green_heart: |  findbugs  |   5m 58s |  the patch passed  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  |  91m 35s |  hbase-server in the patch failed.  |
   | +1 :green_heart: |  unit  |  10m 27s |  hbase-backup in the patch passed.  
|
   | +1 :green_heart: |  asflicense  |   0m 51s |  The patch does not generate 
ASF License warnings.  |
   |  |   | 170m 35s |   |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hbase.regionserver.wal.TestLogRollingNoCluster 
|
   |   | hadoop.hbase.regionserver.TestDefaultMemStore |
   |   | hadoop.hbase.regionserver.TestCompactingMemStore |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | Client=19.03.6 Server=19.03.6 base: 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/3/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hbase/pull/1225 |
   | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs 
shadedjars hadoopcheck hbaseanti checkstyle compile |
   | uname | Linux 265dd6839b32 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 
08:06:28 UTC 2019 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | 
/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1225/out/precommit/personality/provided.sh
 |
   | git revision | master / a420f0482e |
   | Default Java | 1.8.0_181 |
   | unit | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/3/artifact/out/patch-unit-hbase-server.txt
 |
   |  Test Results | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/3/testReport/
 |
   | Max. process+thread count | 6887 (vs. ulimit of 1) |
   | modules | C: hbase-server hbase-backup U: . |
   | Console output | 
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1225/3/console |
   | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[jira] [Commented] (HBASE-23743) Release 1.4.13

2020-02-29 Thread Sakthi (Jira)


[ 
https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048255#comment-17048255
 ] 

Sakthi commented on HBASE-23743:


Sure [~zhangduo], stable-1 pointer has been moved to 1.4.13.
1.4.12 has been removed from releases.

> Release 1.4.13
> --
>
> Key: HBASE-23743
> URL: https://issues.apache.org/jira/browse/HBASE-23743
> Project: HBase
>  Issue Type: Task
>  Components: build, community
>Affects Versions: 1.4.13
>Reporter: Sean Busbey
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.4.13
>
>
> over due for 1.4.13. I'll try to use the release manager scripts this time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)