[jira] [Created] (HBASE-22471) Our nightly jobs for master and branch-2 are still using hadoop-2.7.1 in integration test

2019-05-25 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-22471:
-

 Summary: Our nightly jobs for master and branch-2 are still using 
hadoop-2.7.1 in integration test
 Key: HBASE-22471
 URL: https://issues.apache.org/jira/browse/HBASE-22471
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Duo Zhang


We use ls to get the hadoop 2 jars, so maybe the problem is that the 2.7.1 jars 
are already there for a long time. We need to clean the workspace.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Guanghao Zhang
The base patch didn't commit to branch-2.2? I found HBASE-21991 because it
was a reopened issue and fix version was 2.2.0. Then I helped to commit the
addendum to branch-2.2 and rolled 2.2.0RC4..

张铎(Duo Zhang)  于2019年5月26日周日 上午10:39写道:

> So we need to roll new RCs for both 2.1.5 and 2.2.0?
>
> Peter Somogyi 于2019年5月26日 周日05:46写道:
>
> > Apologies, I misinterpreted git log and JIRA previously. There is no need
> > for a new issue to move the LossyCounting class to IA.Private.
> >
> > What is needed:
> > branch-2.1: commit missing addendum and add 2.1.5 to fixed versions
> > branch-2.2: commit missing base patch
> >
> > On Sat, May 25, 2019 at 10:30 PM Andrew Purtell <
> andrew.purt...@gmail.com>
> > wrote:
> >
> > > We have decided this is not part of the public API so all that is
> needed
> > > is to change the annotation and post new RCs with that change with an
> > > update to release notes. It doesn’t matter if there was an incompatible
> > > change to the class made or not. A simple audience annotation mistake
> is
> > > taking disproportionate attention away from more important efforts.
> > >
> > > If an annotation change to one class is the only update in a new RC you
> > > can have confidence in porting your votes from the last RC to the new
> one
> > > after confirming sums and signatures. For your consideration.
> > >
> > > > On May 25, 2019, at 12:37 PM, Peter Somogyi
> > >  wrote:
> > > >
> > > > Unfortunately, that would require a new RC for 2.1.5 and I'm afraid
> > > > Guanghao already started the process for 2.0.0.
> > > >
> > > > On Sat, May 25, 2019 at 12:21 PM 张铎(Duo Zhang) <
> palomino...@gmail.com>
> > > > wrote:
> > > >
> > > >> A new issue can always solve the problem, I believe. I mean, revert
> > the
> > > >> addendum from branch-2.2-, and open a new issue, which just changes
> > the
> > > >> annotation for branch-2.2-, and commit the addendum again, with a
> new
> > > >> commit message.
> > > >>
> > > >> Peter Somogyi 于2019年5月25日 周六16:42写道:
> > > >>
> > > >>> On the 2.1.5RC0 testing I noticed that the release notes do not
> state
> > > >> that
> > > >>> LossyCounting was moved to IA.Private. This is tricky since the
> > > original
> > > >>> commit which introduced the incompatibility was not committed to
> > > >>> branch-2.1, only the addendum which only modified the IA
> annotation.
> > > For
> > > >>> 2.2.0 we have the same problem, however, 2.2.0 was added as fixed
> > > version
> > > >>> to HBASE-21991 even though only the addendum was committed to that
> > > branch
> > > >>> as well.
> > > >>>
> > > >>> Do we have any best practices for such a case?
> > > >>>
> > > >>> Thanks,
> > > >>> Peter
> > > >>>
> > > >>> On Fri, May 10, 2019 at 6:57 PM Sakthi  >
> > > >> wrote:
> > > >>>
> > >  Looks good to me.
> > > 
> > >  Sakthi
> > > 
> > >  On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) <
> > palomino...@gmail.com>
> > >  wrote:
> > > 
> > > > +1.
> > > >
> > > > Jan Hentschel  于2019年5月10日周五
> > > >>> 下午9:08写道:
> > > >
> > > >> Also +1 for making it IA.Private.
> > > >>
> > > >> From: Peter Somogyi 
> > > >> Reply-To: "dev@hbase.apache.org" 
> > > >> Date: Friday, May 10, 2019 at 1:41 PM
> > > >> To: "dev@hbase.apache.org" 
> > > >> Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
> > >  IA.Private
> > > >> in all maintenance branches
> > > >>
> > > >> +1 on moving LossyCounting to IA.Private
> > > >>
> > > >> On Fri, May 10, 2019 at 7:54 AM Stack  > > >> st...@duboce.net>> wrote:
> > > >>
> > > >> Looks good to me.
> > > >> S
> > > >>
> > > >> On Thu, May 9, 2019 at 7:02 PM Sean Busbey  > > >>>  > > >> bus...@apache.org>> wrote:
> > > >>
> > > >>> Hi folks!
> > > >>>
> > > >>> just a heads up that a few of us are planning to move a class
> out
> > > >>> of
> > > >>> the public API without a deprecation cycle.
> > > >>>
> > > >>> From the planned release note:
> > > >>>
> > >  The class LossyCounting was unintentionally marked Public but
> > > >> was
> > > > never
> > >  intended to be part of our public API. This oversight has been
> > > >> corrected
> > >  and LossyCounting is now marked as Private and going forward
> > > >> may
> > > >>> be
> > >  subject to additional breaking changes or removal without
> > > >> notice.
> > >  If
> > > >> you
> > >  have taken a dependency on this class we recommend cloning it
> > >  locally
> > > >>> into
> > >  your project before upgrading to this release.
> > > >>>
> > > >>> This class was came in via HBASE-19722 and was published in
> HBase
> > > >>> 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> > > >>> 2.2.0).
> > > >>>
> > > >>> It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5
> and
> > >  later
> > > >>> (maybe 2.2.1 depending on 

Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Duo Zhang
So we need to roll new RCs for both 2.1.5 and 2.2.0?

Peter Somogyi 于2019年5月26日 周日05:46写道:

> Apologies, I misinterpreted git log and JIRA previously. There is no need
> for a new issue to move the LossyCounting class to IA.Private.
>
> What is needed:
> branch-2.1: commit missing addendum and add 2.1.5 to fixed versions
> branch-2.2: commit missing base patch
>
> On Sat, May 25, 2019 at 10:30 PM Andrew Purtell 
> wrote:
>
> > We have decided this is not part of the public API so all that is needed
> > is to change the annotation and post new RCs with that change with an
> > update to release notes. It doesn’t matter if there was an incompatible
> > change to the class made or not. A simple audience annotation mistake is
> > taking disproportionate attention away from more important efforts.
> >
> > If an annotation change to one class is the only update in a new RC you
> > can have confidence in porting your votes from the last RC to the new one
> > after confirming sums and signatures. For your consideration.
> >
> > > On May 25, 2019, at 12:37 PM, Peter Somogyi
> >  wrote:
> > >
> > > Unfortunately, that would require a new RC for 2.1.5 and I'm afraid
> > > Guanghao already started the process for 2.0.0.
> > >
> > > On Sat, May 25, 2019 at 12:21 PM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > >> A new issue can always solve the problem, I believe. I mean, revert
> the
> > >> addendum from branch-2.2-, and open a new issue, which just changes
> the
> > >> annotation for branch-2.2-, and commit the addendum again, with a new
> > >> commit message.
> > >>
> > >> Peter Somogyi 于2019年5月25日 周六16:42写道:
> > >>
> > >>> On the 2.1.5RC0 testing I noticed that the release notes do not state
> > >> that
> > >>> LossyCounting was moved to IA.Private. This is tricky since the
> > original
> > >>> commit which introduced the incompatibility was not committed to
> > >>> branch-2.1, only the addendum which only modified the IA annotation.
> > For
> > >>> 2.2.0 we have the same problem, however, 2.2.0 was added as fixed
> > version
> > >>> to HBASE-21991 even though only the addendum was committed to that
> > branch
> > >>> as well.
> > >>>
> > >>> Do we have any best practices for such a case?
> > >>>
> > >>> Thanks,
> > >>> Peter
> > >>>
> > >>> On Fri, May 10, 2019 at 6:57 PM Sakthi 
> > >> wrote:
> > >>>
> >  Looks good to me.
> > 
> >  Sakthi
> > 
> >  On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) <
> palomino...@gmail.com>
> >  wrote:
> > 
> > > +1.
> > >
> > > Jan Hentschel  于2019年5月10日周五
> > >>> 下午9:08写道:
> > >
> > >> Also +1 for making it IA.Private.
> > >>
> > >> From: Peter Somogyi 
> > >> Reply-To: "dev@hbase.apache.org" 
> > >> Date: Friday, May 10, 2019 at 1:41 PM
> > >> To: "dev@hbase.apache.org" 
> > >> Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
> >  IA.Private
> > >> in all maintenance branches
> > >>
> > >> +1 on moving LossyCounting to IA.Private
> > >>
> > >> On Fri, May 10, 2019 at 7:54 AM Stack  > >> st...@duboce.net>> wrote:
> > >>
> > >> Looks good to me.
> > >> S
> > >>
> > >> On Thu, May 9, 2019 at 7:02 PM Sean Busbey  > >>>  > >> bus...@apache.org>> wrote:
> > >>
> > >>> Hi folks!
> > >>>
> > >>> just a heads up that a few of us are planning to move a class out
> > >>> of
> > >>> the public API without a deprecation cycle.
> > >>>
> > >>> From the planned release note:
> > >>>
> >  The class LossyCounting was unintentionally marked Public but
> > >> was
> > > never
> >  intended to be part of our public API. This oversight has been
> > >> corrected
> >  and LossyCounting is now marked as Private and going forward
> > >> may
> > >>> be
> >  subject to additional breaking changes or removal without
> > >> notice.
> >  If
> > >> you
> >  have taken a dependency on this class we recommend cloning it
> >  locally
> > >>> into
> >  your project before upgrading to this release.
> > >>>
> > >>> This class was came in via HBASE-19722 and was published in HBase
> > >>> 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> > >>> 2.2.0).
> > >>>
> > >>> It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
> >  later
> > >>> (maybe 2.2.1 depending on RC timing). The class already has
> > >>> backwards-incompatible changes set to happen in upcoming releases
> > >>> 1.4.10, 1.5.0, and 2.2.0.
> > >>>
> > >>> Please speak up sooner rather than later if you'll have a problem
> > >>> voting on RCs that include this change, either here or on
> >  HBASE-21991.
> > >>>
> > >>
> > >>
> > >>
> > >
> > 
> > >>>
> > >>
> >
>


[jira] [Resolved] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements

2019-05-25 Thread Peter Somogyi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi resolved HBASE-21991.
---
   Resolution: Fixed
 Hadoop Flags: Incompatible change
Fix Version/s: 2.1.5

Pushed addendum to branch-2.1 and main patch to branch-2.2.

FYI [~zghaobac], [~stack]

> Fix MetaMetrics issues - [Race condition, Faulty remove logic], few 
> improvements
> 
>
> Key: HBASE-21991
> URL: https://issues.apache.org/jira/browse/HBASE-21991
> Project: HBase
>  Issue Type: Bug
>  Components: Coprocessors, metrics
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.3.0, 2.1.5
>
> Attachments: hbase-21991.addendum.patch, 
> hbase-21991.branch-1.001.patch, hbase-21991.branch-1.002.patch, 
> hbase-21991.master.001.patch, hbase-21991.master.002.patch, 
> hbase-21991.master.003.patch, hbase-21991.master.004.patch, 
> hbase-21991.master.005.patch, hbase-21991.master.006.patch
>
>
> Here is a list of the issues related to the MetaMetrics implementation:
> +*Bugs*+:
>  # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*: 
> Under certain conditions, we might end up storing/exposing all the meters 
> rather than top-k-ish
>  # MetaMetrics can throw NPE resulting in aborting of the RS because of a 
> *Race Condition*.
> +*Improvements*+:
>  # With high number of regions in the cluster, exposure of metrics for each 
> region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of 
> regions. It's better to use *lossy counting to maintain top-k for region 
> metrics* as well.
>  # As the lossy meters do not represent actual counts, I think, it'll be 
> better to *rename the meters to include "lossy" in the name*. It would be 
> more informative while monitoring the metrics and there would be less 
> confusion regarding actual counts to lossy counts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor

2019-05-25 Thread Peter Somogyi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi resolved HBASE-21800.
---
   Resolution: Fixed
Fix Version/s: 2.1.4

Pushed to branch-2.2.

FYI [~zghaobac]

> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
>  Issue Type: Bug
>  Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
>  Labels: Meta
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.3.0, 2.1.4
>
> Attachments: hbase-21800.branch-1.001.patch, 
> hbase-21800.branch-1.002.patch, hbase-21800.branch-1.003.patch, 
> hbase-21800.branch-1.004.patch, hbase-21800.master.001.patch, 
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics 
> from MetaMetrics, when I bumped into this issue. Though currently we are not 
> capturing "Top K" table metrics, but we can encounter this issue because of 
> the "Top k Clients" that is implemented using the LossyAlgo.
>  
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The 
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR 
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw 
> java.lang.NullPointerException
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR 
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020] 
> regionserver.HRegionServer: * ABORTING region server 
> 10.0.0.24,16020,1548747043814: The coprocessor 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw 
> java.lang.NullPointerException *
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
>   at 
> 

[jira] [Reopened] (HBASE-21800) RegionServer aborted due to NPE from MetaTableMetrics coprocessor

2019-05-25 Thread Peter Somogyi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi reopened HBASE-21800:
---

Reopening for pushing missing commit to branch-2.2.

> RegionServer aborted due to NPE from MetaTableMetrics coprocessor
> -
>
> Key: HBASE-21800
> URL: https://issues.apache.org/jira/browse/HBASE-21800
> Project: HBase
>  Issue Type: Bug
>  Components: Coprocessors, meta, metrics, Operability
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Critical
>  Labels: Meta
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.3.0
>
> Attachments: hbase-21800.branch-1.001.patch, 
> hbase-21800.branch-1.002.patch, hbase-21800.branch-1.003.patch, 
> hbase-21800.branch-1.004.patch, hbase-21800.master.001.patch, 
> hbase-21800.master.002.patch, hbase-21800.master.003.patch
>
>
> I was just playing around the code, trying to capture "Top k" table metrics 
> from MetaMetrics, when I bumped into this issue. Though currently we are not 
> capturing "Top K" table metrics, but we can encounter this issue because of 
> the "Top k Clients" that is implemented using the LossyAlgo.
>  
> RegionServer gets aborted due to a NPE from MetaTableMetrics coprocessor. The 
> log looks somewhat like this:
> {code:java}
> 2019-01-28 23:31:10,311 ERROR 
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw 
> java.lang.NullPointerException
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> 2019-01-28 23:31:10,314 ERROR 
> [RpcServer.priority.FPBQ.Fifo.handler=19,queue=1,port=16020] 
> regionserver.HRegionServer: * ABORTING region server 
> 10.0.0.24,16020,1548747043814: The coprocessor 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics threw 
> java.lang.NullPointerException *
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.markMeterIfPresent(MetaTableMetrics.java:123)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.tableMetricRegisterAndMark2(MetaTableMetrics.java:233)
>   at 
> org.apache.hadoop.hbase.coprocessor.MetaTableMetrics$ExampleRegionObserverMeta.preGetOp(MetaTableMetrics.java:82)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:840)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:551)
>   at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:625)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:837)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2608)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2547)
>   at 
> 

[jira] [Reopened] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements

2019-05-25 Thread Peter Somogyi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi reopened HBASE-21991:
---

Reopening to push missing base patch to branch-2.2 and addendum to branch-2.1.

> Fix MetaMetrics issues - [Race condition, Faulty remove logic], few 
> improvements
> 
>
> Key: HBASE-21991
> URL: https://issues.apache.org/jira/browse/HBASE-21991
> Project: HBase
>  Issue Type: Bug
>  Components: Coprocessors, metrics
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.3.0
>
> Attachments: hbase-21991.addendum.patch, 
> hbase-21991.branch-1.001.patch, hbase-21991.branch-1.002.patch, 
> hbase-21991.master.001.patch, hbase-21991.master.002.patch, 
> hbase-21991.master.003.patch, hbase-21991.master.004.patch, 
> hbase-21991.master.005.patch, hbase-21991.master.006.patch
>
>
> Here is a list of the issues related to the MetaMetrics implementation:
> +*Bugs*+:
>  # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*: 
> Under certain conditions, we might end up storing/exposing all the meters 
> rather than top-k-ish
>  # MetaMetrics can throw NPE resulting in aborting of the RS because of a 
> *Race Condition*.
> +*Improvements*+:
>  # With high number of regions in the cluster, exposure of metrics for each 
> region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of 
> regions. It's better to use *lossy counting to maintain top-k for region 
> metrics* as well.
>  # As the lossy meters do not represent actual counts, I think, it'll be 
> better to *rename the meters to include "lossy" in the name*. It would be 
> more informative while monitoring the metrics and there would be less 
> confusion regarding actual counts to lossy counts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Peter Somogyi
Apologies, I misinterpreted git log and JIRA previously. There is no need
for a new issue to move the LossyCounting class to IA.Private.

What is needed:
branch-2.1: commit missing addendum and add 2.1.5 to fixed versions
branch-2.2: commit missing base patch

On Sat, May 25, 2019 at 10:30 PM Andrew Purtell 
wrote:

> We have decided this is not part of the public API so all that is needed
> is to change the annotation and post new RCs with that change with an
> update to release notes. It doesn’t matter if there was an incompatible
> change to the class made or not. A simple audience annotation mistake is
> taking disproportionate attention away from more important efforts.
>
> If an annotation change to one class is the only update in a new RC you
> can have confidence in porting your votes from the last RC to the new one
> after confirming sums and signatures. For your consideration.
>
> > On May 25, 2019, at 12:37 PM, Peter Somogyi
>  wrote:
> >
> > Unfortunately, that would require a new RC for 2.1.5 and I'm afraid
> > Guanghao already started the process for 2.0.0.
> >
> > On Sat, May 25, 2019 at 12:21 PM 张铎(Duo Zhang) 
> > wrote:
> >
> >> A new issue can always solve the problem, I believe. I mean, revert the
> >> addendum from branch-2.2-, and open a new issue, which just changes the
> >> annotation for branch-2.2-, and commit the addendum again, with a new
> >> commit message.
> >>
> >> Peter Somogyi 于2019年5月25日 周六16:42写道:
> >>
> >>> On the 2.1.5RC0 testing I noticed that the release notes do not state
> >> that
> >>> LossyCounting was moved to IA.Private. This is tricky since the
> original
> >>> commit which introduced the incompatibility was not committed to
> >>> branch-2.1, only the addendum which only modified the IA annotation.
> For
> >>> 2.2.0 we have the same problem, however, 2.2.0 was added as fixed
> version
> >>> to HBASE-21991 even though only the addendum was committed to that
> branch
> >>> as well.
> >>>
> >>> Do we have any best practices for such a case?
> >>>
> >>> Thanks,
> >>> Peter
> >>>
> >>> On Fri, May 10, 2019 at 6:57 PM Sakthi 
> >> wrote:
> >>>
>  Looks good to me.
> 
>  Sakthi
> 
>  On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) 
>  wrote:
> 
> > +1.
> >
> > Jan Hentschel  于2019年5月10日周五
> >>> 下午9:08写道:
> >
> >> Also +1 for making it IA.Private.
> >>
> >> From: Peter Somogyi 
> >> Reply-To: "dev@hbase.apache.org" 
> >> Date: Friday, May 10, 2019 at 1:41 PM
> >> To: "dev@hbase.apache.org" 
> >> Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
>  IA.Private
> >> in all maintenance branches
> >>
> >> +1 on moving LossyCounting to IA.Private
> >>
> >> On Fri, May 10, 2019 at 7:54 AM Stack  >> st...@duboce.net>> wrote:
> >>
> >> Looks good to me.
> >> S
> >>
> >> On Thu, May 9, 2019 at 7:02 PM Sean Busbey  >>>  >> bus...@apache.org>> wrote:
> >>
> >>> Hi folks!
> >>>
> >>> just a heads up that a few of us are planning to move a class out
> >>> of
> >>> the public API without a deprecation cycle.
> >>>
> >>> From the planned release note:
> >>>
>  The class LossyCounting was unintentionally marked Public but
> >> was
> > never
>  intended to be part of our public API. This oversight has been
> >> corrected
>  and LossyCounting is now marked as Private and going forward
> >> may
> >>> be
>  subject to additional breaking changes or removal without
> >> notice.
>  If
> >> you
>  have taken a dependency on this class we recommend cloning it
>  locally
> >>> into
>  your project before upgrading to this release.
> >>>
> >>> This class was came in via HBASE-19722 and was published in HBase
> >>> 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> >>> 2.2.0).
> >>>
> >>> It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
>  later
> >>> (maybe 2.2.1 depending on RC timing). The class already has
> >>> backwards-incompatible changes set to happen in upcoming releases
> >>> 1.4.10, 1.5.0, and 2.2.0.
> >>>
> >>> Please speak up sooner rather than later if you'll have a problem
> >>> voting on RCs that include this change, either here or on
>  HBASE-21991.
> >>>
> >>
> >>
> >>
> >
> 
> >>>
> >>
>


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Andrew Purtell
We have decided this is not part of the public API so all that is needed is to 
change the annotation and post new RCs with that change with an update to 
release notes. It doesn’t matter if there was an incompatible change to the 
class made or not. A simple audience annotation mistake is taking 
disproportionate attention away from more important efforts. 

If an annotation change to one class is the only update in a new RC you can 
have confidence in porting your votes from the last RC to the new one after 
confirming sums and signatures. For your consideration. 

> On May 25, 2019, at 12:37 PM, Peter Somogyi  
> wrote:
> 
> Unfortunately, that would require a new RC for 2.1.5 and I'm afraid
> Guanghao already started the process for 2.0.0.
> 
> On Sat, May 25, 2019 at 12:21 PM 张铎(Duo Zhang) 
> wrote:
> 
>> A new issue can always solve the problem, I believe. I mean, revert the
>> addendum from branch-2.2-, and open a new issue, which just changes the
>> annotation for branch-2.2-, and commit the addendum again, with a new
>> commit message.
>> 
>> Peter Somogyi 于2019年5月25日 周六16:42写道:
>> 
>>> On the 2.1.5RC0 testing I noticed that the release notes do not state
>> that
>>> LossyCounting was moved to IA.Private. This is tricky since the original
>>> commit which introduced the incompatibility was not committed to
>>> branch-2.1, only the addendum which only modified the IA annotation. For
>>> 2.2.0 we have the same problem, however, 2.2.0 was added as fixed version
>>> to HBASE-21991 even though only the addendum was committed to that branch
>>> as well.
>>> 
>>> Do we have any best practices for such a case?
>>> 
>>> Thanks,
>>> Peter
>>> 
>>> On Fri, May 10, 2019 at 6:57 PM Sakthi 
>> wrote:
>>> 
 Looks good to me.
 
 Sakthi
 
 On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) 
 wrote:
 
> +1.
> 
> Jan Hentschel  于2019年5月10日周五
>>> 下午9:08写道:
> 
>> Also +1 for making it IA.Private.
>> 
>> From: Peter Somogyi 
>> Reply-To: "dev@hbase.apache.org" 
>> Date: Friday, May 10, 2019 at 1:41 PM
>> To: "dev@hbase.apache.org" 
>> Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
 IA.Private
>> in all maintenance branches
>> 
>> +1 on moving LossyCounting to IA.Private
>> 
>> On Fri, May 10, 2019 at 7:54 AM Stack > st...@duboce.net>> wrote:
>> 
>> Looks good to me.
>> S
>> 
>> On Thu, May 9, 2019 at 7:02 PM Sean Busbey >> > bus...@apache.org>> wrote:
>> 
>>> Hi folks!
>>> 
>>> just a heads up that a few of us are planning to move a class out
>>> of
>>> the public API without a deprecation cycle.
>>> 
>>> From the planned release note:
>>> 
 The class LossyCounting was unintentionally marked Public but
>> was
> never
 intended to be part of our public API. This oversight has been
>> corrected
 and LossyCounting is now marked as Private and going forward
>> may
>>> be
 subject to additional breaking changes or removal without
>> notice.
 If
>> you
 have taken a dependency on this class we recommend cloning it
 locally
>>> into
 your project before upgrading to this release.
>>> 
>>> This class was came in via HBASE-19722 and was published in HBase
>>> 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
>>> 2.2.0).
>>> 
>>> It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
 later
>>> (maybe 2.2.1 depending on RC timing). The class already has
>>> backwards-incompatible changes set to happen in upcoming releases
>>> 1.4.10, 1.5.0, and 2.2.0.
>>> 
>>> Please speak up sooner rather than later if you'll have a problem
>>> voting on RCs that include this change, either here or on
 HBASE-21991.
>>> 
>> 
>> 
>> 
> 
 
>>> 
>> 


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Peter Somogyi
Unfortunately, that would require a new RC for 2.1.5 and I'm afraid
Guanghao already started the process for 2.0.0.

On Sat, May 25, 2019 at 12:21 PM 张铎(Duo Zhang) 
wrote:

> A new issue can always solve the problem, I believe. I mean, revert the
> addendum from branch-2.2-, and open a new issue, which just changes the
> annotation for branch-2.2-, and commit the addendum again, with a new
> commit message.
>
> Peter Somogyi 于2019年5月25日 周六16:42写道:
>
> > On the 2.1.5RC0 testing I noticed that the release notes do not state
> that
> > LossyCounting was moved to IA.Private. This is tricky since the original
> > commit which introduced the incompatibility was not committed to
> > branch-2.1, only the addendum which only modified the IA annotation. For
> > 2.2.0 we have the same problem, however, 2.2.0 was added as fixed version
> > to HBASE-21991 even though only the addendum was committed to that branch
> > as well.
> >
> > Do we have any best practices for such a case?
> >
> > Thanks,
> > Peter
> >
> > On Fri, May 10, 2019 at 6:57 PM Sakthi 
> wrote:
> >
> > > Looks good to me.
> > >
> > > Sakthi
> > >
> > > On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) 
> > > wrote:
> > >
> > > > +1.
> > > >
> > > > Jan Hentschel  于2019年5月10日周五
> > 下午9:08写道:
> > > >
> > > > > Also +1 for making it IA.Private.
> > > > >
> > > > > From: Peter Somogyi 
> > > > > Reply-To: "dev@hbase.apache.org" 
> > > > > Date: Friday, May 10, 2019 at 1:41 PM
> > > > > To: "dev@hbase.apache.org" 
> > > > > Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
> > > IA.Private
> > > > > in all maintenance branches
> > > > >
> > > > > +1 on moving LossyCounting to IA.Private
> > > > >
> > > > > On Fri, May 10, 2019 at 7:54 AM Stack  > > > > st...@duboce.net>> wrote:
> > > > >
> > > > > Looks good to me.
> > > > > S
> > > > >
> > > > > On Thu, May 9, 2019 at 7:02 PM Sean Busbey  >  > > > > bus...@apache.org>> wrote:
> > > > >
> > > > > > Hi folks!
> > > > > >
> > > > > > just a heads up that a few of us are planning to move a class out
> > of
> > > > > > the public API without a deprecation cycle.
> > > > > >
> > > > > > From the planned release note:
> > > > > >
> > > > > > > The class LossyCounting was unintentionally marked Public but
> was
> > > > never
> > > > > > > intended to be part of our public API. This oversight has been
> > > > > corrected
> > > > > > > and LossyCounting is now marked as Private and going forward
> may
> > be
> > > > > > > subject to additional breaking changes or removal without
> notice.
> > > If
> > > > > you
> > > > > > > have taken a dependency on this class we recommend cloning it
> > > locally
> > > > > > into
> > > > > > > your project before upgrading to this release.
> > > > > >
> > > > > > This class was came in via HBASE-19722 and was published in HBase
> > > > > > 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> > > > > > 2.2.0).
> > > > > >
> > > > > > It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
> > > later
> > > > > > (maybe 2.2.1 depending on RC timing). The class already has
> > > > > > backwards-incompatible changes set to happen in upcoming releases
> > > > > > 1.4.10, 1.5.0, and 2.2.0.
> > > > > >
> > > > > > Please speak up sooner rather than later if you'll have a problem
> > > > > > voting on RCs that include this change, either here or on
> > > HBASE-21991.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>


Re: Using libraries licensed under LGPL

2019-05-25 Thread Toshihiro Suzuki
Thank you for the comments, Stack and Josh.

It looks like the author agreed that they give us special permission to
use Lanterna under Apache 2.0:
https://github.com/mabe02/lanterna/issues/415

In this case, how to proceed? We need to make a note of this
(maybe in http://hbase.apache.org/license.html)? Or we can just use
the library without any notes?

Sorry to bother you, but I'm not familiar with license things.


On Fri, May 24, 2019 at 1:09 PM Josh Elser  wrote:

> The only exception to this rule is that if it's an "optional"
> dependency. Meaning, if you could use some other dependency in place of
> the LGPL one, it's OK. That does not apply in this situation :)
>
> Hopefully the author will agree! The demo of your tool on Tuesday was
> quite nice!!
>
> On 5/24/19 8:06 AM, Toshihiro Suzuki wrote:
> > Thank you for your reply, Duo and Nick.
> >
> > I understood that we can't use libraries licensed under LGPL at least in
> > HBase.
> > Before considering whether I create a new project, I will try to ask the
> > Lanterna
> > author to change the license of the library to Apache License or MIT.
> >
> > On Thu, May 23, 2019 at 11:57 AM Nick Dimiduk 
> wrote:
> >
> >> My understanding of [0] is that various versions of LGPL are explicitly
> >> prohibited from use as a dependency in Apache Foundation projects.
> >>
> >> I’m not a lawyer.
> >>
> >> [0]:
> >> https://apache.org/legal/resolved.html#category-x
> >>
> >> On Thu, May 23, 2019 at 7:18 AM 张铎(Duo Zhang) 
> >> wrote:
> >>
> >>> HBase itself can not depend on libraries which are licensed under
> LGPL. A
> >>> possible way is to create a new project, which depend on both HBase and
> >> the
> >>> LGPL library?
> >>>
> >>> Toshihiro Suzuki  于2019年5月23日周四 下午10:03写道:
> >>>
>  Hi folks!
> 
>  I'm building htop in HBASE-11062 now and using Lanterna library to
> >> make a
>  Unix top like user interface:
>  https://github.com/mabe02/lanterna
> 
>  Lanterna is a Java library allowing you to write easy semi-graphical
> >> user
>  interfaces in a text-only environment, very similar to the C library
>  curses.
> 
>  However, I found Lanterna library is licensed under the LGPL.
>  https://github.com/mabe02/lanterna/blob/master/License.txt
> 
>  According to the Apache website (
>  http://www.apache.org/legal/3party.html#criteriaandcategories), it
> >> looks
>  like LGPL License in incompatible with Apache License, but I'm not
> sure
> >>> if
>  we should not use libraries licensed under LGPL.
> 
>  Could anyone advise me on it?
> 
>  Regards,
>  Toshi
> 
> >>>
> >>
> >
>


Re: [ANNOUNCE] New HBase committer Yi Mei

2019-05-25 Thread Toshihiro Suzuki
Congratulations, Yi Mei!

On Sat, May 25, 2019 at 12:49 AM Nihal Jain  wrote:

> Congrats :)
>
> On Sat, 25 May, 2019, 12:36 PM 张铎(Duo Zhang), 
> wrote:
>
> > Congratulations!
> >
> > Xu Cang  于2019年5月25日周六 上午5:14写道:
> >
> > > Congratulations, Yi Mei!
> > >
> > >
> > >
> > > On Fri, May 24, 2019 at 1:44 PM Esteban Gutierrez
> > >  wrote:
> > >
> > > > Congratulations, Yi Mei!
> > > >
> > > > --
> > > > Cloudera, Inc.
> > > >
> > > >
> > > >
> > > > On Fri, May 24, 2019 at 2:12 AM Guanghao Zhang 
> > > wrote:
> > > >
> > > > > On behalf of the Apache HBase PMC, I am pleased to announce that Yi
> > Mei
> > > > has
> > > > > accepted the PMC's invitation to become a committer on the project.
> > We
> > > > > appreciate all of Yi Mei's generous contributions thus far and look
> > > > forward
> > > > > to Yi Mei's continued involvement.
> > > > >
> > > > > Congratulations and welcome, Yi Mei!
> > > > >
> > > >
> > >
> >
>


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Duo Zhang
A new issue can always solve the problem, I believe. I mean, revert the
addendum from branch-2.2-, and open a new issue, which just changes the
annotation for branch-2.2-, and commit the addendum again, with a new
commit message.

Peter Somogyi 于2019年5月25日 周六16:42写道:

> On the 2.1.5RC0 testing I noticed that the release notes do not state that
> LossyCounting was moved to IA.Private. This is tricky since the original
> commit which introduced the incompatibility was not committed to
> branch-2.1, only the addendum which only modified the IA annotation. For
> 2.2.0 we have the same problem, however, 2.2.0 was added as fixed version
> to HBASE-21991 even though only the addendum was committed to that branch
> as well.
>
> Do we have any best practices for such a case?
>
> Thanks,
> Peter
>
> On Fri, May 10, 2019 at 6:57 PM Sakthi  wrote:
>
> > Looks good to me.
> >
> > Sakthi
> >
> > On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) 
> > wrote:
> >
> > > +1.
> > >
> > > Jan Hentschel  于2019年5月10日周五
> 下午9:08写道:
> > >
> > > > Also +1 for making it IA.Private.
> > > >
> > > > From: Peter Somogyi 
> > > > Reply-To: "dev@hbase.apache.org" 
> > > > Date: Friday, May 10, 2019 at 1:41 PM
> > > > To: "dev@hbase.apache.org" 
> > > > Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
> > IA.Private
> > > > in all maintenance branches
> > > >
> > > > +1 on moving LossyCounting to IA.Private
> > > >
> > > > On Fri, May 10, 2019 at 7:54 AM Stack  > > > st...@duboce.net>> wrote:
> > > >
> > > > Looks good to me.
> > > > S
> > > >
> > > > On Thu, May 9, 2019 at 7:02 PM Sean Busbey   > > > bus...@apache.org>> wrote:
> > > >
> > > > > Hi folks!
> > > > >
> > > > > just a heads up that a few of us are planning to move a class out
> of
> > > > > the public API without a deprecation cycle.
> > > > >
> > > > > From the planned release note:
> > > > >
> > > > > > The class LossyCounting was unintentionally marked Public but was
> > > never
> > > > > > intended to be part of our public API. This oversight has been
> > > > corrected
> > > > > > and LossyCounting is now marked as Private and going forward may
> be
> > > > > > subject to additional breaking changes or removal without notice.
> > If
> > > > you
> > > > > > have taken a dependency on this class we recommend cloning it
> > locally
> > > > > into
> > > > > > your project before upgrading to this release.
> > > > >
> > > > > This class was came in via HBASE-19722 and was published in HBase
> > > > > 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> > > > > 2.2.0).
> > > > >
> > > > > It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
> > later
> > > > > (maybe 2.2.1 depending on RC timing). The class already has
> > > > > backwards-incompatible changes set to happen in upcoming releases
> > > > > 1.4.10, 1.5.0, and 2.2.0.
> > > > >
> > > > > Please speak up sooner rather than later if you'll have a problem
> > > > > voting on RCs that include this change, either here or on
> > HBASE-21991.
> > > > >
> > > >
> > > >
> > > >
> > >
> >
>


Re: [DISCUSS] Moving IA.Public class LossyCounting to IA.Private in all maintenance branches

2019-05-25 Thread Peter Somogyi
On the 2.1.5RC0 testing I noticed that the release notes do not state that
LossyCounting was moved to IA.Private. This is tricky since the original
commit which introduced the incompatibility was not committed to
branch-2.1, only the addendum which only modified the IA annotation. For
2.2.0 we have the same problem, however, 2.2.0 was added as fixed version
to HBASE-21991 even though only the addendum was committed to that branch
as well.

Do we have any best practices for such a case?

Thanks,
Peter

On Fri, May 10, 2019 at 6:57 PM Sakthi  wrote:

> Looks good to me.
>
> Sakthi
>
> On Fri, May 10, 2019 at 6:18 AM 张铎(Duo Zhang) 
> wrote:
>
> > +1.
> >
> > Jan Hentschel  于2019年5月10日周五 下午9:08写道:
> >
> > > Also +1 for making it IA.Private.
> > >
> > > From: Peter Somogyi 
> > > Reply-To: "dev@hbase.apache.org" 
> > > Date: Friday, May 10, 2019 at 1:41 PM
> > > To: "dev@hbase.apache.org" 
> > > Subject: Re: [DISCUSS] Moving IA.Public class LossyCounting to
> IA.Private
> > > in all maintenance branches
> > >
> > > +1 on moving LossyCounting to IA.Private
> > >
> > > On Fri, May 10, 2019 at 7:54 AM Stack  > > st...@duboce.net>> wrote:
> > >
> > > Looks good to me.
> > > S
> > >
> > > On Thu, May 9, 2019 at 7:02 PM Sean Busbey  > > bus...@apache.org>> wrote:
> > >
> > > > Hi folks!
> > > >
> > > > just a heads up that a few of us are planning to move a class out of
> > > > the public API without a deprecation cycle.
> > > >
> > > > From the planned release note:
> > > >
> > > > > The class LossyCounting was unintentionally marked Public but was
> > never
> > > > > intended to be part of our public API. This oversight has been
> > > corrected
> > > > > and LossyCounting is now marked as Private and going forward may be
> > > > > subject to additional breaking changes or removal without notice.
> If
> > > you
> > > > > have taken a dependency on this class we recommend cloning it
> locally
> > > > into
> > > > > your project before upgrading to this release.
> > > >
> > > > This class was came in via HBASE-19722 and was published in HBase
> > > > 1.4.6, 2.0.2, and 2.1.3 (and depending on RC timing might be in
> > > > 2.2.0).
> > > >
> > > > It will move to IA.Private as of 1.4.10, 1.5.0, 2.0.6, 2.1.5 and
> later
> > > > (maybe 2.2.1 depending on RC timing). The class already has
> > > > backwards-incompatible changes set to happen in upcoming releases
> > > > 1.4.10, 1.5.0, and 2.2.0.
> > > >
> > > > Please speak up sooner rather than later if you'll have a problem
> > > > voting on RCs that include this change, either here or on
> HBASE-21991.
> > > >
> > >
> > >
> > >
> >
>


[jira] [Created] (HBASE-22470) Corrupt Surefire test reports

2019-05-25 Thread Peter Somogyi (JIRA)
Peter Somogyi created HBASE-22470:
-

 Summary: Corrupt Surefire test reports
 Key: HBASE-22470
 URL: https://issues.apache.org/jira/browse/HBASE-22470
 Project: HBase
  Issue Type: Bug
  Components: test
Affects Versions: 3.0.0, 2.2.0, 2.1.5
Reporter: Peter Somogyi
Assignee: Peter Somogyi


Jenkins is not able to read surefire test reports occasionally because the 
generated XML file is corrupted. In this case Jenkins shows the following error 
message:

TEST-org.apache.hadoop.hbase.replication.TestMasterReplication.xml.[failed-to-read]

https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.1/1176/testReport/junit/TEST-org.apache.hadoop.hbase.replication.TestMasterReplication/xml/_failed_to_read_/

{noformat}
Failed to read test report file 
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_branch-2.1/output-jdk8-hadoop3/archiver/hbase-server/target/surefire-reports/TEST-org.apache.hadoop.hbase.replication.TestMasterReplication.xml
org.dom4j.DocumentException: Error on line 86 of document  : XML document 
structures must start and end within the same entity. Nested exception: XML 
document structures must start and end within the same entity.{noformat}
The specific XML file is not complete, however, the output file for the test 
contains stdout and stderr output.
{noformat}







java.lang.AssertionError: Waited too much time 
for bulkloaded data replication. Current count=200, expected count=600
at 
org.apache.hadoop.hbase.replication.TestMasterReplication.wait(TestMasterReplication.java:641)
at 
org.apache.hadoop.hbase.replication.TestMasterReplication.loadAndValidateHFileReplication(TestMasterReplication.java:631)
at 
org.apache.hadoop.hbase.replication.TestMasterReplication.testHFileMultiSlaveReplication(TestMasterReplication.java:371)



Re: [ANNOUNCE] New HBase committer Yi Mei

2019-05-25 Thread Nihal Jain
Congrats :)

On Sat, 25 May, 2019, 12:36 PM 张铎(Duo Zhang),  wrote:

> Congratulations!
>
> Xu Cang  于2019年5月25日周六 上午5:14写道:
>
> > Congratulations, Yi Mei!
> >
> >
> >
> > On Fri, May 24, 2019 at 1:44 PM Esteban Gutierrez
> >  wrote:
> >
> > > Congratulations, Yi Mei!
> > >
> > > --
> > > Cloudera, Inc.
> > >
> > >
> > >
> > > On Fri, May 24, 2019 at 2:12 AM Guanghao Zhang 
> > wrote:
> > >
> > > > On behalf of the Apache HBase PMC, I am pleased to announce that Yi
> Mei
> > > has
> > > > accepted the PMC's invitation to become a committer on the project.
> We
> > > > appreciate all of Yi Mei's generous contributions thus far and look
> > > forward
> > > > to Yi Mei's continued involvement.
> > > >
> > > > Congratulations and welcome, Yi Mei!
> > > >
> > >
> >
>


Re: [ANNOUNCE] New HBase committer Yi Mei

2019-05-25 Thread Duo Zhang
Congratulations!

Xu Cang  于2019年5月25日周六 上午5:14写道:

> Congratulations, Yi Mei!
>
>
>
> On Fri, May 24, 2019 at 1:44 PM Esteban Gutierrez
>  wrote:
>
> > Congratulations, Yi Mei!
> >
> > --
> > Cloudera, Inc.
> >
> >
> >
> > On Fri, May 24, 2019 at 2:12 AM Guanghao Zhang 
> wrote:
> >
> > > On behalf of the Apache HBase PMC, I am pleased to announce that Yi Mei
> > has
> > > accepted the PMC's invitation to become a committer on the project. We
> > > appreciate all of Yi Mei's generous contributions thus far and look
> > forward
> > > to Yi Mei's continued involvement.
> > >
> > > Congratulations and welcome, Yi Mei!
> > >
> >
>


[jira] [Resolved] (HBASE-22455) Split TestReplicationStatus

2019-05-25 Thread Duo Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-22455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-22455.
---
  Resolution: Fixed
Hadoop Flags: Reviewed

Pushed to master and branch-2.

Thanks [~openinx] for reviewing.

> Split TestReplicationStatus
> ---
>
> Key: HBASE-22455
> URL: https://issues.apache.org/jira/browse/HBASE-22455
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> The test is a bit strange, we restart the cluster every time when running a 
> test method, and even more, we always shutdown the mini clusters and then 
> restart them in the beginning of each test method...
> Let's just make one class for each test method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)