Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Stack
On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell wrote: > If we do this can we also move out hbck version 1? It would be really weird > in my opinion to have v2 in a separate repo but v1 shipping with the 1.x > releases. That would be a source of understandable confusion. > > My sense is that

Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Stack
On Tue, Jul 24, 2018 at 8:53 AM Josh Elser wrote: > (-cc user as this I'm getting purely into code development topics) > > First off, thanks for working on an hbck2, Umesh! > > I like the idea of having a separate repository for tracking HBCK and > the flexibility it gives us for making releases

Re: [DISCUSS] Kafka Connection, HBASE-15320

2018-07-25 Thread Stack
On Tue, Jul 24, 2018 at 10:01 PM Misty Linville wrote: > I like the idea of a separate connectors repo/release vehicle, but I'm a > little concerned about the need to release all together to update just one > of the connectors. How would that work? What kind of compatibility > guarantees are we

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Stack
On Wed, Jul 25, 2018 at 11:55 AM Josh Elser wrote: > ... > My biggest take-away is that I complicated this document by tying it too > closely with "HBase on Cloud", treating the WAL+Ratis LogService as the > only/biggest thing to figure out. This was inaccurate and overly bold of > me: I

[jira] [Created] (HBASE-20943) Add offline/online region count into metrics

2018-07-25 Thread Tianying Chang (JIRA)
Tianying Chang created HBASE-20943: -- Summary: Add offline/online region count into metrics Key: HBASE-20943 URL: https://issues.apache.org/jira/browse/HBASE-20943 Project: HBase Issue Type:

[jira] [Created] (HBASE-20942) Make RpcServer trace log length configurable

2018-07-25 Thread Mike Drob (JIRA)
Mike Drob created HBASE-20942: - Summary: Make RpcServer trace log length configurable Key: HBASE-20942 URL: https://issues.apache.org/jira/browse/HBASE-20942 Project: HBase Issue Type: Task

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Josh Elser
Thanks, Zach! I like your suggestion about project updates. I sincerely hope that this can be something transparent enough that folks who want to follow-on and participate in implementation can do so. Let me think about how to drive this better. On 7/25/18 3:55 PM, Zach York wrote: +1 to

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Josh Elser
Thanks, Andrew. I was really upset that I was butting heads with you when I would have previously thought that I had a design which was in line with something you would have called "good". I will wholly take the blame in not having an as-clear-as-possible design doc. I am way down in the

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Zach York
+1 to starting the work. I think most of the concerns can be figured out on the JIRAs and we can have a project update every X weeks if enough people are interested. I also agree to frame the feature correctly. Decoupling from a HDFS WAL or WAL on Ratis would be more appropriate names that would

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Andrew Purtell
> My biggest take-away is that I complicated this document by tying it too > closely with "HBase on Cloud", treating the WAL+Ratis LogService as the only/biggest thing to figure out. Understanding this now helps a lot to understand better the positions taken from the doc. At first glance it

[jira] [Created] (HBASE-20941) Cre

2018-07-25 Thread Umesh Agashe (JIRA)
Umesh Agashe created HBASE-20941: Summary: Cre Key: HBASE-20941 URL: https://issues.apache.org/jira/browse/HBASE-20941 Project: HBase Issue Type: Sub-task Reporter: Umesh Agashe

Re: [DISCUSS] Kafka Connection, HBASE-15320

2018-07-25 Thread Hbase Janitor
Hi Misty, As long as the connectors use a public API, we can be flexible. We get the same guarantees app programmers get. Mike On Wed, Jul 25, 2018, 01:01 Misty Linville wrote: > I like the idea of a separate connectors repo/release vehicle, but I'm a > little concerned about the need to

Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Umesh Agashe
bq. Seems like you're saying it's not a problem now, but you're not sure if it would become a problem. Regardless of that, it's a goal to not be version-specific (and thus, we can have generic hbck-v1 and hbck-v2 tools). LMK if I misread, please :) Thats right. On Wed, Jul 25, 2018 at 11:11 AM

Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Umesh Agashe
Thanks Josh! separate 'operator-tools' repo for hbase tools is a great suggestion. We can work towards it starting with hbck2. Each existing tool needs to be looked in detail regarding how much code it shares with HBase. On Wed, Jul 25, 2018 at 11:11 AM Josh Elser wrote: > Thanks, Umesh. Seems

Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc

2018-07-25 Thread Josh Elser
Let me give an update on-list for everyone: First and foremost, thank you very much to everyone who took the time to read this, with an extra thanks to those who participated in discussion. There were lots of great points raised. Some about things that were unclear in the doc, and others

Prep for release candidate 1.5.0 RC0

2018-07-25 Thread Andrew Purtell
I would like to put up the first release candidate for 1.5.0 by the end of August. To that end over the next couple of weeks I will be evaluating test stability, cluster stability under chaos testing, and performance differences (if any) with the latest 1.2, 1.3, and 1.4 releases as measured by

[VOTE] The first HBase 1.4.6 release candidate (RC0) is available

2018-07-25 Thread Andrew Purtell
The first HBase 1.4.6 release candidate (RC0) is available for download at https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.6RC0/ and Maven artifacts are available in the temporary repository https://repository.apache.org/content/repositories/orgapachehbase-1226/ . The git tag corresponding

[jira] [Reopened] (HBASE-20893) Data loss if splitting region while ServerCrashProcedure executing

2018-07-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-20893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-20893: --- Reopening to look at these logs I see running this patch on cluster (Its great it detected recovered.edits...

Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Andrew Purtell
Yes, and in that vein also VerifyReplication and tools of that nature. On Wed, Jul 25, 2018 at 11:11 AM Josh Elser wrote: > Thanks, Umesh. Seems like you're saying it's not a problem now, but > you're not sure if it would become a problem. Regardless of that, it's a > goal to not be

Re: [DISCUSS] Separate Git Repository for HBCK2

2018-07-25 Thread Josh Elser
Thanks, Umesh. Seems like you're saying it's not a problem now, but you're not sure if it would become a problem. Regardless of that, it's a goal to not be version-specific (and thus, we can have generic hbck-v1 and hbck-v2 tools). LMK if I misread, please :) One more thought, it would be

Re: HBase nightly job failing forever

2018-07-25 Thread Allen Wittenauer
> On Jul 25, 2018, at 10:48 AM, Chris Lambertus wrote: > > On-demand resources are certainly being considered (and we had these in the > past,) but I will point out that ephemeral (“on-demand”) cloud builds are in > direct opposition to some of the points brought up by Allen in the other >

Re: HBase nightly job failing forever

2018-07-25 Thread Chris Lambertus
> On Jul 25, 2018, at 10:34 AM, Andrew Purtell wrote: > > public clouds instead. I'm not sure if the ASF is set up to manage on > demand billing for test resources but this could be advantageous. It would > track actual usage not fixed costs. To avoid budget overrun there would be > caps and

Re: HBase nightly job failing forever

2018-07-25 Thread Andrew Purtell
Thanks Joan and Bertrand. > The number of failed builds in our stream that are directly related to this "tragedy of the commons" far exceeds the number of successful builds at this point, and unfortunately Travis CI is having parallel capacity issues that prevent us from moving to them wholesale

Re: HBase nightly job failing forever

2018-07-25 Thread Joan Touzet
I'll speak to CouchDB - the donation is directly in the form of a Jenkins build agent with our tag, no money is changed hands. The donator received a letter from fundraising@a.o allowing for tax deduction on the equivalent amount that the ASF leasing the machine would have cost for a year's

Re: HBase nightly job failing forever

2018-07-25 Thread Bertrand Delacretaz
Hi, On Wed, Jul 25, 2018 at 6:22 PM Andrew Purtell wrote: > ...How does a targeted hardware donation work? I was under the impression that > targeted donations are not accepted by the ASF This has changed, last year IIRC - there's a bit of information at

Re: HBase nightly job failing forever

2018-07-25 Thread Andrew Purtell
How does a targeted hardware donation work? I was under the impression that targeted donations are not accepted by the ASF. Maybe it is different in infrastructure, but this is the first time I've heard of it. Who does the donation on those projects? DataStax for Cassandra? Who for CouchDB? Google

[jira] [Resolved] (HBASE-20746) Release 2.1.0

2018-07-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-20746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-20746. --- Resolution: Fixed > Release 2.1.0 > - > > Key: HBASE-20746 >

Re: [DISCUSS] test-for-tests in precommit

2018-07-25 Thread Sean Busbey
circling back on this, be aware that precommit has been updated so that test-for-tests won't vote -1 on a contribution now. if the plugin can't find tests it'll give an advisory -0. On Fri, Jul 13, 2018 at 10:28 AM, Sean Busbey wrote: > Hi folks! > > Given how often we end up accepting

[jira] [Created] (HBASE-20940) HStore.cansplit should not be allow split to happen if it has references

2018-07-25 Thread Vishal Khandelwal (JIRA)
Vishal Khandelwal created HBASE-20940: - Summary: HStore.cansplit should not be allow split to happen if it has references Key: HBASE-20940 URL: https://issues.apache.org/jira/browse/HBASE-20940

[jira] [Created] (HBASE-20939) There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException

2018-07-25 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-20939: - Summary: There will be race when we call suspendIfNotReady and then throw ProcedureSuspendedException Key: HBASE-20939 URL: https://issues.apache.org/jira/browse/HBASE-20939

Re: HBase nightly job failing forever

2018-07-25 Thread Greg Stein
On Wed, Jul 25, 2018 at 2:36 AM Robert Munteanu wrote: > Hi, > > On Wed, 2018-07-25 at 09:27 +1000, Gav wrote: > > Disk space issues , yes, not on most of the Hadoop and related > > projects > > nodes - H0-H12 do not have disk space issues. As a Hadoop related > > project > > HBase should really

Re: HBase nightly job failing forever

2018-07-25 Thread Robert Munteanu
Hi, On Wed, 2018-07-25 at 09:27 +1000, Gav wrote: > Disk space issues , yes, not on most of the Hadoop and related > projects > nodes - H0-H12 do not have disk space issues. As a Hadoop related > project > HBase should really be concentrating its builds there. A suggestion from the sidelines. We

[jira] [Created] (HBASE-20938) Set version to 2.1.1-SNAPSHOT for branch-2.1

2018-07-25 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-20938: - Summary: Set version to 2.1.1-SNAPSHOT for branch-2.1 Key: HBASE-20938 URL: https://issues.apache.org/jira/browse/HBASE-20938 Project: HBase Issue Type: Sub-task

[jira] [Reopened] (HBASE-20746) Release 2.1.0

2018-07-25 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-20746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-20746: --- > Release 2.1.0 > - > > Key: HBASE-20746 > URL:

[jira] [Created] (HBASE-20937) Update the support matrix in our ref guide about the recent hadoop releases

2018-07-25 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-20937: - Summary: Update the support matrix in our ref guide about the recent hadoop releases Key: HBASE-20937 URL: https://issues.apache.org/jira/browse/HBASE-20937 Project: HBase