[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803003#comment-17803003 ] Shilun Fan commented on HADOOP-13916: - Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662922#comment-16662922 ] Sean Busbey commented on HADOOP-13916: -- moved down from Critical to Major to reflect current prioritization. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608704#comment-16608704 ] Sunil Govindan commented on HADOOP-13916: - Moved to 3.3.0 given there are no progress on this effort, kindy revert if you have different opinions. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366567#comment-16366567 ] Wangda Tan commented on HADOOP-13916: - Moved to 3.2.0 given lack of activities on this effort, please revert if you have different opinions. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16215523#comment-16215523 ] Andrew Wang commented on HADOOP-13916: -- Alright, thanks Sean. This isn't 100% absolutely critical, so I think we can wait until a 3.0.1 or later release if we have to. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213680#comment-16213680 ] Sean Busbey commented on HADOOP-13916: -- HBase has been a bit on fire lately due to a confluence of failures in our testing stuff, so I haven't been keeping this on mind. Let me check my schedule and see if this is doable. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16213190#comment-16213190 ] Andrew Wang commented on HADOOP-13916: -- Hi Sean, is this tracking for GA by the end of the month? > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151283#comment-16151283 ] Sean Busbey commented on HADOOP-13916: -- Thanks Chris! > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151276#comment-16151276 ] Chris Douglas commented on HADOOP-13916: OK. If it'd make sense for BigTop/Yetus to be a consumer or coordinating entity for such things, I'm sure we can work it out later. Thanks [~busbey]. The repo is [created|https://git1-us-west.apache.org/repos/asf?p=hadoop-downstream-tests.git]. You should have all the necessary privs. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150892#comment-16150892 ] Sean Busbey commented on HADOOP-13916: -- I'm not sure. Maybe worth a review of how different projects do this. My intuition says there won't be much overlap in project bits outside of poms since it's so dependent on the individual thing you're testing. But maybe BigTop would make sense, given that even the hbase one already is looking across HBase, Spark, and Hadoop. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[
https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150883#comment-16150883
]
Chris Douglas commented on HADOOP-13916:
Thanks for the pointer to Stack's repo; I get it, now. Requested
{{hadoop-downstream-tests}}.
Does this overlap with Bigtop or Yetus? I mean, if more projects create similar
repositories, we'll likely end up sharing (or at least copy/pasting) similar
machinery.
> Document how downstream clients should make use of the new shaded client
> artifacts
> --
>
> Key: HADOOP-13916
> URL: https://issues.apache.org/jira/browse/HADOOP-13916
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 3.0.0-alpha2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>
> provide a quickstart that walks through using the new shaded dependencies
> with Maven to create a simple downstream project.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[
https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150425#comment-16150425
]
Sean Busbey commented on HADOOP-13916:
--
{quote}
Skipping to the end of HADOOP-11656, the proposal is to publish shaded jars to
maven for downstream projects to build/test against. This repo will be
responsible for...?
{quote}
The repo provides a place where we can show use of the client facing modules,
similar to how Steve mentions the cloud integration stuff.
bq. Does this need to be a separate repo though?
Over in HBase it's been very useful for our similar downstream example to be in
its own repo. For one thing, it ensures that updates to it have attention
called to them. If it were in the main repo it'd be too easy for a dev that
introduced a breaking change to just also update the example at the same time.
Additionally, it has allowed us to recently start expressly show downstream
examples that successfully cross major versions.
https://github.com/saintstack/hbase-downstreamer/
Thus far, there's been no downside to that repo being just some github hosted
thing. I do, however, think it'd be better generally for these things to be in
repos under the control of an appropriate PMC.
bq. Do we need to branch it and release it with Hadoop versions?
One gap on the hbase downstream is that we haven't ever really done releases.
Here in Hadoop we could continue that tradition, but I'd much rather have
something we can refer back to later. That would mean having releases, though
I'd hope at a much lower rate than the main code.
bq. How does precommit work?
This is an excellent question. The most familiar course would be an additional
JIRA tracker and then a copy of the existing tracker specific precommit jobs
would work. That sounds like a fair bit of overhead.
bq. maybe "hadoop-downstream-tests", create one in github as a PoC
I'm happy to do this however the PMC prefers. Our goal is compilable code that
we can have snippets of in project documentation. That documentation should
live in the main repo along with the rest of our web facing stuff. Since the
educational value should be in the documentation that walks through things,
we'd hopefully mitigate the long term risk of e.g. that code coming from a
personal github repo that might cease to be.
> Document how downstream clients should make use of the new shaded client
> artifacts
> --
>
> Key: HADOOP-13916
> URL: https://issues.apache.org/jira/browse/HADOOP-13916
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 3.0.0-alpha2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>
> provide a quickstart that walks through using the new shaded dependencies
> with Maven to create a simple downstream project.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150217#comment-16150217 ] Steve Loughran commented on HADOOP-13916: - maybe "hadoop-downstream-tests", create one in github as a PoC FWIW, I do all my hadoop blobstore & spark integraiion tests in github as there was no home for it. I think Apache Bahir could be the home, but it's easiest right now to keep it public-but-nimble https://github.com/hortonworks-spark/cloud-integration > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149810#comment-16149810 ] Chris Douglas commented on HADOOP-13916: Sorry, I'm out of the loop on this one. [~busbey] can you add some context? Skipping to the end of HADOOP-11656, the proposal is to publish shaded jars to maven for downstream projects to build/test against. This repo will be responsible for...? > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149782#comment-16149782 ] Andrew Wang commented on HADOOP-13916: -- I tried this and it needs to be a PMC chair, not just a PMC member. cc [~chris.douglas]. Does this need to be a separate repo though? Do we need to branch it and release it with Hadoop versions? How does precommit work? We haven't done this before, and I'm worried that the overhead of setting up a new repo will take us past beta1. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[
https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145377#comment-16145377
]
Sean Busbey commented on HADOOP-13916:
--
I need a PMC member to request a new repo, {{hadoop-downstream}}, using the
[asf infra self service tool|https://selfserve.apache.org/git.html]. then I can
post a review for a commit to add stuff.
Here's my plan, someone ping if they want more:
* Generate a configurable number of test files written directly to HDFS (using
SequenceFile and/or Avro)
* Run a MapReduce job over said files to count number of entries
* Run a MapReduce job over said files to a simple aggregate and write results
to HDFS
> Document how downstream clients should make use of the new shaded client
> artifacts
> --
>
> Key: HADOOP-13916
> URL: https://issues.apache.org/jira/browse/HADOOP-13916
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Affects Versions: 3.0.0-alpha2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>
> provide a quickstart that walks through using the new shaded dependencies
> with Maven to create a simple downstream project.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765122#comment-15765122 ] Andrew Wang commented on HADOOP-13916: -- Does having it in the Hadoop repo have any beneficial side-effects, like acting as a compatibility check? Otherwise, what external repo would you recommend? I assume it should be ASF managed somehow, so we can maintain it with the normal protocols. > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13916) Document how downstream clients should make use of the new shaded client artifacts
[ https://issues.apache.org/jira/browse/HADOOP-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15761685#comment-15761685 ] Sean Busbey commented on HADOOP-13916: -- Set conservative target version. will see if we can get alpha2 still. As a part of this I'll presumably be making an actual buildable project for the example. Folks have any preference between doing that within the Hadoop repo or keeping it external (to better isolate changes to Hadoop from changes to it as a downstream consumer)? > Document how downstream clients should make use of the new shaded client > artifacts > -- > > Key: HADOOP-13916 > URL: https://issues.apache.org/jira/browse/HADOOP-13916 > Project: Hadoop Common > Issue Type: Improvement > Components: documentation >Affects Versions: 3.0.0-alpha2 >Reporter: Sean Busbey >Assignee: Sean Busbey > > provide a quickstart that walks through using the new shaded dependencies > with Maven to create a simple downstream project. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
