Re: upstream jenkins build broken?

You could rely on a destructive git clean call instead of maven to do the directory removal. -- Sean On Mar 11, 2015 4:11 PM, Colin McCabe cmcc...@alumni.cmu.edu wrote: Is there a maven plugin or setting we can use to simply remove directories that have no executable permissions on them?

Re: about CHANGES.txt

So long as you include the issue number, you can automate pulling the type from jira directly instead of putting it in the message. On Fri, Mar 13, 2015 at 4:49 PM, Yongjun Zhang yzh...@cloudera.com wrote: Hi, I found that changing CHANGES.txt when committing a jira is error prone because of

Re: Reviving HADOOP-7435: Making Jenkins pre-commit build work with branches

+1 If we can make things look like HBase support for precommit testing on branches (HBASE-12944), that would make it easier for new and occasional contributors who might end up working in other ecosystem projects. AFAICT, Jonathan's proposal for branch names in patch names does this. On Wed,

Re: about CHANGES.txt

/ReleaseNote.jspa?version=12327179styleName=HtmlprojectId=12310240 cheers, esteban. -- Cloudera, Inc. On Fri, Mar 13, 2015 at 3:01 PM, Sean Busbey bus...@cloudera.com wrote: So long as you include the issue number, you can automate pulling the type from jira directly instead

Re: committing HADOOP-11746 test-patch improvements

On Wed, Apr 22, 2015 at 2:10 AM, Allen Wittenauer a...@altiscale.com wrote: * There have been a few runs which seems to indicate that *something* is destroying the artifact directory in the middle of runs…. which is very very odd and something I hadn’t seen in any of my testing. In any

Re: Set minimum version of Hadoop 3 to JDK 8

A few options: * Only change the builds for master to use jdk8 * build with both jdk7 and jdk8 by copying jobs * build with both jdk7 and jdk8 using a jenkins matrix build Robert, if you'd like help with any of these please send me a ping off-list. On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar

Re: upstream jenkins build broken?

code base and a sufficient fledgling community, so I'm going to put together a tlp proposal. Thanks for the feedback thus far from use within Hadoop. I hope we can continue to make things more useful. -Sean On Wed, Mar 11, 2015 at 5:16 PM, Sean Busbey bus...@cloudera.com wrote: HBase's dev

[DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

Sorry for the resend. I figured this deserves a [DISCUSS] flag. On Sat, Jun 6, 2015 at 10:39 PM, Sean Busbey bus...@cloudera.com wrote: Hi Folks! After working on test-patch with other folks for the last few months, I think we've reached the point where we can make the fastest progress

Re: Where to put some code examples?

Could they go under dev-support? On Tue, Jun 23, 2015 at 4:29 PM, Ray Chiang rchi...@cloudera.com wrote: So, as far as I can see, Hadoop has the main developer area for core Hadoop code, unit tests in the test directories, user scripts (like hadoop/mapred/yarn), and build scripts. I've got

Re: [DISCUSS] More Maintenance Releases

On Tue, Jun 23, 2015 at 12:30 PM, Andrew Wang andrew.w...@cloudera.com wrote: There's no reason we have to choose 2.6 xor 2.7. If we have willing RMs and enough PMCs who will vote on releases, there's no reason we can't maintain both. However, based on the discussion at Hadoop Summit with

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

Hi Folks! Work in a feature branch is now being tracked by HADOOP-12111. On Thu, Jun 18, 2015 at 10:07 PM, Sean Busbey bus...@cloudera.com wrote: It looks like we have consensus. I'll start drafting up a proposal for the next board meeting (July 15th). Once we work out the name I'll submit

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

in a branch so that we can start prepping things now? We would want branch commit rights for the proposed new PMC. -Sean On Mon, Jun 15, 2015 at 6:47 PM, Sean Busbey bus...@cloudera.com wrote: Oof. I had meant to push on this again but life got in the way and now the June board meeting is upon us

Re: [DISCUSS] More Maintenance Releases

More maintenance releases would be excellent. If y'all are going to make more releases on the 2.6 line, please consider backporting HADOOP-11710 as without it HBase is unusable on top of HDFS encryption. It's been inconvenient that the fix is only available in a non-production release line.

Re: F 6/19: Jenkins clogged up

Thanks for hte heads up. On Fri, Jun 19, 2015 at 1:43 PM, Chris Nauroth cnaur...@hortonworks.com wrote: Hi everyone, I was just in contact with Apache infrastructure. Jenkins wasn't running jobs for a while, so there is a large backlog in the queue now (over 200 jobs). Infra has fixed the

Re: Pre-integration tests failing

On Tue, Jun 23, 2015 at 10:43 PM, Alan Burlison alan.burli...@oracle.com wrote: On 24/06/2015 04:22, Sean Busbey wrote: Probably not (barring maven attempting to grab SNAPSHOT versions of other modules while building). What are the machine specs like? the complete unit test set requires

Re: Where to put some code examples?

On Wed, Jun 24, 2015 at 2:10 AM, Ray Chiang rchi...@cloudera.com wrote: Thanks, dev-support sounds good. The only question I have is that there isn't a pom.xml there now. Is that something we'd want to have there? And should it at least be linked to the main build via some option, like

Re: Protocol Buffers version

Anyone have a read on how the protobuf folks would feel about that? Apache has a history of not accepting projects that are non-amicable forks. On Mon, Jun 15, 2015 at 9:24 AM, Allen Wittenauer a...@altiscale.com wrote: On Jun 12, 2015, at 1:03 PM, Alan Burlison alan.burli...@oracle.com

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

I'm going to try responding to several things at once here, so apologies if I miss anyone and sorry for the long email. :) On Tue, Jun 16, 2015 at 3:44 PM, Steve Loughran ste...@hortonworks.com wrote: I think it's good to have a general build/test process projects can share, so +1 to pulling

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

trying to solve. It's always amazing the problems that projects have that could be solved by these types of solutions. Let's stop hiding our cool toys in this area. So, what feedback and ideas do you have in this area? Are you a yay or a nay? On Jun 15, 2015, at 4:47 PM, Sean Busbey

Re: set up jenkins test for branch-2

On Mon, Jun 15, 2015 at 1:39 PM, Yongjun Zhang yzh...@cloudera.com wrote: Thanks Sean and Allen! I was not aware of that there is already a way to trigger branch-2 test. Good to know. There are multiple solutions here: 1. When posting patches, we can post two versions of patches, one for

Re: set up jenkins test for branch-2

pre-commit will already test on branch-2 provided you follow the patch naming guidelines. there is also a branch-2 specific jenkins job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-branch2/ I'd suggest starting by looking at that job and filing jiras to address whatever the

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

pmc, sqoop pmc, all around Jenkins expert) * Sean Busbey (ASF member, accumulo pmc, hbase pmc) * Nick Dimiduk (hbase pmc, phoenix pmc) * Chris Nauroth (ASF member, incubator pmc, hadoop pmc) * Andrew Purtell (ASF member, incubator pmc, bigtop pmc, hbase pmc, phoenix pmc) * Allen Wittenauer (hadoop

Re: JIRA admin question: Moving a bug from Hadoop to YARN

you might have to do things in two steps * move from HADOOP tracker to YARN tracker * convert from YARN jira to subtask of YARN-3719 On Wed, May 27, 2015 at 10:52 AM, Ted Yu yuzhih...@gmail.com wrote: When you click on More button, you should see an action called Move. You can move the

edit rights on the hadoop wiki

Hi! Can I get edit rights on the Hadoop wiki? -- Sean

Re: Does repository work correctly?

there appears to be an outage currently. https://issues.apache.org/jira/browse/INFRA-9934 On Mon, Jul 6, 2015 at 3:02 PM, Colin P. McCabe cmcc...@apache.org wrote: I am getting the same error now. Did we ever find the root cause of this problem? cmccabe@keter:~/hadoop2 git push Counting

Re: Planning Hadoop 2.6.1 release

If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened

Re: [DISCUSS] Additional maintenance releases for Hadoop 2.y versions

On Thu, Jul 16, 2015 at 9:17 AM, Karthik Kambatla ka...@cloudera.com wrote: On Thu, Jul 16, 2015 at 4:59 AM, Steve Loughran ste...@hortonworks.com wrote: -any change to the signature of an API, including exception types text -changes to wire formats These two should hold for minor

Re: [DISCUSS] Additional maintenance releases for Hadoop 2.y versions

, Sangjin On Tue, Jul 14, 2015 at 10:42 PM, Tsuyoshi Ozawa oz...@apache.org wrote: Thank you for the notification. Trying to back port bug fixes. - Tsuyoshi On Wed, Jul 15, 2015 at 3:45 AM, Sean Busbey bus...@cloudera.com wrote: Hi Hadoopers! Over in HBase we've been

Re: [DISCUSS] Additional maintenance releases for Hadoop 2.y versions

On Wed, Jul 15, 2015 at 3:45 AM, Sean Busbey bus...@cloudera.com wrote: Hi Hadoopers! Over in HBase we've been discussing the impact of our dependencies on our downstream users. As our most fundamental dependency, Hadoop plays a big role in the operational cost of running

Re: [Test-Patch TLP] consensus on naming

'); wrote: Hi Sean, +1 for Yetus since it sounds good name to me. Thanks - Tsuyoshi On Wed, Jul 8, 2015 at 2:42 PM, Sean Busbey bus...@cloudera.com wrote: Hi folks! It's almost time for the July board meeting, so we need to get the ball rolling on the proposal for a new TLP

[DISCUSS] Additional maintenance releases for Hadoop 2.y versions

Hi Hadoopers! Over in HBase we've been discussing the impact of our dependencies on our downstream users. As our most fundamental dependency, Hadoop plays a big role in the operational cost of running an HBase instance. Currently the HBase 1.y release line supports Hadoop 2.4, 2.5, and 2.6[1].

Re: Github integration for Hadoop

It looks like there's pretty good consensus. Why do we need a VOTE thread? Perhaps better for someone to submit a patch with proposed text for hte contribution guide[1]? -Sean [1]: http://wiki.apache.org/hadoop/HowToContribute On Thu, Oct 29, 2015 at 2:01 PM, Xiaoyu Yao

Re: Github integration for Hadoop

Hi Colin! If Yetus is working on an issue and can't tell what the intended branch is it points folks to project specific contribution guides. For Hadoop, the patch naming for specific branches should be covered in this section of Hadoop's contribution guide:

Re: Github integration for Hadoop

On Sun, Nov 1, 2015 at 12:52 PM, Allen Wittenauer wrote: > >> On Nov 1, 2015, at 6:05 AM, Tsuyoshi Ozawa wrote: >> >> Thank you for starting this discussion. It's good for us to rethink >> our workflow to grow community. >> However, at the moment, my concern

Re: Github integration for Hadoop

On Fri, Oct 30, 2015 at 1:22 PM, Allen Wittenauer wrote: > >> * Have we tried our precommit on PRs yet? Does it work for multiple >> branches? Is there a way to enforce rebase+squash vs. merge on the PR, >> since, per Allen, Yetus requires one commit to work? > > > I

Re: IMPORTANT: automatic changelog creation

On Jul 8, 2015 2:13 AM, Tsuyoshi Ozawa oz...@apache.org wrote: +1, thanks Allen and Andrew for taking lots effort! Is there any possibility that, we can restrict someone from editing the issue in jira once its marked as closed after release? Vinay's comment looks considerable for us to

Re: [DISCUSS] project for pre-commit patch testing (was Re: upstream jenkins build broken?)

As mentioned on HADOOP-12111, there is now an incubator-style proposal: http://wiki.apache.org/incubator/YetusProposal On Wed, Jun 24, 2015 at 9:41 AM, Sean Busbey bus...@cloudera.com wrote: Hi Folks! Work in a feature branch is now being tracked by HADOOP-12111. On Thu, Jun 18, 2015 at 10

[Test-Patch TLP] consensus on naming

Hi folks! It's almost time for the July board meeting, so we need to get the ball rolling on the proposal for a new TLP focused on QA[1]. Once issue outstanding from the original discussion is consensus on a name. We need to get some consensus together so that I can start verifying that the name

Re: [YETUS] let's start getting some releases out the door

just to provide an alternative, maybe one that needs a new thread, is there a reason that a version-info-plugin in Yetus is a better idea than trying to get it into the Maven project itself? On Fri, Aug 28, 2015 at 10:09 AM, Sean Busbey bus...@cloudera.com wrote: How about the Version-Info

Re: [YETUS] let's start getting some releases out the door

in other places (like if we add a maven plugin that runs releasedocmaker, or when building their api docs). On Thu, Aug 27, 2015 at 3:53 PM, Allen Wittenauer a...@altiscale.com wrote: On Aug 27, 2015, at 8:21 AM, Sean Busbey bus...@cloudera.com wrote: Allen, you've been chugging away

[YETUS] Yetus TLP approved

Hi Folks! At yesterday's ASF board meeting the Apache Yetus TLP was approved. There's still some ASF Infra work to get done[1] before we can start transitioning our mailing list, jira, and code over. Thanks to all the folks in Hadoop who've helped us along this process. I look forward to our

Re: [YETUS] Yetus TLP approved

The Apache Yetus dev list is now active: http://mail-archives.apache.org/mod_mbox/yetus-dev/ The first post there has a pointer to the rest of the project resources and the status of getting things set up. On Thu, Sep 17, 2015 at 10:59 AM, Sean Busbey <bus...@cloudera.com> wrote: >

Re: continuing releases on Apache Hadoop 2.6.x

Early december would be great, presuming the RC process doesn't take too long. By then it'll already have over a month since the 2.6.2 release and I'm sure the folks contributing the 18 patches we already have in would like to see their work out there. On Fri, Nov 20, 2015 at 7:51 AM, Junping Du

Re: hadoop-build-tools/src/main/resources/META-INF/

o include it. I'm > happy to try out more elegant solutions if you have any suggestions. > > Thanks! > > > -Xiao > > On Fri, Jun 17, 2016 at 7:34 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> If it's generated and we're following The Maven Way, it sh

Re: hadoop-build-tools/src/main/resources/META-INF/

If it's generated and we're following The Maven Way, it should be in target. probably in target/generated-sources On Fri, Jun 17, 2016 at 9:33 AM, Steve Loughran wrote: > > I see (presumably from the licensing work), that I'm now getting >

[DISCUSS] 2.7.3 release?

It's been about 2 months since 2.7.2 came out, and there are currently 135 issues in resolved status with a fixVersion of 2.7.3: * 32 Common * 52 HDFS * 33 YARN * 18 MAPREDUCE Could we move towards a release soon? -- busbey

Re: 2.7.3 release plan

On Wed, Apr 6, 2016 at 6:26 PM, Colin McCabe wrote: > In general, the only bundled native component I can see is lz4. I guess > debatably we should add tree.h to the NOTICE file as well, since it came > from BSD and is licensed under that license. > > Please keep in mind

Re: Branch policy question

VOTE threads tend to get more eyes than random JIRAs. On Tue, Mar 22, 2016 at 1:23 PM, Andrew Wang wrote: > A branch sounds fine, but how are we going to get 3 +1's to merge it? If > it's hard to find one reviewer, seems even harder to find two. > > On Tue, Mar 22,

Re: [DISCUSS] 2.7.3 release?

t; +Vinod > >> On Mar 29, 2016, at 9:18 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> It's been about 2 months since 2.7.2 came out, and there are currently >> 135 issues in resolved status with a fixVersion of 2.7.3: >> >> * 32 Common >>

Re: 2.7.3 release plan

. In addition, I didn't see any blocker issue to bring it into > 2.6.5 now. > Just my 2 cents. > > Thanks, > > Junping > > > From: Sean Busbey <bus...@cloudera.com> > Sent: Thursday, March 31, 2016 2:57 PM > To: h

Re: 2.7.3 release plan

A layout change in a maintenance release sounds very risky. I saw some discussion on the JIRA about those risks, but the consensus seemed to be "we'll leave it up to the 2.6 and 2.7 release managers." I thought we did RMs per release rather than per branch? No one claiming to be a release manager

Re: ASF OS X Build Infrastructure

Some talk about the MSDN-for-committers program recently passed by on a private list. It's still active, it just changed homes within Microsoft. The info should still be in the committer repo. If something is amiss please let me know and I'll pipe up to the folks already plugged in to confirming

Re: Compatibility guidelines for toString overrides

As a downstream user of Hadoop, it would be much clearer if the toString functions included the appropriate annotations to say they're non-public, evolving, or whatever. Most downstream users of Hadoop aren't going to remember in-detail exceptions to the java API compatibility rules, once they

Re: [VOTE] Merge feature branch HADOOP-12930

+1 (non-binding) reviewed everything, filed an additional subtask for a very trivial typo in the docs. should be fine to make a full issue after close and then fix. tried merging locally, tried running through new shell tests (both with and without bats installed), tried making an example custom

Re: Yes/No newbie question on contributing

The current HowToContribute guide expressly tells folks that they should ensure all the tests run and pass before and after their change. Sounds like we're due for an update if the expectation is now that folks should be using -DskipTests and runs on particular modules. Maybe we could instruct

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

thanks for bringing this up! big +1 on upgrading dependencies for 3.0. I have an updated patch for HADOOP-11804 ready to post this week. I've been updating HBase's master branch to try to make use of it, but could use some other reviews. On Thu, Jul 21, 2016 at 4:30 AM, Tsuyoshi Ozawa

Re: [DISCUSS] Official Docker Image at release time

Folks might want to take a look at https://issues.apache.org/jira/browse/HBASE-12721 and its associated review board posting of aggregate work. One of hte HBase community members has been creating infra (geared towards test ATM) for spinning up clusters of docker images. from what I understand,

Re: Setting JIRA fix versions for 3.0.0 releases

> Longer-term, I assume the 2.x line is not ending with 2.8. So we'd still > have the issue of things committed for 2.9.0 that will be appearing for the > first time in 3.0.0-alpha1. Assuming a script exists to fix up 2.9 JIRAs, > it's only incrementally more work to also fix up 2.8 and other

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

My work on HADOOP-11804 *only* helps processes that sit outside of YARN. :) On Fri, Jul 22, 2016 at 10:48 AM, Allen Wittenauer <a...@effectivemachines.com> wrote: > > Does any of this work actually help processes that sit outside of YARN? > >> On Jul 21, 2016, at 12:29

Re: Setting JIRA fix versions for 3.0.0 releases

On Thu, Jul 21, 2016 at 4:32 PM, Vinod Kumar Vavilapalli wrote: >> I really, really want a 3.0.0-alpha1 ASAP, since it's basically impossible >> for downstreams to test incompat changes and new features without a release >> artifact. I've been doing test builds, and

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

they've been puppetizing the Y! hosted machines, so it's probably related. Can someone point me to the exact jenkins job? I htink I fixed this for HBase once already. On Thu, Jul 21, 2016 at 3:31 PM, Allen Wittenauer wrote: > > I've already asked on builds and infra

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

, just a heads up). On Thu, Jul 21, 2016 at 4:54 PM, Akira Ajisaka <ajisa...@oss.nttdata.co.jp> wrote: > Failing jenkins job > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/ > > -Akira > > > On 7/21/16 14:43, Sean Busbey wrote: >> >> they've

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

u, Jul 21, 2016 at 6:49 PM, Sean Busbey <bus...@cloudera.com> wrote: > Okay, I have something hacked together as of this build: > > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/109 > > It's terrible, but it will get us through when infra uses puppet > a

Re: Setting JIRA fix versions for 3.0.0 releases

Yes, the Java API Compliance Checker allows specifying Annotations to pare down where incompatible changes happen. It was added some time ago based on feedback from the Apache HBase project. The limitations I've found are: 1) at least earlier versions only supported annotations at the class level

Re: Setting JIRA fix versions for 3.0.0 releases

, a >> guide about migration from 2.x to 3.x will be very helpful, and it can also >> help for people to better understand what have changed (Just like >> http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hado

Re: [DISCUSS] 2.6.x line releases

The HBase community would like more 2.6.z releases. On Wed, Jul 20, 2016 at 2:00 PM, Ravi Prakash wrote: > We for one are not using 2.6.* > > On Tue, Jul 19, 2016 at 1:21 PM, Sangjin Lee wrote: > >> It's been a while since we had a release on the 2.6.x

Re: [DISCUSS] Official Docker Image at release time

FWIW, there is a "blessed" apache area on docker hub now, and it's just an INFRA request to point out the needed Dockerfile in the repo. PMCs can also request write access to bintray hosting of docker images for PMC members. Info on INFRA-8441, example on INFRA-12019. A Docker image that starts

Re: Jenkins Node Labelling Documentation

On Thu, Aug 4, 2016 at 4:16 PM, Gav <gmcdon...@apache.org> wrote: > > > On Fri, Aug 5, 2016 at 3:14 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> > Why? yahoo-not-h2 is really not required since H2 is the same as all the >> > other H* nodes. >&

Where do we track our Jenkins stuff

? Or is there some particular mailing list thread or a wiki page or something? -- Sean Busbey

Re: Jenkins Node Labelling Documentation

> Why? yahoo-not-h2 is really not required since H2 is the same as all the > other H* nodes. The yahoo-not-h2 label exists because the H2 node was misconfigured for a long time and would fail builds as a result. What label will jobs taht are currently configured to avoid H2 be migrated to? Will

Re: Where do we track our Jenkins stuff

Hi folks! Any response here? The notice from infra about projects needing to update for Maven and Java had a three day migration period. I'm happy to just make something up, but I want to make sure it's useful for the other person or people who currently update build stuff. -- Sean Busbey

Re: Where do we track our Jenkins stuff

Okay, I'll get something together today. Thanks for the response Chris! -- Sean Busbey On Aug 6, 2016 10:48, "Chris Nauroth" <cnaur...@hortonworks.com> wrote: > Hello Sean, > > Thank you for bringing this up. The lack of response could indicate that > we’

Re: Jenkins Node Labelling Documentation

puppetized? On Thu, Aug 4, 2016 at 5:05 PM, Gav <gmcdon...@apache.org> wrote: > > > On Fri, Aug 5, 2016 at 7:52 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> On Thu, Aug 4, 2016 at 4:16 PM, Gav <gmcdon...@apache.org> wrote: >> > >> &

Re: HADOOP-13410

ServiceLoader API stuff won't load out of the unpacked version, right? On Tue, Aug 9, 2016 at 11:00 AM, Sangjin Lee wrote: > I'd like to get feedback from the community (especially those who might > remember this) on HADOOP-13410: >

Re: [DICUSS] Upgrading Guice to 4.0(HADOOP-12064)

At the very least, I'm running through an updated shaded hadoop client this week[1] (HBase is my test application and it wandered onto some private things that broke in branch-2). And Sangjin has a good lead on an lower-short-term-cost incremental improvement for runtime isolation of apps built on

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

It's also the key Andrew has in the project's KEYS file: http://www.apache.org/dist/hadoop/common/KEYS On Tue, Aug 30, 2016 at 4:12 PM, Andrew Wang wrote: > Hi Eric, thanks for trying this out, > > I tried this gpg command to get my key, seemed to work: > > # gpg

Re: Upgrading Hadoop dependencies and catching potential incompatibilities for HBase

On Wed, Sep 28, 2016 at 1:55 PM, Enis Söztutar wrote: > Can Hadoop please shade ALL of the dependencies (including PB) in Hadoop-3 > so that we do not have this mess going forward. > > Enis I'm working on this under HADOOP-11804, using HBase as my test application. --

[NOTICE] breaking precommit checks

Hi folks! a host of precommit checks are currently timing out due to an update to our job configs (the timeout is currently set to 50 minutes). I'm in the process of giving things more time based on our historic usage, but if your check fails in the mean time and 1) the total run time is close

Re: H9 build slave is bad

Is this HADOOP-13951? On Tue, Mar 7, 2017 at 8:32 PM, Andrew Wang wrote: > A little ping that H9 hit the same error again, and I'm again going to > clean it out. One more time and I'll ask infra about either removing or > reimaging this node. > > On Mon, Mar 6, 2017 at

Re: H9 build slave is bad

On Wed, Mar 8, 2017 at 2:04 PM, Allen Wittenauer <a...@effectivemachines.com> wrote: > >> On Mar 8, 2017, at 9:34 AM, Sean Busbey <bus...@cloudera.com> wrote: >> >> Is this HADOOP-13951? > > Almost certainly. Here's the run that broke it again: >

Re: H9 build slave is bad

All the precommit builds should be doing the correct thing now for making sure we don't render nodes useless. They don't flag the problem yet and someone will still need to run the "cleanup" job on nodes broken before jenkins runs pick up the new configuration changes. Probably best if we move to

Re: Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

disallowing force pushes to trunk was done back in: * August 2014: INFRA-8195 * February 2016: INFRA-11136 On Mon, Apr 17, 2017 at 11:18 AM, Jason Lowe wrote: > I found at least one commit that was dropped, MAPREDUCE-6673. I was able to > cherry-pick the original

Re: zstd compression

I know that the HBase community is also looking at what to do about our inclusion of zstd. We've had it in releases since late 2016. My plan was to request that they relicense it. Perhaps the Hadoop PMC could join HBase in the request? On Sun, Jul 16, 2017 at 8:11 PM, Allen Wittenauer

Re: Pre-Commit build is failing

-dev@yetus to bcc, since I think this is a Hadoop issue and not a yetus issue. Please review/commit HADOOP-14686 (which I am providing as a volunteer/contributor on the Hadoop project). On Tue, Jul 25, 2017 at 7:54 PM, Allen Wittenauer wrote: > > Again: just

Re: zstd compression

>> > below to make the user opt in, and if we are not shipping zstd with our >> products (source code releases), >> > then this is an acceptable usage. >> > >> > Cheers, >> > Chris >> >> >> So I think we are in the clear with respec

[DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

Moving discussion here from HADOOP-14654. Short synopsis: * HADOOP-14654 updated commons-httplient to a new patch release in hadoop-project * Precommit checked the modules that changed (i.e. not many) * nightly had Azure support break due to a change in behavior. Is this just the cost of our

Re: Trunk fails

On Wed, Sep 20, 2017 at 5:12 AM, Steve Loughran wrote: > > > > > What we could do is have a patch submission process which says "if you are > playing with packaging, you must declare at the time of patch submission > that you have run a full mvn clean install". And a

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On 2017-09-14 15:36, Chris Douglas wrote: > This has gotten bad enough that people are dismissing legitimate test > failures among the noise. > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > wrote: > > Someone should probably

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

t; On Sep 14, 2017, at 8:03 AM, Sean Busbey <bus...@cloudera.com> wrote: > > > > * HADOOP-14654 updated commons-httplient to a new patch release in > > hadoop-project > > * Precommit checked the modules that changed (i.e. not many) > > * nightly had Azure support

Re: [DISCUSS] Can we make our precommit test robust to dependency changes while staying usable?

On Thu, Sep 14, 2017 at 4:23 PM, Andrew Wang wrote: > > > > > I discussed this on yetus-dev a while back and Allen thought it'd be > non-trivial: > > https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c01084 >

[DISCUSS] moving to Apache Yetus Audience Annotations

When Apache Yetus formed, it started with several key pieces of Hadoop that looked reusable. In addition to our contribution testing infra, the project also stood up a version of our audience annotations for delineating the public facing API[1]. I recently got the Apache HBase community onto the

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

wrote: > Is this itself an incompatible change? I imagine the bytecode will be > different. > > I think we're too late to do this for beta1 given that I want to cut an > RC0 today. > > On Fri, Sep 22, 2017 at 7:03 AM, Sean Busbey <bus...@cloudera.com> wrote: > >

Re: Moving Java Forward Faster

ugh. this will be rough for cross-jdk compatibility, unless they update the target jre options of javac to support more than the last 2 major versions. > Question: Does GPL licensing of the JDK/JVM affect us negatively? Nope. all the openjdk bits we rely on were already going to be under the

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

Just curious, Junping what would "solid evidence" look like? Is the supposition here that the memory leak is within HDFS test code rather than library runtime code? How would such a distinction be shown? On Tue, Oct 24, 2017 at 4:06 PM, Junping Du wrote: > Allen, > Do

Re: [DISCUSS] Branch Proposal: HADOOP 15407: ABFS

apologies, copying back in common-dev@ with my question about the code. On Tue, May 15, 2018 at 2:36 PM, Sean Busbey <bus...@cloudera.com> wrote: > > Internal constraints prevented this feature from being developed in > Apache, so we want to ensure that all the code is discusse

Re: yetus build rejecting shaded artifacts ....

yep! I'll walk through how to find it, skip to "tl;dr:" if you just want the answer. Start with the "Console output" line in the footer of the QABot post Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/14777/console Search the output for "Checking client artifacts".

Re: site doc cleanup

IMHO dump the docs from the beta release as well. anyone on an alpha/beta release should move on to a GA release and beta1 should have been API frozen compared to GA. 3.1.0 was labeled "not ready for production" in its release notes[1]. Seems that means 3.0.3 is the stable3 release? Speaking

Re: Do we still have nightly (or even weekly) unit test run for Hadoop projects?

Here's the email from last night to common-dev@hadoop: https://s.apache.org/ARe1 On Wed, Oct 18, 2017 at 10:42 PM, Akira Ajisaka wrote: > Yes, qbt runs nightly and it sends e-mail to dev lists. > https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/ > > Regards,

Re: PreCommit-Admin fails to fetch issue list from jira

FYI Duo, I believe https://issues.apache.org/jira/browse/YETUS-594 is also slated to fix this. it just needs a review. On Tue, Jan 16, 2018 at 4:38 AM, Duo Zhang wrote: > Started from this build > > https://builds.apache.org/job/PreCommit-Admin/329113/ > > Have dug a bit,

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

I really, really like the approach of defaulting to only non-routeable IPs allowed. it seems like a good tradeoff for complexity of implementation, pain to reconfigure, and level of protection. On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon wrote: > The approach we took in Apache Kudu is that, if

  1   2   >