JAVA_HOME on mac (regarding launch_container.sh)

I'm working on a MiniMRCluster-based unit tests, and I'm seeing that tests consistently fail because launch_container.sh fails with exit code 127 due to being unable to see JAVA_HOME in the script. I tracked this a little bit, and I am seeing that subprocesses that are launched via JDK's

Re: [VOTE] Release hadoop-2.0.3-alpha

I do see that the commons-daemon dependency is still at 1.0.3. This particular version has a busted group id (POM says org.apache.commons when the maven coordinates say commons-daemon), and it causes havoc with some builds when this dependency is included. Could you please update the

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

Hi Arun, Would it be possible to include HADOOP-9407 in the release? It's been resolved for a while, and it'd be good to have this dependency problem squared away... Thanks for the consideration! https://issues.apache.org/jira/browse/HADOOP-9407 Regards, Sangjin On Wed, Apr 10, 2013 at 2:19

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

+1 (non-binding) Thanks, Sangjin On Fri, May 24, 2013 at 8:48 PM, Konstantin Boudnik c...@apache.org wrote: All, I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I would like to release. This is a stabilization release that includes fixed for a couple a of issues

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

+1 (non-binding) Thanks Cos. Sangjin On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org wrote: All, I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I would like to release. This is a stabilization release that includes fixed for a couple a of

fair scheduler event log filling up?

We noticed that the fair scheduler event log directory was growing over time (this is with hadoop 2.x), and it seems that there is no configuration to limit the number of files there. The fair scheduler event logging is using DailyRollingFileAppender, and it seems like it does not support the max

branch development for HADOOP-9639

We have been having discussions on HADOOP-9639 (shared cache for jars) and the proposed design there for some time now. We are going to start work on this and have it vetted and reviewed by the community. I have just filed some more implementation JIRAs for this feature: YARN-1465, MAPREDUCE-5662,

Re: [VOTE] Release Apache Hadoop 2.3.0

If it's not too late, I'd like to bring your attention to YARN-1692. We get a ConcurrentModificationException with the fair scheduler when it updates demand, and we saw RM get into a pretty bad shape around the time this happened. It would be great if this could make 2.3.0. I provided a patch for

very old dependencies

Hi folks, Even as 2.3 was released, several dependencies of hadoop are quite dated. And more and more of them are causing friction for hadoop-related libraries and hadoop users in general, as these dependency versions are often incompatibly different than the versions most people use these days.

Re: very old dependencies

changing them could break jobs that rely on them. For jobs that want to use newer versions, the recommendation is to use mapreduce.user.classpath.first or turn on classpath isolation with mapreduce.job.classloader. -Sandy On Fri, Mar 28, 2014 at 7:59 AM, Sangjin Lee sjl...@gmail.com wrote: Hi

Re: hadoop-2.5 - June end?

It would be great if we can consider the shared cache (YARN-1492) as part of 2.5. It obviously will depend on the reviews from folks, but the feature itself should be reasonably solid. Thanks, Sangjin On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy a...@hortonworks.com wrote: Thanks Karthik!

Re: [VOTE] Migration from subversion to git for version control

+1 (non-binding) On Sat, Aug 9, 2014 at 9:34 AM, Sandy Ryza sandy.r...@cloudera.com wrote: +1 (binding) On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla ka...@cloudera.com wrote: I have put together this proposal based on recent discussion on this topic. Please vote on the

should clients be able to override *-site.xml?

tl;dr This was borne out of an investigation of a hadoop issue at our company. It appears that prior to https://issues.apache.org/jira/browse/HADOOP-9450 the hadoop configuration directory (HADOOP_CONF_DIR) was always the first entry in the classpath, regardless of HADOOP_USER_CLASSPATH_FIRST or

Re: should clients be able to override *-site.xml?

/ On Wed, Sep 10, 2014 at 10:55 AM, Allen Wittenauer a...@altiscale.com wrote: On Sep 10, 2014, at 10:14 AM, Sangjin Lee sj...@apache.org wrote: if HADOOP_USER_CLASSPATH_FIRST is set and the user provides his/her version of *-site.xml through HADOOP_CLASSPATH, the user would end up

Re: Thinking ahead to hadoop-2.6

I am also +1 for the end of next week. YARN-1492 (shared cache) is near the final stages of the review, and with a little more time it would make it easier to put it in. On Tue, Sep 23, 2014 at 4:48 PM, Subramaniam V K subru...@gmail.com wrote: +1 for end of next week. We have got all the

Re: Guava

FYI, we have an existing ApplicationClassLoader implementation that is used to isolate client/task classes from the rest. If we're going down the route of classloader isolation on this, it would be good to come up with a coherent strategy regarding both of these. As a more practical step, I like

Re: [DISCUSS] Developing features in branches

Ah, I missed that part (obviously). Fantastic! On Tue, Apr 28, 2015 at 5:31 PM, Sean Busbey bus...@cloudera.com wrote: On Apr 28, 2015 5:59 PM, Sangjin Lee sjl...@gmail.com wrote: That said, in a way we're deferring the cost of cleaning things up towards the end of the branch

Re: Hadoop YARN Birds of a Feather (BOF) Session at Hadoop Summit San Jose 2015

I also put up the timeline service next gen (YARN-2928) deck on slideshare: http://www.slideshare.net/sjlee0/timeline-service-next-gen-yarn2928-yarn-bof Comments are welcome. Thanks! On Mon, Jun 8, 2015 at 2:06 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: +1 Thanks +Vinod

Re: YARN-2928 branch reset

From: Naganarasimha G R (Naga) garlanaganarasi...@huawei.commailto: garlanaganarasi...@huawei.com Sent: Friday, August 14, 2015 3:59 PM To: Li Lu; Vinod Kumar Vavilapalli Cc: yarn-dev@hadoop.apache.orgmailto:yarn-dev@hadoop.apache.org; Zhijie Shen; Junping Du; Sangjin Lee; Vrushali

Re: Planning Hadoop 2.6.1 release

as Sangjin mentioned) Hi Vinod, could you mark the issues in 2) as 2.6.1-candidate? I'd like to freeze the candidate list in about 7 days and start backporting them. Do you have any thoughts? Regards, Akira On 7/25/15 10:32, Sangjin Lee wrote: Out of the JIRAs we proposed, please remove HDFS

Re: Planning Hadoop 2.6.1 release

at 10:07 AM, Sangjin Lee sj...@apache.org wrote: Thanks Akira. I'd like to make one small correction. If we're getting HDFS-7704, then we should also get HDFS-7916. My earlier comment was assuming HDFS-7704 was not included in the list. But if is (and I think it should), then we should also get

Re: Planning Hadoop 2.6.1 release

the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin

Re: Planning Hadoop 2.6.1 release

add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee

Re: Planning Hadoop 2.6.1 release

, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got

Re: Hadoop 2.6.1 Release process thread

and the dependent tickets. Thanks +Vinod On Aug 11, 2015, at 4:37 PM, Sangjin Lee sjl...@gmail.commailto: sjl...@gmail.com wrote: Could you double check HDFS-7916? HDFS-7916 is needed only if HDFS-7704 makes it. However, I see commits for HDFS-7916 in this list, but not for HDFS-7704

Re: Hadoop 2.6.1 Release process thread

Could you double check HDFS-7916? HDFS-7916 is needed only if HDFS-7704 makes it. However, I see commits for HDFS-7916 in this list, but not for HDFS-7704. If HDFS-7704 is not in the list, we should not backport HDFS-7916 as it fixes an issue introduced by HDFS-7704. On Tue, Aug 11, 2015 at 4:10

Re: Planning Hadoop 2.6.1 release

Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin On Wed, Jul 22, 2015 at 4:02 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: I’ve added them all to the 2.6.1-candidate list. I included everything even though some of them are

[VOTE] Release Apache Hadoop 2.6.2

Hi all, I have created a release candidate (RC0) for Hadoop 2.6.2. The RC is available at: http://people.apache.org/~sjlee/hadoop-2.6.2-RC0/ The RC tag in git is: release-2.6.2-RC0 The list of JIRAs committed for 2.6.2:

Re: [VOTE] Release Apache Hadoop 2.6.2

; > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > > > Also only PMC votes are binding on releases, so I think we currently > still > > stand at 0 binding +1s. > > > > On Mon, Oct 26, 2015 at 1:28 PM, Sangjin Lee <sjl...@gmail.com

[YARN-2928] first drop on trunk

In the past several months, the timeline service v.2 team has made a tremendous progress. We have a working storage implementation based on HBase, timeline collectors, timeline readers with some of the more important queries and filters, integration with RM and NM, distributed shell and mapreduce,

Re: [VOTE] Release Apache Hadoop 2.6.2

eating RC for 2.6.2. +1 (non-binding) > > - Verified signatures and checksums > - Deployed a single node cluster > - Built Tez 0.7.0 with Hadoop 2.6.2 pom > - Built Hive 1.2.1 with Hadoop 2.6.2 pom > - Ran some Hive on Tez queries successfully > > Regards, > Akira > &

[YARN-2928] working on rebase with trunk

I am working on rebasing the YARN-2928 branch (timeline service v.2) with trunk. So far it seems fairly straightforward, but I noted a few things that require some investigation and discussion. (1) TestDistributedShell failure (and timing out) It appears that after the rebase the

Re: [YARN-2928] working on rebase with trunk

I pushed the new branch (feature-YARN-2928). Please start using this branch from now on to see the timeline service v.2 work. If you're working on a patch, please use the new branch name in the file name. Thanks, Sangjin On Mon, Nov 9, 2015 at 4:53 PM, Sangjin Lee <sj...@apache.org>

Re: [YARN-2928] working on rebase with trunk

. Unless I hear objections soon, I'll push out that branch. The implication is that any patches we generate for YARN-2928 from now on would need to have the name such as "YARN-4053-feature-YARN-2928.001.patch". Sangjin On Mon, Nov 9, 2015 at 4:53 PM, Sangjin Lee <sj...@apache.org

Re: [DISCUSS] Looking to a 2.8.0 release

On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli wrote: > — YARN Timeline Service Next generation: YARN-2928: Lots of momentum, > but clearly a work in progress. Two options here > — If it is safe to ship it into 2.8 in a disable manner, we can > get the

Re: [YARN-2928] first drop on trunk

gt; +1 >> >> In favor of merging a version working end-to-end at alpha quality to >> trunk. >> >> On Mon, Nov 2, 2015 at 10:51 AM, Sangjin Lee <sj...@apache.org> wrote: >> >>> Just a reminder that I'd like input from folks that are working on this &

Re: [DISCUSS] Looking to a 2.8.0 release

challenging given the suggested timeframe. We also need to complete several major tasks before it is ready. Sangjin On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sjl...@gmail.com> wrote: > > On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli < > vino...@hortonworks.com> w

Re: [YARN-2928] working on rebase with trunk

arun Saxena <vsaxena.va...@gmail.com> > wrote: > > > Thanks a lot Sangjin ! > > > > Varun Saxena. > > > > On Thu, Nov 12, 2015 at 6:52 AM, Sangjin Lee <sj...@apache.org> wrote: > > > >> I pushed the new branch (feature-YARN-2928). Please start usi

Re: [YARN-2928] working on rebase with trunk

I will go ahead and complete the rebase and force push based on what I described above. It's likely you'll need to recreate your local YARN-2928 branch, and also rebase/adjust any private branch you may have had. FYI. Thanks! Sangjin On Mon, Nov 9, 2015 at 11:36 AM, Sangjin Lee <sj...@apache.

Re: [YARN-2928] working on rebase with trunk

__ > From: sjl...@gmail.com [sjl...@gmail.com] on behalf of Sangjin Lee [ > sj...@apache.org] > Sent: Tuesday, November 10, 2015 04:33 > To: yarn-dev@hadoop.apache.org > Subject: Re: [YARN-2928] working on rebase with trunk > > I will go ahead

Re: [YARN-2928] first drop on trunk

n Wed, Oct 28, 2015 at 10:30 AM, Varun Saxena <vsaxena.va...@gmail.com> wrote: > Hi Sangjin, > > Thanks for initiating the discussion. > I think YARN-3862 needs to go in as well. YARN-3862 and YARN-3863 are > somewhat related. > > Regards, > Varun Saxena. > > On

continuing releases on Apache Hadoop 2.6.x

As you may have seen, 2.6.2 is out . I have also retargeted all open issues that were targeted for 2.6.2 to 2.6.3. Continuing the discussion in the email thread here , I'd like us to maintain the cadence of

Re: [DISCUSS] About the details of JDK-8 support

+1 On Thu, Oct 15, 2015 at 7:57 AM, Steve Loughran wrote: > > > On 15 Oct 2015, at 14:42, Karthik Kambatla wrote: > > > > On Wed, Oct 14, 2015 at 4:28 PM, Allen Wittenauer > wrote: > > > >> > >> If people want, I could setup a

Re: Planning for Apache Hadoop 2.6.2

Another friendly reminder that I'll be cutting the branch and creating the RC soon. I'm targeting tomorrow. Thanks! Sangjin On Mon, Oct 12, 2015 at 11:40 AM, Sangjin Lee <sj...@apache.org> wrote: > Hi all, > > We are targeting next week to create the first RC for 2.6.2. Currently

Re: Planning for Apache Hadoop 2.6.2

If you have backported bugfixes to 2.7.2, please take a moment to consider if it is relevant (and important) for 2.6.x too, and if so backport it to branch-2.6. Thanks! On Tue, Oct 20, 2015 at 9:33 AM, Sangjin Lee <sj...@apache.org> wrote: > Another friendly reminder that I'll b

Re: Planning for Apache Hadoop 2.6.2

to review them and commit them to branch-2.6. Also, if you have JIRAs that are targeted to 2.6.2 and are close to being done, this might be a good time to complete them. Do let me know if you have any questions. Thanks, Sangjin On Sat, Sep 26, 2015 at 9:19 AM, Sangjin Lee <sj...@apache.org>

[YARN-2928] rebasing to trunk

Hi folks, I have completed the rebase of YARN-2928 (this time cherry-picks really) to the trunk as of last Saturday. I resolved 10 merge conflicts most of which were minor. But I do want to call out a few of them, and would like you to review how I resolved those conflicts before I make the

Re: [DISCUSS] About the details of JDK-8 support

Yes, at least for us, dropping the java 7 support (e.g. moving to java 8 source-wise) **at this point** would be an issue. I concur with the sentiment that we should preserve the java 7 support on branch-2 (not not move to java 8 source level) but can consider it for trunk. My 2 cents. Thanks,

Re: [YARN-2928] rebasing to trunk

. I’m not an expert in HTrace or > > HDFS, but so far the fix works on our side. We may need to revert our > > current fix after HDFS-9187 is officially done, though. For now I’m +1 on > > the temp fix in YARN-2928 branch. > > > > Li Lu > > > > On Oct 12, 201

Re: [YARN-2928] rebasing to trunk

I have forced pushed YARN-2928 with the latest. As before, since this is a force push, you'll want to recreate your local YARN-2928 branch, and recreate your private branch based off of YARN-2928. Let me know if you have issues. Allen, thanks for catching that issue. I filed YARN-4261 for that.

Re: [YARN-2928] rebasing to trunk

t looks good to me. I’m not an expert in HTrace or > HDFS, but so far the fix works on our side. We may need to revert our > current fix after HDFS-9187 is officially done, though. For now I’m +1 on > the temp fix in YARN-2928 branch. > > Li Lu > > On Oct 12, 2015, at 18:02,

Re: continuing releases on Apache Hadoop 2.6.x

kport, but if you do end up backporting patches after that, you > should set the fix-version to be 2.6.1. > > Thanks > +Vinod > > > > On Nov 2, 2015, at 11:29 AM, Sangjin Lee <sj...@apache.org> wrote: > > > > As you may have seen, 2.6.2 is out > > <http:

Re: [VOTE] Release Apache Hadoop 2.6.1 RC0

arly month long [1] toil, with loads of help from Sangjin Lee > and Akira Ajisaka, and 153 commits later, I've created a release candidate > RC0 for hadoop-2.6.1. > > The RC is available at: > http://people.apache.org/~vinodkv/hadoop-2.6.1-RC0/ > > The RC tag in git is: relea

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

MR jobs. > >>> > >>> Xuan Gong > >>> > >>>> On Sep 16, 2015, at 7:10 PM, Vinod Kumar Vavilapalli < > >> vino...@apache.org> wrote: > >>>> > >>>> Hi all, > >>>> > >>>>

Re: Planning for Apache Hadoop 2.6.2

, Sangjin Lee <sj...@apache.org> wrote: > Per Vinod's suggestion, in order to reduce the amount of movement I'll > pick commits from branch-2.6 onto the tip of branch-2.6.1 rather than the > other way around. This means I'll need to move branch-2.6 and force push > that change. &g

Re: Planning for Apache Hadoop 2.6.2

quired. >> >> Thanks, >> Vinay >> >> On Fri, Sep 25, 2015 at 12:16 AM, Vinod Vavilapalli < >> vino...@hortonworks.com >> >>> wrote: >>> >> >> +1. Please take it over, I’ll standby for any help needed. >>> >>>

Re: Planning for Apache Hadoop 2.6.2

done relocating the branch? I'll let you know when I'm done with that exercise. I expect I'll be done with this in 24 hours or so. Let me know if you have any concerns. Thanks, Sangjin On Fri, Sep 25, 2015 at 12:53 PM, Sangjin Lee <sj...@apache.org> wrote: > Thanks folks. I'll ge

Re: Planning for Apache Hadoop 2.6.2

Thanks Vinod for starting the discussion for 2.6.2! I'd like to volunteer as the release manager for 2.6.2 unless there is an objection. As I've worked with Vinod and Akira for 2.6.1 and triaged a number of issues, I feel I should be in a good position to take care of 2.6.2. Let me know your

Re: [Timeline V2 branch] Latest timeline v2 and SMP problem

Can you bisect the commits to see if you can isolate which commit introduced the issue? On Mon, Dec 14, 2015 at 5:39 PM, Li Lu wrote: > Hi YARN developers working on Timeline v2 (YARN-2928) branch, > > I just realized I’ve accidentally turned off SMP for my local Timeline

Re: [VOTE] Release Apache Hadoop 2.6.3 RC0

+1 (non-binding) - downloaded source and binary and verified the signatures (although I didn't connect with Junping via web of trust) - started a pseudo-distributed cluster and ran test jobs - browsed the RM and NN UI - looked through the daemon logs Thanks Junping. Sangjin On Wed, Dec 16,

Re: [YARN-2928] first drop on trunk

t is fine, can we have YARN-4343 too in the list ? >> >> + Naga >> >> From: Varun Saxena [vsaxena.va...@gmail.com] >> Sent: Saturday, November 14, 2015 01:46 >> To: Sangjin Lee >> Cc: Joep Rottinghuis; Karthik Kambatla; yarn-dev@ha

Re: continuing releases on Apache Hadoop 2.6.x

ating new fixes to 2.6.3 unless they are > critical enough as blocker. We can nominate more fixes later in 2.6.4. > Thoughts? > > Thanks, > > Junping > > From: sjl...@gmail.com <sjl...@gmail.com> on behalf of Sangjin Lee <

Re: continuing releases on Apache Hadoop 2.6.x

have a reasonable number of fixes if we start an RC early december. > > > > In the mean while, we should also review 2.7.3 and 2.8.0 blocker / > > critical list and see if it makes sense to backport any of those into > 2.6.3. > > > > +Vinod > > > > > >

Re: [DISCUSS] Increased use of feature branches

Having worked on a major feature in a feature branch, I have some thoughts and observations on feature branch development. IMO feature branch development v. direct commits to trunk in piecemeal is really a choice of *granularity*. Do we want a series of fine-grained state changes on trunk or

Re: [DISCUSS] Increased use of feature branches

ge with a 3-day voting period > instead of 7, or treat it just like today’s commit to trunk – but with 2 > people signing-off? > > What I am suggesting is reducing the administrative overheads of using a > branch to encourage use of branching. > Right now it feels like Apache’

Re: [DISCUSS] what exactly are the stability guarantees of the YARN APIs

I think there is more to it. The InterfaceStability javadoc states: Incompatible changes must not be made to classes marked as stable. And in practice, I don't think the class annotation can be considered a simple sum of method annotations. There is a notion of class compatibility distinct

Re: [DISCUSS] what exactly are the stability guarantees of the YARN APIs

Steve Loughran <ste...@hortonworks.com> > >wrote: > > > >> > >> On 31 May 2016, at 05:44, Karthik Kambatla <ka...@cloudera.com >> ka...@cloudera.com>> wrote: > >> > >> Inline. > >> > >> On Sat, May 28, 2016 at 11

Re: [DISCUSS] what exactly are the stability guarantees of the YARN APIs

On Mon, May 30, 2016 at 9:44 PM, Karthik Kambatla <ka...@cloudera.com> wrote: > Inline. > > On Sat, May 28, 2016 at 11:34 AM, Sangjin Lee <sjl...@gmail.com> wrote: > >> I think there is more to it. The InterfaceStability javadoc states: >> Incompatible

Re: [YARN-2928] working towards trunk merge

(YARN-2928) as well as the documentation (YARN-3150; still WIP). Thanks! Regards, Sangjin On Thu, Apr 28, 2016 at 1:53 PM, Sangjin Lee <sj...@apache.org> wrote: > Thanks Vinod. I'm working on a document that describes the timeline > service v.2: https://issues.apache.org/jira/browse/YARN-31

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

se we would not know where to read the data from, especially while > > fetching flow level information. > > > We can however, potentially change the Local File System based > > implementation to a HDFS based implementation and have it as an alternate > > for non-production us

[DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

Hi all, I’d like to open a discussion on merging the Timeline Service v.2 feature to trunk (YARN-2928 and MAPREDUCE-6331) [1][2]. We have been developing the feature in a feature branch (YARN-2928 [3]) for a while, and we are reasonably confident that the state of the feature meets the criteria

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

ches, include: Sangjin, Li, > Vrushali, > > Naga, Varun, Joep and Zhijie. > > > > Thanks, > > > > Junping > > > > 2016-06-23 13:32 GMT-07:00 Sangjin Lee <sj...@apache.org>: > >> > >> Thanks folks for the good discussion! > &

Re: [DISCUSS] merging YARN-2928 (Timeline Service v.2) to trunk

Yes of course. The discussion is solely about the merge to trunk. Thanks, Sangjin On Fri, Jun 24, 2016 at 10:37 AM, Li Lu <l...@hortonworks.com> wrote: > > On Jun 24, 2016, at 09:59, Sangjin Lee <sj...@apache.org sj...@apache.org>> wrote: > > Also for my und

Re: feature branch merge policy

t; correctly. > > > https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.apache.org%3E > > I never got around to adding this text to the website though, TODO. > > Best, > Andrew > > On Thu, Jun 16

Re: [UPDATE] New ASF git policy on force-pushes / Tags / Stale branches

ame the latest one > feature-YARN-2928 to YARN-2928 to follow our practice for branch > development. Thoughts? > > > Thanks, > > Junping > > From: sjl...@gmail.com <sjl...@gmail.com> on behalf of Sangjin Lee < > s

Re: [UPDATE] New ASF git policy on force-pushes / Tags / Stale branches

Thanks Vinod for the proposal. I deleted my branch (sjlee/hdfs-merge) the other day. Sangjin On Thu, Jan 14, 2016 at 1:26 PM, Vinod Kumar Vavilapalli wrote: > Hi all, > > As some of you have noticed, we have an update from ASF infra on git > branching policy: We no longer

Re: [VOTE] Release Apache Hadoop 2.6.4 RC0

+1 (not binding) - downloaded both the source and the binary tarballs, and checked the checksums and signatures - started a pseudo-distributed cluster and ran test MR jobs - spot checked the URI and the logs Sangjin On Mon, Feb 8, 2016 at 4:17 PM, Jason Lowe wrote:

[YARN-2928] timetable for the first merge to trunk

Last week the folks working on YARN-2928 (YARN Timeline Service v.2) had a discussion on the timing and the content of the first merge to trunk. As mentioned in this email thread previously [1], we are aiming to merge a complete end-to-end feature with a HBase backend storage and basic UI based on

Re: [YARN-2928] rebasing to trunk (again)

I have recreated branch YARN-2928 and pushed it to the repo. I also tagged the result of the rebase. Please use that branch (YARN-2928) going forward. Branch feature-YARN-2928 will be removed in the near future. Thanks, Sangjin On Wed, Jan 20, 2016 at 1:31 AM, Li Lu wrote:

Re: Looking to a Hadoop 3 release

Another thing to throw in there is the dependency/classpath isolation (HADOOP-11656). Some efforts have already been made by Sean, and it'd be great to complete this to have a much better dependency isolation solution for 3.x. On Thu, Feb 18, 2016 at 5:33 PM, Gangumalla, Uma

[YARN-2928] working towards trunk merge

This is an early heads-up that the friendly folks who are working on the YARN Timeline Service v.2 (YARN-2928) are nearing the code complete stage for a first merge to trunk. See an earlier email thread [1] for more details on this merge. This is an alpha-quality (preview) release so that people

Re: [YARN-2928] working towards trunk merge

> +1! > > Thanks to all the contributors involved in this significant piece of work! > > Shall we also post some of the docs / notes on how others can play with it? > > Thanks > +Vinod > > > > On Apr 27, 2016, at 5:44 PM, Sangjin Lee <sj...@apache.

Re: Getting consistent build failures for YARN-2928 branch on Jenkins Slave H8

The console output appears to indicate that it is trying to rebase YARN-2928 on top of the trunk? I wonder why H8 is doing that. On Tue, May 24, 2016 at 8:06 AM, Varun Saxena wrote: > Hi All, > > We are getting consistent build failures for YARN-2928 branch on Jenkins >

Re: [Release thread] 2.8.0 release activities

someone else can help with reviewing Common/HDFS > JIRAs. > > Thanks, > Wangda > > > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote: > > > Where do we stand in terms of closing out blocker/critical issues for > > 2.8.0? I still see 50 open J

Re: [Release thread] 2.8.0 release activities

Where do we stand in terms of closing out blocker/critical issues for 2.8.0? I still see 50 open JIRAs in Vinod's list: https://issues.apache.org/jira/issues/?filter=12334985 But I see a lot of JIRAs with no patches or very stale patches. It would be a good exercise to come up with the list of

Re: [Release thread] 2.8.0 release activities

; > Thanks, > Wangda > > > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote: > >> How about this? I'll review the HADOOP/HDFS bugs in that list to come up >> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll >&g

Re: Different JIRA permissions for HADOOP and HDFS

mitter list before and I > just add you into committer roles for 4 projects. Hope it works for > you now.​ > > > Thanks, > > > Junping > -- > *From:* sjl...@gmail.com <sjl...@gmail.com> on behalf of Sangjin Lee < > sj...@apache.or

Re: Different JIRA permissions for HADOOP and HDFS

I also find myself unable to edit most of the JIRA fields, and that is across projects (HADOOP, YARN, MAPREDUCE, and HDFS). Commenting and the workflow buttons seem to work, however. On Mon, May 16, 2016 at 8:14 AM, Zhihai Xu wrote: > Great, Thanks Junping! Yes, the JIRA

Re: [DISCUSS] Merge Jersey upgrade(HADOOP-9613) to trunk

We're also checking from the timeline service v.2 perspective (YARN-2928). On Fri, May 13, 2016 at 3:33 PM, Tsuyoshi Ozawa wrote: > Hi YARN developers, > > Could you check this thread? I think YARN web UI is affected by this > change. > > Thanks, > - Tsuyoshi > > --

Re: [DISCUSS] Merge Jersey upgrade(HADOOP-9613) to trunk

gt; > I'll wait for your reply until today - after raising this issue to ML, > we had a time longer than 1 week, I think it's time to merge. > > Thanks, > - Tsuyoshi > > On Sat, May 14, 2016 at 9:24 AM, Sangjin Lee <sjl...@gmail.com> wrote: > > We're also checking f

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

Thanks Tsuyoshi for opening the discussion. One benefit of the dependency/classpath isolation work is that it can open up a possibility of having diverging dependencies in a safe manner so that upgrading libraries may have less impact. I'll spend some more time on HADOOP-13070 to make some

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

The work on HADOOP-13070 and the ApplicationClassLoader are generic and go beyond YARN. It can be used in any JVM that uses hadoop. The current use cases are MR containers, hadoop's RunJar (as in "hadoop jar"), and the YARN node manager auxiliary services. I'm not sure if that's what you were

Re: Setting JIRA fix versions for 3.0.0 releases

On Thu, Jul 21, 2016 at 3:58 PM, Andrew Wang wrote: > Thanks for the input Vinod, inline: > > > > Similarly the list of features we are enabling in this alpha would be > good > > - may be update the Roadmap wiki. Things like classpath-isolation which > > were part of

Re: [DISCUSS] The order of classpath isolation work and updating/shading dependencies on trunk

sspath isolation doesn't really have an impact on that. > > > On Jul 22, 2016, at 9:57 AM, Sangjin Lee <sj...@apache.org> wrote: > > > > The work on HADOOP-13070 and the ApplicationClassLoader are generic and > go beyond YARN. It can be used in any JVM that uses hadoop. T

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

+1 (binding) - downloaded both source and binary tarballs and verified the signatures - set up a pseudo-distributed cluster - ran some simple mapreduce jobs - checked the basic web UI Sangjin On Wed, Jul 27, 2016 at 12:57 PM, John Zhuge wrote: > +1 (non-binding) > > -

Re: [Release thread] 2.6.5 release activities

Thanks folks for opening up the discussion on our EOL policy. That's also exactly what I wanted to discuss when I opened up the 2.6.5 discussion: I also want to gauge the community's interest in maintaining the 2.6.x > line. How long do we maintain this line? What would be a sensible EOL >

Re: [DISCUSS] YARN-5079 : Native YARN framework layer for services and Apache Slider

+1. It would be great if it becomes the default way for services to run on YARN. That would also make it easier to make YARN features support services better. As you mentioned, there would be a lot of logistics to work out, and we'll need to talk about them in more detail as they come up.

Re: [VOTE] merge feature branch YARN-2928

initely should be followed up on > post-merge. > > On 2016-07-01 09:00 (-0700), Sangjin Lee <s...@apache.org> wrote: > > Hi folks,> > > > > Per earlier discussion [1], I'd like to start a formal vote to merge> > > feature branch YARN-2928 [2] (Timeline S

Re: [VOTE] merge feature branch YARN-2928

FYI, the hadoop-aws pom has been fixed in YARN-5316. We're going to clean up the documentation more post-merge. Regards, Sangjin On Tue, Jul 5, 2016 at 1:15 PM, Sangjin Lee <sj...@apache.org> wrote: > Thanks for taking a look at it and providing your comments Allen. > > As for

Re: [VOTE] merge feature branch YARN-2928

been exciting working on this and am looking forward to the merge to > > trunk. > > > > On Fri, Jul 1, 2016 at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote: > > > > > Hi folks, > > > > > > Per earlier discussion [1], I'd like to start a formal vote to

  1   2   3   >