Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

2021-07-15 Thread epa...@apache.org
> I usually use PR comments to discuss about the patch submitted. My concern is that still leaves multiple places to look in order to get a full picture of an issue. -Eric On Wednesday, July 14, 2021, 7:07:30 PM CDT, Masatake Iwasaki wrote: > - recently, JIRA became some sort of a

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

2021-07-14 Thread epa...@apache.org
+1 for a review of the backlog!-Eric On Wednesday, July 14, 2021, 10:02:39 AM CDT, Wei-Chiu Chuang wrote: We have more than 400 open PRs. I would be happy to find a way to reduce that number to a more manageable size. Otherwise it just becomes another JIRA where issues are filed and

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

2021-07-14 Thread epa...@apache.org
Ahmed has pinpointed my main concern with both JIRA and PR for the same issue.I often go back to older issues to try to understand the reasons behind the designs.It is somewhat cumbersome to try and follow discussions between JIRA and PRs.Thanks,-Eric On Wednesday, July 14, 2021, 9:50:47

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-11 Thread epa...@apache.org
+1 (binding) Eric On Tuesday, June 1, 2021, 5:29:49 AM CDT, Wei-Chiu Chuang wrote: Hi community, This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker issues have been resolved [1] again. There are 2 additional issues resolved for RC3: * Revert "MAPREDUCE-7303.

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-07 Thread epa...@apache.org
+1 (binding) -Eric On Thursday, June 3, 2021, 1:14:51 AM CDT, Akira Ajisaka wrote: Dear Hadoop developers, Given the feedback from the discussion thread [1], I'd like to start an official vote thread for the community to vote and start the 3.1 EOL process. What this entails: (1) an

Re: Do I need a +1 to merge a backport PR?

2021-06-02 Thread epa...@apache.org
It is okay to go ahead and backport as long as there are no major refactoring necessary.Minor conflict fixes should be fine.-Eric On Tuesday, June 1, 2021, 11:43:44 PM CDT, Wei-Chiu Chuang wrote: I'm curious about the GitHub PR conventions we use today... say I want to backport a commit

Re: [DISCUSS] check style changes

2021-05-15 Thread epa...@apache.org
I would be fine with a discussion and vote on relaxing some checkstyle restrictions. Regarding line length, my personal preference is to leave it at 80, but 80 is arbitrary and I would not oppose 100 if that's what people want. Another one that I think should be relaxed is the limit on number

Re: [ANNOUNCE] Hui Fei is a new Apache Hadoop Committer

2020-09-24 Thread epa...@apache.org
Congratulations Hui Fei! On Wednesday, September 23, 2020, 1:07:11 PM CDT, Wei-Chiu Chuang wrote: I am pleased to announce that Hui Fei has accepted the invitation to become a Hadoop committer. He started contributing to the project in October 2016. Over the past 4 years he has contributed a

Re: [Virtual MEETUP]: Migration to Hadoop 3

2020-08-27 Thread epa...@apache.org
proaches from the community documentation: > > express upgrade and rolling upgrade. Rolling upgrade was selected. > > > > Yahoo Japan was trying out from hadoop-2.6 to hadop-3.2.1 > > > > https://techblog.yahoo.co.jp/entry/20191206786320/ > > > > On Wed,

Re: [Virtual MEETUP]: Migration to Hadoop 3

2020-08-26 Thread epa...@apache.org
and rolling upgrade. Rolling upgrade was selected. Yahoo Japan was trying out from hadoop-2.6 to hadop-3.2.1 https://techblog.yahoo.co.jp/entry/20191206786320/ On Wed, Aug 26, 2020 at 6:56 PM epa...@apache.org wrote: > Hello. Just a reminder that today I would like to invite you

Re: [Virtual MEETUP]: Migration to Hadoop 3

2020-08-26 Thread epa...@apache.org
Hello. Just a reminder that today I would like to invite you all to discuss your experiences migrating from Hadoop 2 to Hadoop 3. -Eric On Monday, August 24, 2020, 1:58:37 PM CDT, epa...@apache.org wrote: Hello everyone! We are considering migrating to Hadoop 3, and we would be very

[Virtual MEETUP]: Migration to Hadoop 3

2020-08-24 Thread epa...@apache.org
Hello everyone! We are considering migrating to Hadoop 3, and we would be very interested to hear about your experiences. If you have migrated from Hadoop 2 to Hadoop 3 and can provide insights, please kindly consider attending the following: Date: Wednesday, Aug 26, 2020 Time: 10:00 A.M. PDT /

Bi-weekly (U.S.) Yarn Call today

2020-08-12 Thread epa...@apache.org
Hello everyone! This is just a gentle reminder that the bi-weekly Apache YARN call is today at 10 PDT/12 CDT/1 EDT. -Eric Zoom: https://cloudera.zoom.us/j/880548968 Notes: https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M/edit

[ANNOUNCE] Jim Brennan is a new Hadoop Committer

2020-08-03 Thread epa...@apache.org
I am pleased to announce that Jim Brennan has accepted the invitation to become a Hadoop committer focusing on the YARN space. Please reach out to Jim and welcome him in his new role. Congratulations, Jim! Well-deserved! -Eric Payne

Re: Why capacity scheduler not support DRF?

2020-02-07 Thread epa...@apache.org
Hi, I didn't see anyone respond to your question. If you already got a response, please ignore this one. The Capacity Scheduler does support DRF. You can specify which resource calculator to use by setting the yarn.scheduler.capacity.resource-calculator property:      

U.S. Yarn Call today

2020-01-29 Thread epa...@apache.org
Hello everyone! This is just a gentle reminder that the bi-weekly Apache YARN call is today at 10 PST/12 CST/1 EST. -Eric Zoom: https://cloudera.zoom.us/j/880548968 Notes: https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M/edit

[DISCUSS] Guidelines for Code cleanup JIRAs

2020-01-09 Thread epa...@apache.org
There was some discussion on https://issues.apache.org/jira/browse/YARN-9052 about concerns surrounding the costs/benefits of code cleanup JIRAs. This email is to get the discussion going within a wider audience. The positive points for code cleanup JIRAs: - Clean up tech debt - Make code more

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-19 Thread epa...@apache.org
-2. But this proposal would reduce a lot of confusion IMO. > > Jonathan Hung > > > On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org > wrote: > > > Thanks Jonathan for opening the discussion. > > > > I am not in favor of this proposal. 2.10 was very recently r

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-15 Thread epa...@apache.org
Thanks Jonathan for opening the discussion. I am not in favor of this proposal. 2.10 was very recently released, and moving to 2.10 will take some time for the community. It seems premature to make a decision at this point that there will never be a need for a 2.11 release. -Eric On

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-29 Thread epa...@apache.org
Compatibility testing has gone well for me. - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0 - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-29 Thread epa...@apache.org
is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread. Jonathan Hung On Mon, Oct 28, 2019 at 1:06 PM epa...@apache.org wrote: > Compatibility testing has gone well for me. > > - In a 4-node cluster, I ran YARN rolling upgrade tests betw

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-28 Thread epa...@apache.org
Compatibility testing has gone well for me. - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.8.5 and 2.10.0 - In a 4-node cluster, I ran YARN rolling upgrade tests between 2.10.0 and trunk - With one 4-node cluster running 2.10.0 and one 4-node cluster running trunk, I ran a

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-27 Thread epa...@apache.org
mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions? Jonathan Hung On Sat, Oct 26, 2019 at 8:55 AM epa...@apache.org wrote: I ran a few compatibility tests between

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-26 Thread epa...@apache.org
I ran a few compatibility tests between 2.10.0 and 3.3.0 (trunk) Unfortunately, I ran into the following problem: Running with 2.10 RM and 3.3.0 (trunk) NM fails attempts with the following error: 2019-10-26 15:44:06,885 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-23 Thread epa...@apache.org
Hi Jonathan, Thanks very much for all of your work on this release. I have a concern about cross-queue (inter-queue) preemption in 2.10. In 2.8, on a 6 node pseudo-cluster, preempting from one queue to meet the needs of another queue seems to work as expected. However, 2.10 in the same

Re: [VOTE] Release Hadoop-3.1.3-RC0

2019-09-19 Thread epa...@apache.org
+1 (binding) Thanks Zhankun for all of your hard work on this release. I downloaded and built the source and ran it on an insecure multi-node pseudo cluster. I performed various YARN manual tests, including creating custom resources, creating queue submission ACLs, and queue refreshes.

Re: branch-3. Was it a created by mistake?

2019-08-28 Thread epa...@apache.org
Yes, I think it must have been. branch-3 is not needed right now. I would be in favor of its removal, since it is confusing for committers. Thanks,-Eric On Tuesday, August 27, 2019, 6:24:09 PM CDT, Wei-Chiu Chuang wrote: I just realized there is a branch-3 in the Hadoop repo. Was

Re: Hadoop Community Sync Up Schedule

2019-08-21 Thread epa...@apache.org
Let's go with bi-weekly (every 2 weeks). Sometimes this gives us 3 sync-ups in one month, which I think is fine. -Eric Payne On Wednesday, August 21, 2019, 5:01:52 AM CDT, Wangda Tan wrote: > > For folks in other US time zones: how about 11am PDT, is it better or 10am > PDT will be better? I

Re: Hadoop Community Sync Up Schedule

2019-08-20 Thread epa...@apache.org
Hi Wangda, Thank you for continuing to keep us moving forward and refining these vital sync-ups. > 3) Update the US [YARN/MapReduce] sync up time from 9AM to 10AM PDT. That puts it at noon central time, which is during our lunch hour. However, I am +1 for this if we are able to allow greater