Re: [DISCUSS] Making 2.10 the last minor 2.x release

2020-04-16 Thread Jonathan Hung
Source code has been deleted from branch-2. Thanks Akira for taking this up! Jonathan Hung On Thu, Apr 16, 2020 at 11:40 AM Jonathan Hung wrote: > Makes sense. I've cherry-picked the commits in branch-2 that were missed > in branch-2.10. > > Jonathan Hung > > > On Wed, Ap

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2020-04-16 Thread Jonathan Hung
Makes sense. I've cherry-picked the commits in branch-2 that were missed in branch-2.10. Jonathan Hung On Wed, Apr 15, 2020 at 2:25 AM Akira Ajisaka wrote: > Hi folks, > > I am still seeing some changes are being committed to branch-2. > I'd like to delete the source code f

[jira] [Reopened] (HADOOP-14918) Remove the Local Dynamo DB test option

2020-02-25 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung reopened HADOOP-14918: > Remove the Local Dynamo DB test opt

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-12-09 Thread Jonathan Hung
old branch-2.10 - Rename branch-2 to (new) branch-2.10 - Set version in new branch-2.10 to 2.10.1-SNAPSHOT - Renamed fix versions from 2.11.0 to 2.10.1 - Removed 2.11.0 as a version in HADOOP/YARN/HDFS/MAPREDUCE Jonathan Hung On Wed, Dec 4, 2019 at 10:55 AM Jonathan Hung wrote: > FYI, s

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-12-04 Thread Jonathan Hung
FYI, starting the rename process, beginning with INFRA-19521. Jonathan Hung On Wed, Nov 27, 2019 at 12:15 PM Konstantin Shvachko wrote: > Hey guys, > > I think we diverged a bit from the initial topic of this discussion, which > is removing branch-2.10, and changing the version

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-21 Thread Jonathan Hung
to use java 11 on branch-2, we can always revive branch-2. But for now I think the convenience of not needing to port to both branch-2 and branch-2.10 (and below) outweighs the cost of potentially needing to revive branch-2. Jonathan Hung On Wed, Nov 20, 2019 at 10:50 AM Eric Yang wrote: >

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-18 Thread Jonathan Hung
. But this proposal would reduce a lot of confusion IMO. Jonathan Hung On Fri, Nov 15, 2019 at 11:41 AM epa...@apache.org wrote: > Thanks Jonathan for opening the discussion. > > I am not in favor of this proposal. 2.10 was very recently released, and > moving to 2.10 will take some time for t

Re: [DISCUSS] Making 2.10 the last minor 2.x release

2019-11-14 Thread Jonathan Hung
Some other additional items we would need: - Mark all fix-versions in YARN/HDFS/MAPREDUCE/HADOOP from 2.11.0 to 2.10.1 - Remove 2.11.0 as a version in these projects Jonathan Hung On Thu, Nov 14, 2019 at 6:51 PM Jonathan Hung wrote: > Hi folks, > > Given the release

[DISCUSS] Making 2.10 the last minor 2.x release

2019-11-14 Thread Jonathan Hung
This way we get all the current branch-2 fixes into the 2.10.x release line. Then the commit chain will look like: trunk -> branch-3.2 -> branch-3.1 -> branch-2.10 -> branch-2.9 -> branch-2.8 Thoughts? Jonathan Hung [1] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html

[ANNOUNCE] Apache Hadoop 2.10.0 release

2019-10-31 Thread Jonathan Hung
Hi all, I am happy to announce that the Apache Hadoop 2.10.0 has been released. Apache Hadoop 2.10.0 is the first release in the Apache Hadoop 2.10 line. The release details, including links to downloads, list of major features, release notes, and changelog, are on the 2.10.0 announcement page

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-29 Thread Jonathan Hung
+1 from me too. The vote passed, so I'll continue with the rest of the release. Thanks everyone! Jonathan Hung On Tue, Oct 29, 2019 at 1:40 PM Giovanni Matteo Fumarola < giovanni.fumar...@gmail.com> wrote: > +1 (non-binding). > > - Built from source on Ubuntu with OpenJDK 11.0

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-28 Thread Jonathan Hung
Thanks Eric! I sent out an RC1 earlier last week, not sure if you saw that. The only diff between RC1 and RC0 is HDFS-14667. If RC1 looks good to you then it'd be great to get your testing results on that thread. Jonathan Hung On Mon, Oct 28, 2019 at 1:06 PM epa...@apache.org wrote

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-26 Thread Jonathan Hung
Hi Eric, I took a quick look, are you using mapreduce.application.framework.path to run your MR jobs? If not, this seems like expected behavior if AM and tasks get launched on different NMs with different locally installed hadoop versions? Jonathan Hung On Sat, Oct 26, 2019 at 8:55 AM epa

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-25 Thread Jonathan Hung
be up to date, but I think that's fine. I'll go ahead with this plan if no objections. Jonathan Hung On Fri, Oct 25, 2019 at 12:19 PM Jonathan Hung wrote: > Thanks for looking Erik. > > For the release notes, yeah I think it's because there's no release notes > for the corresponding

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-25 Thread Jonathan Hung
to be correct. I think we could easily overwrite the current RC1 artifacts with ones containing proper release notes. Not sure what to do about the javadoc issue though, that would require overwriting the release-2.10.0-RC1 tag which I don't want to do. What do others think? Jonathan Hung On Fri

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC1)

2019-10-23 Thread Jonathan Hung
in dominant resource calculator, but root cause remains to be seen. Jonathan Hung On Wed, Oct 23, 2019 at 9:02 AM epa...@apache.org wrote: > Hi Jonathan, > > Thanks very much for all of your work on this release. > > I have a concern about cross-queue (inter-queue) preemption in 2

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-22 Thread Jonathan Hung
Hi Eric, we've run some basic HDFS commands with a 3.2.1 namenode and 2.10.0 clients and datanodes. Everything worked as expected. Jonathan Hung On Tue, Oct 22, 2019 at 3:04 PM Eric Badger wrote: > Hi Jonathan, > > Thanks for putting this RC together. You stated that there are >

Re: [VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-22 Thread Jonathan Hung
Thanks Konstantin and Zhankun. Unfortunately a feature slipped our radar (HDFS-14667). Since this is the first of a minor release, we would like to get it into 2.10.0. HDFS-14667 has been committed to branch-2.10.0, I will be rolling an RC1 shortly. Jonathan Hung On Tue, Oct 22, 2019 at 1:39

[VOTE] Release Apache Hadoop 2.10.0 (RC0)

2019-10-16 Thread Jonathan Hung
/repositories/orgapachehadoop-1241/ My public key is available here: https://dist.apache.org/repos/dist/release/hadoop/common/KEYS The vote will run for 5 weekdays, until Wednesday, October 23 at 6:00 pm PDT. Thanks, Jonathan Hung [1] https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-16 Thread Jonathan Hung
I've moved all jiras with target version 2.10.0 to 2.10.1. Also I've created branch-2.10 and branch-2.10.0, please commit any 2.10.x bug fixes to branch-2.10. I'll send out a vote thread for 2.10.0-RC0 shortly. Jonathan Hung On Fri, Oct 11, 2019 at 10:32 AM Jonathan Hung wrote: > Edit: se

[jira] [Created] (HADOOP-16655) Use http when fetching tomcat tarball for branch-2

2019-10-15 Thread Jonathan Hung (Jira)
Jonathan Hung created HADOOP-16655: -- Summary: Use http when fetching tomcat tarball for branch-2 Key: HADOOP-16655 URL: https://issues.apache.org/jira/browse/HADOOP-16655 Project: Hadoop Common

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-11 Thread Jonathan Hung
Edit: seems a 2.10.0 blocker was reopened (HDFS-14305). I'll continue watching this jira and start the release once this is resolved. Jonathan Hung On Thu, Oct 10, 2019 at 5:13 PM Jonathan Hung wrote: > Hi folks, as of now all 2.10.0 blockers have been resolved [1]. So I'll >

Re: [DISCUSS] Hadoop 2.10.0 release plan

2019-10-10 Thread Jonathan Hung
Hi folks, as of now all 2.10.0 blockers have been resolved [1]. So I'll start the release process soon (cutting branches, updating target versions, etc). [1] https://issues.apache.org/jira/issues/?filter=12346975 Jonathan Hung On Mon, Aug 26, 2019 at 10:19 AM Jonathan Hung wrote: > Hi fo

[jira] [Created] (HADOOP-16636) No rule to make target PROTOBUF_PROTOC_EXECUTABLE-NOTFOUND when compiling trunk

2019-10-05 Thread Jonathan Hung (Jira)
Jonathan Hung created HADOOP-16636: -- Summary: No rule to make target PROTOBUF_PROTOC_EXECUTABLE-NOTFOUND when compiling trunk Key: HADOOP-16636 URL: https://issues.apache.org/jira/browse/HADOOP-16636

Re: Release plans for Hadoop 2.10.0

2019-09-26 Thread Jonathan Hung
, Jonathan Hung On Thu, Sep 26, 2019 at 6:34 PM Viral Bajaria wrote: > (Cross posting from user list based on feedback by Sean Busbey) > > All, > > Just saw the announcement for new release for Hadoop 3.2.1, congratulations > team and thanks for all the hard work! > > Are we go

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
2.9+. But rolling upgrade would still work IIUC. Jonathan Hung On Tue, Sep 24, 2019 at 2:52 PM Eric Badger wrote: > * For YARN-6616, for branch-2.8 and below, it was only committed to > 2.7.8/2.8.6 which have not been released (as I understand). Perhaps we can > revert YARN-6616 fro

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
Sorry, let me edit my first point. We can just create addendums for YARN-6616 in branch-2.7 and branch-2.8 to edit the submitTime field to the correct id 28. We don’t need to revert YARN-6616 from these branches completely. Jonathan From: Jonathan Hung Sent

Re: Incompatible changes between branch-2.8 and branch-2.9

2019-09-24 Thread Jonathan Hung
be to move the intraQueuePreemptionDisabled field from id 12 to id 13 in branch-2.8, then users would upgrade from 2.8.4/2.8.5 to 2.8.6 (someone would have to release this), then upgrade from 2.8.6 to 2.9+. Jonathan Hung On Tue, Sep 24, 2019 at 9:23 AM Eric Badger wrote: > We (Verizon Media) are currently m

Re: [VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-29 Thread Jonathan Hung
Thanks all, +1 from me too. There's three binding +1, two non-binding +1, and no -1 so I'll merge YARN-8200 to branch-2 shortly. I'll skip branch-3.0 since it's EOL as others have mentioned. Jonathan Hung On Tue, Aug 27, 2019 at 11:49 AM Konstantin Shvachko wrote: > +1 for the merge. >

[DISCUSS] Hadoop 2.10.0 release plan

2019-08-26 Thread Jonathan Hung
. We're targeting a release at end of September. Please share any thoughts you have about this. Thanks! [1] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29461.html [2] https://www.mail-archive.com/mapreduce-dev@hadoop.apache.org/msg21293.html [3] https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg33440.html Jonathan Hung

[VOTE] Merge YARN-8200 to branch-2 and branch-3.0

2019-08-22 Thread Jonathan Hung
=csrqgysvpdf7mh-iu17femgx89dhr+1...@mail.gmail.com%3e Jonathan Hung

Re: [VOTE] Mark 2.6, 2.7, 3.0 release lines EOL

2019-08-20 Thread Jonathan Hung
+1. Thanks! Jonathan Hung On Tue, Aug 20, 2019 at 8:03 PM Wangda Tan wrote: > Hi all, > > This is a vote thread to mark any versions smaller than 2.7 (inclusive), > and 3.0 EOL. This is based on discussions of [1] > > This discussion runs for 7 days and will conclude on Aug

Re: [DISCUSS] Hadoop 2019 Release Planning

2019-08-12 Thread Jonathan Hung
Hi Wangda, Thanks for starting the discussion. We would also like to release 2.10.0 which was discussed previously <https://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg29479.html> and at various contributor meetups. I'm interested in being release manager for that. Thanks, Jonatha

Re: [VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-07 Thread Jonathan Hung
My non-binding +1 to finish. This vote passes with 6 binding +1, 3 non-binding +1, and no vetoes. We will make the changes as part of HADOOP-15711, please follow there. Thanks all! Jonathan Hung On Tue, Feb 5, 2019 at 11:38 PM Akira Ajisaka wrote: > +1 > > -Akira > > On Wed, F

[VOTE] Moving branch-2 precommit/nightly test builds to java 8

2019-02-04 Thread Jonathan Hung
-Build/ [3] https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/ Jonathan Hung

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-04 Thread Jonathan Hung
. Jonathan Hung On Mon, Feb 4, 2019 at 10:45 AM Anu Engineer wrote: > Konstantin, > > Just a nitpicky thought, if we move this branch to Java-8 on Jenkins, but > still hope to release code that can run on Java 7, how will we detect > Java 8 only changes? I am asking because till

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-04 Thread Jonathan Hung
Yeah, it's possible with yetus, there's one example here <https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/60/console> which runs compilation on openjdk7 (and openjdk8), and runs tests on openjdk8 only. Jonathan Hung On Mon, Feb 4, 2019 at 10

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

2019-02-01 Thread Jonathan Hung
+1. Thanks Wangda. Jonathan Hung On Fri, Feb 1, 2019 at 2:25 PM Dinesh Chitlangia < dchitlan...@hortonworks.com> wrote: > +1 (non binding), thanks Wangda for organizing this. > > Regards, > Dinesh > > > > On 2/1/19, 5:24 PM, "Wangda Tan" wrote: >

Re: [DISCUSS] Moving branch-2 to java 8

2019-02-01 Thread Jonathan Hung
compatibility we can do that at a later point. Also based on discussion with others in the community at the contributors meetup this past Wednesday, seems we are generally in favor of testing against java8. I'll start a vote soon. Jonathan Hung On Tue, Jan 29, 2019 at 4:11 AM Steve Loughran wrote

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Jonathan Hung
+1. This is important for improving the deep learning on hadoop story. There's recently a lot of momentum for this, and decoupling submarine/hadoop will help it continue. Jonathan Hung On Thu, Jan 31, 2019 at 11:04 AM Wangda Tan wrote: > Hi devs, > > Since we started submarine-relat

[DISCUSS] Moving branch-2 to java 8

2019-01-28 Thread Jonathan Hung
source and binary compatibility. I don't currently have a great answer, but one initial thought is to build source/binary against java 7 to ensure compatibility and run the rest of the build as java 8. Thoughts? Jonathan Hung

Re: [VOTE - 2] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

2018-12-18 Thread Jonathan Hung
+1! Jonathan Hung On Sat, Dec 15, 2018 at 8:26 AM Zhe Zhang wrote: > +1 > > Thanks for addressing concerns from the previous vote. > > On Fri, Dec 14, 2018 at 6:24 PM Konstantin Shvachko > wrote: > > > Hi Hadoop developers, > > > > I would like to propo

[DISCUSS] Merging YARN-8200 to branch-2

2018-12-18 Thread Jonathan Hung
branch-2, syncing any bug fixes in our internal fork which did not make it to the feature branch, etc. Assuming no objections, once it's ready we will start a vote to merge. Thanks, Jonathan Hung

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-28 Thread Jonathan Hung
- verified refreshQueues mutation is disabled in this case Jonathan Hung On Thu, Mar 22, 2018 at 9:10 AM, Wangda Tan <wheele...@gmail.com> wrote: > Thanks @Bharat for the quick check, the previously staged repository has > some issues. I re-deployed jars to nexus. > > Here's

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

2017-12-13 Thread Jonathan Hung
Thanks Konstantin for working on this. +1 (non-binding) - Downloaded binary and verified md5 - Deployed RM HA and tested failover Jonathan Hung On Wed, Dec 13, 2017 at 11:02 AM, Eric Payne <erichadoo...@yahoo.com.invalid > wrote: > Thanks for the hard work on this release, Konstan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

2017-12-12 Thread Jonathan Hung
on restart/failover - Verified "yarn rmadmin -refreshQueues" works when scheduler configuration API disabled, and does not work when scheduler configuration API enabled Jonathan Hung On Tue, Dec 12, 2017 at 5:44 PM, Junping Du <j...@hortonworks.com> wrote: > Thanks Andrew

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-14 Thread Jonathan Hung
"yarn rmadmin -refreshQueues" enabled when configuration mutation API disabled (and vice-versa) - Tested queue admin configuration mutation policy Jonathan Hung On Mon, Nov 13, 2017 at 4:10 PM, Arun Suresh <asur...@apache.org> wrote: > Hi Folks, > > Apache Hadoop 2

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC0)

2017-11-07 Thread Jonathan Hung
API disabled (and vice-versa) Jonathan Hung On Tue, Nov 7, 2017 at 2:56 PM, Eric Badger <ebad...@oath.com> wrote: > +1 (non-binding) pending the issue that Sunil/Rohith pointed out > > - Verified all hashes and checksums > - Built from source on macOS 10.12.6, Java 1.8.0u65 >

Re: [VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2

2017-10-09 Thread Jonathan Hung
Thanks for the votes and discussion. It is now past Monday Oct 9 11:00AM PDT so the vote has ended. There were 4 +1 and no -1, so vote passes. This feature will be merged to trunk, branch-3.0, and branch-2 shortly (16 subtasks). Thanks everyone! Jonathan Hung On Mon, Oct 9, 2017 at 9:18 AM

[VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2

2017-10-02 Thread Jonathan Hung
hou. [1] http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201709.mbox/%3CCAHzWLgfEAgczjcEOUCg-03ma3ROtO=pkec9dpggyx9rzf3n...@mail.gmail.com%3E Jonathan Hung

[DISCUSS] Merging API-based scheduler configuration to trunk/branch-2

2017-09-28 Thread Jonathan Hung
was to merge this to trunk (which is what the YARN-7241 diff is based on), and port to branch-2 before the 2.9 release. @Andrew, what are your thoughts on also merging this to branch-3.0? Thanks! Jonathan Hung

Re: [DISCUSS] Looking to a 2.9.0 release

2017-09-05 Thread Jonathan Hung
on the last week of Sept. Please let me know if you have any concerns. Thanks! Jonathan Hung On Wed, Jul 26, 2017 at 11:23 AM, J. Rottinghuis <jrottingh...@gmail.com> wrote: > Thanks Vrushali for being entirely open as to the current status of ATSv2. > I appreciate that we want to e

[jira] [Resolved] (HADOOP-14828) RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum time

2017-09-05 Thread Jonathan Hung (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved HADOOP-14828. Resolution: Duplicate > RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum t

[jira] [Created] (HADOOP-14828) RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum time

2017-09-01 Thread Jonathan Hung (JIRA)
Jonathan Hung created HADOOP-14828: -- Summary: RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum time Key: HADOOP-14828 URL: https://issues.apache.org/jira/browse/HADOOP-14828 Project

Re: Branch merges and 3.0.0-beta1 scope

2017-08-25 Thread Jonathan Hung
re is also turned off by default). Hoping to get this in before 3.0.0-GA. Just wanted to send this note so that we are not caught off guard by this feature. Thanks! Jonathan Hung On Fri, Aug 25, 2017 at 11:06 AM, Wangda Tan <wheele...@gmail.com> wrote: > Resource profile is similar t

[jira] [Reopened] (HADOOP-12825) Log slow name resolutions

2017-05-30 Thread Jonathan Hung (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung reopened HADOOP-12825: > Log slow name resolutions > -- > >

[jira] [Created] (HADOOP-14463) Port HADOOP-12954 to branch-2.8, branch-2.7

2017-05-27 Thread Jonathan Hung (JIRA)
Jonathan Hung created HADOOP-14463: -- Summary: Port HADOOP-12954 to branch-2.8, branch-2.7 Key: HADOOP-14463 URL: https://issues.apache.org/jira/browse/HADOOP-14463 Project: Hadoop Common