Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
Thanks to Karthik’s +1, HADOOP-13161 was committed last night and appears to be working. I’ve got almost the entire list of stuff finished. I’m now going to go through the java8 jobs and actually sync them up, since they all do slightly different things. :/ > On May 16, 2016, at 8:02 PM, Allen Wittenauer wrote: > > OK, it looks like if someone commits HADOOP-13161, then trunk only uses > JDK8 and branch-2 will use JDK7 and JDK8 during precommit with no changes > required to Apache Yetus. :D > > >> On May 16, 2016, at 5:38 PM, Allen Wittenauer >> wrote: >> >> >> There’s a bunch of stuff that needs to happen at the Jenkins level: >> >> * Kill off the JDK7 trunk builds for HADOOP, HDFS, MAPRED, YARN >> * Remove JDK7 from pre-commit for HADOOP, HDFS, MAPRED, YARN >> >> One thing that needs to happen in the Apache Yetus project: >> * Wait until YETUS-369 has been written and committed to re-enable JDK7 for >> pre-commit (This effectively means that *ALL* JDK7 testing will *ONLY* be >> happening in the regularly scheduled builds) >> >> One thing that really should happen in the Apache Hadoop project: >> * Remove JDK7 from trunk Dockerfile >> >> I’ll start banging on this stuff over the next few days. >> >> >>> On May 16, 2016, at 3:58 PM, Andrew Wang wrote: >>> >>> Very happy to announce that we've committed HADOOP-11858. I'm looking >>> forward to writing my first lambda in Java. I also attached a video to the >>> JIRA so we can all relive this moment in Hadoop development history. >>> >>> It sounds like there's some precommit work to align test-patch with this >>> change. I'm hoping Allen will take point on this, but ping me if I can be >>> of any assistance. >>> >>> On Thu, May 12, 2016 at 11:53 AM, Li Lu wrote: >>> I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083 does the maven change and in yarn-api there are ~5000 javadoc warnings. Li Lu > On May 10, 2016, at 08:32, Akira AJISAKA wrote: > > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > Given this is a critical change, I'm thinking we should get the consensus first. > > One concern I think is, when the minimum version is set to JDK8, we need to configure Jenkins to disable multi JDK test only in trunk. > > Any thoughts? > > Thanks, > Akira > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > >> >> >> - >> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org >> For additional commands, e-mail: common-dev-h...@hadoop.apache.org >> > > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
> On 16 May 2016, at 23:58, Andrew Wang wrote: > > Very happy to announce that we've committed HADOOP-11858. I'm looking > forward to writing my first lambda in Java. I also attached a video to the > JIRA so we can all relive this moment in Hadoop development history. > > It sounds like there's some precommit work to align test-patch with this > change. I'm hoping Allen will take point on this, but ping me if I can be > of any assistance. > Everyone who worked on this deserves a lot of credit, especially anyone working on library incompatibilities What does it mean? 1. it means that if you are working on trunk, you only need to worry about Java 8. 2. You can use java 8 language features: time to learn your lambda expressions. Apparently this it a good tutorial http://www.cse.chalmers.se/research/group/logic/TypesSS05/Extra/geuvers.pdf 3. You can use java 8 library features. It also means: you can no longer backport stuff from trunk to branch-2 and expect it to always compile If you are working on code which you hope to get into branch-2, you need to show it will work in Java 7 Tactics 1. work on branch-2; forward port to trunk 2. before backporting from trunk, submit a HADOOP-1234-branch-2-001.patch ; yetus will see the branch-2 marker and test against branch2. I would recommend always doing #2, irrespective of which branch you work on. Why? It means you can just switch the java version on your path to 1.8, with branch-2 still being in java 7 language —but the java library picked up will be the 1.8 lib. The only way to be confident that you haven't picked up java 8 features is to build and test against JDK7, which Yetus does if you name your patch right. I know this will be a bit of extra work, and I expect we will all make mistakes here. If anyone does accidentally commit JDK8+ code to branch-2, then it should be bounced out as soon as it's noticed, then a new patch nurtured in via Yetus & Jenkins. We want to keep branch-2 building. On that topic, Jenkins isn't happy these days...I think we need to spend some monitoring it and fixing. At least now we can cull the java7 trunk builds, leaving less to ignore. There's one test, TestDNS which fails intermittently (always), but more and more we are learning to ignore. That's dangerous. - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
OK, it looks like if someone commits HADOOP-13161, then trunk only uses JDK8 and branch-2 will use JDK7 and JDK8 during precommit with no changes required to Apache Yetus. :D > On May 16, 2016, at 5:38 PM, Allen Wittenauer > wrote: > > > There’s a bunch of stuff that needs to happen at the Jenkins level: > > * Kill off the JDK7 trunk builds for HADOOP, HDFS, MAPRED, YARN > * Remove JDK7 from pre-commit for HADOOP, HDFS, MAPRED, YARN > > One thing that needs to happen in the Apache Yetus project: > * Wait until YETUS-369 has been written and committed to re-enable JDK7 for > pre-commit (This effectively means that *ALL* JDK7 testing will *ONLY* be > happening in the regularly scheduled builds) > > One thing that really should happen in the Apache Hadoop project: > * Remove JDK7 from trunk Dockerfile > > I’ll start banging on this stuff over the next few days. > > >> On May 16, 2016, at 3:58 PM, Andrew Wang wrote: >> >> Very happy to announce that we've committed HADOOP-11858. I'm looking >> forward to writing my first lambda in Java. I also attached a video to the >> JIRA so we can all relive this moment in Hadoop development history. >> >> It sounds like there's some precommit work to align test-patch with this >> change. I'm hoping Allen will take point on this, but ping me if I can be >> of any assistance. >> >> On Thu, May 12, 2016 at 11:53 AM, Li Lu wrote: >> >>> I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083 >>> does the maven change and in yarn-api there are ~5000 javadoc warnings. >>> >>> Li Lu >>> On May 10, 2016, at 08:32, Akira AJISAKA >>> wrote: Hi developers, Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Given this is a critical change, I'm thinking we should get the >>> consensus first. One concern I think is, when the minimum version is set to JDK8, we need >>> to configure Jenkins to disable multi JDK test only in trunk. Any thoughts? Thanks, Akira - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> > > > - > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
There’s a bunch of stuff that needs to happen at the Jenkins level: * Kill off the JDK7 trunk builds for HADOOP, HDFS, MAPRED, YARN * Remove JDK7 from pre-commit for HADOOP, HDFS, MAPRED, YARN One thing that needs to happen in the Apache Yetus project: * Wait until YETUS-369 has been written and committed to re-enable JDK7 for pre-commit (This effectively means that *ALL* JDK7 testing will *ONLY* be happening in the regularly scheduled builds) One thing that really should happen in the Apache Hadoop project: * Remove JDK7 from trunk Dockerfile I’ll start banging on this stuff over the next few days. > On May 16, 2016, at 3:58 PM, Andrew Wang wrote: > > Very happy to announce that we've committed HADOOP-11858. I'm looking > forward to writing my first lambda in Java. I also attached a video to the > JIRA so we can all relive this moment in Hadoop development history. > > It sounds like there's some precommit work to align test-patch with this > change. I'm hoping Allen will take point on this, but ping me if I can be > of any assistance. > > On Thu, May 12, 2016 at 11:53 AM, Li Lu wrote: > >> I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083 >> does the maven change and in yarn-api there are ~5000 javadoc warnings. >> >> Li Lu >> >>> On May 10, 2016, at 08:32, Akira AJISAKA >> wrote: >>> >>> Hi developers, >>> >>> Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. >>> Given this is a critical change, I'm thinking we should get the >> consensus first. >>> >>> One concern I think is, when the minimum version is set to JDK8, we need >> to configure Jenkins to disable multi JDK test only in trunk. >>> >>> Any thoughts? >>> >>> Thanks, >>> Akira >>> >>> - >>> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org >>> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org >>> >>> >> >> - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
Very happy to announce that we've committed HADOOP-11858. I'm looking forward to writing my first lambda in Java. I also attached a video to the JIRA so we can all relive this moment in Hadoop development history. It sounds like there's some precommit work to align test-patch with this change. I'm hoping Allen will take point on this, but ping me if I can be of any assistance. On Thu, May 12, 2016 at 11:53 AM, Li Lu wrote: > I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083 > does the maven change and in yarn-api there are ~5000 javadoc warnings. > > Li Lu > > > On May 10, 2016, at 08:32, Akira AJISAKA > wrote: > > > > Hi developers, > > > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > > Given this is a critical change, I'm thinking we should get the > consensus first. > > > > One concern I think is, when the minimum version is set to JDK8, we need > to configure Jenkins to disable multi JDK test only in trunk. > > > > Any thoughts? > > > > Thanks, > > Akira > > > > - > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > >
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
I’d like to bring YARN-4977 into attention for using Java 8. HADOOP-13083 does the maven change and in yarn-api there are ~5000 javadoc warnings. Li Lu > On May 10, 2016, at 08:32, Akira AJISAKA wrote: > > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > Given this is a critical change, I'm thinking we should get the consensus > first. > > One concern I think is, when the minimum version is set to JDK8, we need to > configure Jenkins to disable multi JDK test only in trunk. > > Any thoughts? > > Thanks, > Akira > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > >
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
+1 Masatake On 5/12/16 13:11, Gangumalla, Uma wrote: +1 Regards, Uma On 5/10/16, 2:24 PM, "Andrew Wang" wrote: +1 On Tue, May 10, 2016 at 12:36 PM, Ravi Prakash wrote: +1. Thanks for driving this Akira On Tue, May 10, 2016 at 10:25 AM, Tsuyoshi Ozawa wrote: Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Sounds good. To do so, we need to check the blockers of 3.0.0-alpha RC, especially upgrading all dependencies which use refractions at first. Thanks, - Tsuyoshi On Tue, May 10, 2016 at 8:32 AM, Akira AJISAKA wrote: Hi developers, Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Given this is a critical change, I'm thinking we should get the consensus first. One concern I think is, when the minimum version is set to JDK8, we need to configure Jenkins to disable multi JDK test only in trunk. Any thoughts? Thanks, Akira - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
bq. We'll need to be strict after the switch: all patches to go into branch 2 will have to go through yetus as branch-2 patches, then cherry picked to trunk, or a separate patch on the same JIRA done for the branch-2. Assuming yetus still tests the branch-2 stuff on JDK7, that will check version compatibility pre-commit. +1. That's something I want to say also. We should have separate process for backport effort between Trunk and branch-2 then. Otherwise, we could involve Java 7 related bug to branch-2 after the backport. Thanks, Junping From: Steve Loughran Sent: Wednesday, May 11, 2016 5:20 PM To: Akira AJISAKA Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858) > > On 10 May 2016, at 16:32, Akira AJISAKA wrote: > > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. +1 Given Robert Kanter first filed the patch to do this —why not give him the honour. In fact, why not have a webex screen share of the occasion so we can all celebrate? > Given this is a critical change, I'm thinking we should get the consensus > first. > > One concern I think is, when the minimum version is set to JDK8, we need to > configure Jenkins to disable multi JDK test only in trunk. > LGTM We'll need to be strict after the switch: all patches to go into branch 2 will have to go through yetus as branch-2 patches, then cherry picked to trunk, or a separate patch on the same JIRA done for the branch-2. Assuming yetus still tests the branch-2 stuff on JDK7, that will check version compatibility pre-commit - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
> > On 10 May 2016, at 16:32, Akira AJISAKA wrote: > > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. +1 Given Robert Kanter first filed the patch to do this —why not give him the honour. In fact, why not have a webex screen share of the occasion so we can all celebrate? > Given this is a critical change, I'm thinking we should get the consensus > first. > > One concern I think is, when the minimum version is set to JDK8, we need to > configure Jenkins to disable multi JDK test only in trunk. > LGTM We'll need to be strict after the switch: all patches to go into branch 2 will have to go through yetus as branch-2 patches, then cherry picked to trunk, or a separate patch on the same JIRA done for the branch-2. Assuming yetus still tests the branch-2 stuff on JDK7, that will check version compatibility pre-commit - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
+1 On Tue, May 10, 2016 at 12:36 PM, Ravi Prakash wrote: > +1. Thanks for driving this Akira > > On Tue, May 10, 2016 at 10:25 AM, Tsuyoshi Ozawa wrote: > > > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > > > > Sounds good. To do so, we need to check the blockers of 3.0.0-alpha > > RC, especially upgrading all dependencies which use refractions at > > first. > > > > Thanks, > > - Tsuyoshi > > > > On Tue, May 10, 2016 at 8:32 AM, Akira AJISAKA > > wrote: > > > Hi developers, > > > > > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > > > Given this is a critical change, I'm thinking we should get the > consensus > > > first. > > > > > > One concern I think is, when the minimum version is set to JDK8, we > need > > to > > > configure Jenkins to disable multi JDK test only in trunk. > > > > > > Any thoughts? > > > > > > Thanks, > > > Akira > > > > > > - > > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > > > > - > > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > > > > >
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
+1. Thanks for driving this Akira On Tue, May 10, 2016 at 10:25 AM, Tsuyoshi Ozawa wrote: > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > > Sounds good. To do so, we need to check the blockers of 3.0.0-alpha > RC, especially upgrading all dependencies which use refractions at > first. > > Thanks, > - Tsuyoshi > > On Tue, May 10, 2016 at 8:32 AM, Akira AJISAKA > wrote: > > Hi developers, > > > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > > Given this is a critical change, I'm thinking we should get the consensus > > first. > > > > One concern I think is, when the minimum version is set to JDK8, we need > to > > configure Jenkins to disable multi JDK test only in trunk. > > > > Any thoughts? > > > > Thanks, > > Akira > > > > - > > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > > > > - > To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org > >
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
> Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Sounds good. To do so, we need to check the blockers of 3.0.0-alpha RC, especially upgrading all dependencies which use refractions at first. Thanks, - Tsuyoshi On Tue, May 10, 2016 at 8:32 AM, Akira AJISAKA wrote: > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. > Given this is a critical change, I'm thinking we should get the consensus > first. > > One concern I think is, when the minimum version is set to JDK8, we need to > configure Jenkins to disable multi JDK test only in trunk. > > Any thoughts? > > Thanks, > Akira > > - > To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org > - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
Re: [DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
> On May 10, 2016, at 8:32 AM, Akira AJISAKA wrote: > > Hi developers, > > Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. +1 yes please. > One concern I think is, when the minimum version is set to JDK8, we need to > configure Jenkins to disable multi JDK test only in trunk. Two points: * The feature to do this in test-patch is already on the roadmap, since HBase asked already: YETUS-369 . We’re really close to releasing Apache Yetus 0.3.0, so it won’t show up until 0.4.0 though. It’s a non-trivial change on it’s own, and based on some conversations I’ve been having this week with other Yetus users, I’m thinking we may want to rewrite how multijdkmode works anyway. * It’s feasible to just disable multijdkmode in precommit and then rely upon the regularly scheduled builds to tell us if branch-2 gets broken by a JDK8-only patch. (FWIW: I’ll be talking about it during ApacheCon, but I’ve been working on a Yetus-ified replacement for those “build the world” jobs such that they are actually useful, less frequent, easier to read, less likely to break due to running in a docker container, etc, etc.) - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[DISCUSS] Set minimum version of Hadoop 3 to JDK8 (HADOOP-11858)
Hi developers, Before cutting 3.0.0-alpha RC, I'd like to drop JDK7 support in trunk. Given this is a critical change, I'm thinking we should get the consensus first. One concern I think is, when the minimum version is set to JDK8, we need to configure Jenkins to disable multi JDK test only in trunk. Any thoughts? Thanks, Akira - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
