I am bringing the subject to the user mailing list to get some feedback because it makes sense anyway to discuss this there. But I also agree with Kenneth about the fact that runner authors must weight in on this subject.
On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles <k...@google.com.invalid> wrote: > +1 to having runner maintainers weigh in as proxies. Added a few in case > they haven't followed this thread. > > On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov < > kirpic...@google.com.invalid> wrote: > >> Agreed that polling Dataflow users makes sense, though I think they are >> very unlikely to have concerns, because unlike Spark/Flink users they >> wouldn't have a "cluster" that they need to migrate to a new JVM: they'd >> only need to recompile their pipelines with JDK 8. >> >> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid> >> wrote: >> >> > I think the Flink and Spark runner maintainers can weigh in here; given >> > that both of those systems are moving to Java 8, I doubt they will have >> > concerns. Same is true for the Dataflow runner - we should probably poll >> > Dataflow users to make sure this is not a problem for any of them. >> > >> > On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov < >> > kirpic...@google.com.invalid> wrote: >> > >> > > Reuven - do you mean e.g. a poll on the Flink mailing list asking >> whether >> > > there are Flink users who use Beam with Java 7? Or just people who use >> > > Flink with Java 7? (the latter question I'd assume was settled by the >> > poll >> > > about making Flink itself Java8-only?) >> > > >> > > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid> >> > > wrote: >> > > >> > > > I don't know if a vote in @user is sufficient, as it's unfortunately >> > not >> > > > representative of all Beam users. I think the runner communities >> should >> > > be >> > > > polled as well (though I suspect the answer will be the same, that we >> > can >> > > > go ahead and deprecate Java 7 support). >> > > > >> > > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov < >> > > > kirpic...@google.com.invalid> wrote: >> > > > >> > > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you >> be >> > > > > interested in driving this process, since you're already working on >> > > Java9 >> > > > > support and hence you have a good understanding of what's involved >> > in a >> > > > JDK >> > > > > version migration for a large project? >> > > > > >> > > > > As due diligence, we can look at how the other data processing >> > systems >> > > > > handled dropping Java7. >> > > > > >> > > > > Flink: >> > > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242 >> > > > > Discussion >> > > > > http://apache-flink-user-mailing-list-archive.2336050. >> > > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html >> > > > > >> > > > > They also did a Twitter poll in addition to the mailing list poll, >> > > which >> > > > > seems like a good idea. >> > > > > Note that Flink had a number of strong reasons for dropping Java7 >> > that >> > > do >> > > > > not apply in Beam. >> > > > > >> > > > > Spark: >> > > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493 >> > > > > Discussion >> > > > > >> > > > http://apache-spark-developers-list.1001551.n3. >> > > nabble.com/discuss-ending- >> > > > > support-for-Java-7-in-Spark-2-0-td16808.html >> > > > > (I >> > > > > couldn't find a formal poll of the user list rather than developer >> > > list) >> > > > > >> > > > > Hadoop: >> > > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion >> of >> > > > where >> > > > > that decision was made. >> > > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f >> 5439839 >> > > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev. >> > > > > hadoop.apache.org%3E >> > > > > might >> > > > > be it. >> > > > > >> > > > > Kafka is considering it: >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >> > > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 >> > > > > and >> > > > > quotes a number of other open-source projects that have switched >> > > > > http://markmail.org/message/l7s276y3xkga2eqf >> > > > > >> > > > > So basically these projects all did a mailing list poll, and one >> did >> > > > also a >> > > > > twitter poll. >> > > > > >> > > > > Beam has the advantage of being a relatively young project with >> > > perhaps a >> > > > > smaller base of users entrenched in using old versions of Java; >> > > moreover, >> > > > > Java version would matter only for the smaller subset of users who >> > use >> > > > Beam >> > > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which >> is >> > > > likely >> > > > > an even more "early adopter"-ish group of users, as these runners >> > > > generally >> > > > > receive less support. >> > > > > >> > > > > It may be a good idea to have at least 1 release pass between >> > > announcing >> > > > > the intention to drop Java8 and actually dropping it (e.g. if we >> > > decided >> > > > it >> > > > > now, then 2.4 would drop Java7). Also, we could start by switching >> > > tests >> > > > to >> > > > > compile/run with java8 (Maven allows this). This is, I think, >> pretty >> > > much >> > > > > safe to do immediately. >> > > > > >> > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ieme...@gmail.com> >> > > wrote: >> > > > > >> > > > > > Any progress on this? What is the proposed way to validate if >> users >> > > > > > are still interested on Java 7? A vote on user or something >> > > different? >> > > > > > >> > > > > > >> > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles >> > > > <k...@google.com.invalid >> > > > > > >> > > > > > wrote: >> > > > > > > Agree with polling Beam users as well as each runner's >> community >> > in >> > > > > > > aggregate. >> > > > > > > >> > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré < >> > > > j...@nanthrax.net >> > > > > > >> > > > > > > wrote: >> > > > > > > >> > > > > > >> Definitely agree >> > > > > > >> >> > > > > > >> >> > > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote: >> > > > > > >> >> > > > > > >>> I also think that it's time to seriously consider dropping >> > > support >> > > > > for >> > > > > > >>> Java 7. >> > > > > > >>> >> > > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira >> > > > > > >>> <danolive...@google.com.invalid> wrote: >> > > > > > >>> >> > > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right >> now >> > > > > despite >> > > > > > >>>> that >> > > > > > >>>> (I believe) we don't use the extension that's breaking. >> > > > > > >>>> >> > > > > > >>>> As for other ways to solve this, if there is a way to avoid >> > > > > compiling >> > > > > > the >> > > > > > >>>> advanced features of AutoValue that might be worth a try. We >> > > could >> > > > > > also >> > > > > > >>>> try >> > > > > > >>>> to get a release of AutoValue with the fix that works in >> Java >> > 7. >> > > > > > However >> > > > > > >>>> I >> > > > > > >>>> feel that slowly moving over to Java 8 is the most >> > future-proof >> > > > > > solution >> > > > > > >>>> if >> > > > > > >>>> it's possible. >> > > > > > >>>> >> > > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía < >> > > ieme...@gmail.com> >> > > > > > wrote: >> > > > > > >>>> >> > > > > > >>>> The current issue is that compilation fails on master >> because >> > > > beam's >> > > > > > >>>>> parent pom is configured to fail if it finds warnings): >> > > > > > >>>>> >> > > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag> >> > > > > > >>>>> >> > > > > > >>>>> However if you remove that line from the parent pom the >> > > > compilation >> > > > > > >>>>> passes. >> > > > > > >>>>> >> > > > > > >>>>> Of course this does not mean that everything is solved for >> > Java >> > > > 9, >> > > > > > >>>>> there are some tests that break and other issues because of >> > > other >> > > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are >> not >> > > part >> > > > > of >> > > > > > >>>>> this discussion. >> > > > > > >>>>> >> > > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov >> > > > > > >>>>> <kirpic...@google.com.invalid> wrote: >> > > > > > >>>>> >> > > > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. >> > > Does >> > > > > that >> > > > > > >>>>>> mean >> > > > > > >>>>>> this issue is moot? I didn't quite understand from your >> > email >> > > > > > whether >> > > > > > >>>>>> it >> > > > > > >>>>>> >> > > > > > >>>>> is >> > > > > > >>>>> >> > > > > > >>>>>> a compilation blocker for Beam or not. >> > > > > > >>>>>> >> > > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía < >> > > ieme...@gmail.com >> > > > > >> > > > > > >>>>>> wrote: >> > > > > > >>>>>> >> > > > > > >>>>>> Great that you are also working on this too Daniel and >> > thanks >> > > > for >> > > > > > >>>>>>> bringing this subject to the mailing list, I was waiting >> to >> > > my >> > > > > > return >> > > > > > >>>>>>> to office next week, but you did it first :) >> > > > > > >>>>>>> >> > > > > > >>>>>>> Eugene for reference (This is the issue on the migration >> to >> > > > Java >> > > > > > 9), >> > > > > > >>>>>>> notice that here the goal is first that beam passes mvn >> > clean >> > > > > > install >> > > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to >> > > rewrite >> > > > > > >>>>>>> anything to use the new stuff (e.g. modules): >> > > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530 >> > > > > > >>>>>>> >> > > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more >> > details >> > > > on >> > > > > > the >> > > > > > >>>>>>> issue, this is a warning so I don't know if it is really >> > > > critical >> > > > > > in >> > > > > > >>>>>>> particular because we are not using Memoization (do we?). >> > > > > > >>>>>>> https://github.com/google/auto/issues/503 >> > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > > > >> > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d >> > > > > > >>>>>>> >> > > > > > >>>>>> 02247b8532 >> > > > > > >>>>> >> > > > > > >>>>>> >> > > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the >> > > > > google.auto >> > > > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible >> way >> > > and >> > > > > > >>>>>>> 'voila' ? >> > > > > > >>>>>>> >> > > > > > >>>>>>> However I agree that moving to Java 8 is an excellent >> idea >> > > and >> > > > as >> > > > > > >>>>>>> Eugene mentions there is less friction now since most >> > > projects >> > > > > are >> > > > > > >>>>>>> moving, only pending issue are existing clusters on java >> 7 >> > in >> > > > the >> > > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway >> this >> > > > > > discussion >> > > > > > >>>>>>> is really important (maybe even worth a vote). Because >> > moving >> > > > to >> > > > > > Java >> > > > > > >>>>>>> 8 will allow us also to move some of the dependencies >> that >> > we >> > > > are >> > > > > > >>>>>>> keeping in old versions and in general to move forward. >> > > > > > >>>>>>> >> > > > > > >>>>>>> What do the others think ? >> > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov >> > > > > > >>>>>>> <kirpic...@google.com.invalid> wrote: >> > > > > > >>>>>>> >> > > > > > >>>>>>>> Very excited to hear that there's work on JDK9 support - >> > is >> > > > > there >> > > > > > a >> > > > > > >>>>>>>> >> > > > > > >>>>>>> public >> > > > > > >>>>>>> >> > > > > > >>>>>>>> description of the plans for this work somewhere? >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> In general, Beam could probably drop Java7 support >> > > altogether >> > > > at >> > > > > > some >> > > > > > >>>>>>>> >> > > > > > >>>>>>> point >> > > > > > >>>>>>> >> > > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not >> > receiving >> > > > > even >> > > > > > >>>>>>>> >> > > > > > >>>>>>> security >> > > > > > >>>>>>> >> > > > > > >>>>>>>> patches) 2 years ago, and all major players in the data >> > > > > processing >> > > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, >> > Hadoop), >> > > > so >> > > > > I >> > > > > > >>>>>>>> >> > > > > > >>>>>>> presume >> > > > > > >>>>> >> > > > > > >>>>>> the demand for Java7 support in the data processing >> industry >> > > is >> > > > > low. >> > > > > > >>>>>>>> >> > > > > > >>>>>>> By >> > > > > > >>>>> >> > > > > > >>>>>> the >> > > > > > >>>>>>> >> > > > > > >>>>>>>> way: would a Java8 migration be in the scope of your >> work >> > in >> > > > > > general? >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> However, until we say that Beam requires Java8, what >> would >> > > be >> > > > > the >> > > > > > >>>>>>>> implications of using a version of AutoValue that can >> only >> > > be >> > > > > > >>>>>>>> compiled >> > > > > > >>>>>>>> >> > > > > > >>>>>>> with >> > > > > > >>>>>>> >> > > > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a >> > > > compiler >> > > > > > bug, >> > > > > > >>>>>>>> >> > > > > > >>>>>>> and >> > > > > > >>>>> >> > > > > > >>>>>> if we use a Java8 compiler but configured to use source >> and >> > > > target >> > > > > > >>>>>>>> >> > > > > > >>>>>>> versions >> > > > > > >>>>>>> >> > > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then >> > the >> > > > > > resulting >> > > > > > >>>>>>>> >> > > > > > >>>>>>> Beam >> > > > > > >>>>>>> >> > > > > > >>>>>>>> artifacts will be usable by people who don't have Java8? >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira >> > > > > > >>>>>>>> <danolive...@google.com.invalid> wrote: >> > > > > > >>>>>>>> >> > > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I >> > have a >> > > > bug >> > > > > > in >> > > > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue >> > > > > dependency >> > > > > > to >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> the >> > > > > > >>>>> >> > > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to >> be >> > > > banned >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> for >> > > > > > >>>>> >> > > > > > >>>>>> Beam >> > > > > > >>>>>>> >> > > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should >> > still >> > > be >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> possible >> > > > > > >>>>> >> > > > > > >>>>>> to >> > > > > > >>>>>>> >> > > > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler >> > by >> > > > > > building >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> with >> > > > > > >>>>>>> >> > > > > > >>>>>>>> the correct arguments. So the fix to this bug would >> > > > essentially >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> require >> > > > > > >>>>> >> > > > > > >>>>>> Java 8 compilers even for compiling Java 7 code. >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if >> not >> > I >> > > > > would >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>> like to >> > > > > > >>>>> >> > > > > > >>>>>> continue with this fix. >> > > > > > >>>>>>>>> >> > > > > > >>>>>>>>> >> > > > > > >>>>>>> >> > > > > > >>>>> >> > > > > > >> -- >> > > > > > >> Jean-Baptiste Onofré >> > > > > > >> jbono...@apache.org >> > > > > > >> http://blog.nanthrax.net >> > > > > > >> Talend - http://www.talend.com >> > > > > > >> >> > > > > > >> > > > > >> > > > >> > > >> > >>
