+1 to having runner maintainers weigh in as proxies. Added a few in case they haven't followed this thread.
On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov < kirpic...@google.com.invalid> wrote: > Agreed that polling Dataflow users makes sense, though I think they are > very unlikely to have concerns, because unlike Spark/Flink users they > wouldn't have a "cluster" that they need to migrate to a new JVM: they'd > only need to recompile their pipelines with JDK 8. > > On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid> > wrote: > > > I think the Flink and Spark runner maintainers can weigh in here; given > > that both of those systems are moving to Java 8, I doubt they will have > > concerns. Same is true for the Dataflow runner - we should probably poll > > Dataflow users to make sure this is not a problem for any of them. > > > > On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov < > > kirpic...@google.com.invalid> wrote: > > > > > Reuven - do you mean e.g. a poll on the Flink mailing list asking > whether > > > there are Flink users who use Beam with Java 7? Or just people who use > > > Flink with Java 7? (the latter question I'd assume was settled by the > > poll > > > about making Flink itself Java8-only?) > > > > > > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid> > > > wrote: > > > > > > > I don't know if a vote in @user is sufficient, as it's unfortunately > > not > > > > representative of all Beam users. I think the runner communities > should > > > be > > > > polled as well (though I suspect the answer will be the same, that we > > can > > > > go ahead and deprecate Java 7 support). > > > > > > > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov < > > > > kirpic...@google.com.invalid> wrote: > > > > > > > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you > be > > > > > interested in driving this process, since you're already working on > > > Java9 > > > > > support and hence you have a good understanding of what's involved > > in a > > > > JDK > > > > > version migration for a large project? > > > > > > > > > > As due diligence, we can look at how the other data processing > > systems > > > > > handled dropping Java7. > > > > > > > > > > Flink: > > > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242 > > > > > Discussion > > > > > http://apache-flink-user-mailing-list-archive.2336050. > > > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html > > > > > > > > > > They also did a Twitter poll in addition to the mailing list poll, > > > which > > > > > seems like a good idea. > > > > > Note that Flink had a number of strong reasons for dropping Java7 > > that > > > do > > > > > not apply in Beam. > > > > > > > > > > Spark: > > > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493 > > > > > Discussion > > > > > > > > > http://apache-spark-developers-list.1001551.n3. > > > nabble.com/discuss-ending- > > > > > support-for-Java-7-in-Spark-2-0-td16808.html > > > > > (I > > > > > couldn't find a formal poll of the user list rather than developer > > > list) > > > > > > > > > > Hadoop: > > > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion > of > > > > where > > > > > that decision was made. > > > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f > 5439839 > > > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev. > > > > > hadoop.apache.org%3E > > > > > might > > > > > be it. > > > > > > > > > > Kafka is considering it: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 > > > > > and > > > > > quotes a number of other open-source projects that have switched > > > > > http://markmail.org/message/l7s276y3xkga2eqf > > > > > > > > > > So basically these projects all did a mailing list poll, and one > did > > > > also a > > > > > twitter poll. > > > > > > > > > > Beam has the advantage of being a relatively young project with > > > perhaps a > > > > > smaller base of users entrenched in using old versions of Java; > > > moreover, > > > > > Java version would matter only for the smaller subset of users who > > use > > > > Beam > > > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which > is > > > > likely > > > > > an even more "early adopter"-ish group of users, as these runners > > > > generally > > > > > receive less support. > > > > > > > > > > It may be a good idea to have at least 1 release pass between > > > announcing > > > > > the intention to drop Java8 and actually dropping it (e.g. if we > > > decided > > > > it > > > > > now, then 2.4 would drop Java7). Also, we could start by switching > > > tests > > > > to > > > > > compile/run with java8 (Maven allows this). This is, I think, > pretty > > > much > > > > > safe to do immediately. > > > > > > > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ieme...@gmail.com> > > > wrote: > > > > > > > > > > > Any progress on this? What is the proposed way to validate if > users > > > > > > are still interested on Java 7? A vote on user or something > > > different? > > > > > > > > > > > > > > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles > > > > <k...@google.com.invalid > > > > > > > > > > > > wrote: > > > > > > > Agree with polling Beam users as well as each runner's > community > > in > > > > > > > aggregate. > > > > > > > > > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré < > > > > j...@nanthrax.net > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Definitely agree > > > > > > >> > > > > > > >> > > > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote: > > > > > > >> > > > > > > >>> I also think that it's time to seriously consider dropping > > > support > > > > > for > > > > > > >>> Java 7. > > > > > > >>> > > > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira > > > > > > >>> <danolive...@google.com.invalid> wrote: > > > > > > >>> > > > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right > now > > > > > despite > > > > > > >>>> that > > > > > > >>>> (I believe) we don't use the extension that's breaking. > > > > > > >>>> > > > > > > >>>> As for other ways to solve this, if there is a way to avoid > > > > > compiling > > > > > > the > > > > > > >>>> advanced features of AutoValue that might be worth a try. We > > > could > > > > > > also > > > > > > >>>> try > > > > > > >>>> to get a release of AutoValue with the fix that works in > Java > > 7. > > > > > > However > > > > > > >>>> I > > > > > > >>>> feel that slowly moving over to Java 8 is the most > > future-proof > > > > > > solution > > > > > > >>>> if > > > > > > >>>> it's possible. > > > > > > >>>> > > > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía < > > > ieme...@gmail.com> > > > > > > wrote: > > > > > > >>>> > > > > > > >>>> The current issue is that compilation fails on master > because > > > > beam's > > > > > > >>>>> parent pom is configured to fail if it finds warnings): > > > > > > >>>>> > > > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag> > > > > > > >>>>> > > > > > > >>>>> However if you remove that line from the parent pom the > > > > compilation > > > > > > >>>>> passes. > > > > > > >>>>> > > > > > > >>>>> Of course this does not mean that everything is solved for > > Java > > > > 9, > > > > > > >>>>> there are some tests that break and other issues because of > > > other > > > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are > not > > > part > > > > > of > > > > > > >>>>> this discussion. > > > > > > >>>>> > > > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov > > > > > > >>>>> <kirpic...@google.com.invalid> wrote: > > > > > > >>>>> > > > > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. > > > Does > > > > > that > > > > > > >>>>>> mean > > > > > > >>>>>> this issue is moot? I didn't quite understand from your > > email > > > > > > whether > > > > > > >>>>>> it > > > > > > >>>>>> > > > > > > >>>>> is > > > > > > >>>>> > > > > > > >>>>>> a compilation blocker for Beam or not. > > > > > > >>>>>> > > > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía < > > > ieme...@gmail.com > > > > > > > > > > > >>>>>> wrote: > > > > > > >>>>>> > > > > > > >>>>>> Great that you are also working on this too Daniel and > > thanks > > > > for > > > > > > >>>>>>> bringing this subject to the mailing list, I was waiting > to > > > my > > > > > > return > > > > > > >>>>>>> to office next week, but you did it first :) > > > > > > >>>>>>> > > > > > > >>>>>>> Eugene for reference (This is the issue on the migration > to > > > > Java > > > > > > 9), > > > > > > >>>>>>> notice that here the goal is first that beam passes mvn > > clean > > > > > > install > > > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to > > > rewrite > > > > > > >>>>>>> anything to use the new stuff (e.g. modules): > > > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530 > > > > > > >>>>>>> > > > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more > > details > > > > on > > > > > > the > > > > > > >>>>>>> issue, this is a warning so I don't know if it is really > > > > critical > > > > > > in > > > > > > >>>>>>> particular because we are not using Memoization (do we?). > > > > > > >>>>>>> https://github.com/google/auto/issues/503 > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > > > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d > > > > > > >>>>>>> > > > > > > >>>>>> 02247b8532 > > > > > > >>>>> > > > > > > >>>>>> > > > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the > > > > > google.auto > > > > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible > way > > > and > > > > > > >>>>>>> 'voila' ? > > > > > > >>>>>>> > > > > > > >>>>>>> However I agree that moving to Java 8 is an excellent > idea > > > and > > > > as > > > > > > >>>>>>> Eugene mentions there is less friction now since most > > > projects > > > > > are > > > > > > >>>>>>> moving, only pending issue are existing clusters on java > 7 > > in > > > > the > > > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway > this > > > > > > discussion > > > > > > >>>>>>> is really important (maybe even worth a vote). Because > > moving > > > > to > > > > > > Java > > > > > > >>>>>>> 8 will allow us also to move some of the dependencies > that > > we > > > > are > > > > > > >>>>>>> keeping in old versions and in general to move forward. > > > > > > >>>>>>> > > > > > > >>>>>>> What do the others think ? > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> > > > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov > > > > > > >>>>>>> <kirpic...@google.com.invalid> wrote: > > > > > > >>>>>>> > > > > > > >>>>>>>> Very excited to hear that there's work on JDK9 support - > > is > > > > > there > > > > > > a > > > > > > >>>>>>>> > > > > > > >>>>>>> public > > > > > > >>>>>>> > > > > > > >>>>>>>> description of the plans for this work somewhere? > > > > > > >>>>>>>> > > > > > > >>>>>>>> In general, Beam could probably drop Java7 support > > > altogether > > > > at > > > > > > some > > > > > > >>>>>>>> > > > > > > >>>>>>> point > > > > > > >>>>>>> > > > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not > > receiving > > > > > even > > > > > > >>>>>>>> > > > > > > >>>>>>> security > > > > > > >>>>>>> > > > > > > >>>>>>>> patches) 2 years ago, and all major players in the data > > > > > processing > > > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, > > Hadoop), > > > > so > > > > > I > > > > > > >>>>>>>> > > > > > > >>>>>>> presume > > > > > > >>>>> > > > > > > >>>>>> the demand for Java7 support in the data processing > industry > > > is > > > > > low. > > > > > > >>>>>>>> > > > > > > >>>>>>> By > > > > > > >>>>> > > > > > > >>>>>> the > > > > > > >>>>>>> > > > > > > >>>>>>>> way: would a Java8 migration be in the scope of your > work > > in > > > > > > general? > > > > > > >>>>>>>> > > > > > > >>>>>>>> However, until we say that Beam requires Java8, what > would > > > be > > > > > the > > > > > > >>>>>>>> implications of using a version of AutoValue that can > only > > > be > > > > > > >>>>>>>> compiled > > > > > > >>>>>>>> > > > > > > >>>>>>> with > > > > > > >>>>>>> > > > > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a > > > > compiler > > > > > > bug, > > > > > > >>>>>>>> > > > > > > >>>>>>> and > > > > > > >>>>> > > > > > > >>>>>> if we use a Java8 compiler but configured to use source > and > > > > target > > > > > > >>>>>>>> > > > > > > >>>>>>> versions > > > > > > >>>>>>> > > > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then > > the > > > > > > resulting > > > > > > >>>>>>>> > > > > > > >>>>>>> Beam > > > > > > >>>>>>> > > > > > > >>>>>>>> artifacts will be usable by people who don't have Java8? > > > > > > >>>>>>>> > > > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira > > > > > > >>>>>>>> <danolive...@google.com.invalid> wrote: > > > > > > >>>>>>>> > > > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I > > have a > > > > bug > > > > > > in > > > > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue > > > > > dependency > > > > > > to > > > > > > >>>>>>>>> > > > > > > >>>>>>>> the > > > > > > >>>>> > > > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to > be > > > > banned > > > > > > >>>>>>>>> > > > > > > >>>>>>>> for > > > > > > >>>>> > > > > > > >>>>>> Beam > > > > > > >>>>>>> > > > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should > > still > > > be > > > > > > >>>>>>>>> > > > > > > >>>>>>>> possible > > > > > > >>>>> > > > > > > >>>>>> to > > > > > > >>>>>>> > > > > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler > > by > > > > > > building > > > > > > >>>>>>>>> > > > > > > >>>>>>>> with > > > > > > >>>>>>> > > > > > > >>>>>>>> the correct arguments. So the fix to this bug would > > > > essentially > > > > > > >>>>>>>>> > > > > > > >>>>>>>> require > > > > > > >>>>> > > > > > > >>>>>> Java 8 compilers even for compiling Java 7 code. > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if > not > > I > > > > > would > > > > > > >>>>>>>>> > > > > > > >>>>>>>> like to > > > > > > >>>>> > > > > > > >>>>>> continue with this fix. > > > > > > >>>>>>>>> > > > > > > >>>>>>>>> > > > > > > >>>>>>> > > > > > > >>>>> > > > > > > >> -- > > > > > > >> Jean-Baptiste Onofré > > > > > > >> jbono...@apache.org > > > > > > >> http://blog.nanthrax.net > > > > > > >> Talend - http://www.talend.com > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
