Reuven - do you mean e.g. a poll on the Flink mailing list asking whether there are Flink users who use Beam with Java 7? Or just people who use Flink with Java 7? (the latter question I'd assume was settled by the poll about making Flink itself Java8-only?)
On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid> wrote: > I don't know if a vote in @user is sufficient, as it's unfortunately not > representative of all Beam users. I think the runner communities should be > polled as well (though I suspect the answer will be the same, that we can > go ahead and deprecate Java 7 support). > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov < > kirpic...@google.com.invalid> wrote: > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be > > interested in driving this process, since you're already working on Java9 > > support and hence you have a good understanding of what's involved in a > JDK > > version migration for a large project? > > > > As due diligence, we can look at how the other data processing systems > > handled dropping Java7. > > > > Flink: > > JIRA https://issues.apache.org/jira/browse/FLINK-7242 > > Discussion > > http://apache-flink-user-mailing-list-archive.2336050. > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html > > > > They also did a Twitter poll in addition to the mailing list poll, which > > seems like a good idea. > > Note that Flink had a number of strong reasons for dropping Java7 that do > > not apply in Beam. > > > > Spark: > > JIRA https://issues.apache.org/jira/browse/SPARK-19493 > > Discussion > > > http://apache-spark-developers-list.1001551.n3.nabble.com/discuss-ending- > > support-for-Java-7-in-Spark-2-0-td16808.html > > (I > > couldn't find a formal poll of the user list rather than developer list) > > > > Hadoop: > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of > where > > that decision was made. > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839 > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev. > > hadoop.apache.org%3E > > might > > be it. > > > > Kafka is considering it: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 > > and > > quotes a number of other open-source projects that have switched > > http://markmail.org/message/l7s276y3xkga2eqf > > > > So basically these projects all did a mailing list poll, and one did > also a > > twitter poll. > > > > Beam has the advantage of being a relatively young project with perhaps a > > smaller base of users entrenched in using old versions of Java; moreover, > > Java version would matter only for the smaller subset of users who use > Beam > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is > likely > > an even more "early adopter"-ish group of users, as these runners > generally > > receive less support. > > > > It may be a good idea to have at least 1 release pass between announcing > > the intention to drop Java8 and actually dropping it (e.g. if we decided > it > > now, then 2.4 would drop Java7). Also, we could start by switching tests > to > > compile/run with java8 (Maven allows this). This is, I think, pretty much > > safe to do immediately. > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ieme...@gmail.com> wrote: > > > > > Any progress on this? What is the proposed way to validate if users > > > are still interested on Java 7? A vote on user or something different? > > > > > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles > <k...@google.com.invalid > > > > > > wrote: > > > > Agree with polling Beam users as well as each runner's community in > > > > aggregate. > > > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré < > j...@nanthrax.net > > > > > > > wrote: > > > > > > > >> Definitely agree > > > >> > > > >> > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote: > > > >> > > > >>> I also think that it's time to seriously consider dropping support > > for > > > >>> Java 7. > > > >>> > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira > > > >>> <danolive...@google.com.invalid> wrote: > > > >>> > > > >>>> Yes, just as Ismaël said it's a compilation blocker right now > > despite > > > >>>> that > > > >>>> (I believe) we don't use the extension that's breaking. > > > >>>> > > > >>>> As for other ways to solve this, if there is a way to avoid > > compiling > > > the > > > >>>> advanced features of AutoValue that might be worth a try. We could > > > also > > > >>>> try > > > >>>> to get a release of AutoValue with the fix that works in Java 7. > > > However > > > >>>> I > > > >>>> feel that slowly moving over to Java 8 is the most future-proof > > > solution > > > >>>> if > > > >>>> it's possible. > > > >>>> > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ieme...@gmail.com> > > > wrote: > > > >>>> > > > >>>> The current issue is that compilation fails on master because > beam's > > > >>>>> parent pom is configured to fail if it finds warnings): > > > >>>>> > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag> > > > >>>>> > > > >>>>> However if you remove that line from the parent pom the > compilation > > > >>>>> passes. > > > >>>>> > > > >>>>> Of course this does not mean that everything is solved for Java > 9, > > > >>>>> there are some tests that break and other issues because of other > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are not part > > of > > > >>>>> this discussion. > > > >>>>> > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov > > > >>>>> <kirpic...@google.com.invalid> wrote: > > > >>>>> > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does > > that > > > >>>>>> mean > > > >>>>>> this issue is moot? I didn't quite understand from your email > > > whether > > > >>>>>> it > > > >>>>>> > > > >>>>> is > > > >>>>> > > > >>>>>> a compilation blocker for Beam or not. > > > >>>>>> > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ieme...@gmail.com > > > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>> Great that you are also working on this too Daniel and thanks > for > > > >>>>>>> bringing this subject to the mailing list, I was waiting to my > > > return > > > >>>>>>> to office next week, but you did it first :) > > > >>>>>>> > > > >>>>>>> Eugene for reference (This is the issue on the migration to > Java > > > 9), > > > >>>>>>> notice that here the goal is first that beam passes mvn clean > > > install > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite > > > >>>>>>> anything to use the new stuff (e.g. modules): > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530 > > > >>>>>>> > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more details > on > > > the > > > >>>>>>> issue, this is a warning so I don't know if it is really > critical > > > in > > > >>>>>>> particular because we are not using Memoization (do we?). > > > >>>>>>> https://github.com/google/auto/issues/503 > > > >>>>>>> > > > >>>>>>> > > > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d > > > >>>>>>> > > > >>>>>> 02247b8532 > > > >>>>> > > > >>>>>> > > > >>>>>>> Wouldn't the easiest way be that you guys convince the > > google.auto > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible way and > > > >>>>>>> 'voila' ? > > > >>>>>>> > > > >>>>>>> However I agree that moving to Java 8 is an excellent idea and > as > > > >>>>>>> Eugene mentions there is less friction now since most projects > > are > > > >>>>>>> moving, only pending issue are existing clusters on java 7 in > the > > > >>>>>>> hadoop world, but those are less frequent now. Anyway this > > > discussion > > > >>>>>>> is really important (maybe even worth a vote). Because moving > to > > > Java > > > >>>>>>> 8 will allow us also to move some of the dependencies that we > are > > > >>>>>>> keeping in old versions and in general to move forward. > > > >>>>>>> > > > >>>>>>> What do the others think ? > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov > > > >>>>>>> <kirpic...@google.com.invalid> wrote: > > > >>>>>>> > > > >>>>>>>> Very excited to hear that there's work on JDK9 support - is > > there > > > a > > > >>>>>>>> > > > >>>>>>> public > > > >>>>>>> > > > >>>>>>>> description of the plans for this work somewhere? > > > >>>>>>>> > > > >>>>>>>> In general, Beam could probably drop Java7 support altogether > at > > > some > > > >>>>>>>> > > > >>>>>>> point > > > >>>>>>> > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving > > even > > > >>>>>>>> > > > >>>>>>> security > > > >>>>>>> > > > >>>>>>>> patches) 2 years ago, and all major players in the data > > processing > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), > so > > I > > > >>>>>>>> > > > >>>>>>> presume > > > >>>>> > > > >>>>>> the demand for Java7 support in the data processing industry is > > low. > > > >>>>>>>> > > > >>>>>>> By > > > >>>>> > > > >>>>>> the > > > >>>>>>> > > > >>>>>>>> way: would a Java8 migration be in the scope of your work in > > > general? > > > >>>>>>>> > > > >>>>>>>> However, until we say that Beam requires Java8, what would be > > the > > > >>>>>>>> implications of using a version of AutoValue that can only be > > > >>>>>>>> compiled > > > >>>>>>>> > > > >>>>>>> with > > > >>>>>>> > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a > compiler > > > bug, > > > >>>>>>>> > > > >>>>>>> and > > > >>>>> > > > >>>>>> if we use a Java8 compiler but configured to use source and > target > > > >>>>>>>> > > > >>>>>>> versions > > > >>>>>>> > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the > > > resulting > > > >>>>>>>> > > > >>>>>>> Beam > > > >>>>>>> > > > >>>>>>>> artifacts will be usable by people who don't have Java8? > > > >>>>>>>> > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira > > > >>>>>>>> <danolive...@google.com.invalid> wrote: > > > >>>>>>>> > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I have a > bug > > > in > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue > > dependency > > > to > > > >>>>>>>>> > > > >>>>>>>> the > > > >>>>> > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be > banned > > > >>>>>>>>> > > > >>>>>>>> for > > > >>>>> > > > >>>>>> Beam > > > >>>>>>> > > > >>>>>>>> due to requiring Java 8 compilers. However, it should still be > > > >>>>>>>>> > > > >>>>>>>> possible > > > >>>>> > > > >>>>>> to > > > >>>>>>> > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler by > > > building > > > >>>>>>>>> > > > >>>>>>>> with > > > >>>>>>> > > > >>>>>>>> the correct arguments. So the fix to this bug would > essentially > > > >>>>>>>>> > > > >>>>>>>> require > > > >>>>> > > > >>>>>> Java 8 compilers even for compiling Java 7 code. > > > >>>>>>>>> > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I > > would > > > >>>>>>>>> > > > >>>>>>>> like to > > > >>>>> > > > >>>>>> continue with this fix. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>> > > > >>>>> > > > >> -- > > > >> Jean-Baptiste Onofré > > > >> jbono...@apache.org > > > >> http://blog.nanthrax.net > > > >> Talend - http://www.talend.com > > > >> > > > > > >
