+1 For Eugene’s arguments waiting for Beam 3.0 seems still far away, and starting to improve Beam to offer a Java 8 friendly experience seems like an excellent idea.
I understand the backwards compatibility argument. We should do the poll in twitter + try to reach more users for comments. If you consider that it is worth, I can open a second poll at user@. In any case we should try to move forward, even if we have more than 5% of users who want to stay on Java 7 we can consider to maintain minor releases of a backwards compatible version where we can backport only critical fixes e.g. security/data related errors but nothing new, in case some user really needs to have them. Of course this can be some extra work (to be discussed). On Tue, Dec 5, 2017 at 7:24 AM, Jean-Baptiste Onofré <[email protected]> wrote: > +1, and sorry again, I thought we got an consensus. > > Regards > JB > > On 12/05/2017 07:10 AM, Kenneth Knowles wrote: >> >> +1 to the poll and also to Reuven's point. >> >> Those without a support contract would have been using JDK 7 without >> security updates for years. IMO it seems harmful, as a netizen, to encourage >> its use/existence. >> >> If there's no noise from the prior thread, then I would assume no one on >> user@ has any objection. Anyone else with customers should reach out to >> them. >> >> Kenn >> >> On Mon, Dec 4, 2017 at 9:49 PM, Reuven Lax <[email protected] >> <mailto:[email protected]>> wrote: >> >> Technically it's a backwards-incompatible change, however if we are >> convinced the risk is low we could do it. >> >> As mentioned on the original thread, it's not clear that all Beam >> users read >> user@ - e.g. most Dataflow users definitely do not. I think we need to >> separately reach out to users of each runner through runner-specific >> channels. >> >> Reuven >> >> On Mon, Dec 4, 2017 at 9:37 PM, Eugene Kirpichov <[email protected] >> <mailto:[email protected]>> wrote: >> >> On the original thread >> >> https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E >> >> <https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E> >> , >> Robert and Ismaël were in favor of no major version change [Ismaël >> said:/Also I am afraid that if we wait/ >> /until we have enough changes to switch Beam to a new major >> version the >> switch to Java 8 will happen too late, probably after Java 8's end >> of >> life. And I am not exaggerating, Java 8 is planned to EOL next >> march >> 2018!/]; JB and now Reuven are in favor of a major version change; >> >> nobody so far argued against switching to Java8 in general. >> >> I'm personally in favor of no major version change (i.e. not >> waiting >> until all other large changes for Beam 3.0 converge, which will >> likely >> be many months), because: >> - Reasons Ismaël cited; plus the reason that most people are >> likely >> already using Java 8. >> - Going Java8-only earlier will make other Beam 3.0 APIs better >> for >> Java8 users, because we (Beam contributors) will have experience >> working >> with them within the SDK in Java8 (e.g. writing tests with use of >> lambdas and noticing whether it's clunky, or whether some other >> Beam >> APIs need better Java8 support). >> - Going Java8 will make it more reasonable to include (mostly or >> only) >> Java8 snippets in Beam documentation, which will obviously look >> more >> concise and attractive, addressing one of the common concerns of >> Beam >> users that it has a heavyweight API compared to functional-style >> APIs of >> Spark etc. >> >> I think resolving this via a poll of users would be reasonable. >> I'd >> suggest e.g. the following phrasing: >> >> Apache Beam is considering dropping support for Java 7, and >> supporting >> only Java 8 and above in a subsequent release. How would it impact >> your >> usage of Beam? >> - I am already using only Java 8+ for building my Beam code >> - I am using Java 7 for building my Beam code, but I would have no >> trouble switching to Java 8 >> - I am using Java 7 for building my Beam code, and dropping Java 7 >> would >> be a blocker or hindrance to adopting the new release for me >> >> We could tweet this poll on Apache Beam twitter and publish on >> user@, >> and, say, if we receive 5% or fewer votes for option 3 after >> keeping it >> open for 2 weeks, then adopt Java 8 without a major version >> change. >> >> WDYT? >> >> On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré >> <[email protected] >> <mailto:[email protected]>> wrote: >> >> Good idea ! Definitely +1 >> >> Regards >> JB >> >> On 12/05/2017 05:25 AM, Reuven Lax wrote: >> > We should bring this up on the Beam 3.0 thread. Since it's >> technically a >> > backwards-incompatible change, it might make a good item >> for Beam >> 3.0. >> > >> > Reuven >> > >> > On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré >> <[email protected] <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>>> wrote: >> > >> > My apologizes, I thought we had a consensus already. >> > >> > Regards >> > JB >> > >> > On 12/04/2017 11:22 PM, Eugene Kirpichov wrote: >> > >> > Thanks JB for sending the detailed notes about new >> stuff >> in 2.2.0! A lot >> > of exciting things indeed. >> > >> > Regarding Java 8: I thought our consensus was to >> have the >> release notes >> > say that we're *considering* going Java8-only, and >> use >> that to get more >> > opinions from the user community - but I can't find >> the >> emails that made >> > me think so. >> > >> > +Ismaël Mejía <mailto:[email protected] >> <mailto:[email protected]> <mailto:[email protected] >> <mailto:[email protected]>>> - do >> > you think we should formally conclude the vote on >> the >> thread [VOTE] >> > [DISCUSSION] Remove support for Java 7? >> > Or should we take more steps - e.g. perhaps tweet a >> link >> to that thread >> > from the Beam twitter account, ask people to chime >> in, >> and wait for say >> > 2 weeks before declaring a conclusion? >> > >> > Let's also have a process JIRA for going Java8. >> I've >> filed one: >> > https://issues.apache.org/jira/browse/BEAM-3285 >> <https://issues.apache.org/jira/browse/BEAM-3285> >> > <https://issues.apache.org/jira/browse/BEAM-3285 >> <https://issues.apache.org/jira/browse/BEAM-3285>> >> > >> > On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré >> <[email protected] <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>> >> <mailto:[email protected] <mailto:[email protected]> >> > <mailto:[email protected] >> <mailto:[email protected]>>>> >> wrote: >> > >> > Just an important note that we forgot to >> mention. >> > >> > !! The 2.2.0 release will be the last one >> supporting >> Spark 1.x and >> > Java 7 !! >> > >> > Starting from Beam 2.3.0, the Spark runner >> will work >> only with >> > Spark 2.x and we >> > will focus only Java 8. >> > >> > Regards >> > JB >> > >> > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré >> wrote: >> > > Thanks Reuven ! >> > > >> > > I would like to emphasize on some >> highlights in >> 2.2.0 release: >> > > >> > > - New IOs have been introduced: >> > > * TikaIO leveraging Apache Tika, allowing >> the >> deal with a lot >> > of different >> > > data formats >> > > * RedisIO to read and write key/value >> pairs >> from a Redis >> > server. This >> > IO will >> > > be soon extended to Redis PubSub. >> > > * FileIO provides transforms for working >> with >> files (raw). >> > Especially, it >> > > provides matching file patterns and read on >> patterns. It can be >> > easily >> > extended >> > > for a specific format (like we do in AvroIO >> or >> TextIO now). >> > > * SolrIO to interact with Apache Solr >> (Lucene) >> > > >> > > - On the other hand, improvements have been >> performed on >> > existing IOs: >> > > * We started to introduce readAll pattern >> in >> IOs (AvroIO, >> > TextIO, JdbcIO, >> > > ...), allowing to pass "request" arguments >> via an >> input PCollection. >> > > * ElasticsearchIO has an improved support >> of >> different >> > Elasticsearch >> > version >> > > (including Elasticsearch 5.x). It also now >> supports SSL/TLS. >> > > * HBaseIO is now able to do dynamic work >> rebalancing >> > > * KinesisIO uses a more accurate >> watermark >> (based on >> > approximateArrivalTimestamp) >> > > * TextIO now supports custom delimiter >> and like >> AvroIO, >> > supports the >> > readAll >> > > pattern, >> > > * Performance improvements on JdbcIO when >> it >> has to read lot >> > of rows >> > > * Kafka write supports Exactly-Once >> pattern >> (introduce in >> > Kafka 0.11.x) >> > > >> > > - A new DSL has been introduced: the SQL >> DSL ! >> > > >> > > We are now focus on 2.3.0 release with new >> improvements and >> > features ! >> > > >> > > Stay tuned ! >> > > >> > > JB on behalf of the Apache Beam community. >> > > >> > > On 12/02/2017 11:40 PM, Reuven Lax wrote: >> > >> The Apache Beam community is pleased to >> announce the >> > availability of the >> > >> 2.2.0 release. >> > >> >> > >> This release adds support for generic file >> sources and sinks >> > (beyond TextIO >> > >> and AvroIO) using FileIO, including >> support for >> dynamic >> > filenames using >> > >> readAll; this allows streaming pipelines >> to now >> read from files by >> > >> continuously monitoring a directory for >> new >> filw. Many other >> > IOs are >> > improved, >> > >> notably including exactly-once support for >> the >> Kafka sink. Initial >> > support for >> > >> BEAM-SQL is also included in this release. >> For a >> more-complete >> > list of major >> > >> changes in the release, please refer to >> the >> release notes [2]. >> > >> >> > >> The 2.2.0 release is now the recommended >> version; we encourage >> > everyone to >> > >> upgrade from any earlier releases. >> > >> >> > >> We’d like to invite everyone to try out >> Apache >> Beam today and >> > consider >> > >> joining our vibrant community. We welcome >> feedback, >> > contribution and >> > >> participation through our mailing lists, >> issue >> tracker, pull >> > requests, and >> > >> events. >> > >> >> > >> - Reuven Lax, on behalf of the Apache Beam >> community. >> > >> >> > >> [1] >> https://beam.apache.org/get-started/downloads/ >> <https://beam.apache.org/get-started/downloads/> >> > <https://beam.apache.org/get-started/downloads/ >> <https://beam.apache.org/get-started/downloads/>> >> > >> [2] >> > >> >> > >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 >> >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044> >> > >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 >> <https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044>> >> > >> >> > > >> > >> > -- >> > Jean-Baptiste Onofré >> > [email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>> >> > <mailto:[email protected] >> <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>> >> > http://blog.nanthrax.net >> > Talend - http://www.talend.com >> > >> > >> > -- >> > Jean-Baptiste Onofré >> > [email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>> >> > http://blog.nanthrax.net >> > Talend - http://www.talend.com >> > >> > >> >> -- >> Jean-Baptiste Onofré >> [email protected] <mailto:[email protected]> >> http://blog.nanthrax.net >> Talend - http://www.talend.com >> >> >> > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com
