Okay, then let's go forward. Seems that we should: - Open a new poll on user@, in light of 2.2 having been released - Open a twitter poll - Tweet that there's also a poll going on on user@ - Runner authors will reach out to respective runner user communities - 2 weeks later we gather results and decide ?
On Wed, Dec 6, 2017 at 6:16 AM Ismaël Mejía <[email protected]> wrote: > +1 For Eugene’s arguments waiting for Beam 3.0 seems still far away, > and starting to improve Beam to offer a Java 8 friendly experience > seems like an excellent idea. > > I understand the backwards compatibility argument. We should do the > poll in twitter + try to reach more users for comments. If you > consider that it is worth, I can open a second poll at user@. > > In any case we should try to move forward, even if we have more than > 5% of users who want to stay on Java 7 we can consider to maintain > minor releases of a backwards compatible version where we can backport > only critical fixes e.g. security/data related errors but nothing new, > in case some user really needs to have them. Of course this can be > some extra work (to be discussed). > > > On Tue, Dec 5, 2017 at 7:24 AM, Jean-Baptiste Onofré <[email protected]> > wrote: > > +1, and sorry again, I thought we got an consensus. > > > > Regards > > JB > > > > On 12/05/2017 07:10 AM, Kenneth Knowles wrote: > >> > >> +1 to the poll and also to Reuven's point. > >> > >> Those without a support contract would have been using JDK 7 without > >> security updates for years. IMO it seems harmful, as a netizen, to > encourage > >> its use/existence. > >> > >> If there's no noise from the prior thread, then I would assume no one on > >> user@ has any objection. Anyone else with customers should reach out to > >> them. > >> > >> Kenn > >> > >> On Mon, Dec 4, 2017 at 9:49 PM, Reuven Lax <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Technically it's a backwards-incompatible change, however if we are > >> convinced the risk is low we could do it. > >> > >> As mentioned on the original thread, it's not clear that all Beam > >> users read > >> user@ - e.g. most Dataflow users definitely do not. I think we > need to > >> separately reach out to users of each runner through runner-specific > >> channels. > >> > >> Reuven > >> > >> On Mon, Dec 4, 2017 at 9:37 PM, Eugene Kirpichov < > [email protected] > >> <mailto:[email protected]>> wrote: > >> > >> On the original thread > >> > >> > https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E > >> > >> < > https://lists.apache.org/thread.html/2e1890c62d9f022f09b20e9f12f130fe9f1042e391979087f725d2e0@%3Cuser.beam.apache.org%3E > > > >> , > >> Robert and Ismaël were in favor of no major version change > [Ismaël > >> said:/Also I am afraid that if we wait/ > >> /until we have enough changes to switch Beam to a new major > >> version the > >> switch to Java 8 will happen too late, probably after Java 8's > end > >> of > >> life. And I am not exaggerating, Java 8 is planned to EOL next > >> march > >> 2018!/]; JB and now Reuven are in favor of a major version > change; > >> > >> nobody so far argued against switching to Java8 in general. > >> > >> I'm personally in favor of no major version change (i.e. not > >> waiting > >> until all other large changes for Beam 3.0 converge, which will > >> likely > >> be many months), because: > >> - Reasons Ismaël cited; plus the reason that most people are > >> likely > >> already using Java 8. > >> - Going Java8-only earlier will make other Beam 3.0 APIs better > >> for > >> Java8 users, because we (Beam contributors) will have experience > >> working > >> with them within the SDK in Java8 (e.g. writing tests with use > of > >> lambdas and noticing whether it's clunky, or whether some other > >> Beam > >> APIs need better Java8 support). > >> - Going Java8 will make it more reasonable to include (mostly or > >> only) > >> Java8 snippets in Beam documentation, which will obviously look > >> more > >> concise and attractive, addressing one of the common concerns of > >> Beam > >> users that it has a heavyweight API compared to functional-style > >> APIs of > >> Spark etc. > >> > >> I think resolving this via a poll of users would be reasonable. > >> I'd > >> suggest e.g. the following phrasing: > >> > >> Apache Beam is considering dropping support for Java 7, and > >> supporting > >> only Java 8 and above in a subsequent release. How would it > impact > >> your > >> usage of Beam? > >> - I am already using only Java 8+ for building my Beam code > >> - I am using Java 7 for building my Beam code, but I would have > no > >> trouble switching to Java 8 > >> - I am using Java 7 for building my Beam code, and dropping > Java 7 > >> would > >> be a blocker or hindrance to adopting the new release for me > >> > >> We could tweet this poll on Apache Beam twitter and publish on > >> user@, > >> and, say, if we receive 5% or fewer votes for option 3 after > >> keeping it > >> open for 2 weeks, then adopt Java 8 without a major version > >> change. > >> > >> WDYT? > >> > >> On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré > >> <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Good idea ! Definitely +1 > >> > >> Regards > >> JB > >> > >> On 12/05/2017 05:25 AM, Reuven Lax wrote: > >> > We should bring this up on the Beam 3.0 thread. Since > it's > >> technically a > >> > backwards-incompatible change, it might make a good item > >> for Beam > >> 3.0. > >> > > >> > Reuven > >> > > >> > On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré > >> <[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>>> > wrote: > >> > > >> > My apologizes, I thought we had a consensus already. > >> > > >> > Regards > >> > JB > >> > > >> > On 12/04/2017 11:22 PM, Eugene Kirpichov wrote: > >> > > >> > Thanks JB for sending the detailed notes about > new > >> stuff > >> in 2.2.0! A lot > >> > of exciting things indeed. > >> > > >> > Regarding Java 8: I thought our consensus was to > >> have the > >> release notes > >> > say that we're *considering* going Java8-only, > and > >> use > >> that to get more > >> > opinions from the user community - but I can't > find > >> the > >> emails that made > >> > me think so. > >> > > >> > +Ismaël Mejía <mailto:[email protected] > >> <mailto:[email protected]> <mailto:[email protected] > >> <mailto:[email protected]>>> - do > >> > you think we should formally conclude the vote on > >> the > >> thread [VOTE] > >> > [DISCUSSION] Remove support for Java 7? > >> > Or should we take more steps - e.g. perhaps > tweet a > >> link > >> to that thread > >> > from the Beam twitter account, ask people to > chime > >> in, > >> and wait for say > >> > 2 weeks before declaring a conclusion? > >> > > >> > Let's also have a process JIRA for going Java8. > >> I've > >> filed one: > >> > https://issues.apache.org/jira/browse/BEAM-3285 > >> <https://issues.apache.org/jira/browse/BEAM-3285> > >> > <https://issues.apache.org/jira/browse/BEAM-3285 > >> <https://issues.apache.org/jira/browse/BEAM-3285>> > >> > > >> > On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste > Onofré > >> <[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected] > >> > >> <mailto:[email protected] <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected] > >>>> > >> wrote: > >> > > >> > Just an important note that we forgot to > >> mention. > >> > > >> > !! The 2.2.0 release will be the last one > >> supporting > >> Spark 1.x and > >> > Java 7 !! > >> > > >> > Starting from Beam 2.3.0, the Spark runner > >> will work > >> only with > >> > Spark 2.x and we > >> > will focus only Java 8. > >> > > >> > Regards > >> > JB > >> > > >> > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré > >> wrote: > >> > > Thanks Reuven ! > >> > > > >> > > I would like to emphasize on some > >> highlights in > >> 2.2.0 release: > >> > > > >> > > - New IOs have been introduced: > >> > > * TikaIO leveraging Apache Tika, > allowing > >> the > >> deal with a lot > >> > of different > >> > > data formats > >> > > * RedisIO to read and write key/value > >> pairs > >> from a Redis > >> > server. This > >> > IO will > >> > > be soon extended to Redis PubSub. > >> > > * FileIO provides transforms for > working > >> with > >> files (raw). > >> > Especially, it > >> > > provides matching file patterns and read > on > >> patterns. It can be > >> > easily > >> > extended > >> > > for a specific format (like we do in > AvroIO > >> or > >> TextIO now). > >> > > * SolrIO to interact with Apache Solr > >> (Lucene) > >> > > > >> > > - On the other hand, improvements have > been > >> performed on > >> > existing IOs: > >> > > * We started to introduce readAll > pattern > >> in > >> IOs (AvroIO, > >> > TextIO, JdbcIO, > >> > > ...), allowing to pass "request" > arguments > >> via an > >> input PCollection. > >> > > * ElasticsearchIO has an improved > support > >> of > >> different > >> > Elasticsearch > >> > version > >> > > (including Elasticsearch 5.x). It also > now > >> supports SSL/TLS. > >> > > * HBaseIO is now able to do dynamic > work > >> rebalancing > >> > > * KinesisIO uses a more accurate > >> watermark > >> (based on > >> > approximateArrivalTimestamp) > >> > > * TextIO now supports custom delimiter > >> and like > >> AvroIO, > >> > supports the > >> > readAll > >> > > pattern, > >> > > * Performance improvements on JdbcIO > when > >> it > >> has to read lot > >> > of rows > >> > > * Kafka write supports Exactly-Once > >> pattern > >> (introduce in > >> > Kafka 0.11.x) > >> > > > >> > > - A new DSL has been introduced: the SQL > >> DSL ! > >> > > > >> > > We are now focus on 2.3.0 release with > new > >> improvements and > >> > features ! > >> > > > >> > > Stay tuned ! > >> > > > >> > > JB on behalf of the Apache Beam > community. > >> > > > >> > > On 12/02/2017 11:40 PM, Reuven Lax wrote: > >> > >> The Apache Beam community is pleased to > >> announce the > >> > availability of the > >> > >> 2.2.0 release. > >> > >> > >> > >> This release adds support for generic > file > >> sources and sinks > >> > (beyond TextIO > >> > >> and AvroIO) using FileIO, including > >> support for > >> dynamic > >> > filenames using > >> > >> readAll; this allows streaming pipelines > >> to now > >> read from files by > >> > >> continuously monitoring a directory for > >> new > >> filw. Many other > >> > IOs are > >> > improved, > >> > >> notably including exactly-once support > for > >> the > >> Kafka sink. Initial > >> > support for > >> > >> BEAM-SQL is also included in this > release. > >> For a > >> more-complete > >> > list of major > >> > >> changes in the release, please refer to > >> the > >> release notes [2]. > >> > >> > >> > >> The 2.2.0 release is now the recommended > >> version; we encourage > >> > everyone to > >> > >> upgrade from any earlier releases. > >> > >> > >> > >> We’d like to invite everyone to try out > >> Apache > >> Beam today and > >> > consider > >> > >> joining our vibrant community. We > welcome > >> feedback, > >> > contribution and > >> > >> participation through our mailing lists, > >> issue > >> tracker, pull > >> > requests, and > >> > >> events. > >> > >> > >> > >> - Reuven Lax, on behalf of the Apache > Beam > >> community. > >> > >> > >> > >> [1] > >> https://beam.apache.org/get-started/downloads/ > >> <https://beam.apache.org/get-started/downloads/> > >> > <https://beam.apache.org/get-started/downloads/ > >> <https://beam.apache.org/get-started/downloads/>> > >> > >> [2] > >> > >> > >> > > >> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > >> > >> < > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > > > >> > > >> < > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > >> < > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044 > >> > >> > >> > >> > > > >> > > >> > -- > >> > Jean-Baptiste Onofré > >> > [email protected] <mailto:[email protected]> > >> <mailto:[email protected] <mailto:[email protected]>> > >> > <mailto:[email protected] > >> <mailto:[email protected]> > >> <mailto:[email protected] <mailto:[email protected]>>> > >> > http://blog.nanthrax.net > >> > Talend - http://www.talend.com > >> > > >> > > >> > -- > >> > Jean-Baptiste Onofré > >> > [email protected] <mailto:[email protected]> > >> <mailto:[email protected] <mailto:[email protected]>> > >> > http://blog.nanthrax.net > >> > Talend - http://www.talend.com > >> > > >> > > >> > >> -- > >> Jean-Baptiste Onofré > >> [email protected] <mailto:[email protected]> > >> http://blog.nanthrax.net > >> Talend - http://www.talend.com > >> > >> > >> > > > > -- > > Jean-Baptiste Onofré > > [email protected] > > http://blog.nanthrax.net > > Talend - http://www.talend.com >
