Good idea ! Definitely +1
Regards
JB
On 12/05/2017 05:25 AM, Reuven Lax wrote:
We should bring this up on the Beam 3.0 thread. Since it's technically a
backwards-incompatible change, it might make a good item for Beam 3.0.
Reuven
On Mon, Dec 4, 2017 at 8:20 PM, Jean-Baptiste Onofré <[email protected]
<mailto:[email protected]>> wrote:
My apologizes, I thought we had a consensus already.
Regards
JB
On 12/04/2017 11:22 PM, Eugene Kirpichov wrote:
Thanks JB for sending the detailed notes about new stuff in 2.2.0! A lot
of exciting things indeed.
Regarding Java 8: I thought our consensus was to have the release notes
say that we're *considering* going Java8-only, and use that to get more
opinions from the user community - but I can't find the emails that made
me think so.
+Ismaël Mejía <mailto:[email protected] <mailto:[email protected]>> - do
you think we should formally conclude the vote on the thread [VOTE]
[DISCUSSION] Remove support for Java 7?
Or should we take more steps - e.g. perhaps tweet a link to that thread
from the Beam twitter account, ask people to chime in, and wait for say
2 weeks before declaring a conclusion?
Let's also have a process JIRA for going Java8. I've filed one:
https://issues.apache.org/jira/browse/BEAM-3285
<https://issues.apache.org/jira/browse/BEAM-3285>
On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré <[email protected]
<mailto:[email protected]> <mailto:[email protected]
<mailto:[email protected]>>> wrote:
Just an important note that we forgot to mention.
!! The 2.2.0 release will be the last one supporting Spark 1.x and
Java 7 !!
Starting from Beam 2.3.0, the Spark runner will work only with
Spark 2.x and we
will focus only Java 8.
Regards
JB
On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote:
> Thanks Reuven !
>
> I would like to emphasize on some highlights in 2.2.0 release:
>
> - New IOs have been introduced:
> * TikaIO leveraging Apache Tika, allowing the deal with a lot
of different
> data formats
> * RedisIO to read and write key/value pairs from a Redis
server. This
IO will
> be soon extended to Redis PubSub.
> * FileIO provides transforms for working with files (raw).
Especially, it
> provides matching file patterns and read on patterns. It can be
easily
extended
> for a specific format (like we do in AvroIO or TextIO now).
> * SolrIO to interact with Apache Solr (Lucene)
>
> - On the other hand, improvements have been performed on
existing IOs:
> * We started to introduce readAll pattern in IOs (AvroIO,
TextIO, JdbcIO,
> ...), allowing to pass "request" arguments via an input
PCollection.
> * ElasticsearchIO has an improved support of different
Elasticsearch
version
> (including Elasticsearch 5.x). It also now supports SSL/TLS.
> * HBaseIO is now able to do dynamic work rebalancing
> * KinesisIO uses a more accurate watermark (based on
approximateArrivalTimestamp)
> * TextIO now supports custom delimiter and like AvroIO,
supports the
readAll
> pattern,
> * Performance improvements on JdbcIO when it has to read lot
of rows
> * Kafka write supports Exactly-Once pattern (introduce in
Kafka 0.11.x)
>
> - A new DSL has been introduced: the SQL DSL !
>
> We are now focus on 2.3.0 release with new improvements and
features !
>
> Stay tuned !
>
> JB on behalf of the Apache Beam community.
>
> On 12/02/2017 11:40 PM, Reuven Lax wrote:
>> The Apache Beam community is pleased to announce the
availability of the
>> 2.2.0 release.
>>
>> This release adds support for generic file sources and sinks
(beyond TextIO
>> and AvroIO) using FileIO, including support for dynamic
filenames using
>> readAll; this allows streaming pipelines to now read from
files by
>> continuously monitoring a directory for new filw. Many other
IOs are
improved,
>> notably including exactly-once support for the Kafka sink.
Initial
support for
>> BEAM-SQL is also included in this release. For a more-complete
list of major
>> changes in the release, please refer to the release notes [2].
>>
>> The 2.2.0 release is now the recommended version; we encourage
everyone to
>> upgrade from any earlier releases.
>>
>> We’d like to invite everyone to try out Apache Beam today and
consider
>> joining our vibrant community. We welcome feedback,
contribution and
>> participation through our mailing lists, issue tracker, pull
requests, and
>> events.
>>
>> - Reuven Lax, on behalf of the Apache Beam community.
>>
>> [1] https://beam.apache.org/get-started/downloads/
<https://beam.apache.org/get-started/downloads/>
>> [2]
>>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044
<https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044>
>>
>
--
Jean-Baptiste Onofré
[email protected] <mailto:[email protected]>
<mailto:[email protected] <mailto:[email protected]>>
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected] <mailto:[email protected]>
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
