I'm super excited about this release! Great work everyone involved! On Mon, Dec 4, 2017 at 10:58 AM, Jean-Baptiste Onofré <[email protected]> wrote:
> Just an important note that we forgot to mention. > > !! The 2.2.0 release will be the last one supporting Spark 1.x and Java 7 > !! > > Starting from Beam 2.3.0, the Spark runner will work only with Spark 2.x > and we will focus only Java 8. > > Regards > JB > > > On 12/04/2017 10:15 AM, Jean-Baptiste Onofré wrote: > >> Thanks Reuven ! >> >> I would like to emphasize on some highlights in 2.2.0 release: >> >> - New IOs have been introduced: >> * TikaIO leveraging Apache Tika, allowing the deal with a lot of >> different data formats >> * RedisIO to read and write key/value pairs from a Redis server. This >> IO will be soon extended to Redis PubSub. >> * FileIO provides transforms for working with files (raw). Especially, >> it provides matching file patterns and read on patterns. It can be easily >> extended for a specific format (like we do in AvroIO or TextIO now). >> * SolrIO to interact with Apache Solr (Lucene) >> >> - On the other hand, improvements have been performed on existing IOs: >> * We started to introduce readAll pattern in IOs (AvroIO, TextIO, >> JdbcIO, ...), allowing to pass "request" arguments via an input PCollection. >> * ElasticsearchIO has an improved support of different Elasticsearch >> version (including Elasticsearch 5.x). It also now supports SSL/TLS. >> * HBaseIO is now able to do dynamic work rebalancing >> * KinesisIO uses a more accurate watermark (based on >> approximateArrivalTimestamp) >> * TextIO now supports custom delimiter and like AvroIO, supports the >> readAll pattern, >> * Performance improvements on JdbcIO when it has to read lot of rows >> * Kafka write supports Exactly-Once pattern (introduce in Kafka 0.11.x) >> >> - A new DSL has been introduced: the SQL DSL ! >> >> We are now focus on 2.3.0 release with new improvements and features ! >> >> Stay tuned ! >> >> JB on behalf of the Apache Beam community. >> >> On 12/02/2017 11:40 PM, Reuven Lax wrote: >> >>> The Apache Beam community is pleased to announce the availability of the >>> 2.2.0 release. >>> >>> This release adds support for generic file sources and sinks (beyond >>> TextIO and AvroIO) using FileIO, including support for dynamic filenames >>> using readAll; this allows streaming pipelines to now read from files by >>> continuously monitoring a directory for new filw. Many other IOs are >>> improved, notably including exactly-once support for the Kafka sink. >>> Initial support for BEAM-SQL is also included in this release. For a >>> more-complete list of major changes in the release, please refer to the >>> release notes [2]. >>> >>> The 2.2.0 release is now the recommended version; we encourage everyone >>> to >>> upgrade from any earlier releases. >>> >>> We’d like to invite everyone to try out Apache Beam today and consider >>> joining our vibrant community. We welcome feedback, contribution and >>> participation through our mailing lists, issue tracker, pull requests, >>> and >>> events. >>> >>> - Reuven Lax, on behalf of the Apache Beam community. >>> >>> [1] https://beam.apache.org/get-started/downloads/ >>> [2] >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje >>> ctId=12319527&version=12341044 >>> >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
