Thanks for the headsup.
On Tue, Aug 13, 2019 at 3:08 AM Maximilian Michels wrote:
> Dear users of the Flink Runner,
>
> Just a heads-up that we may remove Flink 1.5/1.6 support for future
> version of Beam (>=2.16.0): https://jira.apache.org/jira/browse/BEAM-7962
>
> Why? We support 1.5 - 1.8 a
Ok I think I have an understanding of what happens - somehow.
Flink switched their RocksDB fork in the 1.8 release, this is why the
dependency must now be explicitly added to a project. [0]
I did both actually, adding this dependency to my projects pom (resulting
in beam_pipelines.jar) and to the l
This is a major issue for us as we are no longer able to do a
clean-shutdown of the pipelines right now - only cancelling them hard is
possible.
On Tue, Aug 13, 2019 at 2:46 PM Kaymak, Tobias
wrote:
> I just rolled out the upgraded and working 1.8.0/2.14.0 combination to
> production and noticed
I just rolled out the upgraded and working 1.8.0/2.14.0 combination to
production and noticed that when I try to cleanly shutdown a pipeline via
the stop button in the web-interface of Flink 1.8.0 I get exactly the same
error:
java.lang.NoSuchMethodError:
org.rocksdb.ColumnFamilyHandle.getDescript
(Better late than never) Thank you Jeff !
I could manage the throughput for Kafka to Avro files on GCS by changing
the worker disk type to SSD, disk size of 100 GB was enough.
Even with these changes, couldn't manage the throughput for direct writes
to Bigquery using FILE_LOADS.
Best
Ziyad
On F
Dear users of the Flink Runner,
Just a heads-up that we may remove Flink 1.5/1.6 support for future
version of Beam (>=2.16.0): https://jira.apache.org/jira/browse/BEAM-7962
Why? We support 1.5 - 1.8 and have a 1.9 version in the making. The
increased number of supported versions manifests in a m
Hello everyone,
Latest release of Apache Beam (2.14.0) has announced [1] support for Python
3.6, 3.7, and we have made several improvements [2] since 2.11.0 when
support of Python 3.5 was first announced.
I would like to encourage Beam Python users to start using Beam on Python 3
for the followin
