Re: Checkpoints timing out upgrading from Beam version 2.29 with Flink 1.12 to Beam 2.38 and Flink 1.14

2022-07-26 Thread Kenneth Knowles
Bumping this and adding +John Casey who knows about KafkaIO and unbounded sources, though probably less about the FlinkRunner. It seems you have isolated it to the Flink translation logic. I'm not sure who would be the best expert to evaluate if that logic is still OK. Kenn On Wed, Jun 29, 2022

Re: [CDAP IO] SparkReceiverIO integration testing

2022-07-26 Thread Chamikara Jayalath via dev
On Tue, Jul 26, 2022 at 11:24 AM Elizaveta Lomteva < elizaveta.lomt...@akvelon.com> wrote: > Hi, community! > Our team has prepared SparkReceiverIO Read via SDF PR [1]. We have started > working on integration tests for the SparkReceiverIO connector which will > allow to read data from Custom

Re: BigTable reader for Python?

2022-07-26 Thread Sachin Agarwal via dev
On Tue, Jul 26, 2022 at 6:12 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > > > On Mon, Jul 25, 2022 at 12:53 PM Lina Mårtensson via dev < > dev@beam.apache.org> wrote: > >> Hi dev, >> >> We're starting to incorporate BigTable in our stack and I've delighted >> my co-workers with

Re: BigTable reader for Python?

2022-07-26 Thread Chamikara Jayalath via dev
On Mon, Jul 25, 2022 at 12:53 PM Lina Mårtensson via dev < dev@beam.apache.org> wrote: > Hi dev, > > We're starting to incorporate BigTable in our stack and I've delighted > my co-workers with how easy it was to create some BigTables with > Beam... but there doesn't appear to be a reader for

[CDAP IO] SparkReceiverIO integration testing

2022-07-26 Thread Elizaveta Lomteva
Hi, community! Our team has prepared SparkReceiverIO Read via SDF PR [1]. We have started working on integration tests for the SparkReceiverIO connector which will allow to read data from Custom Spark Receivers in Apache Beam pipeline. A general Apache Beam recommendation is to implement “

Beam High Priority Issue Report

2022-07-26 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/22440 [Bug]: Python