Re: BigTable reader for Python?

On Thu, Jul 28, 2022 at 4:51 PM Lina Mårtensson wrote: > Thanks for the detailed answers! > > I totally get the points about development & maintenance cost, and, > from a user perspective, about getting the performance right. > > I decided to try out the Spanner connector to get a sense of how

Re: BigTable reader for Python?

Thanks for the detailed answers! I totally get the points about development & maintenance cost, and, from a user perspective, about getting the performance right. I decided to try out the Spanner connector to get a sense of how well the x-language approach works in our world, since that's an

Re: [Release] 2.41.0 release update

Quick update for today: I'm still working through the validation tests, but we currently have 2 open issues: https://github.com/apache/beam/issues/22454 https://github.com/apache/beam/issues/22188 On Wed, Jul 27, 2022 at 5:03 PM Kiley Sok wrote: > Hi all, > > I've cut the release branch: >

Re: [RFC] State & Timers API Design for Go SDK

Looks great! On Thu, Jul 28, 2022 at 10:54 AM Jack McCluskey via dev wrote: > Great write-up on state and timers! The solution you chose feels very > in-line with how the Go SDK works. Make sure the design doc makes it onto > the wiki once you've addressed any feedback! > > On Thu, Jul 28, 2022

Re: [RFC] State & Timers API Design for Go SDK

Great write-up on state and timers! The solution you chose feels very in-line with how the Go SDK works. Make sure the design doc makes it onto the wiki once you've addressed any feedback! On Thu, Jul 28, 2022 at 1:49 PM Kerry Donny-Clark via dev < dev@beam.apache.org> wrote: > I think this a

Re: Runner benchmarks in portable mode

Your question isn't for me, but I just want to say that I am really happy to hear you are doing this. I would like to get more continuous benchmarking so we can reduce any overheads Beam might introduce, for example on Samza in your case. And I would like to basically focus entirely on portable

Re: [RFC] State & Timers API Design for Go SDK

I think this a perfect example of a clear design doc. Great, deeply detailed alternatives considered and why they were rejected. This makes review easy, and lets us follow your thought process. I think this is a good implementation, and I support the chosen approach. Kerry On Thu, Jul 28, 2022 at

Re: [RFC] State & Timers API Design for Go SDK

Really thorough. Love it! On Thu, Jul 28, 2022 at 9:02 AM Ritesh Ghorse via dev wrote: > Hey everyone, > > Danny and I have been working on > designing the state and timers for Go SDK. We wrote a design doc with > user-facing API, execution details, and different alternatives considered. > It

Beam Dependency Check Report (2022-07-28)

<<< text/html; charset=UTF-8: Unrecognized >>>

Re: Output after Pipeline replaceAll

Yes, this is expected. The goal of replaceAll is to replace a transform with a different subgraph that implements precisely the same semantics. And since the rest of the graph depends on the PCollections, the new expanded transform is wired directly to the old outputs. First point: Certainly the

Runner benchmarks in portable mode

Hi, We are currently working on making beam portable mode mainstream in addition to supporting classic mode for Samza runner. I was looking at OSS benchmarks on how other runners performed in portable mode in comparison with the classic mode. However, all I found was performance numbers and

Re: [ANNOUNCE] New committer: Steven Niemitz

Congrats Steve！ On Thu, Jul 28, 2022 at 9:46 AM Ahmet Altay via dev wrote: > Congratulations Steve! > > On Thu, Jul 21, 2022 at 10:31 AM Steve Niemitz via dev < > dev@beam.apache.org> wrote: > >> Thanks everyone! >> >> On Thu, Jul 21, 2022 at 2:23 AM Moritz Mack wrote: >> >>> Congrats, Steven!

Re: [ANNOUNCE] New committer: Steven Niemitz

Congratulations Steve! On Thu, Jul 21, 2022 at 10:31 AM Steve Niemitz via dev wrote: > Thanks everyone! > > On Thu, Jul 21, 2022 at 2:23 AM Moritz Mack wrote: > >> Congrats, Steven! >> >> >> >> On 21.07.22, 05:25, "Evan Galpin" wrote: >> >> >> >> Congrats! Well deserved! On Wed, Jul 20, 2022

[RFC] State & Timers API Design for Go SDK

Hey everyone, Danny and I have been working on designing the state and timers for Go SDK. We wrote a design doc with user-facing API, execution details, and different alternatives considered. It would be really helpful if we could get your suggestions/feedback/comments on the design. Design

Oracle Database Connection Pool creation from Beam

Hi Team, We have an issue with the Oracle connections being used up and we have tried to implement a pooled data source using PooledDataSourceFactory, somehow we are ending up with "Invalid Universal Connection Pool Configuration: oracle.ucp.UniversalConnectionPoolException: Universal

Re: Checkpoints timing out upgrading from Beam version 2.29 with Flink 1.12 to Beam 2.38 and Flink 1.14

Hi Sandeep, looking into the code, can you please elaborate on how the reading thread holds the lock for ever? From what I understand from the code the lock is released after each call to reader.advance(). Therefore the checkpoint should not be blocked "for ever". Am I missing something?

Beam High Priority Issue Report

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/22440 [Bug]: Python

Output after Pipeline replaceAll

Hi all, Wondering if somebody could help and shed some lights on the behavior of Pipeline.replaceAll, particularly the outputs to expect after the replacement. I’m currently looking into supporting VR tests for SparkRunner in streaming mode [1]. Unfortunately, I didn’t succeed replacing