Understanding watermark

2020-01-14 Thread Cam Mach
Hello Beam expert, We have a pipeline (run on Flink) that read both bounded and unbounded sources and our understanding is that when the bounded sources complete they should get a watermark of +inf and then we should be able to take a savepoint and safely restart the pipeline. However, we have

Re: [DISCUSS] AWS IOs V1 Deprecation Plan

2019-12-06 Thread Cam Mach
and document to encourage forks to use V2. Thanks again for your contributions to this thread Cam Mach On Fri, Nov 29, 2019 at 4:41 AM Cam Mach wrote: > Thanks Alex for the summary, and all for your contributes. > I have been waiting for a couple of days, and so far we don't have any > o

Re: [DISCUSS] AWS IOs V1 Deprecation Plan

2019-11-29 Thread Cam Mach
t;> they are ready to be stable in terms of API? Perhaps, this topic >>>>>> deserves a >>>>>> new discussion if there are several opinions on that. >>>>>> >>>>> >> Probably, decision to move component APIs (for example, an IO con

Re: [DISCUSS] AWS IOs V1 Deprecation Plan

2019-11-26 Thread Cam Mach
e marked > experimental which will be a problem for Phase 2. > > What is the feasibility of making the V1 transforms wrappers around V2? > > On Mon, Nov 25, 2019 at 1:46 PM Cam Mach wrote: > >> Hello Beam Devs, >> >> I have been working on the migration of Ama

[DISCUSS] AWS IOs V1 Deprecation Plan

2019-11-25 Thread Cam Mach
a notification to the community 3 months before we remove them - Completely get rid of V1 IOs Please let me know what you think or if you see any potential issues? Thanks, Cam Mach

Re: DynamoDBIO related issue

2019-10-24 Thread Cam Mach
Hi Pradeep, What is your enhancement? Can you create a ticket and describe it? Thanks, Cam On Thu, Oct 24, 2019 at 1:58 PM Pradeep Bhosale < bhosale.pradeep1...@gmail.com> wrote: > Hi, > > This is Pradeep. I am using DynamoDB IO to write data to dynamo DB. > I would like to report one

Re: [VOTE] Sign a pledge to discontinue support of Python 2 in 2020.

2019-10-01 Thread Cam Mach
+1 On Tue, Oct 1, 2019 at 9:44 AM Udi Meiri wrote: > +1 > > On Tue, Oct 1, 2019 at 3:22 AM Łukasz Gajowy > wrote: > >> +1 >> >> wt., 1 paź 2019 o 11:29 Maximilian Michels napisał(a): >> >>> +1 >>> >>> On 30.09.19 23:03, Reza Rokni wrote: >>> > +1 >>> > >>> > On Tue, 1 Oct 2019 at 13:54,

Beam KinesisIO Migration V1 to V2

2019-09-30 Thread Cam Mach
Hello Beam Dev, I have discussed with a couple of Beam dev regarding this topic. We found something interesting in the new AWS Kinesis SDK and Libraries V2, so like to propose a design for this migration. Here is the design doc:

Re: How to run DynamoDBIOTest?

2019-07-20 Thread Cam Mach
Hi Anton, It should not be a machine issue, since you have the container got started. I would say give it around 5 - 10 minutes, it would throw out connection time out error log. This usually happens when the integration tests can't connect to the container. As you may notice, in the tests we