Re: Local Test Beam Portable Flink Runner in mac

Hi Lydia, The LOOPBACK environment is nothing but an EXTERNAL environment with automated setup of SDK Harness process manager in the pipeline submission process. The LOOPBACK environment setup a server on a random port to start the SDK Harness process on a random port. This is what you are

Re: [KafkaIO] Use of sinkGroupId with Exactly Once Semantics

Thanks for the input folks! Sounds like a singular value across my application (despite that application writing to multiple topics) is what I want. On Fri, Nov 11, 2022 at 1:34 PM Byron Ellis via user wrote: > The Kafka consumer offset key >

Re: Reading from AWS Kinesis Stream Cross account

If I’m not mistaken, it’s not supported in the current implementation of KinesisIO. PS: Btw, if you still use KinesisIO connector based on AWS SDK v1 [1] then it’s highly recommended to switch to one based on AWS SDK v2 [2] since former is deprecated. [1]

Local Test Beam Portable Flink Runner in mac

Hi, I am working on implementing a local beam flink runner for faster development. I have made the docker image which contains the required flink and Beam dependencies, and then launched different containers:Job manager, task manager and beam job server, via docker-compose. I am using the bridge