Re: Need suggestion/help for use case (usage of the side input pattern and sliding window)

Cool ..Thanks again for you help/suggestions Luke. Regards Mohil On Tue, Jun 2, 2020 at 10:42 AM Luke Cwik wrote: > Using side inputs is fine and is a common pattern. You should take a look > at "slowly changing side inputs"[1] as there is some example code there. > > 1: >

Re: Pipeline Processing Time

Note: you need to tag the timestamp parameter to @ProcessElement with the @Timestamp annotation. On Mon, Jun 1, 2020 at 3:31 PM Luke Cwik wrote: > You can configure KafkaIO to use some data from the record as the elements > timestamp. See the KafkaIO javadoc around the TimestampPolicy[1], the >

Re: Error restoring Flink checkpoint

> KafkaIO or Protobuf. *I meant to say "Avro or Protobuf". On 03.06.20 18:00, Maximilian Michels wrote: > Hi Ivan, > > Moving to the new type serializer snapshot interface is not going to > solve this problem because we cannot version the coder through the Beam > coder interface. That is only

Re: Error restoring Flink checkpoint

Hi Ivan, Moving to the new type serializer snapshot interface is not going to solve this problem because we cannot version the coder through the Beam coder interface. That is only possible through Flink. However, it is usually not trivial. In Beam, when you evolve your data model, the only way

Re: Beam First Steps Workshop - 9 June

Awesome! On 02.06.20 22:09, Austin Bennett wrote: > Hi Beam Users, > > Wanted to share the Workshop that I'll give at Berlin Buzzword's next > week:   > https://berlinbuzzwords.de/session/first-steps-apache-beam-writing-portable-pipelines-using-java-python-go > > Do consider joining if you are

Error restoring Flink checkpoint

Hi, we have a Beam application running with Flink runner and we are struggling using Flink checkpoints. Everytime we evolve the source code modifying a Java model, an exception is thrown when trying to restore last checkpoint taken: Caused by: java.lang.IllegalStateException: Could not

Re: KafkaIO write in case on topic name present in PCollection

NP, I’m happy to hear that it works for you! > On 3 Jun 2020, at 01:37, Mohil Khare wrote: > > Hey Alexey, > > Thanks a lot for your quick response. This worked for me :). Awesome. > > Regards > Mohil > > > On Tue, Jun 2, 2020 at 6:31 AM Alexey Romanenko >