Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
Hi Sriram, You can read and modify savepoints using StateProcessor API [1]. Alternatively, you can modify a code of your function/operator for which you want to modify the state. For example in the `org.apache.flink.streaming.api.checkpoint.CheckpointedFunction#initializeState` method you could

Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
to use StateProcessor API. Best, Piotrek pt., 21 paź 2022 o 10:54 Sriram Ganesh napisał(a): > Thanks !. Will try this. > > On Fri, Oct 21, 2022 at 2:22 PM Piotr Nowojski > wrote: > >> Hi Sriram, >> >> You can read and modify savepoints using StateProcessor API [1]. >

Re: Modify savepoints in Flink

2022-10-21 Thread Piotr Nowojski
ent connector can have different > serialisation and de-serlisation technique right?. Wouldn't that impact?. > If I use StateProcessor API, would that be agnostic to all the sources and > sinks?. > > On Fri, Oct 21, 2022, 18:00 Piotr Nowojski wrote: > >> ops >> >>

Re: Kafka transactions drastically limit usability of Flink savepoints

2022-11-17 Thread Piotr Nowojski
:", however this is not what I observe, instead the job fails. I > am attaching the relevant part of the log. This error happens upon > trying to recover from a one month old savepoint. > > Regards, > Yordan > > On Tue, 15 Nov 2022 at 18:53, Piotr Nowojski wrote: > > >

Re: Kafka transactions drastically limit usability of Flink savepoints

2022-11-15 Thread Piotr Nowojski
Hi Yordan, I don't understand where the problem is, why do you think savepoints are unusable? If you recover with `ignoreFailuresAfterTransactionTimeout` enabled, the current Flink behaviour shouldn't cause any problems (except for maybe some logged errors). Best, Piotrek wt., 15 lis 2022 o

Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

2023-03-30 Thread Piotr Nowojski
Hey, > 1. The Flink community agrees that we upgrade Kryo to a later version, which means breaking all checkpoint/savepoint compatibility and releasing a Flink 2.0 with Java 17 support added and Java 8 and Flink Scala API support dropped. This is probably the quickest way, but would still mean

Re: java.lang.StackOverflowError

2020-01-23 Thread Piotr Nowojski
Hi, Thanks for reporting the issue. Could you first try to upgrade to Flink 1.6.4? This might be a known issue fixed in a later bug fix release [1]. Also, are you sure you are using (unmodified) Flink 1.6.2? Stack traces somehow do not match with the 1.6.2 release tag in the repository, for

<    2   3   4   5   6   7