GitHub user sangkyoonnam added a comment to the discussion: Planning Flink Agents 1.0 - Brainstorming
I tried the upgrade path from your clarification with the Kafka store, and on 0.4 I couldn't find a way to restore a savepoint without reusing the old ActionState journal while a store stays configured. With an operator test harness against a real Kafka broker (testcontainers, on `main` and on `release-0.4.0-rc1`), I took a checkpoint, completed one action, and in separate same-version runs restored the checkpoint four ways, resubmitting the input after each successful restore: - Same topic: the restore replays the old journal, so the action isn't called again. - New topic: the restore fails in `KafkaActionStateStore.validateVersionedMarker` with "recovery marker references topic …, but the configured topic is …". - Same topic, deleted and recreated: the restore fails with "… the topic may have been recreated". - No store: the restore works and the action runs again, but the upgraded job has no durable execution. The checks are deliberate, since they keep a job from silently losing records, and `DurableExecutionManager.handleRecovery` passes the restored markers to the store whenever one is configured. So for these 0.4 Kafka markers, the routes left are reusing the journal or dropping the store. I haven't tried a 0.3 savepoint, whose markers are plain offset maps, or the Fluss store. One option is an explicit upgrade mode that replaces the old journal's recovery markers and, where cleanup is configured, its cleanup metadata. It would have to keep replaying from the new journal on later failures, including any before the first new checkpoint. Would something like that fit the 1.0 plan, or would you rather document a different procedure? GitHub link: https://github.com/apache/flink-agents/discussions/1327#discussioncomment-19248278 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
