Asthenia0412 commented on PR #12567: URL: https://github.com/apache/seatunnel/pull/12567#issuecomment-6147079264
@DanielLeens Read your follow-up carefully. Issue 3 (restart position / restored generation after `IncrementalSplitAssigner#restore`) is the one I haven't addressed yet — you're right that it's a design-level problem and I under-estimated it earlier. Issues 1 and 2 (the HashSet lookup and monotonic offsets) are in the tree and I kept the last real code commit (e18f31cf as of your review) plus the set-based `toSourceSplit` / assigner lookup changes. The restart-position blocker: let me trace it from your description — when a restore creates a fresh `IncrementalSplitState` seeded from checkpoint, and that state's offsets then feed `IncrementalSplitAssigner#restore`, the assigner would compare its own checkpoint offset against the split's offsets. If the per-table path goes through `IncrementalSplitAssigner` lines 309–324 in the same way, we'd need a consistent seeding of per-table checkpoint offsets on restore too. I want to implement that before pushing further — want me to sketch it out here or on #12698? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
