SEZ9 commented on issue #12361: URL: https://github.com/apache/seatunnel/issues/12361#issuecomment-6514104065
@goutamadwant thanks for picking this up. Spill-to-temporary-storage plus lazy emission in the exactly-once path is the right direction for the problem @201811510411lw traced: the terminal `ChunkRange.of(chunkStart, null)` split keeps growing after planning (812,737 rows against a `snapshot.split.size` of 81,920 in the reported case), and `pollSplitRecordsIfExactlyOnce()` holding the whole split in a `LinkedHashMap` turns that growth directly into heap pressure. Before I review #13027 in depth, a few things I'd like to see covered in the PR, since they are what makes this fix trustworthy rather than just a larger effective buffer: 1. **Watermark correctness is unchanged.** The low/high watermark boundaries and the binlog backfill merge for the terminal split must behave exactly as in 8c9d47f5ea7e634d2d27952c7ee7a55269218950, only with records resident on disk instead of heap. Please call out explicitly how upsert/delete merging by key is preserved when records are spilled, since the `LinkedHashMap` currently does that de-duplication in memory. 2. **Checkpoint/restore on the open-ended split.** The reported failure loop is: OOM → recover → retry the same unfinished range → OOM again. Please make sure the MySQL checkpoint/restore test you mention actually grows the table after enumeration and restores mid-split, and confirm what happens to spill files on restart (cleanup, no reuse of stale data). 3. **Bounded retained heap as an assertion, not just a measurement.** A regression test that asserts a bound on in-heap records for a terminal split, plus no lost/duplicate rows across snapshot and binlog, is what I'd want to gate this on. 4. **Validation limits.** You mention remaining validation limits — please list them concretely in the PR description so we know what still needs manual verification. @201811510411lw if you are able to try the PR build against a workload shaped like your incident (long-running inserts past the planning-time max key during the final split), that would be very valuable feedback. The sanitized `SHOW CREATE TABLE`, full source split options and GC/heap evidence requested earlier would also help us match the test data shape to what you saw. <!-- streview-comment:2145 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
