DanielLeens commented on issue #12097: URL: https://github.com/apache/seatunnel/issues/12097#issuecomment-5551704114
Thanks @programmerloverun. I rechecked current dev and the open PR set. The report still identifies three distinct costs: the JDBC sampling path drains the complete result set, the enumerator materializes and assigns all generated splits, and each reader appends its assignment to an unbounded deque. There is no open PR covering this combined path. GitHub did not accept an assignee update for this account, so this comment records the accepted coordination claim rather than asserting that issue metadata changed. Please keep the investigation and implementation order explicit. The first slice must show how a bounded handoff can work without changing split predicates, stable split IDs, checkpoint restore, or read completeness. The current enumerator does not implement split requests, so a cap that merely withholds assignments would strand remaining work. Add deterministic tests for split progression, null and duplicate boundaries, empty ranges, restore, and skew before adding dialect-specific sampling SQL. Any database-side sampling or probing must be an explicit dialect capability with a fallback that never silently returns to a full client-side scan. Attach the measured planning scan volume, generated and pending split counts, peak retained metadata, and comparable before/after results to the focused PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
