zhangshenghang commented on issue #10344: URL: https://github.com/apache/seatunnel/issues/10344#issuecomment-3755175843
<!-- code-pr-reviewer --> This is a SeaTunnel engine-level constraint rather than ClickHouse-specific behavior. The ClickHouse source is a `BOUNDED` source: - `connector-clickhouse/.../ClickhouseSource.java:60-62` returns `Boundedness.BOUNDED` - `seatunnel-api/.../SourceReader.java:54-63` API doc states: "If the source is bounded, checkpoint is not triggered" - `seatunnel-engine-server/.../CheckpointCoordinator.java:592-623` shows savepoint waits for pending operations to complete The connector implements checkpoint interfaces (`ClickhouseSourceReader:112-114`), but the engine doesn't invoke `snapshotState()` for bounded sources during execution. Savepoints require waiting for consistency. **Current workarounds:** 1. Split large jobs using `partition_list` or `filter_query` to process data in batches 2. Consider streaming mode with CDC if incremental sync is required **Questions for you:** - Is this for failure recovery or manual pause/resume? - Would splitting the job into multiple smaller tasks be acceptable for your use case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
