hailin0 commented on PR #5904: URL: https://github.com/apache/seatunnel/pull/5904#issuecomment-1826301727
> Similar to [5722](https://github.com/apache/seatunnel/issues/5722), when I use ES's scroll query or read large files, the checkpoint timeout problem will also occur. > I found that it was caused by the synchronized lock granularity in pollNext method being too large. This is correct, split will prevent the checkpoint from acquiring the lock whenever the split is read, until the split read is complete. This is the basis for maintaining data consistency. Timeouts can be solved by reducing the data block size or increasing the timeout. In #5914 we allow batch tasks to turn off checkpoints -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
