goutamadwant commented on issue #11306: URL: https://github.com/apache/seatunnel/issues/11306#issuecomment-4900080062
@DanielLeens @Echo18322276280 I would like to take a look at this one. I will first compare it with #11312 so we do not duplicate the current file discovery/update compare work. From the issue details, the part I want to focus on is the submit-time blocking before jobId is returned, especially because the same delay shows up in full mode also. Plan is to trace the SftpFile source preparation path, add a small local reproduction around large file enumeration, and keep the first PR scoped to reducing or moving the expensive source discovery out of the REST submit path where possible. I will also add timing coverage/logs so the behavior is easier to verify with the @Echo18322276280's sanitized logs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
