nfsantos commented on PR #728: URL: https://github.com/apache/jackrabbit-oak/pull/728#issuecomment-1271251150
Just a few general comments about error handling. 1. Imagine that there is some network problem that makes all download attempts fail. The error could be raised immediately upon trying the connection (like hostname not found) or it could be a timeout and raised only after 1 minute of each connection attempt. My understanding of the current implementation is that it would not abort early and would try to download every file. And it seems the only error reporting is at the end, once every file was processed by the downloader. So in the case of a timeout connecting or read from the blob store, the tool could stay for a long time (hours) trying to download blobs without any success, while showing no indication that something is going wrong. At a bare minimum, the tool should log any errors as soon as they happen, so the operator can abort the transfer. Maybe even abort the execution if a file fails to download. 2. I don't see any logic to deal with transient errors. If downloading a dataset may take many hours, the chance of some transient error is very high. What will happen in this case? Will the whole transfer have to started over again from start? This could easily lead to situations where it becomes close to impossible to download a large dataset because of random transient failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
