rickchengx commented on PR #13654: URL: https://github.com/apache/dolphinscheduler/pull/13654#issuecomment-1459355483
> In addition, i just find that if uploading fail, whether caused by network error or OOM etc, ds only records an error log, user may not sense this error. Can we have some optimization about it? > > https://github.com/apache/dolphinscheduler/blob/fd720880c77d9ef60367b9da49a4bc0c257d0d37/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/log/remote/OssRemoteLogHandler.java#L66-L72 > > > Another concern is that if we see local files as caches of remote logs, we have to try the best to make sure logs can be successfully uploaded to remote storage. Can we introduce some mechanism, such as, split log file into slices or support breakpoint retransmission to handle that? Hi, @Radeity , thanks for your comment. 1. About the error log: Currently, uploading logs to remote storage is an asynchronous process (via thread pool). We can't make the user aware of this error on the front end. But I think it's a good idea, maybe we can show this error in local task log and provide button for user to upload again. I will implement this function in another PR. 2. About the error handling for upload or download logs, this is a big topic that needs further discussion and design. Work is in progress on this as well, but maybe it's out of scope for this PR and I'll be refining it in a future PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
