[GitHub] [dolphinscheduler] rickchengx commented on pull request #13332: [Feature-13331][Remote Logging] Add support for writing task logs to OSS

[GitBox]

rickchengx commented on PR #13332:
URL: 
https://github.com/apache/dolphinscheduler/pull/13332#issuecomment-1384937430

   Hi, @zhongjiajie @Radeity, thanks again for your kind review. I have 
modified this PR according to your comments and suggestions.
   
   Here are some brief changes:
   
   1. Remove the `Thread.sleep()` in `WorkerTaskExecuteRunnable`
   * send the task log to the remote storage in `clear()` method in 
`AbstractCommandExecutor`, so there is no need to sleep anymore.
   <img width="798" alt="截屏2023-01-17 15 07 36" 
src="https://user-images.githubusercontent.com/38122586/212832390-140878be-117c-4a2e-b2c3-e6a75c4b0f8f.png";>
   
   * Note that not all tasks will use `logHandle` and 
`FINALIZE_SESSION_MARKER`, such as `ZEPPELIN` task and the tasks executed on 
master. So this PR also sends the task log to the remote storage in 
`afterExecuted()` and `afterThrowing()` on worker (if the task does not use 
`logHandle`) and `taskFinished` on master (if the task is executed on master).
   
   Here is some examples:
   
   * Shell Task (use `logHandle`)
   <img width="1058" alt="截屏2023-01-16 14 08 29" 
src="https://user-images.githubusercontent.com/38122586/212834530-0cdbc222-a831-43b3-a1a2-5e7100db5b01.png";>
   
   * Zeppelin Task (does not use `logHandle`)
   <img width="1055" alt="截屏2023-01-16 14 08 50" 
src="https://user-images.githubusercontent.com/38122586/212834590-cb0476d3-a900-4a06-88c6-778761702cf4.png";>
   
   
   2. Add the doc
   <img width="1355" alt="截屏2023-01-17 15 18 36" 
src="https://user-images.githubusercontent.com/38122586/212834059-ac499f1f-ea42-45dd-b4f4-85daff4fb5d8.png";>
   
   * I am not sure if the location of this doc is appropriate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org