rickchengx commented on code in PR #13332:
URL:
https://github.com/apache/dolphinscheduler/pull/13332#discussion_r1102219710
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/AbstractCommandExecutor.java:
##########
@@ -100,6 +101,10 @@ public
AbstractCommandExecutor(Consumer<LinkedBlockingQueue<String>> logHandler,
this.taskRequest = taskRequest;
this.logger = logger;
this.logBuffer = new LinkedBlockingQueue<>();
+
+ if (this.taskRequest != null) {
+ this.taskRequest.setLogHandleEnable(true);
Review Comment:
I've changed `logHandleEnable` to `logBufferEnable` for better readability.
This PR records whether the `logBuffer` is enabled in the task context for
the following resaons:
There are two types of tasks:
1. Tasks use `logBuffer` to cache logs (tasks that use
`ShellCommandExecutor` which extends `AbstractCommandExecutor`)
* At the end of this type of task, the log is not completely written to the
file system, so it needs to write the log to the remote storage when the cache
is finally emptied
* it sends the task log to the remote storage in `clear()` method in
`AbstractCommandExecutor` as below
<img width="798" alt="截屏2023-01-17 15 07 36"
src="https://user-images.githubusercontent.com/38122586/212832390-140878be-117c-4a2e-b2c3-e6a75c4b0f8f.png";>
2. Tasks that not use `logBuffer`
* Note that not all tasks will use `logBuffer` , such as `ZEPPELIN` task and
the tasks executed on master. So this PR also sends the task log to the remote
storage in `afterExecuted()` and `afterThrowing()` on worker (if the task does
not use `logBuffer`) and `taskFinished` on master (if the task is executed on
master).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://user-images.githubusercontent.com/38122586/212832390-140878be-117c-4a2e-b2c3-e6a75c4b0f8f.png"](https://user-images.githubusercontent.com/38122586/212832390-140878be-117c-4a2e-b2c3-e6a75c4b0f8f.png"){kind=link}