Gallardot commented on code in PR #14934:
URL:
https://github.com/apache/dolphinscheduler/pull/14934#discussion_r1336739386
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/k8s/impl/K8sTaskExecutor.java:
##########
@@ -257,6 +258,12 @@ private void parsePodLogOutput() {
try (BufferedReader reader = new BufferedReader(new
InputStreamReader(watcher.getOutput()))) {
while ((line = reader.readLine()) != null) {
log.info("[K8S-pod-log] {}", line);
+
+ if (line.contains(VarPoolUtils.VAR_PREFIX_DOLLAR)
+ ||
line.contains(VarPoolUtils.VAR_PREFIX_HASH)) {
+ varPool.append(VarPoolUtils.findVarPool(line));
+ varPool.append(VarPoolUtils.VAR_DELIMITER);
Review Comment:
After discussing with @Radeity , we concluded that using the `contains`
method to find magic code is not performant. However, using the startswith
method, like in a shell task, is not suitable for this k8s task scenario
because Java, Golang, and other languages have specific logging frameworks that
cannot be determined by the first character, making it less universal.
Therefore, we decided to introduce a new magic code format
`${(key=value)dsVal}`, and require that this character appear at the end of a
line of log. This approach balances performance and universality, making it
suitable for use in k8s tasks.
@EricGao888 @ruanwenjun @SbloodyS @qingwli WDYT?
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/k8s/impl/K8sTaskExecutor.java:
##########
@@ -257,6 +258,12 @@ private void parsePodLogOutput() {
try (BufferedReader reader = new BufferedReader(new
InputStreamReader(watcher.getOutput()))) {
while ((line = reader.readLine()) != null) {
log.info("[K8S-pod-log] {}", line);
+
+ if (line.contains(VarPoolUtils.VAR_PREFIX_DOLLAR)
+ ||
line.contains(VarPoolUtils.VAR_PREFIX_HASH)) {
+ varPool.append(VarPoolUtils.findVarPool(line));
+ varPool.append(VarPoolUtils.VAR_DELIMITER);
Review Comment:
After discussing with @Radeity , we concluded that using the `contains`
method to find magic code is not performant. However, using the startswith
method, like in a shell task, is not suitable for this k8s task scenario
because Java, Golang, and other languages have specific logging frameworks that
cannot be determined by the first character, making it less universal.
Therefore, we decided to introduce a new magic code format
`${(key=value)dsVal}`, and require that this character appear at the end of a
line of log. This approach balances performance and universality, making it
suitable for use in k8s tasks.
@EricGao888 @ruanwenjun @SbloodyS @qingwli WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
