[PR] [Improvement] Audit log Improvement [dolphinscheduler]
qingwli opened a new pull request, #15554: URL: https://github.com/apache/dolphinscheduler/pull/15554 - close #15423 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
rickchengx commented on code in PR #15553:
URL:
https://github.com/apache/dolphinscheduler/pull/15553#discussion_r1475610039
##
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/operator/BaseTaskExecuteRunnableTimeoutOperator.java:
##
@@ -38,18 +38,8 @@ public
BaseTaskExecuteRunnableTimeoutOperator(TaskInstanceDao taskInstanceDao) {
@Override
public void operate(DefaultTaskExecuteRunnable taskExecuteRunnable) {
-// Right now, if the task is running in worker, the timeout strategy
will be handled at worker side.
-// if the task is in master, the timeout strategy will be handled at
master side.
-// todo: we should unify this, the master only need to handle the
timeout strategy. and send request to worker
-// to kill the task, if the strategy is timeout_failed.
TaskInstance taskInstance = taskExecuteRunnable.getTaskInstance();
TaskTimeoutStrategy taskTimeoutStrategy =
taskInstance.getTaskDefine().getTimeoutNotifyStrategy();
-if (TaskTimeoutStrategy.FAILED != taskTimeoutStrategy
Review Comment:
if `taskTimeoutStrategy ` == `TaskTimeoutStrategy.FAILED`, it would not step
into this block, I don't get how this PR could fix this problem, could you
please add some description?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(dolphinscheduler) branch dev updated: [Doc][Docker] fix typo on start with docker (#15534)
This is an automated email from the ASF dual-hosted git repository.
zihaoxiang pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
The following commit(s) were added to refs/heads/dev by this push:
new 1b42d45fcd [Doc][Docker] fix typo on start with docker (#15534)
1b42d45fcd is described below
commit 1b42d45fcdd22ccac7692fac4a64d096736f5e7c
Author: John Huang
AuthorDate: Fri Feb 2 14:17:11 2024 +0800
[Doc][Docker] fix typo on start with docker (#15534)
Co-authored-by: xiangzihao <460888...@qq.com>
Co-authored-by: Eric Gao
---
docs/docs/en/guide/start/docker.md | 8
docs/docs/zh/guide/start/docker.md | 8
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs/docs/en/guide/start/docker.md
b/docs/docs/en/guide/start/docker.md
index 7326ff42a5..b98d0572ae 100644
--- a/docs/docs/en/guide/start/docker.md
+++ b/docs/docs/en/guide/start/docker.md
@@ -81,7 +81,7 @@ $ docker run -d --name dolphinscheduler-tools \
# Starting DolphinScheduler service
$ docker run -d --name dolphinscheduler-master \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -90,7 +90,7 @@ $ docker run -d --name dolphinscheduler-master \
-d apache/dolphinscheduler-master:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-worker \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -99,7 +99,7 @@ $ docker run -d --name dolphinscheduler-worker \
-d apache/dolphinscheduler-worker:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-api \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -108,7 +108,7 @@ $ docker run -d --name dolphinscheduler-api \
-d apache/dolphinscheduler-api:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-alert-server \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
diff --git a/docs/docs/zh/guide/start/docker.md
b/docs/docs/zh/guide/start/docker.md
index 9b60ccb84a..2a3a577a73 100644
--- a/docs/docs/zh/guide/start/docker.md
+++ b/docs/docs/zh/guide/start/docker.md
@@ -74,7 +74,7 @@ $ docker run -d --name dolphinscheduler-tools \
# 启动 DolphinScheduler 对应的服务
$ docker run -d --name dolphinscheduler-master \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -83,7 +83,7 @@ $ docker run -d --name dolphinscheduler-master \
-d apache/dolphinscheduler-master:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-worker \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -92,7 +92,7 @@ $ docker run -d --name dolphinscheduler-worker \
-d apache/dolphinscheduler-worker:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-api \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e SPRING_JACKSON_TIME_ZONE="UTC" \
@@ -101,7 +101,7 @@ $ docker run -d --name dolphinscheduler-api \
-d apache/dolphinscheduler-api:"${DOLPHINSCHEDULER_VERSION}"
$ docker run -d --name dolphinscheduler-alert-server \
-e DATABASE="postgresql" \
--e
SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/dolphinscheduler" \
+-e SPRING_DATASOURCE_URL="jdbc:postgresql://localhost:5432/" \
-e SPRING_DATASOURCE_USERNAME="" \
-e SPRING_DATASOURCE_PASSWORD="" \
-e
Re: [I] [Doc][Docker] fix typo on start with docker [dolphinscheduler]
SbloodyS closed issue #15533: [Doc][Docker] fix typo on start with docker URL: https://github.com/apache/dolphinscheduler/issues/15533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Doc][Docker] fix typo on start with docker [dolphinscheduler]
SbloodyS merged PR #15534: URL: https://github.com/apache/dolphinscheduler/pull/15534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(dolphinscheduler-website) branch asf-site updated: Automated deployment: 548502ad3bb70a4ac21e287df70757144088d9a0
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git The following commit(s) were added to refs/heads/asf-site by this push: new c292ec5ee4 Automated deployment: 548502ad3bb70a4ac21e287df70757144088d9a0 c292ec5ee4 is described below commit c292ec5ee442f62158c42a606800f85c120b4f0a Author: github-actions[bot] AuthorDate: Fri Feb 2 05:06:07 2024 + Automated deployment: 548502ad3bb70a4ac21e287df70757144088d9a0 --- python/2.0.5/.buildinfo | 2 +- python/2.0.6/.buildinfo | 2 +- python/2.0.7/.buildinfo | 2 +- python/3.0.0/.buildinfo | 2 +- python/3.0.1/.buildinfo | 2 +- python/3.1.0/.buildinfo | 2 +- python/4.0.0/.buildinfo | 2 +- python/4.0.1/.buildinfo | 2 +- python/4.0.2/.buildinfo | 2 +- python/4.0.3/.buildinfo | 2 +- python/4.0.4/.buildinfo | 2 +- python/main/.buildinfo | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/python/2.0.5/.buildinfo b/python/2.0.5/.buildinfo index 85f3443f0b..bf98be134a 100644 --- a/python/2.0.5/.buildinfo +++ b/python/2.0.5/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: da44fd3444fd02cab872fc35f760eda5 +config: 132ba0720fbdcadfac96f3a7cea68ba0 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/2.0.6/.buildinfo b/python/2.0.6/.buildinfo index 8e2084b2b9..86a69b4d9e 100644 --- a/python/2.0.6/.buildinfo +++ b/python/2.0.6/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: ce80a3106dc7f6d20024f0493302bf01 +config: 36bcc3bbe7558f0156632463fc02d5d6 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/2.0.7/.buildinfo b/python/2.0.7/.buildinfo index 1c3ad85a77..b7d7307c07 100644 --- a/python/2.0.7/.buildinfo +++ b/python/2.0.7/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 4aaa19ef9494494548f2a7044f8c501a +config: 2619eae83bce9fd8bc44893e1ce117db tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.0.0/.buildinfo b/python/3.0.0/.buildinfo index 57882d25b7..6a0b973422 100644 --- a/python/3.0.0/.buildinfo +++ b/python/3.0.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 820dfd56c735d2139fd9247a0653055d +config: 42ec7dd909eda77ebf7fe9125e3f7d98 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.0.1/.buildinfo b/python/3.0.1/.buildinfo index f857785fce..b4bb065a77 100644 --- a/python/3.0.1/.buildinfo +++ b/python/3.0.1/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: b06f20731cce2497ad21ffc21e5e9de5 +config: a42a548fcf54c1ea50a47e43b50f3361 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.1.0/.buildinfo b/python/3.1.0/.buildinfo index 416024689d..2eee63122f 100644 --- a/python/3.1.0/.buildinfo +++ b/python/3.1.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 0cbd22c7eaa6141529a018be6e641e5b +config: c41fd05276db76c8e0e8417bdabbcf58 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.0/.buildinfo b/python/4.0.0/.buildinfo index 82bffc..cb9074a366 100644 --- a/python/4.0.0/.buildinfo +++ b/python/4.0.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 666b981638e7530396e9d0099f675eac +config: 8c2743fe1f1859dadb5a71bbb1e2f09f tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.1/.buildinfo b/python/4.0.1/.buildinfo index 8b91deefc3..09de80725c 100644 --- a/python/4.0.1/.buildinfo +++ b/python/4.0.1/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: d6ebf33abe1b5187a7aa6310273af903 +config: 00bf6f16ddca36d687182645b9baac42 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.2/.buildinfo b/python/4.0.2/.buildinfo index e89abda08c..23d5200d70 100644 --- a/python/4.0.2/.buildinfo +++ b/python/4.0.2/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: a15520ac0deca7c1fb470311fa2d93dd
Re: [PR] fix: data quality can not use [dolphinscheduler]
sonarcloud[bot] commented on PR #15551: URL: https://github.com/apache/dolphinscheduler/pull/15551#issuecomment-1922799247 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15551) **Quality Gate failed** Failed conditions [22.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler=15551=new_coverage=list) (required ≥ 60%) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15551) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix: data quality can not use [dolphinscheduler]
sonarcloud[bot] commented on PR #15551: URL: https://github.com/apache/dolphinscheduler/pull/15551#issuecomment-1922798293 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15551) **Quality Gate failed** Failed conditions [22.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler=15551=new_coverage=list) (required ≥ 60%) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15551) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] add two blog [dolphinscheduler-website]
pinkfloyds commented on PR #943: URL: https://github.com/apache/dolphinscheduler-website/pull/943#issuecomment-1922780334 Thanks for your reply,I hive submited article file,and README.md modification I will create another PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [datax] Dolphinscheduler3.2.0 version, datax component failed when using global variables [dolphinscheduler]
GaoXSen commented on issue #15519:
URL:
https://github.com/apache/dolphinscheduler/issues/15519#issuecomment-1922739186
我也遇到过类似问题，将${PYTHON_LAUNCHER}
${DATAX_LAUNCHER}配置在项目环境变量里就可以了。但是为什么在3.2.0中需要配置这两个默认变量呢？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] [Bug] [datax] Dolphinscheduler3.2.0 version, datax component failed when using global variables [dolphinscheduler]
GaoXSen commented on issue #15519:
URL:
https://github.com/apache/dolphinscheduler/issues/15519#issuecomment-1922738955
I have also encountered similar problems. Just configure ${PYTHON_LAUNCHER}
${DATAX_LAUNCHER} in the project environment variables. But why do we need to
configure these two default variables in 3.2.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] fix: data quality can not use [dolphinscheduler]
zhongjiajie commented on code in PR #15551:
URL:
https://github.com/apache/dolphinscheduler/pull/15551#discussion_r1475476199
##
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-api/src/main/java/org/apache/dolphinscheduler/plugin/datasource/api/utils/CommonUtils.java:
##
@@ -123,14 +132,56 @@ public static boolean loadKerberosConf(String
javaSecurityKrb5Conf, String login
return false;
}
-public static String getDataQualityJarName() {
-String dqsJarName = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
+public static String getDataQualityJarPath() {
+String dqsJarPath = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
-if (StringUtils.isEmpty(dqsJarName)) {
-return "dolphinscheduler-data-quality.jar";
+if (StringUtils.isEmpty(dqsJarPath)) {
+log.info("data quality jar path is empty, will try to get it from
data quality jar name");
+return getDefaultDataQualityJarPath();
}
-return dqsJarName;
+return dqsJarPath;
+}
+
+private static String getDefaultDataQualityJarPath() {
+if (StringUtils.isNotEmpty(DEFAULT_DATA_QUALITY_JAR_PATH)) {
Review Comment:
> It seems that if the location of the jar package has been found once before
correct
> Do we need to consider the situation where the user has modified the name
or location of the jar package
I think is ok to keep this, cause this jar is not in class path, They can
directly replace the jar with the same name if they rebuild the jar. Or if they
want to change de jar name, they should restart worker node, or change
`data-quality.jar.name`(actually a path) in `common.properties`
##
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-api/src/main/java/org/apache/dolphinscheduler/plugin/datasource/api/utils/CommonUtils.java:
##
@@ -123,14 +132,56 @@ public static boolean loadKerberosConf(String
javaSecurityKrb5Conf, String login
return false;
}
-public static String getDataQualityJarName() {
-String dqsJarName = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
+public static String getDataQualityJarPath() {
+String dqsJarPath = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
-if (StringUtils.isEmpty(dqsJarName)) {
-return "dolphinscheduler-data-quality.jar";
+if (StringUtils.isEmpty(dqsJarPath)) {
+log.info("data quality jar path is empty, will try to get it from
data quality jar name");
+return getDefaultDataQualityJarPath();
}
-return dqsJarName;
+return dqsJarPath;
+}
+
+private static String getDefaultDataQualityJarPath() {
+if (StringUtils.isNotEmpty(DEFAULT_DATA_QUALITY_JAR_PATH)) {
Review Comment:
> It seems that if the location of the jar package has been found once before
correct
> Do we need to consider the situation where the user has modified the name
or location of the jar package
I think is ok to keep this, cause this jar is not in class path, They can
directly replace the jar with the same name if they rebuild the jar. Or if they
want to change de jar name, they should restart worker node, or change
`data-quality.jar.name`(actually a path) in `common.properties`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(dolphinscheduler) branch augit-log deleted (was e79ff870e1)
This is an automated email from the ASF dual-hosted git repository. wangyang pushed a change to branch augit-log in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git was e79ff870e1 update This change permanently discards the following revisions: discard e79ff870e1 update discard c95faa4d7a support multi codes discard 27277f820d Merge branch 'dev' into augit-log discard 844585af49 update discard 771e9342c0 init
(dolphinscheduler) 03/05: Merge branch 'dev' into augit-log
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 27277f820d5e6988d7f5303d1cc547f5abc84ac4
Merge: 844585af49 2f66a667ba
Author: Draven
AuthorDate: Thu Feb 1 16:10:47 2024 +0800
Merge branch 'dev' into augit-log
# Conflicts:
#
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ClusterController.java
#
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AuditLogMapperTest.java
.github/workflows/backend.yml | 4 +-
.github/workflows/e2e.yml | 2 +
README.md | 2 +-
README_zh_CN.md| 2 +-
deploy/kubernetes/dolphinscheduler/README.md | 22 +-
deploy/kubernetes/dolphinscheduler/values.yaml | 45 +++-
docs/docs/en/architecture/configuration.md | 80 +++---
docs/docs/en/guide/installation/kubernetes.md | 2 +
docs/docs/en/guide/project/workflow-definition.md | 11 +-
docs/docs/en/guide/task/flink.md | 2 +-
docs/docs/en/guide/task/spark.md | 2 +-
docs/docs/zh/architecture/configuration.md | 226 +
docs/docs/zh/guide/installation/kubernetes.md | 2 +
docs/docs/zh/guide/project/workflow-definition.md | 13 +-
docs/docs/zh/guide/task/flink.md | 2 +-
docs/docs/zh/guide/task/spark.md | 2 +-
docs/img/new_ui/dev/project/workflow-list.png | Bin 143361 -> 155286 bytes
docs/img/new_ui/dev/project/workflow-time03.png| Bin 62574 -> 172800 bytes
docs/img/new_ui/dev/project/workflow-time04.png| Bin 0 -> 200575 bytes
docs/img/scheduler_online.png | Bin 0 -> 1703 bytes
docs/img/timeManagement.png| Bin 11195 -> 0 bytes
.../src/main/bin/jvm_args_env.sh | 1 +
.../apache/dolphinscheduler/alert/AlertServer.java | 4 +
.../dolphinscheduler/alert/config/AlertConfig.java | 8 +-
.../alert/metrics/AlertServerMetrics.java | 6 +
.../alert/registry/AlertHeartbeatTask.java | 17 +-
.../alert/registry/AlertRegistryClient.java| 6 +-
.../src/main/resources/application.yaml| 2 +-
.../api/test/cases/ProcessInstanceAPITest.java | 54 ++--
dolphinscheduler-api/src/main/bin/jvm_args_env.sh | 1 +
.../dolphinscheduler/api/ApiApplicationServer.java | 4 +
.../api/controller/AlertGroupController.java | 12 +-
.../api/controller/BaseController.java | 2 +-
.../api/controller/CloudController.java| 4 +-
.../api/controller/ClusterController.java | 2 +-
.../api/controller/DataAnalysisController.java | 12 +-
.../api/controller/DataSourceController.java | 6 +-
.../api/controller/DynamicTaskTypeController.java | 2 +-
.../api/controller/K8sNamespaceController.java | 6 +-
.../api/controller/LoginController.java| 2 +-
.../api/controller/MonitorController.java | 2 +-
.../api/controller/ProjectController.java | 2 +-
.../api/controller/ResourcesController.java| 2 +-
.../api/controller/SchedulerController.java| 2 +-
.../api/controller/TaskGroupController.java| 4 +-
.../api/controller/TenantController.java | 8 +-
.../api/controller/WorkFlowLineageController.java | 2 +-
.../api/controller/WorkerGroupController.java | 2 +-
.../v2/ProcessTaskRelationV2Controller.java| 2 +-
.../api/controller/v2/ProjectV2Controller.java | 2 +-
.../apache/dolphinscheduler/api/enums/Status.java | 2 +-
.../api/metrics/ApiServerMetrics.java | 8 +
.../api/service/DataAnalysisService.java | 18 +-
.../api/service/K8sNamespaceService.java | 4 +-
.../api/service/impl/DataAnalysisServiceImpl.java | 36 +--
.../api/service/impl/DataSourceServiceImpl.java| 1 +
.../api/service/impl/ExecutorServiceImpl.java | 2 +
.../api/service/impl/K8SNamespaceServiceImpl.java | 4 +-
.../api/service/impl/LoggerServiceImpl.java| 7 +-
.../service/impl/ProcessDefinitionServiceImpl.java | 1 +
.../api/service/impl/SchedulerServiceImpl.java | 12 +-
.../service/impl/TaskDefinitionServiceImpl.java| 4 +-
.../api/service/impl/TaskInstanceServiceImpl.java | 9 +-
.../api/vo/{ScheduleVo.java => ScheduleVO.java}| 4 +-
...TaskDefinitionVo.java => TaskDefinitionVO.java} | 10 +-
...stanceCountVo.java => TaskInstanceCountVO.java} | 8 +-
...CountVo.java => WorkflowDefinitionCountVO.java} | 10 +-
...ceCountVo.java => WorkflowInstanceCountVO.java} | 8 +-
.../src/main/resources/i18n/messages.properties| 2 +-
.../main/resources/i18n/messages_en_US.properties | 2 +-
(dolphinscheduler) 01/05: init
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 771e9342c01e520f5efa7bbfcfaab7c2d627a153
Author: Draven
AuthorDate: Mon Jan 15 15:34:31 2024 +0800
init
---
.../dolphinscheduler/api/audit/AuditMessage.java | 60 +++-
.../api/audit/AuditPublishService.java | 92 --
.../api/audit/AuditSubscriber.java | 28 --
.../api/audit/AuditSubscriberImpl.java | 42
.../dolphinscheduler/api/audit/OperatorLog.java| 21
.../api/audit/OperatorLogAspect.java | 108 +
.../api/controller/AuditLogController.java | 8 +-
.../controller/ProcessDefinitionController.java| 4 +
.../api/controller/ProjectController.java | 10 ++
.../dolphinscheduler/api/service/AuditService.java | 20 ++--
.../api/service/impl/AuditServiceImpl.java | 38
.../dolphinscheduler/api/utils/AuditLogUtils.java | 80 +++
.../src/main/resources/application.yaml| 23 ++---
.../src/main/resources/logback-spring.xml | 7 +-
.../api/audit/AuditSubscriberTest.java | 9 +-
.../api/service/AuditServiceTest.java | 4 +-
dolphinscheduler-bom/pom.xml | 1 -
.../common/enums/Audit/AuditObjectType.java| 102 +++
.../enums/{ => Audit}/AuditOperationType.java | 34 ---
.../common/enums/AuditResourceType.java| 61
.../dolphinscheduler/dao/entity/AuditLog.java | 62 ++--
.../dao/mapper/AuditLogMapper.java | 6 +-
.../dolphinscheduler/dao/mapper/AuditLogMapper.xml | 22 ++---
.../dao/mapper/AuditLogMapperTest.java | 1 -
.../service/process/ProcessServiceImpl.java| 1 +
25 files changed, 419 insertions(+), 425 deletions(-)
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
index 0d936f0ad5..893460590d 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
@@ -17,70 +17,32 @@
package org.apache.dolphinscheduler.api.audit;
-import org.apache.dolphinscheduler.common.enums.AuditOperationType;
-import org.apache.dolphinscheduler.common.enums.AuditResourceType;
+import lombok.Data;
+import org.apache.dolphinscheduler.common.enums.Audit.AuditObjectType;
+import org.apache.dolphinscheduler.common.enums.Audit.AuditOperationType;
import org.apache.dolphinscheduler.dao.entity.User;
import java.util.Date;
+@Data
public class AuditMessage {
private User user;
private Date auditDate;
-private AuditResourceType resourceType;
+private AuditObjectType objectType;
-private AuditOperationType operation;
+private AuditOperationType operationType;
private Integer resourceId;
-public AuditMessage(User user, Date auditDate, AuditResourceType
resourceType, AuditOperationType operation,
+public AuditMessage(User user, Date auditDate, AuditObjectType objectType,
AuditOperationType operationType,
Integer resourceId) {
this.user = user;
this.auditDate = auditDate;
-this.resourceType = resourceType;
-this.operation = operation;
-this.resourceId = resourceId;
-}
-
-public User getUser() {
-return user;
-}
-
-public void setUser(User user) {
-this.user = user;
-}
-
-public Date getAuditDate() {
-return auditDate;
-}
-
-public void setAuditDate(Date auditDate) {
-this.auditDate = auditDate;
-}
-
-public AuditResourceType getResourceType() {
-return resourceType;
-}
-
-public void setResourceType(AuditResourceType resourceType) {
-this.resourceType = resourceType;
-}
-
-public AuditOperationType getOperation() {
-return operation;
-}
-
-public void setOperation(AuditOperationType operation) {
-this.operation = operation;
-}
-
-public Integer getResourceId() {
-return resourceId;
-}
-
-public void setResourceId(Integer resourceId) {
+this.objectType = objectType;
+this.operationType = operationType;
this.resourceId = resourceId;
}
@@ -89,8 +51,8 @@ public class AuditMessage {
return "AuditMessage{"
+ "user=" + user
+ ", Date=" + auditDate
-+ ", resourceType" + resourceType
-+ ", operation=" + operation
++ ", objectType" + objectType
++ ", operationType=" + operationType
(dolphinscheduler) 05/05: update
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit e79ff870e17099ef918e9481a372c5356dc29e45
Author: Draven
AuthorDate: Fri Feb 2 10:52:28 2024 +0800
update
---
.../dolphinscheduler/api/audit/OperatorLog.java| 7
.../api/audit/OperatorLogAspect.java | 46 --
.../api/controller/AuditLogController.java | 11 --
.../api/dto/auditLog/AuditObjectTypeDto.java | 42
.../api/dto/auditLog/AuditOperationTypeDto.java| 27 +
.../dolphinscheduler/api/service/AuditService.java | 20 +-
.../src/main/resources/sql/dolphinscheduler_h2.sql | 10 +++--
.../main/resources/sql/dolphinscheduler_mysql.sql | 12 --
.../resources/sql/dolphinscheduler_postgresql.sql | 14 ---
.../3.3.0_schema/mysql/dolphinscheduler_ddl.sql| 29 +-
.../postgresql/dolphinscheduler_ddl.sql| 28 -
.../dao/mapper/AuditLogMapperTest.java | 14 +--
12 files changed, 183 insertions(+), 77 deletions(-)
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLog.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLog.java
index 440dd0f997..cd113a826b 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLog.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLog.java
@@ -54,13 +54,6 @@ public @interface OperatorLog {
*/
String[] requestParamName() default {};
-/**
- * The name of the object to be returned after the operation.
- * For operations like create and import, specify the object name to
retrieve details from the response.
- * For update and delete, the necessary information is usually provided in
the request parameters.
- */
-String returnObjectName() default "";
-
/**
* The names of the fields in the returned object to be recorded.
* Represents an array of field names, e.g., ["id", "code"].
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLogAspect.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLogAspect.java
index ec3211641d..141b979435 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLogAspect.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/OperatorLogAspect.java
@@ -30,7 +30,6 @@ import
org.apache.dolphinscheduler.common.enums.AuditObjectType;
import org.apache.dolphinscheduler.common.enums.AuditOperationType;
import org.apache.dolphinscheduler.common.enums.ReleaseState;
import org.apache.dolphinscheduler.dao.entity.*;
-import org.apache.dolphinscheduler.dao.entity.Queue;
import org.apache.dolphinscheduler.dao.mapper.*;
import org.apache.dolphinscheduler.spi.enums.ResourceType;
import org.aspectj.lang.ProceedingJoinPoint;
@@ -49,57 +48,12 @@ public class OperatorLogAspect {
@Autowired
private AuditService auditService;
-@Autowired
-private ProjectMapper projectMapper;
-
-@Autowired
-private ProcessDefinitionMapper processDefinitionMapper;
-
-@Autowired
-private ProcessInstanceMapper processInstanceMapper;
-
-@Autowired
-private TaskDefinitionMapper taskDefinitionMapper;
-
@Autowired
private ScheduleMapper scheduleMapper;
-@Autowired
-private UdfFuncMapper udfFuncMapper;
-
@Autowired
private UserMapper userMapper;
-@Autowired
-private DataSourceMapper dataSourceMapper;
-
-@Autowired
-private TenantMapper tenantMapper;
-
-@Autowired
-private AlertGroupMapper alertGroupMapper;
-
-@Autowired
-private AlertPluginInstanceMapper alertPluginInstanceMapper;
-
-@Autowired
-private WorkerGroupMapper workerGroupMapper;
-
-@Autowired
-private QueueMapper queueMapper;
-
-@Autowired
-private EnvironmentMapper environmentMapper;
-
-@Autowired
-private ClusterMapper clusterMapper;
-
-@Autowired
-private K8sNamespaceMapper k8sNamespaceMapper;
-
-@Autowired
-private AccessTokenMapper accessTokenMapper;
-
@Pointcut("@annotation(OperatorLog)")
public void logPointCut() {
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
index afae94cff1..b198669a82 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
@@ -20,6 +20,8 @@ package
(dolphinscheduler) branch augit-log created (now e79ff870e1)
This is an automated email from the ASF dual-hosted git repository. wangyang pushed a change to branch augit-log in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git at e79ff870e1 update This branch includes the following new commits: new 771e9342c0 init new 844585af49 update new 27277f820d Merge branch 'dev' into augit-log new c95faa4d7a support multi codes new e79ff870e1 update The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(dolphinscheduler) 04/05: support multi codes
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit c95faa4d7a5a8ab777233d7baac9bc457bffd106
Author: Draven
AuthorDate: Thu Feb 1 16:59:07 2024 +0800
support multi codes
---
.../api/controller/AuditLogController.java | 16
.../dolphinscheduler/api/service/AuditService.java | 22 +--
.../api/service/impl/AuditServiceImpl.java | 46 --
.../api/service/AuditServiceTest.java | 8 ++--
.../dao/mapper/AuditLogMapper.java | 5 ++-
.../dolphinscheduler/dao/mapper/AuditLogMapper.xml | 19 ++---
.../dao/mapper/AuditLogMapperTest.java | 4 +-
7 files changed, 67 insertions(+), 53 deletions(-)
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
index b77e7e095d..afae94cff1 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
@@ -59,8 +59,8 @@ public class AuditLogController extends BaseController {
*
* @param loginUser login user
* @param pageNopage number
- * @param resourceType resource type
- * @param operationType operation type
+ * @param objectTypeCodes object type codes
+ * @param operationTypeCodesoperation type codes
* @param startDate start time
* @param endDate end time
* @param userName user name
@@ -71,8 +71,8 @@ public class AuditLogController extends BaseController {
@Parameters({
@Parameter(name = "startDate", description = "START_DATE", schema
= @Schema(implementation = String.class)),
@Parameter(name = "endDate", description = "END_DATE", schema =
@Schema(implementation = String.class)),
-@Parameter(name = "resourceType", description = "RESOURCE_TYPE",
schema = @Schema(implementation = AuditObjectType.class)),
-@Parameter(name = "operationType", description = "OPERATION_TYPE",
schema = @Schema(implementation = AuditOperationType.class)),
+@Parameter(name = "objectTypeCodes", description =
"OBJECT_TYPE_CODES", schema = @Schema(implementation = String.class)),
+@Parameter(name = "operationTypeCodes", description =
"OPERATION_TYPE_CODES", schema = @Schema(implementation = String.class)),
@Parameter(name = "userName", description = "USER_NAME", schema =
@Schema(implementation = String.class)),
@Parameter(name = "pageNo", description = "PAGE_NO", required =
true, schema = @Schema(implementation = int.class, example = "1")),
@Parameter(name = "pageSize", description = "PAGE_SIZE", required
= true, schema = @Schema(implementation = int.class, example = "20"))
@@ -83,16 +83,16 @@ public class AuditLogController extends BaseController {
public Result>
queryAuditLogListPaging(@Parameter(hidden = true) @RequestAttribute(value =
Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize,
-
@RequestParam(value = "resourceType", required = false) AuditObjectType
resourceType,
-
@RequestParam(value = "operationType", required = false) AuditOperationType
operationType,
+
@RequestParam(value = "objectTypeCodes", required = false) String
objectTypeCodes,
+
@RequestParam(value = "operationTypeCodes", required = false) String
operationTypeCodes,
@RequestParam(value = "startDate", required = false) String startDate,
@RequestParam(value = "endDate", required = false) String endDate,
@RequestParam(value = "userName", required = false) String userName) {
checkPageParams(pageNo, pageSize);
PageInfo auditDtoPageInfo = auditService.queryLogListPaging(
loginUser,
-resourceType,
-operationType,
+objectTypeCodes,
+operationTypeCodes,
startDate,
endDate,
userName,
diff --git
Re: [PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
sonarcloud[bot] commented on PR #15553: URL: https://github.com/apache/dolphinscheduler/pull/15553#issuecomment-1922696044 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15553) **Quality Gate passed** Kudos, no new issues were introduced! [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler=15553=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler=15553=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15553) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
sonarcloud[bot] commented on PR #15553: URL: https://github.com/apache/dolphinscheduler/pull/15553#issuecomment-1922695131 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15553) **Quality Gate passed** Kudos, no new issues were introduced! [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler=15553=false=true) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler=15553=false=true) No data about Coverage No data about Duplication [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15553) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
codecov-commenter commented on PR #15553: URL: https://github.com/apache/dolphinscheduler/pull/15553#issuecomment-1922690351 ## [Codecov](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15553?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Comparison is base [(`8974233`)](https://app.codecov.io/gh/apache/dolphinscheduler/commit/89742332a8c19ee8d52a2dc786b460f4fc96ee62?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 38.40% compared to head [(`9d1ea44`)](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15553?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) 38.38%. > :exclamation: Current head 9d1ea44 differs from pull request most recent head 56580ab. Consider uploading reports for the commit 56580ab to get more accurate results Additional details and impacted files ```diff @@ Coverage Diff @@ ##dev #15553 +/- ## - Coverage 38.40% 38.38% -0.03% + Complexity 4753 4749 -4 Files 1305 1305 Lines 4484144801 -40 Branches 4800 4776 -24 - Hits 1722317197 -26 + Misses2574925737 -12 + Partials 1869 1867 -2 ``` [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15553?src=pr=continue_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix: data quality can not use [dolphinscheduler]
rickchengx commented on code in PR #15551:
URL:
https://github.com/apache/dolphinscheduler/pull/15551#discussion_r1475436092
##
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-api/src/main/java/org/apache/dolphinscheduler/plugin/datasource/api/utils/CommonUtils.java:
##
@@ -123,14 +132,56 @@ public static boolean loadKerberosConf(String
javaSecurityKrb5Conf, String login
return false;
}
-public static String getDataQualityJarName() {
-String dqsJarName = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
+public static String getDataQualityJarPath() {
+String dqsJarPath = PropertyUtils.getString(DATA_QUALITY_JAR_NAME);
-if (StringUtils.isEmpty(dqsJarName)) {
-return "dolphinscheduler-data-quality.jar";
+if (StringUtils.isEmpty(dqsJarPath)) {
+log.info("data quality jar path is empty, will try to get it from
data quality jar name");
+return getDefaultDataQualityJarPath();
}
-return dqsJarName;
+return dqsJarPath;
+}
+
+private static String getDefaultDataQualityJarPath() {
+if (StringUtils.isNotEmpty(DEFAULT_DATA_QUALITY_JAR_PATH)) {
Review Comment:
It seems that if the location of the jar package has been found once before,
it will be returned directly. Do we need to consider the situation where the
user has modified the name or location of the jar package?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Bug][Task Api] fix 'MACPATTERN' in ProcessUtils and cover all cases on MacOS in ProcessUtilsTest (#15480) [dolphinscheduler]
rickchengx commented on code in PR #15529:
URL:
https://github.com/apache/dolphinscheduler/pull/15529#discussion_r1475419812
##
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/test/java/org/apache/dolphinscheduler/plugin/task/api/utils/ProcessUtilsTest.java:
##
@@ -70,6 +72,7 @@ public void testGetPidsStr() throws Exception {
String exceptPidsStr3 = "5000 6000 7000 7100";
String command3;
if (SystemUtils.IS_OS_MAC) {
+pids3 = "-+= 5000 sshd -+- 6000 sshd --= 7000 bash --- 7100
python";
Review Comment:
Maybe we could test this method separately according to operating system,
what do u think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [Bug]Alert plug-in: the get request in HTTP does not have a port [dolphinscheduler]
github-actions[bot] commented on PR #11750: URL: https://github.com/apache/dolphinscheduler/pull/11750#issuecomment-1922543787 This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Improvement-#11822]Return more friendly return information in class of ProcessDefinitionServiceImpl,and remove invalid codes [dolphinscheduler]
github-actions[bot] commented on PR #11839: URL: https://github.com/apache/dolphinscheduler/pull/11839#issuecomment-1922543716 This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Improvement][SqlTask] Refactor SqlTask [dolphinscheduler]
github-actions[bot] closed issue #15218: [Improvement][SqlTask] Refactor SqlTask URL: https://github.com/apache/dolphinscheduler/issues/15218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Improvement][SqlTask] Refactor SqlTask [dolphinscheduler]
github-actions[bot] commented on issue #15218: URL: https://github.com/apache/dolphinscheduler/issues/15218#issuecomment-1922543487 This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
pegasas opened a new pull request, #15553: URL: https://github.com/apache/dolphinscheduler/pull/15553 ## Purpose of the pull request resolve https://github.com/apache/dolphinscheduler/issues/15545 ## Brief change log directly kill task if the strategy is timeout_failed. ## Verify this pull request This pull request is code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: (or) If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time [dolphinscheduler]
ruanwenjun commented on code in PR #15528:
URL:
https://github.com/apache/dolphinscheduler/pull/15528#discussion_r1474078555
##
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/taskgroup/TaskGroupCoordinator.java:
##
@@ -0,0 +1,451 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.server.master.runner.taskgroup;
+
+import org.apache.dolphinscheduler.common.constants.Constants;
+import org.apache.dolphinscheduler.common.enums.Flag;
+import org.apache.dolphinscheduler.common.enums.TaskGroupQueueStatus;
+import org.apache.dolphinscheduler.common.enums.WorkflowExecutionStatus;
+import org.apache.dolphinscheduler.common.lifecycle.ServerLifeCycleManager;
+import org.apache.dolphinscheduler.common.thread.BaseDaemonThread;
+import org.apache.dolphinscheduler.common.thread.ThreadUtils;
+import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
+import org.apache.dolphinscheduler.dao.entity.TaskGroup;
+import org.apache.dolphinscheduler.dao.entity.TaskGroupQueue;
+import org.apache.dolphinscheduler.dao.entity.TaskInstance;
+import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
+import org.apache.dolphinscheduler.dao.repository.TaskGroupDao;
+import org.apache.dolphinscheduler.dao.repository.TaskGroupQueueDao;
+import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao;
+import
org.apache.dolphinscheduler.extract.base.client.SingletonJdkDynamicRpcClientProxyFactory;
+import org.apache.dolphinscheduler.extract.master.IWorkflowInstanceService;
+import
org.apache.dolphinscheduler.extract.master.transportor.TaskInstanceWakeupRequest;
+import
org.apache.dolphinscheduler.extract.master.transportor.TaskInstanceWakeupResponse;
+import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus;
+import org.apache.dolphinscheduler.plugin.task.api.utils.LogUtils;
+import org.apache.dolphinscheduler.registry.api.RegistryClient;
+import org.apache.dolphinscheduler.registry.api.enums.RegistryNodeType;
+
+import org.apache.commons.collections4.CollectionUtils;
+import org.apache.commons.lang3.time.StopWatch;
+
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import lombok.extern.slf4j.Slf4j;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+/**
+ * The TaskGroupCoordinator use to manage the task group slot. The task group
slot is used to limit the number of {@link TaskInstance} that can be run at the
same time.
+ *
+ * The {@link TaskGroupQueue} is used to represent the task group slot. When a
{@link TaskGroupQueue} which inQueue is YES means the {@link TaskGroupQueue} is
using by a {@link TaskInstance}.
+ *
+ * When the {@link TaskInstance} need to use task group, we should use @{@link
TaskGroupCoordinator#acquireTaskGroupSlot(TaskInstance)} to acquire the task
group slot,
+ * this method doesn't block should always acquire successfully, and you
should directly stop dispatch the task instance.
+ * When the task group slot is available, the TaskGroupCoordinator will wake
up the waiting {@link TaskInstance} to dispatch.
+ *
+ * if(needAcquireTaskGroupSlot(taskInstance)) {
+ * taskGroupCoordinator.acquireTaskGroupSlot(taskInstance);
+ * return;
+ * }
+ *
+ *
+ * When the {@link TaskInstance} is finished, we should use @{@link
TaskGroupCoordinator#releaseTaskGroupSlot(TaskInstance)} to release the task
group slot.
+ *
+ * if(needToReleaseTaskGroupSlot(taskInstance)) {
+ * taskGroupCoordinator.releaseTaskGroupSlot(taskInstance);
+ * }
+ *
+ */
+@Slf4j
+@Component
+public class TaskGroupCoordinator extends BaseDaemonThread {
+
+@Autowired
+private RegistryClient registryClient;
+
+@Autowired
+private TaskGroupDao taskGroupDao;
+
+@Autowired
+private TaskGroupQueueDao taskGroupQueueDao;
+
+@Autowired
+private TaskInstanceDao taskInstanceDao;
+
+@Autowired
+private ProcessInstanceDao processInstanceDao;
+
+public TaskGroupCoordinator() {
+super("TaskGroupCoordinator");
+
Re: [PR] Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time [dolphinscheduler]
ruanwenjun merged PR #15528: URL: https://github.com/apache/dolphinscheduler/pull/15528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(dolphinscheduler) branch augit-log deleted (was c95faa4d7a)
This is an automated email from the ASF dual-hosted git repository. wangyang pushed a change to branch augit-log in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git was c95faa4d7a support multi codes This change permanently discards the following revisions: discard c95faa4d7a support multi codes discard 27277f820d Merge branch 'dev' into augit-log discard 844585af49 update discard 771e9342c0 init
(dolphinscheduler) 03/04: Merge branch 'dev' into augit-log
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 27277f820d5e6988d7f5303d1cc547f5abc84ac4
Merge: 844585af49 2f66a667ba
Author: Draven
AuthorDate: Thu Feb 1 16:10:47 2024 +0800
Merge branch 'dev' into augit-log
# Conflicts:
#
dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ClusterController.java
#
dolphinscheduler-dao/src/test/java/org/apache/dolphinscheduler/dao/mapper/AuditLogMapperTest.java
.github/workflows/backend.yml | 4 +-
.github/workflows/e2e.yml | 2 +
README.md | 2 +-
README_zh_CN.md| 2 +-
deploy/kubernetes/dolphinscheduler/README.md | 22 +-
deploy/kubernetes/dolphinscheduler/values.yaml | 45 +++-
docs/docs/en/architecture/configuration.md | 80 +++---
docs/docs/en/guide/installation/kubernetes.md | 2 +
docs/docs/en/guide/project/workflow-definition.md | 11 +-
docs/docs/en/guide/task/flink.md | 2 +-
docs/docs/en/guide/task/spark.md | 2 +-
docs/docs/zh/architecture/configuration.md | 226 +
docs/docs/zh/guide/installation/kubernetes.md | 2 +
docs/docs/zh/guide/project/workflow-definition.md | 13 +-
docs/docs/zh/guide/task/flink.md | 2 +-
docs/docs/zh/guide/task/spark.md | 2 +-
docs/img/new_ui/dev/project/workflow-list.png | Bin 143361 -> 155286 bytes
docs/img/new_ui/dev/project/workflow-time03.png| Bin 62574 -> 172800 bytes
docs/img/new_ui/dev/project/workflow-time04.png| Bin 0 -> 200575 bytes
docs/img/scheduler_online.png | Bin 0 -> 1703 bytes
docs/img/timeManagement.png| Bin 11195 -> 0 bytes
.../src/main/bin/jvm_args_env.sh | 1 +
.../apache/dolphinscheduler/alert/AlertServer.java | 4 +
.../dolphinscheduler/alert/config/AlertConfig.java | 8 +-
.../alert/metrics/AlertServerMetrics.java | 6 +
.../alert/registry/AlertHeartbeatTask.java | 17 +-
.../alert/registry/AlertRegistryClient.java| 6 +-
.../src/main/resources/application.yaml| 2 +-
.../api/test/cases/ProcessInstanceAPITest.java | 54 ++--
dolphinscheduler-api/src/main/bin/jvm_args_env.sh | 1 +
.../dolphinscheduler/api/ApiApplicationServer.java | 4 +
.../api/controller/AlertGroupController.java | 12 +-
.../api/controller/BaseController.java | 2 +-
.../api/controller/CloudController.java| 4 +-
.../api/controller/ClusterController.java | 2 +-
.../api/controller/DataAnalysisController.java | 12 +-
.../api/controller/DataSourceController.java | 6 +-
.../api/controller/DynamicTaskTypeController.java | 2 +-
.../api/controller/K8sNamespaceController.java | 6 +-
.../api/controller/LoginController.java| 2 +-
.../api/controller/MonitorController.java | 2 +-
.../api/controller/ProjectController.java | 2 +-
.../api/controller/ResourcesController.java| 2 +-
.../api/controller/SchedulerController.java| 2 +-
.../api/controller/TaskGroupController.java| 4 +-
.../api/controller/TenantController.java | 8 +-
.../api/controller/WorkFlowLineageController.java | 2 +-
.../api/controller/WorkerGroupController.java | 2 +-
.../v2/ProcessTaskRelationV2Controller.java| 2 +-
.../api/controller/v2/ProjectV2Controller.java | 2 +-
.../apache/dolphinscheduler/api/enums/Status.java | 2 +-
.../api/metrics/ApiServerMetrics.java | 8 +
.../api/service/DataAnalysisService.java | 18 +-
.../api/service/K8sNamespaceService.java | 4 +-
.../api/service/impl/DataAnalysisServiceImpl.java | 36 +--
.../api/service/impl/DataSourceServiceImpl.java| 1 +
.../api/service/impl/ExecutorServiceImpl.java | 2 +
.../api/service/impl/K8SNamespaceServiceImpl.java | 4 +-
.../api/service/impl/LoggerServiceImpl.java| 7 +-
.../service/impl/ProcessDefinitionServiceImpl.java | 1 +
.../api/service/impl/SchedulerServiceImpl.java | 12 +-
.../service/impl/TaskDefinitionServiceImpl.java| 4 +-
.../api/service/impl/TaskInstanceServiceImpl.java | 9 +-
.../api/vo/{ScheduleVo.java => ScheduleVO.java}| 4 +-
...TaskDefinitionVo.java => TaskDefinitionVO.java} | 10 +-
...stanceCountVo.java => TaskInstanceCountVO.java} | 8 +-
...CountVo.java => WorkflowDefinitionCountVO.java} | 10 +-
...ceCountVo.java => WorkflowInstanceCountVO.java} | 8 +-
.../src/main/resources/i18n/messages.properties| 2 +-
.../main/resources/i18n/messages_en_US.properties | 2 +-
(dolphinscheduler) 04/04: support multi codes
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit c95faa4d7a5a8ab777233d7baac9bc457bffd106
Author: Draven
AuthorDate: Thu Feb 1 16:59:07 2024 +0800
support multi codes
---
.../api/controller/AuditLogController.java | 16
.../dolphinscheduler/api/service/AuditService.java | 22 +--
.../api/service/impl/AuditServiceImpl.java | 46 --
.../api/service/AuditServiceTest.java | 8 ++--
.../dao/mapper/AuditLogMapper.java | 5 ++-
.../dolphinscheduler/dao/mapper/AuditLogMapper.xml | 19 ++---
.../dao/mapper/AuditLogMapperTest.java | 4 +-
7 files changed, 67 insertions(+), 53 deletions(-)
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
index b77e7e095d..afae94cff1 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/AuditLogController.java
@@ -59,8 +59,8 @@ public class AuditLogController extends BaseController {
*
* @param loginUser login user
* @param pageNopage number
- * @param resourceType resource type
- * @param operationType operation type
+ * @param objectTypeCodes object type codes
+ * @param operationTypeCodesoperation type codes
* @param startDate start time
* @param endDate end time
* @param userName user name
@@ -71,8 +71,8 @@ public class AuditLogController extends BaseController {
@Parameters({
@Parameter(name = "startDate", description = "START_DATE", schema
= @Schema(implementation = String.class)),
@Parameter(name = "endDate", description = "END_DATE", schema =
@Schema(implementation = String.class)),
-@Parameter(name = "resourceType", description = "RESOURCE_TYPE",
schema = @Schema(implementation = AuditObjectType.class)),
-@Parameter(name = "operationType", description = "OPERATION_TYPE",
schema = @Schema(implementation = AuditOperationType.class)),
+@Parameter(name = "objectTypeCodes", description =
"OBJECT_TYPE_CODES", schema = @Schema(implementation = String.class)),
+@Parameter(name = "operationTypeCodes", description =
"OPERATION_TYPE_CODES", schema = @Schema(implementation = String.class)),
@Parameter(name = "userName", description = "USER_NAME", schema =
@Schema(implementation = String.class)),
@Parameter(name = "pageNo", description = "PAGE_NO", required =
true, schema = @Schema(implementation = int.class, example = "1")),
@Parameter(name = "pageSize", description = "PAGE_SIZE", required
= true, schema = @Schema(implementation = int.class, example = "20"))
@@ -83,16 +83,16 @@ public class AuditLogController extends BaseController {
public Result>
queryAuditLogListPaging(@Parameter(hidden = true) @RequestAttribute(value =
Constants.SESSION_USER) User loginUser,
@RequestParam("pageNo") Integer pageNo,
@RequestParam("pageSize") Integer pageSize,
-
@RequestParam(value = "resourceType", required = false) AuditObjectType
resourceType,
-
@RequestParam(value = "operationType", required = false) AuditOperationType
operationType,
+
@RequestParam(value = "objectTypeCodes", required = false) String
objectTypeCodes,
+
@RequestParam(value = "operationTypeCodes", required = false) String
operationTypeCodes,
@RequestParam(value = "startDate", required = false) String startDate,
@RequestParam(value = "endDate", required = false) String endDate,
@RequestParam(value = "userName", required = false) String userName) {
checkPageParams(pageNo, pageSize);
PageInfo auditDtoPageInfo = auditService.queryLogListPaging(
loginUser,
-resourceType,
-operationType,
+objectTypeCodes,
+operationTypeCodes,
startDate,
endDate,
userName,
diff --git
(dolphinscheduler) branch augit-log created (now c95faa4d7a)
This is an automated email from the ASF dual-hosted git repository. wangyang pushed a change to branch augit-log in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git at c95faa4d7a support multi codes This branch includes the following new commits: new 771e9342c0 init new 844585af49 update new 27277f820d Merge branch 'dev' into augit-log new c95faa4d7a support multi codes The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(dolphinscheduler) 01/04: init
This is an automated email from the ASF dual-hosted git repository.
wangyang pushed a commit to branch augit-log
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 771e9342c01e520f5efa7bbfcfaab7c2d627a153
Author: Draven
AuthorDate: Mon Jan 15 15:34:31 2024 +0800
init
---
.../dolphinscheduler/api/audit/AuditMessage.java | 60 +++-
.../api/audit/AuditPublishService.java | 92 --
.../api/audit/AuditSubscriber.java | 28 --
.../api/audit/AuditSubscriberImpl.java | 42
.../dolphinscheduler/api/audit/OperatorLog.java| 21
.../api/audit/OperatorLogAspect.java | 108 +
.../api/controller/AuditLogController.java | 8 +-
.../controller/ProcessDefinitionController.java| 4 +
.../api/controller/ProjectController.java | 10 ++
.../dolphinscheduler/api/service/AuditService.java | 20 ++--
.../api/service/impl/AuditServiceImpl.java | 38
.../dolphinscheduler/api/utils/AuditLogUtils.java | 80 +++
.../src/main/resources/application.yaml| 23 ++---
.../src/main/resources/logback-spring.xml | 7 +-
.../api/audit/AuditSubscriberTest.java | 9 +-
.../api/service/AuditServiceTest.java | 4 +-
dolphinscheduler-bom/pom.xml | 1 -
.../common/enums/Audit/AuditObjectType.java| 102 +++
.../enums/{ => Audit}/AuditOperationType.java | 34 ---
.../common/enums/AuditResourceType.java| 61
.../dolphinscheduler/dao/entity/AuditLog.java | 62 ++--
.../dao/mapper/AuditLogMapper.java | 6 +-
.../dolphinscheduler/dao/mapper/AuditLogMapper.xml | 22 ++---
.../dao/mapper/AuditLogMapperTest.java | 1 -
.../service/process/ProcessServiceImpl.java| 1 +
25 files changed, 419 insertions(+), 425 deletions(-)
diff --git
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
index 0d936f0ad5..893460590d 100644
---
a/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
+++
b/dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/audit/AuditMessage.java
@@ -17,70 +17,32 @@
package org.apache.dolphinscheduler.api.audit;
-import org.apache.dolphinscheduler.common.enums.AuditOperationType;
-import org.apache.dolphinscheduler.common.enums.AuditResourceType;
+import lombok.Data;
+import org.apache.dolphinscheduler.common.enums.Audit.AuditObjectType;
+import org.apache.dolphinscheduler.common.enums.Audit.AuditOperationType;
import org.apache.dolphinscheduler.dao.entity.User;
import java.util.Date;
+@Data
public class AuditMessage {
private User user;
private Date auditDate;
-private AuditResourceType resourceType;
+private AuditObjectType objectType;
-private AuditOperationType operation;
+private AuditOperationType operationType;
private Integer resourceId;
-public AuditMessage(User user, Date auditDate, AuditResourceType
resourceType, AuditOperationType operation,
+public AuditMessage(User user, Date auditDate, AuditObjectType objectType,
AuditOperationType operationType,
Integer resourceId) {
this.user = user;
this.auditDate = auditDate;
-this.resourceType = resourceType;
-this.operation = operation;
-this.resourceId = resourceId;
-}
-
-public User getUser() {
-return user;
-}
-
-public void setUser(User user) {
-this.user = user;
-}
-
-public Date getAuditDate() {
-return auditDate;
-}
-
-public void setAuditDate(Date auditDate) {
-this.auditDate = auditDate;
-}
-
-public AuditResourceType getResourceType() {
-return resourceType;
-}
-
-public void setResourceType(AuditResourceType resourceType) {
-this.resourceType = resourceType;
-}
-
-public AuditOperationType getOperation() {
-return operation;
-}
-
-public void setOperation(AuditOperationType operation) {
-this.operation = operation;
-}
-
-public Integer getResourceId() {
-return resourceId;
-}
-
-public void setResourceId(Integer resourceId) {
+this.objectType = objectType;
+this.operationType = operationType;
this.resourceId = resourceId;
}
@@ -89,8 +51,8 @@ public class AuditMessage {
return "AuditMessage{"
+ "user=" + user
+ ", Date=" + auditDate
-+ ", resourceType" + resourceType
-+ ", operation=" + operation
++ ", objectType" + objectType
++ ", operationType=" + operationType
Re: [I] [Bug] [API] 3.2.0 The restart of MySQL 5.7 can cause an Auto Increment ID reset, potentially resulting in the inability to execute command. [dolphinscheduler]
Zzih96 commented on issue #15543: URL: https://github.com/apache/dolphinscheduler/issues/15543#issuecomment-1920765825 > I think add a code field would be better I noticed that the `job_id` field in the `t_ds_trigger_relation` table also holds the `id` field from the `t_ds_process_instance` table. If I add a `code` field to the `t_ds_command` table, should I also add a `code` field to the `t_ds_process_instance` table for association? This change may have significant implications, so I believe it warrants further discussion. https://github.com/apache/dolphinscheduler/assets/158246610/e9c8c802-7d7e-4678-9961-e10f96bd54eb;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] [Master] task_execute_type column should not be nullable column [dolphinscheduler]
leachli commented on issue #15055: URL: https://github.com/apache/dolphinscheduler/issues/15055#issuecomment-1920760568 I also have this error, but it is caused by the empty status. I have no problem in pseudo-cluster mode. This problem occurs in production cluster, could you please tell me how to solve it? [ERROR] 2024-02-01 15:26:04.471 +0800 org.apache.dolphinscheduler.server.master.runner.WorkflowExecuteRunnable:[317] - [WorkflowInstance-265][TaskInstance-837] - State event handle error, get a unknown exception, will retry this event: TaskStateEvent(processInstanceId=265, taskInstanceId=837, taskCode=0, status=null, type=TASK_STATE_CHANGE, key=267-0-265-837, channel=null, context=null) java.lang.NullPointerException: null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time [dolphinscheduler]
Gallardot commented on code in PR #15528:
URL:
https://github.com/apache/dolphinscheduler/pull/15528#discussion_r1473965372
##
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/taskgroup/TaskGroupCoordinator.java:
##
@@ -0,0 +1,451 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.server.master.runner.taskgroup;
+
+import org.apache.dolphinscheduler.common.constants.Constants;
+import org.apache.dolphinscheduler.common.enums.Flag;
+import org.apache.dolphinscheduler.common.enums.TaskGroupQueueStatus;
+import org.apache.dolphinscheduler.common.enums.WorkflowExecutionStatus;
+import org.apache.dolphinscheduler.common.lifecycle.ServerLifeCycleManager;
+import org.apache.dolphinscheduler.common.thread.BaseDaemonThread;
+import org.apache.dolphinscheduler.common.thread.ThreadUtils;
+import org.apache.dolphinscheduler.dao.entity.ProcessInstance;
+import org.apache.dolphinscheduler.dao.entity.TaskGroup;
+import org.apache.dolphinscheduler.dao.entity.TaskGroupQueue;
+import org.apache.dolphinscheduler.dao.entity.TaskInstance;
+import org.apache.dolphinscheduler.dao.repository.ProcessInstanceDao;
+import org.apache.dolphinscheduler.dao.repository.TaskGroupDao;
+import org.apache.dolphinscheduler.dao.repository.TaskGroupQueueDao;
+import org.apache.dolphinscheduler.dao.repository.TaskInstanceDao;
+import
org.apache.dolphinscheduler.extract.base.client.SingletonJdkDynamicRpcClientProxyFactory;
+import org.apache.dolphinscheduler.extract.master.IWorkflowInstanceService;
+import
org.apache.dolphinscheduler.extract.master.transportor.TaskInstanceWakeupRequest;
+import
org.apache.dolphinscheduler.extract.master.transportor.TaskInstanceWakeupResponse;
+import org.apache.dolphinscheduler.plugin.task.api.enums.TaskExecutionStatus;
+import org.apache.dolphinscheduler.plugin.task.api.utils.LogUtils;
+import org.apache.dolphinscheduler.registry.api.RegistryClient;
+import org.apache.dolphinscheduler.registry.api.enums.RegistryNodeType;
+
+import org.apache.commons.collections4.CollectionUtils;
+import org.apache.commons.lang3.time.StopWatch;
+
+import java.util.Date;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import lombok.extern.slf4j.Slf4j;
+
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+/**
+ * The TaskGroupCoordinator use to manage the task group slot. The task group
slot is used to limit the number of {@link TaskInstance} that can be run at the
same time.
+ *
+ * The {@link TaskGroupQueue} is used to represent the task group slot. When a
{@link TaskGroupQueue} which inQueue is YES means the {@link TaskGroupQueue} is
using by a {@link TaskInstance}.
+ *
+ * When the {@link TaskInstance} need to use task group, we should use @{@link
TaskGroupCoordinator#acquireTaskGroupSlot(TaskInstance)} to acquire the task
group slot,
+ * this method doesn't block should always acquire successfully, and you
should directly stop dispatch the task instance.
+ * When the task group slot is available, the TaskGroupCoordinator will wake
up the waiting {@link TaskInstance} to dispatch.
+ *
+ * if(needAcquireTaskGroupSlot(taskInstance)) {
+ * taskGroupCoordinator.acquireTaskGroupSlot(taskInstance);
+ * return;
+ * }
+ *
+ *
+ * When the {@link TaskInstance} is finished, we should use @{@link
TaskGroupCoordinator#releaseTaskGroupSlot(TaskInstance)} to release the task
group slot.
+ *
+ * if(needToReleaseTaskGroupSlot(taskInstance)) {
+ * taskGroupCoordinator.releaseTaskGroupSlot(taskInstance);
+ * }
+ *
+ */
+@Slf4j
+@Component
+public class TaskGroupCoordinator extends BaseDaemonThread {
+
+@Autowired
+private RegistryClient registryClient;
+
+@Autowired
+private TaskGroupDao taskGroupDao;
+
+@Autowired
+private TaskGroupQueueDao taskGroupQueueDao;
+
+@Autowired
+private TaskInstanceDao taskInstanceDao;
+
+@Autowired
+private ProcessInstanceDao processInstanceDao;
+
+public TaskGroupCoordinator() {
+super("TaskGroupCoordinator");
+
