Re: [PR] [Bug] [Master] kill task when TaskInstance set TimeoutFailed [dolphinscheduler]
rickchengx commented on code in PR #15553:
URL:
https://github.com/apache/dolphinscheduler/pull/15553#discussion_r1477196878
##
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/runner/operator/BaseTaskExecuteRunnableTimeoutOperator.java:
##
@@ -38,18 +38,8 @@ public
BaseTaskExecuteRunnableTimeoutOperator(TaskInstanceDao taskInstanceDao) {
@Override
public void operate(DefaultTaskExecuteRunnable taskExecuteRunnable) {
-// Right now, if the task is running in worker, the timeout strategy
will be handled at worker side.
-// if the task is in master, the timeout strategy will be handled at
master side.
-// todo: we should unify this, the master only need to handle the
timeout strategy. and send request to worker
-// to kill the task, if the strategy is timeout_failed.
TaskInstance taskInstance = taskExecuteRunnable.getTaskInstance();
TaskTimeoutStrategy taskTimeoutStrategy =
taskInstance.getTaskDefine().getTimeoutNotifyStrategy();
-if (TaskTimeoutStrategy.FAILED != taskTimeoutStrategy
Review Comment:
I'm a little confused, I think it will also trigger
`killRemoteTaskInstanceInThreadPool(taskInstance)` in line 45 when
taskTimeoutStrategy == TaskTimeoutStrategy.FAILED
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [fix-#14858][datasource] can not correctly parse the 'other' parameter [dolphinscheduler]
fuchanghai commented on PR #15558: URL: https://github.com/apache/dolphinscheduler/pull/15558#issuecomment-1925609436 same as PR #15551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(dolphinscheduler-website) branch asf-site updated: Automated deployment: f927da5fa0fbf637642914e2ea1a5a43e8829908
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 8ec7cae206 Automated deployment: f927da5fa0fbf637642914e2ea1a5a43e8829908 8ec7cae206 is described below commit 8ec7cae206658f42e1a2709f2d10b2991eb71da9 Author: github-actions[bot] AuthorDate: Sun Feb 4 05:05:53 2024 + Automated deployment: f927da5fa0fbf637642914e2ea1a5a43e8829908 --- python/2.0.5/.buildinfo | 2 +- python/2.0.6/.buildinfo | 2 +- python/2.0.7/.buildinfo | 2 +- python/3.0.0/.buildinfo | 2 +- python/3.0.1/.buildinfo | 2 +- python/3.1.0/.buildinfo | 2 +- python/4.0.0/.buildinfo | 2 +- python/4.0.1/.buildinfo | 2 +- python/4.0.2/.buildinfo | 2 +- python/4.0.3/.buildinfo | 2 +- python/4.0.4/.buildinfo | 2 +- python/main/.buildinfo | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/python/2.0.5/.buildinfo b/python/2.0.5/.buildinfo index dc0a394a84..ebd10cc7c0 100644 --- a/python/2.0.5/.buildinfo +++ b/python/2.0.5/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 947ea2e40cfcfc68f4f61dd913f62df8 +config: 0d0121ee2f30740ed20c10b7a9236e2e tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/2.0.6/.buildinfo b/python/2.0.6/.buildinfo index cd946bb410..c51a6265fb 100644 --- a/python/2.0.6/.buildinfo +++ b/python/2.0.6/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: d90249d0f0b0570920f63b7bcf1d9c7c +config: 06bfacb285cdce43b534f0c8d944bb46 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/2.0.7/.buildinfo b/python/2.0.7/.buildinfo index b4046575af..1fca93ef80 100644 --- a/python/2.0.7/.buildinfo +++ b/python/2.0.7/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 2fbe5863987d16d4ce30404f70a5e048 +config: e0a08abc6f606cc69ea6737a4f089814 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.0.0/.buildinfo b/python/3.0.0/.buildinfo index 7bdeaca27f..641edaddc9 100644 --- a/python/3.0.0/.buildinfo +++ b/python/3.0.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: b56624eb59d7e2818bdaf8c3bb523dd8 +config: a8222c1ff9f223709e7a77f0e4e0d879 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.0.1/.buildinfo b/python/3.0.1/.buildinfo index 05887f1149..25c65a6dee 100644 --- a/python/3.0.1/.buildinfo +++ b/python/3.0.1/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 378bcecb776c3b797f88515ac9295e1a +config: ada31dab9e59e90f410c01681bcc8046 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/3.1.0/.buildinfo b/python/3.1.0/.buildinfo index ae79dd21f0..0f96f59997 100644 --- a/python/3.1.0/.buildinfo +++ b/python/3.1.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: b4970e3ac224026964c9618cda46c4b9 +config: b81e0e78a3df4f3740c6ed92cd662587 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.0/.buildinfo b/python/4.0.0/.buildinfo index 1bcb50e0b0..55f3a9e635 100644 --- a/python/4.0.0/.buildinfo +++ b/python/4.0.0/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 2c91da3d4f27f91436a38172795a +config: e8b26a4250bf9135bbc8e44909c5f192 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.1/.buildinfo b/python/4.0.1/.buildinfo index adbbe9f26e..1914b83410 100644 --- a/python/4.0.1/.buildinfo +++ b/python/4.0.1/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: c0a58c2421f3673810e30e6f97bd4082 +config: 82361802f975596c207f1b48951facc1 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/python/4.0.2/.buildinfo b/python/4.0.2/.buildinfo index 12c35b7adb..b5cea1c7bb 100644 --- a/python/4.0.2/.buildinfo +++ b/python/4.0.2/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 8e91bc23792405248530e5d3d127626d
[PR] [fix-#14858][datasource] can not correctly parse the 'other' parameter [dolphinscheduler]
JohnZp opened a new pull request, #15558: URL: https://github.com/apache/dolphinscheduler/pull/15558 ## Purpose of the pull request fix this bug #14858 ## Brief change log ## Verify this pull request This pull request is code cleanup without any test coverage. *(or)* This pull request is already covered by existing tests, such as *(please describe tests)*. (or) This change added tests and can be verified as follows: (or) If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature][alert] Get notification of task/job/workflow changes [dolphinscheduler]
EricGao888 commented on issue #10482: URL: https://github.com/apache/dolphinscheduler/issues/10482#issuecomment-1925564919 This is an interesting feature. However, as we need to notify users on state change, we need to add some code into core modules to support it. If we handle this inappropriately, it could be really hard to maintain and cause potential critical bugs. Therefore, I do not suggest we do this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [RoadMap][Year 2024] Community RoadMap [dolphinscheduler]
zhongjiajie commented on issue #13459: URL: https://github.com/apache/dolphinscheduler/issues/13459#issuecomment-1925563924 > > * For some topics which need further discussions, we temporarily note the owner as `Community`. > > > > ### RoadMap 2024 > > > > * Cloud Native > > > > * [ ] K8S Executor [[Feature][Executor] Add K8S Executor for task dispatching #13316](https://github.com/apache/dolphinscheduler/issues/13316) > > * User Friendly > > > > * [ ] Dynamic Task Plugin [DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526](https://github.com/apache/dolphinscheduler/issues/12526) > > * [ ] Task Parameters Sharing Optimization > > * [ ] Independent Release of Task Plugin > > * [ ] Audit log improvement design [[Improvement][Audit log] Audit log improvement design #15423](https://github.com/apache/dolphinscheduler/issues/15423) > > * New Important Task or Task Related > > > > * [ ] Support Streaming Task [[DSIP-16][Task] Support stream task #11352](https://github.com/apache/dolphinscheduler/issues/11352) > > * Remote Task > > > > * [ ] [[Feature] DS can support task running on remote host, not just worker server. #11652](https://github.com/apache/dolphinscheduler/issues/11652) > > * [ ] [TriggerPlugin] Introduce Trigger Plugin [[DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330](https://github.com/apache/dolphinscheduler/issues/15330) > > * [ ] Task Sharing Storage > > * [ ] Trigger Workflow by Event > > * [ ] Lifecycle method to TaskPlugin [[DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin #14877](https://github.com/apache/dolphinscheduler/issues/14877) > > * [ ] Unify the task spi about logic task and worker task [[DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task #14823](https://github.com/apache/dolphinscheduler/issues/14823) > > * Test Improvement > > > > * [ ] API Test > > * [ ] UT Improvement [[DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573](https://github.com/apache/dolphinscheduler/issues/10573) > > * [ ] [[DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330](https://github.com/apache/dolphinscheduler/issues/15330) > > * Data Ops > > > > * [ ] Git Ops Support > > * [ ] Git CICD related > > could add loop task？ #14619 not sure whether is like dynamic task, so I add some comment on it, which you can see in https://github.com/apache/dolphinscheduler/issues/14619#issuecomment-1925563753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Feature][dolphinscheduler-master] Add loop task [dolphinscheduler]
zhongjiajie commented on issue #14619: URL: https://github.com/apache/dolphinscheduler/issues/14619#issuecomment-1925563753 is this same or like dynamic task? https://dolphinscheduler.apache.org/en-us/docs/3.2.0/guide/task/dynamic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Improvement][Dao] reduce code specification warnings #11705 [dolphinscheduler]
github-actions[bot] commented on PR #11722: URL: https://github.com/apache/dolphinscheduler/pull/11722#issuecomment-1925499069 This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Improvement][Unit Tests] Improve DolphinScheduler Worker unit tests [dolphinscheduler]
github-actions[bot] commented on issue #15396: URL: https://github.com/apache/dolphinscheduler/issues/15396#issuecomment-1925499035 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Improvement][API] Project List Load too Slow [dolphinscheduler]
github-actions[bot] commented on issue #15425: URL: https://github.com/apache/dolphinscheduler/issues/15425#issuecomment-1925499023 This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(dolphinscheduler) branch dev updated: Fix createFile with permission will not work (#15556)
This is an automated email from the ASF dual-hosted git repository.
wenjun pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
The following commit(s) were added to refs/heads/dev by this push:
new 4d15932ede Fix createFile with permission will not work (#15556)
4d15932ede is described below
commit 4d15932ede770bb65786c697bf16cdd4f2cc02be
Author: Wenjun Ruan
AuthorDate: Sat Feb 3 18:00:33 2024 +0800
Fix createFile with permission will not work (#15556)
---
.../dolphinscheduler/common/utils/FileUtils.java | 139 +
.../common/utils/FileUtilsTest.java| 24 +++-
.../service/utils/ProcessUtils.java| 3 +-
.../plugin/storage/abs/AbsStorageOperator.java | 3 +-
.../plugin/storage/gcs/GcsStorageOperator.java | 3 +-
.../plugin/storage/obs/ObsStorageOperator.java | 3 +-
.../plugin/storage/oss/OssStorageOperator.java | 3 +-
.../plugin/storage/s3/S3StorageOperator.java | 3 +-
.../shell/BaseLinuxShellInterceptorBuilder.java| 8 +-
.../plugin/task/java/JavaTaskTest.java | 3 +-
.../worker/utils/TaskExecutionContextUtils.java| 3 +-
.../utils/TaskExecutionContextUtilsTest.java | 2 +-
12 files changed, 72 insertions(+), 125 deletions(-)
diff --git
a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java
b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java
index c71c5f2e47..1f3ccfed3d 100644
---
a/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java
+++
b/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java
@@ -25,8 +25,6 @@ import static
org.apache.dolphinscheduler.common.constants.Constants.RESOURCE_VI
import static org.apache.dolphinscheduler.common.constants.Constants.UTF_8;
import static
org.apache.dolphinscheduler.common.constants.DateConstants.MMDDHHMMSS;
-import org.apache.dolphinscheduler.common.exception.FileOperateException;
-
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.SystemUtils;
@@ -38,9 +36,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
-import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
-import java.nio.file.attribute.FileAttribute;
import java.nio.file.attribute.PosixFilePermission;
import java.nio.file.attribute.PosixFilePermissions;
import java.util.Set;
@@ -48,11 +44,10 @@ import java.util.zip.CRC32;
import java.util.zip.CheckedInputStream;
import lombok.NonNull;
+import lombok.experimental.UtilityClass;
import lombok.extern.slf4j.Slf4j;
-/**
- * file utils
- */
+@UtilityClass
@Slf4j
public class FileUtils {
@@ -62,14 +57,7 @@ public class FileUtils {
public static final String KUBE_CONFIG_FILE = "config";
-private static final String RWXR_XR_X = "rwxr-xr-x";
-
-private static final FileAttribute>
PERMISSION_755 =
-
PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString(RWXR_XR_X));
-
-private FileUtils() {
-throw new UnsupportedOperationException("Construct FileUtils");
-}
+private static final Set PERMISSION_755 =
PosixFilePermissions.fromString("rwxr-xr-x");
/**
* get download file absolute path and name
@@ -161,34 +149,6 @@ public class FileUtils {
return PropertyUtils.getString(RESOURCE_VIEW_SUFFIXES,
RESOURCE_VIEW_SUFFIXES_DEFAULT_VALUE);
}
-/**
- * create directory if absent
- *
- * @param execLocalPath execute local path
- * @throws IOException errors
- */
-public static void createWorkDirIfAbsent(String execLocalPath) throws
IOException {
-// if work dir exists, first delete
-File execLocalPathFile = new File(execLocalPath);
-
-if (execLocalPathFile.exists()) {
-try {
-org.apache.commons.io.FileUtils.forceDelete(execLocalPathFile);
-} catch (Exception ex) {
-if (ex instanceof NoSuchFileException || ex.getCause()
instanceof NoSuchFileException) {
-// this file is already be deleted.
-} else {
-throw ex;
-}
-}
-}
-
-// create work dir
-org.apache.commons.io.FileUtils.forceMkdir(execLocalPathFile);
-String mkdirLog = "create dir success " + execLocalPath;
-log.info(mkdirLog);
-}
-
/**
* write content to file ,if parent path not exists, it will do one's
utmost to mkdir
*
@@ -231,25 +191,6 @@ public class FileUtils {
org.apache.commons.io.FileUtils.deleteQuietly(new File(filename));
}
-/**
- * Gets all the parent subdirectories of the parentDir directory
- *
- * @param parentDir parent dir
- * @return all
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
ruanwenjun merged PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
ruanwenjun commented on PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556#issuecomment-1925251719 > but we have a secutity scan in this pr It's OK, since the path is controlled by application, the path is not from user input. https://github.com/apache/dolphinscheduler/assets/22415594/c11dffcf-efd3-40f9-8aad-847b93bc3ec9;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] After the service is restarted, new tasks are not scheduled and old tasks are scheduled normally. [dolphinscheduler]
q4q5q6qw commented on issue #15557:
URL:
https://github.com/apache/dolphinscheduler/issues/15557#issuecomment-1925251286
[INFO] 2024-02-03 17:50:49.460 +0800
org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[107] - REQUEST
TRACE_ID:12483001530752, LOGIN_USER:csb_soc_user,
URI:/dolphinscheduler/projects/7848358598592/process-definition/12483001532224/release,
METHOD:POST,
HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.releaseProcessDefinition,
ARGS:{code=12483001532224, projectCode=7848358598592, releaseState=ONLINE}
[INFO] 2024-02-03 17:50:49.641 +0800
org.apache.dolphinscheduler.service.cache.impl.CacheNotifyServiceImpl:[115] -
send result, command:Command [type=CACHE_EXPIRE, opaque=3295, bodyLen=62]
[INFO] 2024-02-03 17:50:49.644 +0800
org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl:[1120]
- Set process definition online, projectCode:7848358598592,
processDefinitionCode:12483001532224.
[INFO] 2024-02-03 17:50:49.671 +0800
org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[122] - Call
POST:/dolphinscheduler/projects/7848358598592/process-definition/12483001532224/release
success, cost: 211ms
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
caishunfeng commented on PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556#issuecomment-1925248816 Is it better to add some docs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
zhongjiajie commented on PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556#issuecomment-1925246004 but we have a secutity scan in this pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] After the service is restarted, new tasks are not scheduled and old tasks are scheduled normally. [dolphinscheduler]
q4q5q6qw commented on issue #15557: URL: https://github.com/apache/dolphinscheduler/issues/15557#issuecomment-1925236660 Complement: In the t_ds_process_definition_log table, the value of release_state of the new task is 0. However, in the t_ds_process_definition table, the value of release_state is 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] After the service is restarted, new tasks are not scheduled and old tasks are scheduled normally. [dolphinscheduler]
q4q5q6qw opened a new issue, #15557: URL: https://github.com/apache/dolphinscheduler/issues/15557 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened I changed the exec_threads of the master and worker services. After the restart, the newly created tasks do not generate new process instances, but the old tasks can continue to be executed. There is no obvious error in the logs of the master and worker. The qrtz_cron_triggers and qrtz_triggers tables contain the data of the new task. NEXT_FIRE_TIME is the time when the task should be executed for the first time. However, the task is not invoked. The t_ds_process_instance and t_ds_task_instance tables do not contain the new task instance. Now the exec_threads before the restore is not able to execute the new task. How can I locate the fault? ### What you expected to happen The new task can be executed properly. ### How to reproduce modified the exec_threads of the master and worker services. and restart ### Anything else _No response_ ### Version 3.2.x ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
sonarcloud[bot] commented on PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556#issuecomment-1925206217 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15556) **Quality Gate failed** Failed conditions [28.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler=15556=new_coverage=list) (required ≥ 60%) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15556) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
sonarcloud[bot] commented on PR #15556: URL: https://github.com/apache/dolphinscheduler/pull/15556#issuecomment-1925206277 ## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15556) **Quality Gate failed** Failed conditions [28.1% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler=15556=new_coverage=list) (required ≥ 60%) [See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler=15556) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix createFile with permission will not work [dolphinscheduler]
github-advanced-security[bot] commented on code in PR #15556:
URL:
https://github.com/apache/dolphinscheduler/pull/15556#discussion_r1477011637
##
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/FileUtils.java:
##
@@ -325,59 +266,47 @@
return crcString;
}
-public static void setFileOwner(Path filePath, String fileOwner) throws
FileOperateException {
-try {
-// We use linux command to set the file owner, since jdk api will
not use sudo.
-String command = String.format("sudo chown %s %s", fileOwner,
filePath.toString());
-Runtime.getRuntime().exec(command);
-Process process = Runtime.getRuntime().exec(command);
-int exitCode = process.waitFor();
-if (0 != exitCode) {
-throw new FileOperateException(
-"Set file: " + filePath + " to owner: " + fileOwner +
" failed, existCode(" + exitCode + ")");
-}
-} catch (FileOperateException ex) {
-throw ex;
-} catch (Exception ex) {
-throw new FileOperateException("Set directory: " + filePath + " to
owner: " + fileOwner + " failed");
-
+public static void createFileWith755(@NonNull Path path) throws
IOException {
+if (SystemUtils.IS_OS_WINDOWS) {
+Files.createFile(path);
+} else {
+Files.createFile(path);
+Files.setPosixFilePermissions(path, PERMISSION_755);
}
}
-public static void setDirectoryOwner(Path filePath, String fileOwner)
throws FileOperateException {
-try {
-// We use linux command to set the file owner, since jdk api will
not use sudo.
-String command = String.format("sudo chown -R %s %s", fileOwner,
filePath.toString());
-Runtime.getRuntime().exec(command);
-Process process = Runtime.getRuntime().exec(command);
-int exitCode = process.waitFor();
-if (0 != exitCode) {
-throw new FileOperateException("Set directory: " + filePath +
" to owner: " + fileOwner
-+ " failed, existCode(" + exitCode + ")");
+public static void createDirectoryWith755(@NonNull Path path) throws
IOException {
+if (path.toFile().exists()) {
+return;
+}
+if (OSUtils.isWindows()) {
+Files.createDirectories(path);
+} else {
+Path parent = path.getParent();
+if (parent != null && !parent.toFile().exists()) {
+createDirectoryWith755(parent);
}
-} catch (FileOperateException ex) {
-throw ex;
-} catch (Exception ex) {
-throw new FileOperateException("Set directory: " + filePath + " to
owner: " + fileOwner + " failed");
+
+Files.createDirectory(path);
Review Comment:
## Uncontrolled data used in path expression
This path depends on a [user-provided value](1).
[Show more
details](https://github.com/apache/dolphinscheduler/security/code-scanning/3894)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
