This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a change to branch 3.2.0-release
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
omit a1cf67e19c [Bug][Worker] Global and local parameters are not replaced
in SQL components (#14926)
omit b9adaf58e0 [Bug][Worker] Fix sql/shell task doesn't replace parameter
(#14876)
new baf7b5ef45 [Improvement-14835][UI] Support to immediately adjust the
fill color of the task node after changing the run flag of the task. (#14837)
new 9d4f9a7457 feat: Add others scopes for release tool (#14844)
new fdc2421817 [Feature-14802][api] Use Casdoor SSO to log in, add admin
user configuration (#14814)
new ce935c5c99 chore: add revert label check (#14853)
new 3a749d0109 [Improve] Fix placeholders should match arguments in log
statement (#14850)
new 984eebd0c4 [Bug] [UI] New tasks with the flag normal show a grey color
(#14862)
new 121bb3d854 Fix session/thread leak at ssh datasource (#14859)
new fd00bb605a [Fix][Resource Center] Remove NONE storage type and set
LOCAL type as default (#14661)
new 6c1d14dcfe [Fix-14811][fix] Fix when using Casdoor SSO, ldap, OAuth2
to log in, the tenant problem of not specifying the currently logged in user
(#14812)
new 5270cc62a4 [Improvement][doc] fix typo in readme (#14878)
new eb9a6ab4ce [Improvement][UI] Optimize task disable node style in dark
theme (#14871)
new e71ae0da55 [Bug][Worker] Fix sql/shell task doesn't replace parameter
(#14876)
new e2c8815457 fix(dolphinscheduler-alert): fix create http request error
(#14793)
new 82926f1850 [Fix-14790][dolphinscheduler-ui] Fix the data quality task,
keyword filtering not supported (#14792)
new 06e5b21b4f Fix java.sql.Date serialize error by convert it to
java.util.Date (#13050)
new b77d5c4b88 [Improvement][API] Support to check if the worker group has
been used by any tasks or schedulers when users delete or rename it. (#14893)
new 3b3e2f1712 [Fix-14907]Fix click to view the history in the task node
of the workflow instance (#14909)
new 8097d74663 [Improvement-14884][Master] Add overload state in master
heartbeat to trigger slot change (#14887)
new 12baee69d8 [Bug][Worker] Global and local parameters are not replaced
in SQL components (#14926)
new 6d1e9d37c4 feat: add task priority in alert info (#14919)
new 4eae5e224f [fix-#14918] add missing ddl (#14921)
new fb7fab0ecd [Bug] Dependent components cannot be justified and viewed
(#14924)
new 60b9abba18 Fix the ephemeralId will not be deleted before
jdbc-registry closed. (#14936)
new 87d6092ef9 [Fix-14546] seatunnel unable submit spark job to spark
(#14617)
new 3ef25e50cc [Fix-14895][API] Keep printing null if task log file not
exist (#14912)
new 23f6494ad6 [Improvement][Api]Change default zookeeper timeout setting
(#14940)
new 06a555e494 fix the #14729 problem (#14902)
new 6b36354ba3 [Improvement-14894][dolphinscheduler-alert] alert message
contains the operator (#14910)
new 2fb488f4ac Set charset in upgrading tools to solve read upgrade sql
mojibake (#14955)
new cb80df4570 Fix the error of using shell task to obtain Home variable
in Ubuntu system. (#14964)
new 62275e932d [Bug] [Api] Logback file STDOUT duplicate (#14966)
new 333866489a Fix typo (#14973)
new c2dd874c82 [feature-#14449] add secret in k8s Task (#14930)
new f2c68bc97a [Improvement][Common] Obtain resource information in the
k8s environm… (#14968)
new 512f94070a Fix task log appender will not be closed immediate after
throwing exception (#14989)
new 9190f23133 [Improvement][Task] Mask password in task log (#14988)
new a1a041082d [Improvement][K8S] K8S task support passing context
parameters to downstream task (#14934)
new 7b43c66811 [Feature][Helm] Supports deploying only partial components
(#14985)
new 33305532bf [Improvement][Alert] HTTP Related Alert plugin add retry
logic (#14972)
new a8272b2606 [Doc]fix typo (#15005)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a1cf67e19c)
\
N -- N -- N refs/heads/3.2.0-release (a8272b2606)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 40 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/mergeable.yml | 4 +-
.../deployment-dolphinscheduler-alert.yaml | 2 +
.../templates/deployment-dolphinscheduler-api.yaml | 2 +
.../dolphinscheduler/templates/ingress.yaml | 2 +-
.../job-dolphinscheduler-schema-initializer.yaml | 2 +
.../templates/keda-autoscaler-worker.yaml | 2 +-
.../templates/pvc-dolphinscheduler-alert.yaml | 2 +-
.../templates/pvc-dolphinscheduler-api.yaml | 2 +-
.../statefulset-dolphinscheduler-master.yaml | 2 +
.../statefulset-dolphinscheduler-worker.yaml | 2 +
.../templates/svc-dolphinscheduler-alert.yaml | 2 +
.../templates/svc-dolphinscheduler-api.yaml | 2 +
.../svc-dolphinscheduler-master-headless.yaml | 2 +
.../svc-dolphinscheduler-worker-headless.yaml | 2 +
deploy/kubernetes/dolphinscheduler/values.yaml | 8 +
docs/docs/en/architecture/configuration.md | 8 +
docs/docs/en/guide/installation/kubernetes.md | 29 ++++
docs/docs/en/guide/parameter/context.md | 11 ++
docs/docs/en/guide/project/workflow-instance.md | 2 +-
docs/docs/en/guide/resource/configuration.md | 4 +-
docs/docs/en/guide/security/authentication-type.md | 16 ++
docs/docs/zh/architecture/configuration.md | 8 +
docs/docs/zh/guide/installation/kubernetes.md | 29 ++++
docs/docs/zh/guide/parameter/context.md | 11 ++
docs/docs/zh/guide/project/workflow-instance.md | 2 +-
docs/docs/zh/guide/resource/configuration.md | 2 +-
docs/docs/zh/guide/security/authentication-type.md | 16 ++
.../img/new_ui/dev/parameter/k8s_context_param.png | Bin 0 -> 95702 bytes
.../alert/api/HttpServiceRetryStrategy.java | 73 ++++++++++
.../plugin/alert/dingtalk/DingTalkSender.java | 6 +-
.../plugin/alert/feishu/HttpRequestUtil.java | 7 +-
.../plugin/alert/http/HttpSender.java | 18 ++-
.../plugin/alert/pagerduty/PagerDutySender.java | 4 +-
.../plugin/alert/slack/SlackSender.java | 5 +-
.../plugin/alert/telegram/TelegramSender.java | 6 +-
.../plugin/alert/webexteams/WebexTeamsSender.java | 4 +-
.../plugin/alert/wechat/WeChatSender.java | 9 +-
.../api/controller/LoginController.java | 3 +-
.../api/controller/TaskInstanceController.java | 9 +-
.../controller/v2/TaskInstanceV2Controller.java | 4 +-
.../dto/taskInstance/TaskInstanceQueryRequest.java | 3 +
.../apache/dolphinscheduler/api/enums/Status.java | 15 +-
.../security/impl/sso/CasdoorAuthenticator.java | 9 +-
.../api/service/TaskInstanceService.java | 2 +
.../api/service/impl/DataSourceServiceImpl.java | 35 +----
.../api/service/impl/LoggerServiceImpl.java | 6 +-
.../api/service/impl/ResourcesServiceImpl.java | 77 ++--------
.../api/service/impl/TaskInstanceServiceImpl.java | 4 +
.../api/service/impl/TenantServiceImpl.java | 14 +-
.../api/service/impl/UdfFuncServiceImpl.java | 18 +--
.../api/service/impl/UsersServiceImpl.java | 9 +-
.../api/service/impl/WorkerGroupServiceImpl.java | 75 +++++++++-
.../src/main/resources/application.yaml | 23 ++-
.../src/main/resources/logback-spring.xml | 1 -
.../api/controller/TaskInstanceControllerTest.java | 5 +-
.../api/controller/TenantControllerTest.java | 1 -
.../v2/TaskInstanceV2ControllerTest.java | 3 +-
.../impl/sso/CasdoorAuthenticatorTest.java | 3 +-
.../api/service/DataSourceServiceTest.java | 14 +-
.../api/service/ResourcesServiceTest.java | 78 +---------
.../api/service/TaskInstanceServiceTest.java | 20 ++-
.../api/service/UdfFuncServiceTest.java | 23 +--
.../api/service/WorkerGroupServiceTest.java | 17 ++-
.../src/test/resources/application.yaml | 3 +
.../common/constants/Constants.java | 4 -
.../common/constants/DataSourceConstants.java | 2 +-
.../common/enums/ResUploadType.java | 2 +-
.../enums/{AlertEvent.java => ServerStatus.java} | 5 +-
.../common/model/MasterHeartBeat.java | 3 +
.../common/model/WorkerHeartBeat.java | 4 +-
.../common/sql/ClasspathSqlScriptParser.java | 137 +++++++++++++++++
.../SqlScriptParser.java} | 9 +-
.../common/sql/SqlScriptRunner.java | 65 +++++++++
.../dolphinscheduler/common/utils/DateUtils.java | 2 +-
.../common/utils/KubernetesUtils.java | 6 +-
.../dolphinscheduler/common/utils/OSUtils.java | 23 ++-
.../common/utils/PropertyUtils.java | 8 -
.../common/utils/ScriptRunner.java | 162 ---------------------
.../src/main/resources/common.properties | 2 +-
.../common/log/SensitiveDataConverterTest.java | 71 ++++++---
.../common/sql/ClasspathSqlScriptParserTest.java | 52 +++++++
.../common/utils/PropertyUtilsTest.java | 5 -
.../common/utils/ScriptRunnerTest.java | 38 -----
.../src/test/resources/common.properties | 2 +-
.../src/test/resources/sql/mysql_ddl.sql | 0
.../src/test/resources/sql/mysql_dml.sql | 0
.../dao/entity/ProcessAlertContent.java | 4 +
.../dao/entity/TaskAlertContent.java | 2 +
.../dao/mapper/TaskInstanceMapper.java | 2 +
.../dao/mapper/TaskInstanceMapper.xml | 6 +
.../3.2.0_schema/mysql/dolphinscheduler_ddl.sql | 72 +++++++++
.../postgresql/dolphinscheduler_ddl.sql | 10 ++
.../dao/mapper/TaskInstanceMapperTest.java | 1 +
.../datasource/AbstractDataSourceProcessor.java | 14 ++
.../api/datasource/DataSourceProcessor.java | 7 +-
.../plugin/datasource/ssh/SshClientWrapper.java | 81 +++++++++++
.../ssh/param/SSHDataSourceProcessor.java | 22 +--
.../datasource/ssh/SSHDataSourceProcessorTest.java | 30 ++--
.../dispatch/host/LowerWeightHostManager.java | 6 +-
...Listener.java => MasterInfoChangeListener.java} | 15 +-
.../server/master/registry/MasterSlotManager.java | 116 +++++++++++++++
.../server/master/registry/ServerNodeManager.java | 98 +++++--------
.../master/runner/MasterSchedulerBootstrap.java | 8 +-
.../runner/WorkflowExecuteContextFactory.java | 8 +-
.../runner/execute/MasterTaskExecuteRunnable.java | 5 +-
.../server/master/task/MasterHeartBeatTask.java | 8 +
.../master/registry/MasterSlotManagerTest.java | 94 ++++++++++++
dolphinscheduler-registry/README.md | 2 +-
.../registry/jdbc/task/EphemeralDateManager.java | 6 +-
.../service/alert/ProcessAlertManager.java | 47 ++++++
.../service/queue/MasterPriorityQueue.java | 4 +-
.../service/alert/ProcessAlertManagerTest.java | 12 ++
.../src/main/resources/application.yaml | 17 ++-
.../plugin/storage/hdfs/HdfsStorageOperator.java | 25 +---
.../storage/hdfs/HdfsStorageOperatorFactory.java | 4 +-
.../plugin/task/api/TaskConstants.java | 2 +
.../plugin/task/api/k8s/AbstractK8sTask.java | 2 +
.../task/api/k8s/AbstractK8sTaskExecutor.java | 6 +-
.../plugin/task/api/k8s/K8sTaskMainParameters.java | 1 +
.../plugin/task/api/k8s/impl/K8sTaskExecutor.java | 21 ++-
.../task/api/parameters/K8sTaskParameters.java | 1 +
.../shell/BaseLinuxShellInterceptorBuilder.java | 2 +-
.../utils/{ShellUtils.java => VarPoolUtils.java} | 35 +++--
.../plugin/task/api/k8s/K8sTaskExecutorTest.java | 11 +-
.../plugin/task/api/utils/VarPoolUtilsTest.java | 50 +++++++
.../plugin/task/flink/FlinkArgsUtils.java | 8 +-
.../dolphinscheduler/plugin/task/k8s/K8sTask.java | 6 +
.../plugin/task/k8s/K8sTaskTest.java | 20 ++-
.../task/seatunnel/spark/SeatunnelSparkTask.java | 5 +-
.../tools/datasource/dao/UpgradeDao.java | 54 +++----
dolphinscheduler-ui/src/locales/en_US/project.ts | 3 +
dolphinscheduler-ui/src/locales/zh_CN/project.ts | 3 +
.../src/views/datasource/list/detail.tsx | 4 +-
.../src/views/datasource/list/use-form.ts | 2 +-
.../projects/task/components/node/detail-modal.tsx | 5 +-
.../task/components/node/fields/use-dependent.ts | 4 +-
.../task/components/node/fields/use-k8s.ts | 10 +-
.../task/components/node/fields/use-rules.ts | 2 +
.../projects/task/components/node/format-data.ts | 1 +
.../views/projects/task/components/node/types.ts | 1 +
.../views/projects/task/instance/batch-task.tsx | 15 ++
.../src/views/projects/task/instance/use-table.ts | 9 +-
.../workflow/components/dag/use-cell-update.ts | 14 ++
.../components/dag/use-custom-cell-builder.ts | 2 +-
.../workflow/components/dag/use-node-status.ts | 7 +-
.../workflow/components/dag/use-task-edit.ts | 6 +
.../projects/workflow/components/dag/x6-style.scss | 8 +
.../server/worker/task/WorkerHeartBeatTask.java | 22 +--
.../worker/utils/TaskExecutionCheckerUtils.java | 5 -
tools/release/github/changelog.py | 10 +-
150 files changed, 1680 insertions(+), 787 deletions(-)
create mode 100644 docs/img/new_ui/dev/parameter/k8s_context_param.png
create mode 100644
dolphinscheduler-alert/dolphinscheduler-alert-plugins/dolphinscheduler-alert-api/src/main/java/org/apache/dolphinscheduler/alert/api/HttpServiceRetryStrategy.java
copy
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/enums/{AlertEvent.java
=> ServerStatus.java} (93%)
create mode 100644
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/sql/ClasspathSqlScriptParser.java
copy
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/{enums/ResourceManagerType.java
=> sql/SqlScriptParser.java} (81%)
create mode 100644
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/sql/SqlScriptRunner.java
delete mode 100644
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/ScriptRunner.java
create mode 100644
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/sql/ClasspathSqlScriptParserTest.java
delete mode 100644
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/ScriptRunnerTest.java
copy
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.2.0_schema/mysql/dolphinscheduler_ddl_post.sql
=> dolphinscheduler-common/src/test/resources/sql/mysql_ddl.sql (100%)
copy
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.2.0_schema/mysql/dolphinscheduler_dml.sql
=> dolphinscheduler-common/src/test/resources/sql/mysql_dml.sql (100%)
create mode 100644
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-ssh/src/main/java/org/apache/dolphinscheduler/plugin/datasource/ssh/SshClientWrapper.java
copy
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/{WorkerInfoChangeListener.java
=> MasterInfoChangeListener.java} (66%)
create mode 100644
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/MasterSlotManager.java
create mode 100644
dolphinscheduler-master/src/test/java/org/apache/dolphinscheduler/server/master/registry/MasterSlotManagerTest.java
copy
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/utils/{ShellUtils.java
=> VarPoolUtils.java} (58%)
create mode 100644
dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/test/java/org/apache/dolphinscheduler/plugin/task/api/utils/VarPoolUtilsTest.java
