This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6638f264a Automated deployment:
0f421125b2fca94615c5755fb5bf30c82f77751f
6638f264a is described below
commit 6638f264a4824c77eacd9740b4e59f41f2de8b50
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Jun 3 05:06:41 2022 +0000
Automated deployment: 0f421125b2fca94615c5755fb5bf30c82f77751f
---
en-us/development/architecture-design.html | 22 +++++++++++-----------
en-us/development/architecture-design.json | 2 +-
en-us/development/e2e-test.html | 10 +++++-----
en-us/development/e2e-test.json | 2 +-
.../guide/project/workflow-definition.html | 20 ++++++++++----------
.../guide/project/workflow-definition.json | 2 +-
python/_static/basic.css | 6 ++++--
zh-cn/development/architecture-design.html | 16 ++++++++--------
zh-cn/development/architecture-design.json | 2 +-
zh-cn/development/e2e-test.html | 10 +++++-----
zh-cn/development/e2e-test.json | 2 +-
.../guide/project/workflow-definition.html | 2 +-
.../guide/project/workflow-definition.json | 2 +-
13 files changed, 50 insertions(+), 48 deletions(-)
diff --git a/en-us/development/architecture-design.html
b/en-us/development/architecture-design.html
index a02f424d4..7f1e31376 100644
--- a/en-us/development/architecture-design.html
+++ b/en-us/development/architecture-design.html
@@ -15,7 +15,7 @@
<h3>1.Noun Interpretation</h3>
<p><strong>DAG:</strong> Full name Directed Acyclic Graph,referred to as
DAG。Tasks in the workflow are assembled in the form of directed acyclic graphs,
which are topologically traversed from nodes with zero indegrees of ingress
until there are no successor nodes. For example, the following picture:</p>
<p align="center">
- <img src="/img/architecture-design/dag_examples.png" alt="dag示例"
width="80%" />
+ <img src="../../../img/architecture-design/dag_examples.png" alt="dag示例"
width="80%" />
<p align="center">
<em>dag example</em>
</p>
@@ -34,7 +34,7 @@
<h3>2.System architecture</h3>
<h4>2.1 System Architecture Diagram</h4>
<p align="center">
- <img src="/img/architecture.jpg" alt="System Architecture Diagram" />
+ <img src="../../../img/architecture.jpg" alt="System Architecture Diagram"
/>
<p align="center">
<em>System Architecture Diagram</em>
</p>
@@ -136,13 +136,13 @@ Interfaces include workflow creation, definition, query,
modification, release,
<li>The core process algorithm for obtaining distributed locks is as
follows</li>
</ol>
<p align="center">
- <img src="/img/architecture-design/distributed_lock.png" alt="Get
Distributed Lock Process" width="70%" />
+ <img src="../../../img/architecture-design/distributed_lock.png" alt="Get
Distributed Lock Process" width="70%" />
</p>
<ol start="2">
<li>Scheduler thread distributed lock implementation flow chart in
DolphinScheduler:</li>
</ol>
<p align="center">
- <img src="/img/architecture-design/distributed_lock_procss.png" alt="Get
Distributed Lock Process" />
+ <img src="../../../img/architecture-design/distributed_lock_procss.png"
alt="Get Distributed Lock Process" />
</p>
<h5>Third, the thread is insufficient loop waiting problem</h5>
<ul>
@@ -150,7 +150,7 @@ Interfaces include workflow creation, definition, query,
modification, release,
<li>If a large number of sub-processes are nested in a large DAG, the
following figure will result in a "dead" state:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/lack_thread.png" alt="Thread is not
enough to wait for loop" width="70%" />
+ <img src="../../../img/architecture-design/lack_thread.png" alt="Thread is
not enough to wait for loop" width="70%" />
</p>
<p>In the above figure, MainFlowThread waits for SubFlowThread1 to end,
SubFlowThread1 waits for SubFlowThread2 to end, SubFlowThread2 waits for
SubFlowThread3 to end, and SubFlowThread3 waits for a new thread in the thread
pool, then the entire DAG process cannot end, and thus the thread cannot be
released. This forms the state of the child parent process loop waiting. At
this point, the scheduling cluster will no longer be available unless a new
Master is started to add threads to brea [...]
<p>It seems a bit unsatisfactory to start a new Master to break the deadlock,
so we proposed the following three options to reduce this risk:</p>
@@ -166,21 +166,21 @@ Interfaces include workflow creation, definition, query,
modification, release,
<h6>1. Downtime fault tolerance</h6>
<p>Service fault tolerance design relies on ZooKeeper's Watcher mechanism. The
implementation principle is as follows:</p>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant.png"
alt="DolphinScheduler Fault Tolerant Design" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant.png"
alt="DolphinScheduler Fault Tolerant Design" width="70%" />
</p>
<p>The Master monitors the directories of other Masters and Workers. If the
remove event is detected, the process instance is fault-tolerant or the task
instance is fault-tolerant according to the specific business logic.</p>
<ul>
<li>Master fault tolerance flow chart:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant_master.png" alt="Master
Fault Tolerance Flowchart" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant_master.png"
alt="Master Fault Tolerance Flowchart" width="70%" />
</p>
<p>After the ZooKeeper Master is fault-tolerant, it is rescheduled by the
Scheduler thread in DolphinScheduler. It traverses the DAG to find the
"Running" and "Submit Successful" tasks, and monitors the
status of its task instance for the "Running" task. You need to
determine whether the Task Queue already exists. If it exists, monitor the
status of the task instance. If it does not exist, resubmit the task
instance.</p>
<ul>
<li>Worker fault tolerance flow chart:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant_worker.png" alt="Worker
Fault Tolerance Flowchart" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant_worker.png"
alt="Worker Fault Tolerance Flowchart" width="70%" />
</p>
<p>Once the Master Scheduler thread finds the task instance as "need to
be fault tolerant", it takes over the task and resubmits.</p>
<p>Note: Because the "network jitter" may cause the node to lose the
heartbeat of ZooKeeper in a short time, the node's remove event occurs. In this
case, we use the easiest way, that is, once the node has timeout connection
with ZooKeeper, it will directly stop the Master or Worker service.</p>
@@ -210,13 +210,13 @@ Interfaces include workflow creation, definition, query,
modification, release,
<li>
<p>The priority of the process definition is that some processes need to be
processed before other processes. This can be configured at the start of the
process or at the time of scheduled start. There are 5 levels, followed by
HIGHEST, HIGH, MEDIUM, LOW, and LOWEST. As shown below</p>
<p align="center">
- <img src="/img/architecture-design/process_priority.png" alt="Process
Priority Configuration" width="40%" />
+ <img src="../../../img/architecture-design/process_priority.png"
alt="Process Priority Configuration" width="40%" />
</p>
</li>
<li>
<p>The priority of the task is also divided into 5 levels, followed by
HIGHEST, HIGH, MEDIUM, LOW, and LOWEST. As shown below</p>
<p align="center">`
- <img src="/img/architecture-design/task_priority.png" alt="task priority
configuration" width="35%" />
+ <img src="../../../img/architecture-design/task_priority.png" alt="task
priority configuration" width="35%" />
</p>
</li>
</ul>
@@ -235,7 +235,7 @@ Interfaces include workflow creation, definition, query,
modification, release,
<li>Considering the lightweightness of DolphinScheduler as much as possible,
gRPC was chosen to implement remote access log information.</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/grpc.png" alt="grpc remote access"
width="50%" />
+ <img src="../../../img/architecture-design/grpc.png" alt="grpc remote
access" width="50%" />
</p>
<ul>
<li>We use a custom Logback FileAppender and Filter function to generate a log
file for each task instance.</li>
diff --git a/en-us/development/architecture-design.json
b/en-us/development/architecture-design.json
index 6917060ba..a1b05d4a0 100644
--- a/en-us/development/architecture-design.json
+++ b/en-us/development/architecture-design.json
@@ -1,6 +1,6 @@
{
"filename": "architecture-design.md",
- "__html": "<h2>Architecture Design</h2>\n<p>Before explaining the
architecture of the schedule system, let us first understand the common nouns
of the schedule system.</p>\n<h3>1.Noun
Interpretation</h3>\n<p><strong>DAG:</strong> Full name Directed Acyclic
Graph,referred to as DAG。Tasks in the workflow are assembled in the form of
directed acyclic graphs, which are topologically traversed from nodes with zero
indegrees of ingress until there are no successor nodes. For example, the fol
[...]
+ "__html": "<h2>Architecture Design</h2>\n<p>Before explaining the
architecture of the schedule system, let us first understand the common nouns
of the schedule system.</p>\n<h3>1.Noun
Interpretation</h3>\n<p><strong>DAG:</strong> Full name Directed Acyclic
Graph,referred to as DAG。Tasks in the workflow are assembled in the form of
directed acyclic graphs, which are topologically traversed from nodes with zero
indegrees of ingress until there are no successor nodes. For example, the fol
[...]
"link": "/dist/en-us/development/architecture-design.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/development/e2e-test.html b/en-us/development/e2e-test.html
index 5d058c51f..b37c39564 100644
--- a/en-us/development/e2e-test.html
+++ b/en-us/development/e2e-test.html
@@ -95,7 +95,7 @@
<span class="hljs-keyword">throw</span> <span
class="hljs-keyword">new</span> UnsupportedOperationException(<span
class="hljs-string">"Unknown tab: "</span> + tab.getName());
}
</code></pre>
-<p><img src="/img/e2e-test/SecurityPage.png" alt="SecurityPage"></p>
+<p><img src="../../../img/e2e-test/SecurityPage.png" alt="SecurityPage"></p>
<p>For navigation bar options jumping, the goToNav method is provided in
<code>org/apache/dolphinscheduler/e2e/pages/common/NavBarPage.java</code>. The
currently supported pages are: ProjectPage, SecurityPage and ResourcePage.</p>
<pre><code class="language-java"> <span class="hljs-keyword">public</span>
<T extends NavBarItem> <span class="hljs-function">T <span
class="hljs-title">goToNav</span><span class="hljs-params">(Class<T>
nav)</span> </span>{
<span class="hljs-keyword">if</span> (nav == ProjectPage.class) {
@@ -124,7 +124,7 @@
</code></pre>
<h3>E2E-Cases</h3>
<p>Current E2E test cases supported include: File Management, Project
Management, Queue Management, Tenant Management, User Management, Worker Group
Management and Workflow Test.</p>
-<p><img src="/img/e2e-test/E2E_Cases.png" alt="E2E_Cases"></p>
+<p><img src="../../../img/e2e-test/E2E_Cases.png" alt="E2E_Cases"></p>
<p>The following is an example of a tenant management test. As explained
earlier, we use docker-compose for deployment, so for each test case, we need
to import the corresponding file in the form of an annotation.</p>
<p>The browser is loaded using the RemoteWebDriver provided with Selenium.
Before each test case is started there is some preparation work that needs to
be done. For example: logging in the user, jumping to the corresponding page
(depending on the specific test case).</p>
<pre><code class="language-java"> <span class="hljs-meta">@BeforeAll</span>
@@ -157,11 +157,11 @@
<p>When running E2E tests locally, the <code>-Dlocal=true</code> parameter can
be configured to connect locally and facilitate changes to the UI.</p>
<p>When running E2E tests with <code>M1</code> chip, you can use
<code>-Dm1_chip=true</code> parameter to configure containers supported by
<code>ARM64</code>.</p>
-<p><img src="/img/e2e-test/Dlocal.png" alt="Dlocal"></p>
+<p><img src="../../../img/e2e-test/Dlocal.png" alt="Dlocal"></p>
<p>If a connection timeout occurs during a local run, increase the load time
to a recommended 30 and above.</p>
-<p><img src="/img/e2e-test/timeout.png" alt="timeout"></p>
+<p><img src="../../../img/e2e-test/timeout.png" alt="timeout"></p>
<p>The test run will be available as an MP4 file.</p>
-<p><img src="/img/e2e-test/MP4.png" alt="MP4"></p>
+<p><img src="../../../img/e2e-test/MP4.png" alt="MP4"></p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>About us</h3><h4>Do you need feedback? Please
contact us through the following ways.</h4></div><div
class="contact-container"><ul><li><a
href="/en-us/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>Email List</p></a></li><li><a
href="https://twitter.com/dolphinschedule"><img class="img-base"
src="/img/twittergray.png [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/en-us/development/e2e-test.json b/en-us/development/e2e-test.json
index a63a91760..9a5911964 100644
--- a/en-us/development/e2e-test.json
+++ b/en-us/development/e2e-test.json
@@ -1,6 +1,6 @@
{
"filename": "e2e-test.md",
- "__html": "<h1>DolphinScheduler E2E Automation Test</h1>\n<h2>I. Preparatory
knowledge</h2>\n<h3>1. The difference between E2E Test and Unit
Test</h3>\n<p>E2E, which stands for "End to End", can be translated
as "end-to-end" testing. It imitates the user, starting from a
certain entry point and progressively performing actions until a certain job is
completed. And unit tests are different, the latter usually requires testing
parameters, types and parameter values, t [...]
+ "__html": "<h1>DolphinScheduler E2E Automation Test</h1>\n<h2>I. Preparatory
knowledge</h2>\n<h3>1. The difference between E2E Test and Unit
Test</h3>\n<p>E2E, which stands for "End to End", can be translated
as "end-to-end" testing. It imitates the user, starting from a
certain entry point and progressively performing actions until a certain job is
completed. And unit tests are different, the latter usually requires testing
parameters, types and parameter values, t [...]
"link": "/dist/en-us/development/e2e-test.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/project/workflow-definition.html
b/en-us/docs/dev/user_doc/guide/project/workflow-definition.html
index c7848e9a0..81316bd5f 100644
--- a/en-us/docs/dev/user_doc/guide/project/workflow-definition.html
+++ b/en-us/docs/dev/user_doc/guide/project/workflow-definition.html
@@ -77,16 +77,16 @@
</ul>
<p><img src="/img/new_ui/dev/project/workflow-run.png" alt="workflow-run"></p>
<p>Description of workflow operating parameters:</p>
-<pre><code> * Failure strategy: When a task node fails to execute, other
parallel task nodes need to execute this strategy. "Continue" means:
after a certain task fails, other task nodes execute normally; "End"
means: terminate all tasks execution, and terminate the entire process
- * Notification strategy: When the process is over, send the process
execution result notification email according to the process status, options
including no send, send if sucess, send of failure, send whatever result
- * Process priority: The priority of process operation, divide into five
levels: highest (HIGHEST), high (HIGH), medium (MEDIUM), low (LOW), and lowest
(LOWEST). When the number of master threads is insufficient, high priority
processes will execute first in the execution queue, and processes with the
same priority will execute in the order of first in, first out;
- * Worker group: The process can only be executed in the specified worker
machine group. The default is `Default`, which can execute on any worker
- * Notification group: select notification strategy||timeout alarm||when
fault tolerance occurs, process result information or email will send to all
members in the notification group
- * Recipient: select notification policy||timeout alarm||when fault tolerance
occurs, process result information or alarm email will be sent to the recipient
list
- * Cc: select notification policy||timeout alarm||when fault tolerance
occurs, the process result information or warning email will be copied to the
CC list
- * Startup parameter: Set or overwrite global parameter values when starting
a new process instance
- * Complement: two modes including serial complement and parallel complement.
Serial complement: within the specified time range, the complements are
executed from the start date to the end date and N process instances are
generated in turn; parallel complement: within the specified time range,
multiple days are complemented at the same time to generate N process instances.
-* You can select complement time range (only support continuous date) when
executing a timing workflow definition. For example, need to fill in the data
from 1st May to 10th May, as shown in the figure below:
+<pre><code>* Failure strategy: When a task node fails to execute, other
parallel task nodes need to execute this strategy. "Continue" means:
after a certain task fails, other task nodes execute normally; "End"
means: terminate all tasks execution, and terminate the entire process
+* Notification strategy: When the process is over, send the process execution
result notification email according to the process status, options including no
send, send if sucess, send of failure, send whatever result
+* Process priority: The priority of process operation, divide into five
levels: highest (HIGHEST), high (HIGH), medium (MEDIUM), low (LOW), and lowest
(LOWEST). When the number of master threads is insufficient, high priority
processes will execute first in the execution queue, and processes with the
same priority will execute in the order of first in, first out;
+* Worker group: The process can only be executed in the specified worker
machine group. The default is `Default`, which can execute on any worker
+* Notification group: select notification strategy||timeout alarm||when fault
tolerance occurs, process result information or email will send to all members
in the notification group
+* Recipient: select notification policy||timeout alarm||when fault tolerance
occurs, process result information or alarm email will be sent to the recipient
list
+* Cc: select notification policy||timeout alarm||when fault tolerance occurs,
the process result information or warning email will be copied to the CC list
+* Startup parameter: Set or overwrite global parameter values when starting a
new process instance
+* Complement: two modes including serial complement and parallel complement.
Serial complement: within the specified time range, the complements are
executed from the start date to the end date and N process instances are
generated in turn; parallel complement: within the specified time range,
multiple days are complemented at the same time to generate N process instances.
+ * You can select complement time range (When the scheduled configuration is
not online, the daily complement will be performed by default according to the
selected time range.If the timing configuration is online, it will be
complemented according to the selected time range in combination with the
timing configuration) when executing a timing workflow definition. For example,
need to fill in the data from 1st May to 10th May, as shown in the figure below:

</code></pre>
diff --git a/en-us/docs/dev/user_doc/guide/project/workflow-definition.json
b/en-us/docs/dev/user_doc/guide/project/workflow-definition.json
index bde683c6c..015833035 100644
--- a/en-us/docs/dev/user_doc/guide/project/workflow-definition.json
+++ b/en-us/docs/dev/user_doc/guide/project/workflow-definition.json
@@ -1,6 +1,6 @@
{
"filename": "workflow-definition.md",
- "__html": "<h1>Workflow Definition</h1>\n<h2>Create workflow
definition</h2>\n<ul>\n<li>\n<p>Click Project Management -> Workflow ->
Workflow Definition, enter the workflow definition page, and click the
"Create Workflow" button to enter the <strong>workflow DAG
edit</strong> page, as shown in the following figure:</p>\n<p><img
src=\"/img/new_ui/dev/project/workflow-dag.png\"
alt=\"workflow-dag\"></p>\n</li>\n<li>\n<p>Drag from the toolbar <img
src=\"/img/tasks/icons/sh [...]
+ "__html": "<h1>Workflow Definition</h1>\n<h2>Create workflow
definition</h2>\n<ul>\n<li>\n<p>Click Project Management -> Workflow ->
Workflow Definition, enter the workflow definition page, and click the
"Create Workflow" button to enter the <strong>workflow DAG
edit</strong> page, as shown in the following figure:</p>\n<p><img
src=\"/img/new_ui/dev/project/workflow-dag.png\"
alt=\"workflow-dag\"></p>\n</li>\n<li>\n<p>Drag from the toolbar <img
src=\"/img/tasks/icons/sh [...]
"link":
"/dist/en-us/docs/dev/user_doc/guide/project/workflow-definition.html",
"meta": {}
}
\ No newline at end of file
diff --git a/python/_static/basic.css b/python/_static/basic.css
index 7243282d2..9039e027c 100644
--- a/python/_static/basic.css
+++ b/python/_static/basic.css
@@ -335,13 +335,13 @@ p.sidebar-title {
font-weight: bold;
}
-div.admonition, div.topic, blockquote {
+div.admonition, div.topic, aside.topic, blockquote {
clear: left;
}
/* -- topics ----------------------------------------------------------------
*/
-div.topic {
+div.topic, aside.topic {
border: 1px solid #ccc;
padding: 7px;
margin: 10px 0 10px 0;
@@ -380,6 +380,7 @@ div.body p.centered {
div.sidebar > :last-child,
aside.sidebar > :last-child,
div.topic > :last-child,
+aside.topic > :last-child,
div.admonition > :last-child {
margin-bottom: 0;
}
@@ -387,6 +388,7 @@ div.admonition > :last-child {
div.sidebar::after,
aside.sidebar::after,
div.topic::after,
+aside.topic::after,
div.admonition::after,
blockquote::after {
display: block;
diff --git a/zh-cn/development/architecture-design.html
b/zh-cn/development/architecture-design.html
index c3bd5788e..b340df719 100644
--- a/zh-cn/development/architecture-design.html
+++ b/zh-cn/development/architecture-design.html
@@ -15,7 +15,7 @@
<h3>1.名词解释</h3>
<p><strong>DAG:</strong> 全称Directed Acyclic
Graph,简称DAG。工作流中的Task任务以有向无环图的形式组装起来,从入度为零的节点进行拓扑遍历,直到无后继节点为止。举例如下图:</p>
<p align="center">
- <img src="/img/architecture-design/dag_examples.png" alt="dag示例"
width="80%" />
+ <img src="../../../img/architecture-design/dag_examples.png" alt="dag示例"
width="80%" />
<p align="center">
<em>dag示例</em>
</p>
@@ -34,7 +34,7 @@
<h3>2.系统架构</h3>
<h4>2.1 系统架构图</h4>
<p align="center">
- <img src="/img/architecture.jpg" alt="系统架构图" />
+ <img src="../../../img/architecture.jpg" alt="系统架构图" />
<p align="center">
<em>系统架构图</em>
</p>
@@ -132,13 +132,13 @@ MasterServer服务启动时向Zookeeper注册临时节点,通过监听Zookeepe
<li>获取分布式锁的核心流程算法如下</li>
</ol>
<p align="center">
- <img src="/img/architecture-design/distributed_lock.png" alt="获取分布式锁流程"
width="70%" />
+ <img src="../../../img/architecture-design/distributed_lock.png"
alt="获取分布式锁流程" width="70%" />
</p>
<ol start="2">
<li>DolphinScheduler中Scheduler线程分布式锁实现流程图:</li>
</ol>
<p align="center">
- <img src="/img/architecture-design/distributed_lock_procss.png"
alt="获取分布式锁流程" />
+ <img src="../../../img/architecture-design/distributed_lock_procss.png"
alt="获取分布式锁流程" />
</p>
<h5>三、线程不足循环等待问题</h5>
<ul>
@@ -146,7 +146,7 @@ MasterServer服务启动时向Zookeeper注册临时节点,通过监听Zookeepe
<li>如果一个大的DAG中嵌套了很多子流程,如下图则会产生“死等”状态:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/lack_thread.png" alt="线程不足循环等待问题"
width="70%" />
+ <img src="../../../img/architecture-design/lack_thread.png"
alt="线程不足循环等待问题" width="70%" />
</p>
上图中MainFlowThread等待SubFlowThread1结束,SubFlowThread1等待SubFlowThread2结束,
SubFlowThread2等待SubFlowThread3结束,而SubFlowThread3等待线程池有新线程,则整个DAG流程不能结束,从而其中的线程也不能释放。这样就形成的子父流程循环等待的状态。此时除非启动新的Master来增加线程来打破这样的”僵局”,否则调度集群将不能再使用。
<p>对于启动新Master来打破僵局,似乎有点差强人意,于是我们提出了以下三种方案来降低这种风险:</p>
@@ -162,21 +162,21 @@ MasterServer服务启动时向Zookeeper注册临时节点,通过监听Zookeepe
<h6>1. 宕机容错</h6>
<p>服务容错设计依赖于ZooKeeper的Watcher机制,实现原理如图:</p>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant.png"
alt="DolphinScheduler容错设计" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant.png"
alt="DolphinScheduler容错设计" width="70%" />
</p>
其中Master监控其他Master和Worker的目录,如果监听到remove事件,则会根据具体的业务逻辑进行流程实例容错或者任务实例容错。
<ul>
<li>Master容错流程图:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant_master.png"
alt="Master容错流程图" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant_master.png"
alt="Master容错流程图" width="70%" />
</p>
ZooKeeper Master容错完成之后则重新由DolphinScheduler中Scheduler线程调度,遍历 DAG
找到”正在运行”和“提交成功”的任务,对”正在运行”的任务监控其任务实例的状态,对”提交成功”的任务需要判断Task
Queue中是否已经存在,如果存在则同样监控任务实例的状态,如果不存在则重新提交任务实例。
<ul>
<li>Worker容错流程图:</li>
</ul>
<p align="center">
- <img src="/img/architecture-design/fault-tolerant_worker.png"
alt="Worker容错流程图" width="70%" />
+ <img src="../../../img/architecture-design/fault-tolerant_worker.png"
alt="Worker容错流程图" width="70%" />
</p>
<p>Master Scheduler线程一旦发现任务实例为” 需要容错”状态,则接管任务并进行重新提交。</p>
<p>注意:由于”
网络抖动”可能会使得节点短时间内失去和ZooKeeper的心跳,从而发生节点的remove事件。对于这种情况,我们使用最简单的方式,那就是节点一旦和ZooKeeper发生超时连接,则直接将Master或Worker服务停掉。</p>
diff --git a/zh-cn/development/architecture-design.json
b/zh-cn/development/architecture-design.json
index e13e4adfb..9039aded5 100644
--- a/zh-cn/development/architecture-design.json
+++ b/zh-cn/development/architecture-design.json
@@ -1,6 +1,6 @@
{
"filename": "architecture-design.md",
- "__html":
"<h2>系统架构设计</h2>\n<p>在对调度系统架构说明之前,我们先来认识一下调度系统常用的名词</p>\n<h3>1.名词解释</h3>\n<p><strong>DAG:</strong>
全称Directed Acyclic
Graph,简称DAG。工作流中的Task任务以有向无环图的形式组装起来,从入度为零的节点进行拓扑遍历,直到无后继节点为止。举例如下图:</p>\n<p
align=\"center\">\n <img src=\"/img/architecture-design/dag_examples.png\"
alt=\"dag示例\" width=\"80%\" />\n <p align=\"center\">\n
<em>dag示例</em>\n
</p>\n</p>\n<p><strong>流程定义</strong>:通过拖拽任务节点并建立任务节点的关联所形成的可视化<strong>DAG</strong></p>\n<p><strong>流程实例</strong>:流程实例是流程定义的实例化
[...]
+ "__html":
"<h2>系统架构设计</h2>\n<p>在对调度系统架构说明之前,我们先来认识一下调度系统常用的名词</p>\n<h3>1.名词解释</h3>\n<p><strong>DAG:</strong>
全称Directed Acyclic
Graph,简称DAG。工作流中的Task任务以有向无环图的形式组装起来,从入度为零的节点进行拓扑遍历,直到无后继节点为止。举例如下图:</p>\n<p
align=\"center\">\n <img
src=\"../../../img/architecture-design/dag_examples.png\" alt=\"dag示例\"
width=\"80%\" />\n <p align=\"center\">\n <em>dag示例</em>\n
</p>\n</p>\n<p><strong>流程定义</strong>:通过拖拽任务节点并建立任务节点的关联所形成的可视化<strong>DAG</strong></p>\n<p><strong>流程实例</strong>:流程实例是
[...]
"link": "/dist/zh-cn/development/architecture-design.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/development/e2e-test.html b/zh-cn/development/e2e-test.html
index ade937465..de619ec8c 100644
--- a/zh-cn/development/e2e-test.html
+++ b/zh-cn/development/e2e-test.html
@@ -95,7 +95,7 @@
<span class="hljs-keyword">throw</span> <span
class="hljs-keyword">new</span> UnsupportedOperationException(<span
class="hljs-string">"Unknown tab: "</span> + tab.getName());
}
</code></pre>
-<p><img src="/img/e2e-test/SecurityPage.png" alt="SecurityPage"></p>
+<p><img src="../../../img/e2e-test/SecurityPage.png" alt="SecurityPage"></p>
<p>对于导航栏选项的跳转,在<code>org/apache/dolphinscheduler/e2e/pages/common/NavBarPage.java</code>
中提供了 goToNav
的方法。当前支持的页面为:项目管理(ProjectPage)、安全中心(SecurityPage)和资源中心(ResourcePage)。</p>
<pre><code class="language-java"> <span class="hljs-keyword">public</span>
<T extends NavBarItem> <span class="hljs-function">T <span
class="hljs-title">goToNav</span><span class="hljs-params">(Class<T>
nav)</span> </span>{
<span class="hljs-keyword">if</span> (nav == ProjectPage.class) {
@@ -124,7 +124,7 @@
</code></pre>
<h3>2、E2E-Cases</h3>
<p>当前所支持的 E2E 测试案例,主要包括:文件管理、项目管理、队列管理、租户管理、用户管理、Worker 分组管理和工作流测试。</p>
-<p><img src="/img/e2e-test/E2E_Cases.png" alt="E2E_Cases"></p>
+<p><img src="../../../img/e2e-test/E2E_Cases.png" alt="E2E_Cases"></p>
<p>下面以租户管理测试为例,前文已经说明,我们使用 docker-compose 进行部署,所以每个测试案例,都需要以注解的形式引入对应的文件。</p>
<p>使用 Selenium 所提供的 RemoteWebDriver
来加载浏览器。在每个测试案例开始之前都需要进行一些准备工作。比如:登录用户、跳转到对应的页面(根据具体的测试案例而定)。</p>
<pre><code class="language-java"> <span class="hljs-meta">@BeforeAll</span>
@@ -155,11 +155,11 @@
<p>在本地运行的时候,首先需要启动相应的本地服务,可以参考该页面: <a
href="https://dolphinscheduler.apache.org/zh-cn/development/development-environment-setup.html">环境搭建</a></p>
<p>在本地运行 E2E 测试的时候,可以配置 <code>-Dlocal=true</code> 参数,用于连接本地,方便对于 UI 界面的更改。</p>
<p>如果是<code>M1</code>芯片的机器,可以使用<code>-Dm1_chip=true</code>
参数,用于配置使用<code>ARM64</code>支持的容器。</p>
-<p><img src="/img/e2e-test/Dlocal.png" alt="Dlocal"></p>
+<p><img src="../../../img/e2e-test/Dlocal.png" alt="Dlocal"></p>
<p>在本地运行过程中,如果出现连接超时,可增大加载时间,建议 30 及其以上。</p>
-<p><img src="/img/e2e-test/timeout.png" alt="timeout"></p>
+<p><img src="../../../img/e2e-test/timeout.png" alt="timeout"></p>
<p>测试的运行过程将会以 MP4 的文件格式存在。</p>
-<p><img src="/img/e2e-test/MP4.png" alt="MP4"></p>
+<p><img src="../../../img/e2e-test/MP4.png" alt="MP4"></p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>联系我们</h3><h4>有问题需要反馈?请通过以下方式联系我们。</h4></div><div
class="contact-container"><ul><li><a
href="/zh-cn/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>邮件列表</p></a></li><li><a
href="https://twitter.com/dolphinschedule"><img class="img-base"
src="/img/twittergray.png"/><img class="img-change"
src="/img/twitterblue.png"/><p [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/zh-cn/development/e2e-test.json b/zh-cn/development/e2e-test.json
index 48c9f6d20..db99b9568 100644
--- a/zh-cn/development/e2e-test.json
+++ b/zh-cn/development/e2e-test.json
@@ -1,6 +1,6 @@
{
"filename": "e2e-test.md",
- "__html": "<h1>DolphinScheduler — E2E
自动化测试</h1>\n<h2>一、前置知识:</h2>\n<h3>1、E2E 测试与单元测试的区别</h3>\n<p>E2E,是“End to
End”的缩写,可以翻译成“端到端”测试。它模仿用户,从某个入口开始,逐步执行操作,直到完成某项工作。与单元测试不同,后者通常需要测试参数、参数类型、参数值、参数数量、返回值、抛出错误等,目的在于保证特定函数能够在任何情况下都稳定可靠完成工作。单元测试假定只要所有函数都正常工作,那么整个产品就能正常工作。</p>\n<p>相对来说,E2E
测试并没有那么强调要覆盖全部使用场景,它关注的<strong>一个完整的操作链是否能够完成</strong>。对于 Web
前端来说,还关注<strong>界面布局、内容信息是否符合预期</strong>。</p>\n<p>比如,登陆界面的 E2E
测试,关注用户是否能够正常输入,正常登录;登陆失败的�
�,是否能够正确显示错误信息。至于输入不合法的内容是否处理,并不是所关注的重点。</p>\n<h3>2、Selen [...]
+ "__html": "<h1>DolphinScheduler — E2E
自动化测试</h1>\n<h2>一、前置知识:</h2>\n<h3>1、E2E 测试与单元测试的区别</h3>\n<p>E2E,是“End to
End”的缩写,可以翻译成“端到端”测试。它模仿用户,从某个入口开始,逐步执行操作,直到完成某项工作。与单元测试不同,后者通常需要测试参数、参数类型、参数值、参数数量、返回值、抛出错误等,目的在于保证特定函数能够在任何情况下都稳定可靠完成工作。单元测试假定只要所有函数都正常工作,那么整个产品就能正常工作。</p>\n<p>相对来说,E2E
测试并没有那么强调要覆盖全部使用场景,它关注的<strong>一个完整的操作链是否能够完成</strong>。对于 Web
前端来说,还关注<strong>界面布局、内容信息是否符合预期</strong>。</p>\n<p>比如,登陆界面的 E2E
测试,关注用户是否能够正常输入,正常登录;登陆失败的�
�,是否能够正确显示错误信息。至于输入不合法的内容是否处理,并不是所关注的重点。</p>\n<h3>2、Selen [...]
"link": "/dist/zh-cn/development/e2e-test.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.html
b/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.html
index 27dfd8bd7..b8c4cf5ce 100644
--- a/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.html
+++ b/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.html
@@ -103,7 +103,7 @@
<li>
<p>补数:包括串行补数、并行补数 2
种模式。串行补数:指定时间范围内,从开始日期至结束日期依次执行补数,依次生成N条流程实例;并行补数:指定时间范围内,多天同时进行补数,同时生成 N
条流程实例。</p>
<ul>
-<li>补数: 执行指定日期的工作流定义,可以选择补数时间范围(目前只支持针对连续的天进行补数),比如需要补 5 月 9 号到 5 月 10
号的数据,如下图所示:</li>
+<li>补数:
执行指定日期的工作流定义,可以选择补数时间范围(当定时配置未上线时默认会根据所选时间范围进行每天一次的补数,如果定时配置已上线则会根据所选的时间范围结合定时配置进行补数),比如需要补
5 月 9 号到 5 月 10 号的数据,如下图所示:</li>
</ul>
<p><img src="/img/new_ui/dev/project/workflow-date.png"
alt="workflow-date"></p>
<blockquote>
diff --git a/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.json
b/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.json
index c66ea1bcf..ddd094c0e 100644
--- a/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.json
+++ b/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.json
@@ -1,6 +1,6 @@
{
"filename": "workflow-definition.md",
- "__html":
"<h1>工作流定义</h1>\n<h2>创建工作流定义</h2>\n<ul>\n<li>\n<p>点击项目管理->工作流->工作流定义,进入工作流定义页面,点击“创建工作流”按钮,进入<strong>工作流DAG编辑</strong>页面,如下图所示:</p>\n<p><img
src=\"/img/new_ui/dev/project/workflow-dag.png\"
alt=\"workflow-dag\"></p>\n</li>\n<li>\n<p>工具栏中拖拽 <img
src=\"/img/tasks/icons/shell.png\" width=\"15\"/>
到画板中,新增一个Shell任务,如下图所示:</p>\n<p><img src=\"/img/tasks/demo/shell.jpg\"
alt=\"demo-shell-simple\"></p>\n</li>\n<li>\n<p><strong>添加 Shell
任务的参数设置:</strong></p>\n<ol>\n<li>填写“节点名称”,“ [...]
+ "__html":
"<h1>工作流定义</h1>\n<h2>创建工作流定义</h2>\n<ul>\n<li>\n<p>点击项目管理->工作流->工作流定义,进入工作流定义页面,点击“创建工作流”按钮,进入<strong>工作流DAG编辑</strong>页面,如下图所示:</p>\n<p><img
src=\"/img/new_ui/dev/project/workflow-dag.png\"
alt=\"workflow-dag\"></p>\n</li>\n<li>\n<p>工具栏中拖拽 <img
src=\"/img/tasks/icons/shell.png\" width=\"15\"/>
到画板中,新增一个Shell任务,如下图所示:</p>\n<p><img src=\"/img/tasks/demo/shell.jpg\"
alt=\"demo-shell-simple\"></p>\n</li>\n<li>\n<p><strong>添加 Shell
任务的参数设置:</strong></p>\n<ol>\n<li>填写“节点名称”,“ [...]
"link":
"/dist/zh-cn/docs/dev/user_doc/guide/project/workflow-definition.html",
"meta": {}
}
\ No newline at end of file