This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 15119c66f Automated deployment:
2649e95f1ffc6bc4cff51741169273f587195bd6
15119c66f is described below
commit 15119c66fb01cb29f7617294e9ea5dbc48e8c288
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Jun 9 05:06:39 2022 +0000
Automated deployment: 2649e95f1ffc6bc4cff51741169273f587195bd6
---
.../dev/user_doc/architecture/configuration.html | 5 +
.../dev/user_doc/architecture/configuration.json | 2 +-
.../{task/switch.html => metrics/metrics.html} | 162 +++++++++++++++++----
en-us/docs/dev/user_doc/guide/metrics/metrics.json | 6 +
en-us/docs/dev/user_doc/guide/task/datax.html | 2 +
en-us/docs/dev/user_doc/guide/task/datax.json | 2 +-
en-us/docs/dev/user_doc/guide/task/jupyter.html | 2 +
en-us/docs/dev/user_doc/guide/task/jupyter.json | 2 +-
en-us/docs/dev/user_doc/guide/task/python.html | 2 +
en-us/docs/dev/user_doc/guide/task/python.json | 2 +-
en-us/docs/dev/user_doc/guide/task/shell.html | 2 +
en-us/docs/dev/user_doc/guide/task/shell.json | 2 +-
en-us/docs/dev/user_doc/guide/task/switch.html | 46 +++---
en-us/docs/dev/user_doc/guide/task/switch.json | 2 +-
img/metrics/metrics-datasource.png | Bin 0 -> 336581 bytes
img/metrics/metrics-master.png | Bin 0 -> 467778 bytes
img/metrics/metrics-worker.png | Bin 0 -> 403432 bytes
img/tasks/demo/switch_01.png | Bin 0 -> 117194 bytes
img/tasks/demo/switch_02.png | Bin 0 -> 141217 bytes
img/tasks/demo/switch_03.png | Bin 0 -> 148408 bytes
img/tasks/demo/switch_04.png | Bin 0 -> 125565 bytes
.../dev/user_doc/architecture/configuration.html | 5 +
.../dev/user_doc/architecture/configuration.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/datax.html | 2 +
zh-cn/docs/dev/user_doc/guide/task/datax.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/jupyter.html | 4 +-
zh-cn/docs/dev/user_doc/guide/task/jupyter.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/python.html | 2 +
zh-cn/docs/dev/user_doc/guide/task/python.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/shell.html | 2 +
zh-cn/docs/dev/user_doc/guide/task/shell.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/switch.html | 52 ++++---
zh-cn/docs/dev/user_doc/guide/task/switch.json | 2 +-
33 files changed, 232 insertions(+), 86 deletions(-)
diff --git a/en-us/docs/dev/user_doc/architecture/configuration.html
b/en-us/docs/dev/user_doc/architecture/configuration.html
index 6041d2a94..cecb22131 100644
--- a/en-us/docs/dev/user_doc/architecture/configuration.html
+++ b/en-us/docs/dev/user_doc/architecture/configuration.html
@@ -454,6 +454,11 @@ Currently, DS just makes a basic config, remember to
config further JVM options
<td>false</td>
<td>specify whether in development state</td>
</tr>
+<tr>
+<td>task.resource.limit.state</td>
+<td>false</td>
+<td>specify whether in resource limit state</td>
+</tr>
</tbody>
</table>
<h3>application-api.properties [API-service log config]</h3>
diff --git a/en-us/docs/dev/user_doc/architecture/configuration.json
b/en-us/docs/dev/user_doc/architecture/configuration.json
index 0cf7663b1..108753610 100644
--- a/en-us/docs/dev/user_doc/architecture/configuration.json
+++ b/en-us/docs/dev/user_doc/architecture/configuration.json
@@ -1,6 +1,6 @@
{
"filename": "configuration.md",
- "__html": "<!-- markdown-link-check-disable
-->\n<h1>Configuration</h1>\n<h2>Preface</h2>\n<p>This document explains the
DolphinScheduler application configurations according to DolphinScheduler-1.3.x
versions.</p>\n<h2>Directory Structure</h2>\n<p>Currently, all the
configuration files are under [conf ] directory.\nCheck the following
simplified DolphinScheduler installation directories to have a direct view
about the position of [conf] directory and configuration files it has.\nThis
[...]
+ "__html": "<!-- markdown-link-check-disable
-->\n<h1>Configuration</h1>\n<h2>Preface</h2>\n<p>This document explains the
DolphinScheduler application configurations according to DolphinScheduler-1.3.x
versions.</p>\n<h2>Directory Structure</h2>\n<p>Currently, all the
configuration files are under [conf ] directory.\nCheck the following
simplified DolphinScheduler installation directories to have a direct view
about the position of [conf] directory and configuration files it has.\nThis
[...]
"link": "/dist/en-us/docs/dev/user_doc/architecture/configuration.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/switch.html
b/en-us/docs/dev/user_doc/guide/metrics/metrics.html
similarity index 70%
copy from en-us/docs/dev/user_doc/guide/task/switch.html
copy to en-us/docs/dev/user_doc/guide/metrics/metrics.html
index ff01d1784..3b7161ff4 100644
--- a/en-us/docs/dev/user_doc/guide/task/switch.html
+++ b/en-us/docs/dev/user_doc/guide/metrics/metrics.html
@@ -3,47 +3,147 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=1.0, user-scalable=no">
- <meta name="keywords" content="switch">
- <meta name="description" content="switch">
- <title>switch</title>
+ <meta name="keywords" content="metrics">
+ <meta name="description" content="metrics">
+ <title>metrics</title>
<link rel="shortcut icon" href="/img/favicon.ico">
<link rel="stylesheet" href="/build/vendor.eeae4ed.css">
</head>
<body>
- <div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
Have queries regarding Apache DolphinScheduler, Join Slack channel to disscuss
them <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/en-us/index.html"><img class="logo"
src=" [...]
-<p>The switch is a conditional judgment node, decide the branch executes
according to the value of <a href="../parameter/global.md">global variable</a>
and the expression result written by the user.</p>
-<h2>Create</h2>
-<p>Drag from the toolbar <img src="/img/switch.png" width="20"/> task node to
canvas to create a task.
-<strong>Note</strong> After created a switch task, you must first configure
the upstream and downstream, then configure the parameter of task branches.</p>
-<h2>Parameter</h2>
+ <div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
Have queries regarding Apache DolphinScheduler, Join Slack channel to disscuss
them <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/en-us/index.html"><img class="logo"
src=" [...]
+<p>Apache DolphinScheduler has export some metrics to monitor the system. We
use micrometer for the exporter facade, and
+the default exporter is prometheus, more exporter is coming soon.</p>
+<h2>Quick Start</h2>
+<p>You can add the following config in master/worker/alert/api's yaml file to
open the metrics exporter.</p>
+<pre><code class="language-yaml"><span class="hljs-attr">metrics:</span>
+ <span class="hljs-attr">enabled:</span> <span
class="hljs-literal">true</span>
+</code></pre>
+<p>Once you open the metrics exporter, you can access the metrics by the url:
<code>http://ip:port/actuator/prometheus</code></p>
+<p>The exporter port is the <code>server.port</code> defined in
application.yaml, e.g: master: <code>server.port: 5679</code>, worker:
<code>server.port: 1235</code>, alert: <code>server.port: 50053</code>, api:
<code>server.port: 12345</code>.</p>
+<p>For example, you can get the master metrics by <code>curl
http://localhost:5679/actuator/prometheus</code></p>
+<p>We have prepared the out-of-the-box Grafana configuration for you, you can
find the Grafana dashboard
+at <code>dolphinscheduler-meter/resources/grafana</code>, you can directly
import these dashboards to grafana.</p>
+<p>If you want to try at docker, you can use the following command to start
the prometheus with grafana:</p>
+<pre><code class="language-shell">cd
dolphinscheduler-meter/src/main/resources/grafana-demo
+docker compose up
+</code></pre>
+<p>Then you can access the grafana by the url:
<code>http://localhost/3001</code></p>
+<p><img src="/img/metrics/metrics-master.png" alt="image.png">
+<img src="/img/metrics/metrics-worker.png" alt="image.png">
+<img src="/img/metrics/metrics-datasource.png" alt="image.png"></p>
+<h2>Master Metrics</h2>
+<p>Master metrics are exported by the DolphinScheduler master server.</p>
+<h3>System Metrics</h3>
<ul>
-<li>Node name: The node name in a workflow definition is unique.</li>
-<li>Run flag: Identifies whether this node schedules normally, if it does not
need to execute, select the <code>prohibition execution</code>.</li>
-<li>Descriptive information: Describe the function of the node.</li>
-<li>Task priority: When the number of worker threads is insufficient, execute
in the order of priority from high to low, and tasks with the same priority
will execute in a first-in first-out order.</li>
-<li>Worker grouping: Assign tasks to the machines of the worker group to
execute. If <code>Default</code> is selected, randomly select a worker machine
for execution.</li>
-<li>Times of failed retry attempts: The number of times the task failed to
resubmit. You can select from drop-down or fill-in a number.</li>
-<li>Failed retry interval: The time interval for resubmitting the task after a
failed task. You can select from drop-down or fill-in a number.</li>
-<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
runs exceed the "timeout", an alarm email will send and the task
execution will fail.</li>
-<li>Condition: You can configure multiple conditions for the switch task. When
the conditions are satisfied, execute the configured branch. You can configure
multiple different conditions to satisfy different businesses.</li>
-<li>Branch flow: The default branch flow, when all the conditions are not
satisfied, execute this branch flow.</li>
+<li>dolphinscheduler_master_overload_count: Indicates the number of times the
master has been overloaded.</li>
+<li>dolphinscheduler_master_consume_command_count: Indicates the number of
commands has consumed.</li>
</ul>
-<h2>Detail</h2>
-<p>Here we have three tasks, the dependencies are <code>A -> B -> [C,
D]</code>, and <code>task_a</code> is a shell task and <code>task_b</code> is a
switch task</p>
+<h3>Process Metrics</h3>
<ul>
-<li>In task A, a global variable named <code>id</code> is defined through <a
href="../parameter/global.md">global variable</a>, and the declaration method
is <code>${setValue(id=1)}</code></li>
-<li>Task B adds conditions and uses global variables declared upstream to
achieve conditional judgment (Note: switch can get the global variables value,
as long as its direct or indirect upstream have already assigned the global
variables before switch acquires). We want to execute task C when <code>id =
1</code>, otherwise run task D
+<li>dolphinscheduler_create_command_count: Indicates the number of command has
been inserted.</li>
+<li>dolphinscheduler_process_instance_submit_count: Indicates the number of
process has been submitted.</li>
+<li>dolphinscheduler_process_instance_running_gauge: Indicates the number of
process are running now.</li>
+<li>dolphinscheduler_process_instance_timeout_count: Indicates the number of
process has been timeout.</li>
+<li>dolphinscheduler_process_instance_finish_count: Indicates the number of
process has been finished, include success or
+failure.</li>
+<li>dolphinscheduler_process_instance_success_count: Indicates the number of
process has been successful.</li>
+<li>dolphinscheduler_process_instance_stop_count: Indicates the number of
process has been stopped.</li>
+<li>dolphinscheduler_process_instance_failover_count: Indicates the number of
process has been failed over.</li>
+</ul>
+<h3>Task Metrics</h3>
+<ul>
+<li>dolphinscheduler_task_timeout_count: Indicates the number of tasks has
been timeout.</li>
+<li>dolphinscheduler_task_finish_count: Indicates the number of tasks has been
finished, include success or failure.</li>
+<li>dolphinscheduler_task_success_count: Indicates the number of tasks has
been successful.</li>
+<li>dolphinscheduler_task_timeout_count: Indicates the number of tasks has
been timeout.</li>
+<li>dolphinscheduler_task_retry_count: Indicates the number of tasks has been
retry.</li>
+<li>dolphinscheduler_task_failover_count: Indicates the number of tasks has
been failover.</li>
+<li>dolphinscheduler_task_dispatch_count: Indicates the number of tasks has
been dispatched to worker.</li>
+<li>dolphinscheduler_task_dispatch_failed_count: Indicates the number of tasks
dispatched failed, if dispatched failed
+will retry.</li>
+<li>dolphinscheduler_task_dispatch_error_count: Indicates the number of tasks
dispatched error, if dispatched error, means
+there are exception occur.</li>
+</ul>
+<h2>Worker Metrics</h2>
+<p>Worker metrics are exported by the DolphinScheduler worker server.</p>
+<h3>System Metrics</h3>
+<ul>
+<li>dolphinscheduler_worker_overload_count: Indicates the number of times the
worker has been overloaded.</li>
+<li>dolphinscheduler_worker_submit_queue_is_full_count: Indicates the number
of times the worker's submit queue has been
+full.</li>
+</ul>
+<h3>Task Metrics</h3>
<ul>
-<li>Configure task C to run when the global variable <code>id=1</code>. Then
edit <code>${id} == 1</code> in the condition of task B, and select
<code>C</code> as branch flow</li>
-<li>For other tasks, select <code>D</code> as branch flow</li>
+<li>dolphinscheduler_task_execute_count: Indicates the number of times a task
has been executed, it contains a tag -
+<code>task_type</code>.</li>
+<li>dolphinscheduler_task_execution_count: Indicates the total number of task
has been executed.</li>
+<li>dolphinscheduler_task_execution_timer: Indicates the time spent executing
tasks.</li>
</ul>
-</li>
+<h2>Default System Metrics</h2>
+<p>In each server, there are some default metrics related to the system
instance.</p>
+<h3>Database Metrics</h3>
+<ul>
+<li>hikaricp_connections_creation_seconds_max: Connection creation time
max.</li>
+<li>hikaricp_connections_creation_seconds_count: Connection creation time
count.</li>
+<li>hikaricp_connections_creation_seconds_sum: Connection creation time
sum.</li>
+<li>hikaricp_connections_acquire_seconds_max: Connection acquire time max.</li>
+<li>hikaricp_connections_acquire_seconds_count: Connection acquire time
count.</li>
+<li>hikaricp_connections_acquire_seconds_sum: Connection acquire time sum.</li>
+<li>hikaricp_connections_usage_seconds_max: Connection usage max.</li>
+<li>hikaricp_connections_usage_seconds_count: Connection usage time count.</li>
+<li>hikaricp_connections_usage_seconds_sum: Connection usage time sum.</li>
+<li>hikaricp_connections_max: Max connections.</li>
+<li>hikaricp_connections_min Min connections</li>
+<li>hikaricp_connections_active: Active connections.</li>
+<li>hikaricp_connections_idle: Idle connections.</li>
+<li>hikaricp_connections_pending: Pending connections.</li>
+<li>hikaricp_connections_timeout_total: Timeout connections.</li>
+<li>hikaricp_connections: Total connections</li>
+<li>jdbc_connections_max: Maximum number of active connections that can be
allocated at the same time.</li>
+<li>jdbc_connections_min: Minimum number of idle connections in the pool.</li>
+<li>jdbc_connections_idle: Number of established but idle connections.</li>
+<li>jdbc_connections_active: Current number of active connections that have
been allocated from the data source.</li>
+</ul>
+<h3>JVM Metrics</h3>
+<ul>
+<li>jvm_buffer_total_capacity_bytes: An estimate of the total capacity of the
buffers in this pool.</li>
+<li>jvm_buffer_count_buffers: An estimate of the number of buffers in the
pool.</li>
+<li>jvm_buffer_memory_used_bytes: An estimate of the memory that the Java
virtual machine is using for this buffer pool.</li>
+<li>jvm_memory_committed_bytes: The amount of memory in bytes that is
committed for the Java virtual machine to use.</li>
+<li>jvm_memory_max_bytes: The maximum amount of memory in bytes that can be
used for memory management.</li>
+<li>jvm_memory_used_bytes: The amount of used memory.</li>
+<li>jvm_threads_peak_threads: The peak live thread count since the Java
virtual machine started or peak was reset.</li>
+<li>jvm_threads_states_threads: The current number of threads having NEW
state.</li>
+<li>jvm_gc_memory_allocated_bytes_total: Incremented for an increase in the
size of the (young) heap memory pool after one GC to before the next.</li>
+<li>jvm_gc_max_data_size_bytes: Max size of long-lived heap memory pool.</li>
+<li>jvm_gc_pause_seconds_count: Time spent count in GC pause.</li>
+<li>jvm_gc_pause_seconds_sum: Time spent sum in GC pause.</li>
+<li>jvm_gc_pause_seconds_max: Time spent max in GC pause.</li>
+<li>jvm_gc_live_data_size_bytes: Size of long-lived heap memory pool after
reclamation.</li>
+<li>jvm_gc_memory_promoted_bytes_total: Count of positive increases in the
size of the old generation memory pool before GC to after GC.</li>
+<li>jvm_classes_loaded_classes: The number of classes that are currently
loaded in the Java virtual machine.</li>
+<li>jvm_threads_live_threads: The current number of live threads including
both daemon and non-daemon threads.</li>
+<li>jvm_threads_daemon_threads: The current number of live daemon threads.</li>
+<li>jvm_classes_unloaded_classes_total: The total number of classes unloaded
since the Java virtual machine has started execution.</li>
+<li>process_cpu_usage: The "recent cpu usage" for the Java Virtual
Machine process.</li>
+<li>process_start_time_seconds: Start time of the process since unix
epoch.</li>
+<li>process_uptime_seconds: The uptime of the Java virtual machine.</li>
+</ul>
+<h2>Other Metrics</h2>
+<ul>
+<li>jetty_threads_config_max: The maximum number of threads in the pool.</li>
+<li>jetty_threads_config_min: The minimum number of threads in the pool.</li>
+<li>jetty_threads_current: The total number of threads in the pool.</li>
+<li>jetty_threads_idle: The number of idle threads in the pool.</li>
+<li>jetty_threads_busy: The number of busy threads in the pool.</li>
+<li>jetty_threads_jobs: Number of jobs queued waiting for a thread.</li>
+<li>process_files_max_files: The maximum file descriptor count.</li>
+<li>process_files_open_files: The open file descriptor count.</li>
+<li>system_cpu_usage: The "recent cpu usage" for the whole
system.</li>
+<li>system_cpu_count: The number of processors available to the Java virtual
machine.</li>
+<li>system_load_average_1m: The sum of the number of runnable entities queued
to available processors and the number of runnable entities running on the
available processors averaged over a period of time.</li>
+<li>logback_events_total: Number of level events that made it to the logs</li>
</ul>
-<p>The following shows the switch task configuration:</p>
-<p><img src="/img/switch_configure.jpg" alt="task-switch-configure"></p>
-<h2>Related Task</h2>
-<p><a href="conditions.md">condition</a>：<a
href="conditions.md">Condition</a>task mainly executes the corresponding branch
based on the execution result status (success, failure) of the upstream node.
-The <a href="switch.md">Switch</a> task mainly executes the corresponding
branch based on the value of the <a href="../parameter/global.md">global
parameter</a> and the judgment expression result written by the user.</p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>About us</h3><h4>Do you need feedback? Please
contact us through the following ways.</h4></div><div
class="contact-container"><ul><li><a
href="/en-us/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>Email List</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/en-us/docs/dev/user_doc/guide/metrics/metrics.json
b/en-us/docs/dev/user_doc/guide/metrics/metrics.json
new file mode 100644
index 000000000..08494c15a
--- /dev/null
+++ b/en-us/docs/dev/user_doc/guide/metrics/metrics.json
@@ -0,0 +1,6 @@
+{
+ "filename": "metrics.md",
+ "__html": "<h1>Introduction</h1>\n<p>Apache DolphinScheduler has export some
metrics to monitor the system. We use micrometer for the exporter facade,
and\nthe default exporter is prometheus, more exporter is coming
soon.</p>\n<h2>Quick Start</h2>\n<p>You can add the following config in
master/worker/alert/api's yaml file to open the metrics
exporter.</p>\n<pre><code class=\"language-yaml\"><span
class=\"hljs-attr\">metrics:</span>\n <span
class=\"hljs-attr\">enabled:</span> <span cla [...]
+ "link": "/dist/en-us/docs/dev/user_doc/guide/metrics/metrics.html",
+ "meta": {}
+}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/datax.html
b/en-us/docs/dev/user_doc/guide/task/datax.html
index f48bbedcc..1d12527f0 100644
--- a/en-us/docs/dev/user_doc/guide/task/datax.html
+++ b/en-us/docs/dev/user_doc/guide/task/datax.html
@@ -28,6 +28,8 @@
<li><strong>Environment Name</strong>: Configure the environment name in which
to run the script.</li>
<li><strong>Number of failed retry attempts</strong>: The number of times the
task failed to be resubmitted.</li>
<li><strong>Failed retry interval</strong>: The time, in cents, interval for
resubmitting the task after a failed task.</li>
+<li><strong>Cpu quota</strong>: Assign the specified CPU time quota to the
task executed. Takes a percentage value. Default -1 means unlimited. For
example, the full CPU load of one core is 100%,and that of 16 cores is 1600%.
This function is controlled by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
+<li><strong>Max memory</strong>：Assign the specified max memory to the task
executed. Exceeding this limit will trigger oom to be killed and will not
automatically retry. Takes an MB value. Default -1 means unlimited. This
function is controlled by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li><strong>Delayed execution time</strong>: The time, in cents, that a task
is delayed in execution.</li>
<li><strong>Timeout alarm</strong>: Check the timeout alarm and timeout
failure. When the task exceeds the "timeout period", an alarm email
will be sent and the task execution will fail.</li>
<li><strong>Custom template</strong>: Custom the content of the DataX node's
json profile when the default data source provided does not meet the required
requirements.</li>
diff --git a/en-us/docs/dev/user_doc/guide/task/datax.json
b/en-us/docs/dev/user_doc/guide/task/datax.json
index eece48440..35a3de567 100644
--- a/en-us/docs/dev/user_doc/guide/task/datax.json
+++ b/en-us/docs/dev/user_doc/guide/task/datax.json
@@ -1,6 +1,6 @@
{
"filename": "datax.md",
- "__html": "<h1>DataX</h1>\n<h2>Overview</h2>\n<p>DataX task type for
executing DataX programs. For DataX nodes, the worker will execute
<code>${DATAX_HOME}/bin/datax.py</code> to analyze the input json
file.</p>\n<h2>Create Task</h2>\n<ul>\n<li>Click Project Management ->
Project Name -> Workflow Definition, and click the "Create
Workflow" button to enter the DAG editing page.</li>\n<li>Drag the <img
src=\"/img/tasks/icons/datax.png\" width=\"15\"/> from the toolbar to [...]
+ "__html": "<h1>DataX</h1>\n<h2>Overview</h2>\n<p>DataX task type for
executing DataX programs. For DataX nodes, the worker will execute
<code>${DATAX_HOME}/bin/datax.py</code> to analyze the input json
file.</p>\n<h2>Create Task</h2>\n<ul>\n<li>Click Project Management ->
Project Name -> Workflow Definition, and click the "Create
Workflow" button to enter the DAG editing page.</li>\n<li>Drag the <img
src=\"/img/tasks/icons/datax.png\" width=\"15\"/> from the toolbar to [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/datax.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/jupyter.html
b/en-us/docs/dev/user_doc/guide/task/jupyter.html
index 491043ba7..74d6878ce 100644
--- a/en-us/docs/dev/user_doc/guide/task/jupyter.html
+++ b/en-us/docs/dev/user_doc/guide/task/jupyter.html
@@ -34,6 +34,8 @@ Click <a href="https://docs.conda.io/en/latest/";>here</a> for
more information a
<li>Worker grouping: Assign tasks to the machines of the worker group to
execute. If <code>Default</code> is selected, randomly select a worker machine
for execution.</li>
<li>Number of failed retry attempts: The failure task resubmitting times. It
supports drop-down and hand-filling.</li>
<li>Failed retry interval: The time interval for resubmitting the task after a
failed task. It supports drop-down and hand-filling.</li>
+<li>Cpu quota: Assign the specified CPU time quota to the task executed. Takes
a percentage value. Default -1 means unlimited. For example, the full CPU load
of one core is 100%,and that of 16 cores is 1600%. This function is controlled
by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
+<li>Max memory：Assign the specified max memory to the task executed. Exceeding
this limit will trigger oom to be killed and will not automatically retry.
Takes an MB value. Default -1 means unlimited. This function is controlled by
<a href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
exceeds the "timeout period", an alarm email will send and the task
execution will fail.</li>
<li>Conda Env Name: Name of conda environment.</li>
<li>Input Note Path: Path of input jupyter note template.</li>
diff --git a/en-us/docs/dev/user_doc/guide/task/jupyter.json
b/en-us/docs/dev/user_doc/guide/task/jupyter.json
index e4cf9a068..b970793f2 100644
--- a/en-us/docs/dev/user_doc/guide/task/jupyter.json
+++ b/en-us/docs/dev/user_doc/guide/task/jupyter.json
@@ -1,6 +1,6 @@
{
"filename": "jupyter.md",
- "__html": "<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p>Use <code>Jupyter
Task</code> to create a jupyter-type task and execute jupyter notes. When the
worker executes <code>Jupyter Task</code>,\nit will use <code>papermill</code>
to evaluate jupyter notes. Click <a
href=\"https://papermill.readthedocs.io/en/latest/\";>here</a> for details about
<code>papermill</code>.</p>\n<h2>Conda Configuration</h2>\n<ul>\n<li>Config
<code>conda.path</code> in <code>common.properties</code> to the path of [...]
+ "__html": "<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p>Use <code>Jupyter
Task</code> to create a jupyter-type task and execute jupyter notes. When the
worker executes <code>Jupyter Task</code>,\nit will use <code>papermill</code>
to evaluate jupyter notes. Click <a
href=\"https://papermill.readthedocs.io/en/latest/\";>here</a> for details about
<code>papermill</code>.</p>\n<h2>Conda Configuration</h2>\n<ul>\n<li>Config
<code>conda.path</code> in <code>common.properties</code> to the path of [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/jupyter.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/python.html
b/en-us/docs/dev/user_doc/guide/task/python.html
index 77b63e198..f611b8b47 100644
--- a/en-us/docs/dev/user_doc/guide/task/python.html
+++ b/en-us/docs/dev/user_doc/guide/task/python.html
@@ -29,6 +29,8 @@ it will generate a temporary python script, and executes the
script by the Linux
<li>Environment Name: Configure the environment name in which to run the
script.</li>
<li>Number of failed retry attempts: The failure task resubmitting times. It
supports drop-down and hand-filling.</li>
<li>Failed retry interval: The time interval for resubmitting the task after a
failed task. It supports drop-down and hand-filling.</li>
+<li>Cpu quota: Assign the specified CPU time quota to the task executed. Takes
a percentage value. Default -1 means unlimited. For example, the full CPU load
of one core is 100%,and that of 16 cores is 1600%. This function is controlled
by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
+<li>Max memory：Assign the specified max memory to the task executed. Exceeding
this limit will trigger oom to be killed and will not automatically retry.
Takes an MB value. Default -1 means unlimited. This function is controlled by
<a href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
exceeds the "timeout period", an alarm email will send and the task
execution will fail.</li>
<li>Script: Python program developed by the user.</li>
<li>Resource: Refers to the list of resource files that need to be called in
the script, and the files uploaded or created by the resource center-file
management.</li>
diff --git a/en-us/docs/dev/user_doc/guide/task/python.json
b/en-us/docs/dev/user_doc/guide/task/python.json
index 4db59a744..a45db08fa 100644
--- a/en-us/docs/dev/user_doc/guide/task/python.json
+++ b/en-us/docs/dev/user_doc/guide/task/python.json
@@ -1,6 +1,6 @@
{
"filename": "python.md",
- "__html": "<h1>Python Node</h1>\n<h2>Overview</h2>\n<p>Use <code>Python
Task</code> to create a python-type task and execute python scripts. When the
worker executes <code>Python Task</code>,\nit will generate a temporary python
script, and executes the script by the Linux user with the same name as the
tenant.</p>\n<h2>Create Task</h2>\n<ul>\n<li>Click Project Management-Project
Name-Workflow Definition, and click the "Create Workflow" button to
enter the DAG editing page.</ [...]
+ "__html": "<h1>Python Node</h1>\n<h2>Overview</h2>\n<p>Use <code>Python
Task</code> to create a python-type task and execute python scripts. When the
worker executes <code>Python Task</code>,\nit will generate a temporary python
script, and executes the script by the Linux user with the same name as the
tenant.</p>\n<h2>Create Task</h2>\n<ul>\n<li>Click Project Management-Project
Name-Workflow Definition, and click the "Create Workflow" button to
enter the DAG editing page.</ [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/python.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/shell.html
b/en-us/docs/dev/user_doc/guide/task/shell.html
index ab301b4d4..6ec0b3232 100644
--- a/en-us/docs/dev/user_doc/guide/task/shell.html
+++ b/en-us/docs/dev/user_doc/guide/task/shell.html
@@ -28,6 +28,8 @@
<li>Environment Name: Configure the environment name in which run the
script.</li>
<li>Times of failed retry attempts: The number of times the task failed to
resubmit. You can select from drop-down or fill-in a number.</li>
<li>Failed retry interval: The time interval for resubmitting the task after a
failed task. You can select from drop-down or fill-in a number.</li>
+<li>Cpu quota: Assign the specified CPU time quota to the task executed. Takes
a percentage value. Default -1 means unlimited. For example, the full CPU load
of one core is 100%,and that of 16 cores is 1600%. This function is controlled
by <a
href="../../architecture/configuration.md">task.resource.limit.state</a></li>
+<li>Max memory：Assign the specified max memory to the task executed. Exceeding
this limit will trigger oom to be killed and will not automatically retry.
Takes an MB value. Default -1 means unlimited. This function is controlled by
<a href="../../architecture/configuration.md">task.resource.limit.state</a></li>
<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
runs exceed the "timeout", an alarm email will send and the task
execution will fail.</li>
<li>Script: Shell program developed by users.</li>
<li>Resource: Refers to the list of resource files that called in the script,
and upload or create files by the Resource Center file management.</li>
diff --git a/en-us/docs/dev/user_doc/guide/task/shell.json
b/en-us/docs/dev/user_doc/guide/task/shell.json
index 5a8164e20..1d11dfdcc 100644
--- a/en-us/docs/dev/user_doc/guide/task/shell.json
+++ b/en-us/docs/dev/user_doc/guide/task/shell.json
@@ -1,6 +1,6 @@
{
"filename": "shell.md",
- "__html": "<h1>Shell</h1>\n<h2>Overview</h2>\n<p>Shell task used to create a
shell task type and execute a series of shell scripts. When the worker run the
shell task, a temporary shell script is generated, and use the Linux user with
the same name as the tenant executes the script.</p>\n<h2>Create
Task</h2>\n<ul>\n<li>Click <code>Project -> Management-Project ->
Name-Workflow Definition</code>, and click the <code>Create Workflow</code>
button to enter the DAG editing page.</li> [...]
+ "__html": "<h1>Shell</h1>\n<h2>Overview</h2>\n<p>Shell task used to create a
shell task type and execute a series of shell scripts. When the worker run the
shell task, a temporary shell script is generated, and use the Linux user with
the same name as the tenant executes the script.</p>\n<h2>Create
Task</h2>\n<ul>\n<li>Click <code>Project -> Management-Project ->
Name-Workflow Definition</code>, and click the <code>Create Workflow</code>
button to enter the DAG editing page.</li> [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/shell.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/switch.html
b/en-us/docs/dev/user_doc/guide/task/switch.html
index ff01d1784..05b39d914 100644
--- a/en-us/docs/dev/user_doc/guide/task/switch.html
+++ b/en-us/docs/dev/user_doc/guide/task/switch.html
@@ -11,10 +11,12 @@
</head>
<body>
<div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
Have queries regarding Apache DolphinScheduler, Join Slack channel to disscuss
them <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/en-us/index.html"><img class="logo"
src=" [...]
-<p>The switch is a conditional judgment node, decide the branch executes
according to the value of <a href="../parameter/global.md">global variable</a>
and the expression result written by the user.</p>
-<h2>Create</h2>
-<p>Drag from the toolbar <img src="/img/switch.png" width="20"/> task node to
canvas to create a task.
-<strong>Note</strong> After created a switch task, you must first configure
the upstream and downstream, then configure the parameter of task branches.</p>
+<p>The switch is a conditional judgment node, decide the branch executes
according to the value of <a href="../parameter/global.md">global variable</a>
and the expression result written by the user.
+<strong>Note</strong> Execute expressions using
javax.script.ScriptEngine.eval.</p>
+<h2>Create Task</h2>
+<p>Click Project -> Management-Project -> Name-Workflow Definition, and
click the Create Workflow button to enter the DAG editing page.
+Drag from the toolbar <img src="/img/switch.png" width="20"/> task node to
canvas to create a task.</p>
+<p><strong>Note</strong> After created a switch task, you must first configure
the upstream and downstream, then configure the parameter of task branches.</p>
<h2>Parameter</h2>
<ul>
<li>Node name: The node name in a workflow definition is unique.</li>
@@ -22,28 +24,32 @@
<li>Descriptive information: Describe the function of the node.</li>
<li>Task priority: When the number of worker threads is insufficient, execute
in the order of priority from high to low, and tasks with the same priority
will execute in a first-in first-out order.</li>
<li>Worker grouping: Assign tasks to the machines of the worker group to
execute. If <code>Default</code> is selected, randomly select a worker machine
for execution.</li>
+<li>Environment name: The environment in Security, if not configured, it will
not be used.</li>
+<li>Task group name: The group in Resources, if not configured, it will not be
used.</li>
<li>Times of failed retry attempts: The number of times the task failed to
resubmit. You can select from drop-down or fill-in a number.</li>
<li>Failed retry interval: The time interval for resubmitting the task after a
failed task. You can select from drop-down or fill-in a number.</li>
+<li>Delay execution time: Task delay execution time.</li>
<li>Timeout alarm: Check the timeout alarm and timeout failure. When the task
runs exceed the "timeout", an alarm email will send and the task
execution will fail.</li>
<li>Condition: You can configure multiple conditions for the switch task. When
the conditions are satisfied, execute the configured branch. You can configure
multiple different conditions to satisfy different businesses.</li>
<li>Branch flow: The default branch flow, when all the conditions are not
satisfied, execute this branch flow.</li>
</ul>
-<h2>Detail</h2>
-<p>Here we have three tasks, the dependencies are <code>A -> B -> [C,
D]</code>, and <code>task_a</code> is a shell task and <code>task_b</code> is a
switch task</p>
-<ul>
-<li>In task A, a global variable named <code>id</code> is defined through <a
href="../parameter/global.md">global variable</a>, and the declaration method
is <code>${setValue(id=1)}</code></li>
-<li>Task B adds conditions and uses global variables declared upstream to
achieve conditional judgment (Note: switch can get the global variables value,
as long as its direct or indirect upstream have already assigned the global
variables before switch acquires). We want to execute task C when <code>id =
1</code>, otherwise run task D
-<ul>
-<li>Configure task C to run when the global variable <code>id=1</code>. Then
edit <code>${id} == 1</code> in the condition of task B, and select
<code>C</code> as branch flow</li>
-<li>For other tasks, select <code>D</code> as branch flow</li>
-</ul>
-</li>
-</ul>
-<p>The following shows the switch task configuration:</p>
-<p><img src="/img/switch_configure.jpg" alt="task-switch-configure"></p>
-<h2>Related Task</h2>
-<p><a href="conditions.md">condition</a>：<a
href="conditions.md">Condition</a>task mainly executes the corresponding branch
based on the execution result status (success, failure) of the upstream node.
-The <a href="switch.md">Switch</a> task mainly executes the corresponding
branch based on the value of the <a href="../parameter/global.md">global
parameter</a> and the judgment expression result written by the user.</p>
+<h2>Task Example</h2>
+<p>This is demonstrated using one switch task and three shell tasks.</p>
+<h3>Create a workflow</h3>
+<p>Create a new switch task, and three shell tasks downstream. The shell task
is not required.
+The switch task needs to be connected with the downstream task to configure
the relationship before the downstream task can be selected.</p>
+<p><img src="/img/tasks/demo/switch_01.png" alt="switch_01"></p>
+<h3>Set conditions</h3>
+<p>Configure the conditions and default branches. If the conditions are met,
the specified branch will be taken. If the conditions are not met, the default
branch will be taken.
+In the figure, if the value of the variable is "A", the branch taskA
is executed, if the value of the variable is "B", the branch taskB is
executed, and default is executed if both are not satisfied.</p>
+<p><img src="/img/tasks/demo/switch_02.png" alt="switch_02"></p>
+<p>Conditions use global variables, please refer to <a
href="../parameter/global.md">Global Parameter</a>.
+The value of the global variable configured here is A.</p>
+<p><img src="/img/tasks/demo/switch_03.png" alt="switch_03"></p>
+<p>If executed correctly, then taskA will be executed correctly.</p>
+<h3>Execute</h3>
+<p>Execute and see if it works as expected. It can be seen that the specified
downstream tasksA are executed as expected.</p>
+<p><img src="/img/tasks/demo/switch_04.png" alt="switch_04"></p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>About us</h3><h4>Do you need feedback? Please
contact us through the following ways.</h4></div><div
class="contact-container"><ul><li><a
href="/en-us/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>Email List</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/en-us/docs/dev/user_doc/guide/task/switch.json
b/en-us/docs/dev/user_doc/guide/task/switch.json
index 99a208639..a5a810640 100644
--- a/en-us/docs/dev/user_doc/guide/task/switch.json
+++ b/en-us/docs/dev/user_doc/guide/task/switch.json
@@ -1,6 +1,6 @@
{
"filename": "switch.md",
- "__html": "<h1>Switch</h1>\n<p>The switch is a conditional judgment node,
decide the branch executes according to the value of <a
href=\"../parameter/global.md\">global variable</a> and the expression result
written by the user.</p>\n<h2>Create</h2>\n<p>Drag from the toolbar <img
src=\"/img/switch.png\" width=\"20\"/> task node to canvas to create a
task.\n<strong>Note</strong> After created a switch task, you must first
configure the upstream and downstream, then configure the parame [...]
+ "__html": "<h1>Switch</h1>\n<p>The switch is a conditional judgment node,
decide the branch executes according to the value of <a
href=\"../parameter/global.md\">global variable</a> and the expression result
written by the user.\n<strong>Note</strong> Execute expressions using
javax.script.ScriptEngine.eval.</p>\n<h2>Create Task</h2>\n<p>Click Project
-> Management-Project -> Name-Workflow Definition, and click the Create
Workflow button to enter the DAG editing page.\nDrag from [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/switch.html",
"meta": {}
}
\ No newline at end of file
diff --git a/img/metrics/metrics-datasource.png
b/img/metrics/metrics-datasource.png
new file mode 100644
index 000000000..7bc96ddd4
Binary files /dev/null and b/img/metrics/metrics-datasource.png differ
diff --git a/img/metrics/metrics-master.png b/img/metrics/metrics-master.png
new file mode 100644
index 000000000..af8c44947
Binary files /dev/null and b/img/metrics/metrics-master.png differ
diff --git a/img/metrics/metrics-worker.png b/img/metrics/metrics-worker.png
new file mode 100644
index 000000000..a45fb93f1
Binary files /dev/null and b/img/metrics/metrics-worker.png differ
diff --git a/img/tasks/demo/switch_01.png b/img/tasks/demo/switch_01.png
new file mode 100644
index 000000000..c99c7edca
Binary files /dev/null and b/img/tasks/demo/switch_01.png differ
diff --git a/img/tasks/demo/switch_02.png b/img/tasks/demo/switch_02.png
new file mode 100644
index 000000000..48aa3edec
Binary files /dev/null and b/img/tasks/demo/switch_02.png differ
diff --git a/img/tasks/demo/switch_03.png b/img/tasks/demo/switch_03.png
new file mode 100644
index 000000000..d4509bac5
Binary files /dev/null and b/img/tasks/demo/switch_03.png differ
diff --git a/img/tasks/demo/switch_04.png b/img/tasks/demo/switch_04.png
new file mode 100644
index 000000000..57144049b
Binary files /dev/null and b/img/tasks/demo/switch_04.png differ
diff --git a/zh-cn/docs/dev/user_doc/architecture/configuration.html
b/zh-cn/docs/dev/user_doc/architecture/configuration.html
index a34ada93c..f838ead1b 100644
--- a/zh-cn/docs/dev/user_doc/architecture/configuration.html
+++ b/zh-cn/docs/dev/user_doc/architecture/configuration.html
@@ -453,6 +453,11 @@
<td>false</td>
<td>是否处于开发模式</td>
</tr>
+<tr>
+<td>task.resource.limit.state</td>
+<td>false</td>
+<td>是否启用资源限制模式</td>
+</tr>
</tbody>
</table>
<h2>5.application-api.properties [API服务配置]</h2>
diff --git a/zh-cn/docs/dev/user_doc/architecture/configuration.json
b/zh-cn/docs/dev/user_doc/architecture/configuration.json
index 5e41e0d3d..eb93baaeb 100644
--- a/zh-cn/docs/dev/user_doc/architecture/configuration.json
+++ b/zh-cn/docs/dev/user_doc/architecture/configuration.json
@@ -1,6 +1,6 @@
{
"filename": "configuration.md",
- "__html": "<!-- markdown-link-check-disable
-->\n<h1>前言</h1>\n<p>本文档为dolphinscheduler配置文件说明文档,针对版本为 dolphinscheduler-1.3.x
版本.</p>\n<h1>目录结构</h1>\n<p>目前dolphinscheduler 所有的配置文件都在 [conf ]
目录中.\n为了更直观的了解[conf]目录所在的位置以及包含的配置文件,请查看下面dolphinscheduler安装目录的简化说明.\n本文主要讲述dolphinscheduler的配置文件.其他部分先不做赘述.</p>\n<p>[注:以下
dolphinscheduler 简称为DS.]</p>\n<pre><code>├── LICENSE\n│\n├── NOTICE\n│\n├──
licenses licenses存放目录\n│\n├── bin
[...]
+ "__html": "<!-- markdown-link-check-disable
-->\n<h1>前言</h1>\n<p>本文档为dolphinscheduler配置文件说明文档,针对版本为 dolphinscheduler-1.3.x
版本.</p>\n<h1>目录结构</h1>\n<p>目前dolphinscheduler 所有的配置文件都在 [conf ]
目录中.\n为了更直观的了解[conf]目录所在的位置以及包含的配置文件,请查看下面dolphinscheduler安装目录的简化说明.\n本文主要讲述dolphinscheduler的配置文件.其他部分先不做赘述.</p>\n<p>[注:以下
dolphinscheduler 简称为DS.]</p>\n<pre><code>├── LICENSE\n│\n├── NOTICE\n│\n├──
licenses licenses存放目录\n│\n├── bin
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/architecture/configuration.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/datax.html
b/zh-cn/docs/dev/user_doc/guide/task/datax.html
index d4a3f123c..b5441b715 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/datax.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/datax.html
@@ -28,6 +28,8 @@
<li>环境名称：配置运行脚本的环境。</li>
<li>失败重试次数：任务失败重新提交的次数。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，以分为单位。</li>
+<li>Cpu 配额:
为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
+<li>最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>延时执行时间：任务延迟执行的时间，以分为单位。</li>
<li>超时警告：勾选超时警告、超时失败，当任务超过“超时时长”后，会发送告警邮件并且任务执行失败。</li>
<li>自定义模板：当默认提供的数据源不满足所需要求的时，可自定义 datax 节点的 json 配置文件内容。</li>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/datax.json
b/zh-cn/docs/dev/user_doc/guide/task/datax.json
index 4619ca34d..1bf558df9 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/datax.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/datax.json
@@ -1,6 +1,6 @@
{
"filename": "datax.md",
- "__html": "<h1>DATAX 节点</h1>\n<h2>综述</h2>\n<p>DataX 任务类型，用于执行 DataX 程序。对于
DataX 节点，worker 会通过执行 <code>${DATAX_HOME}/bin/datax.py</code> 来解析传入的 json
文件。</p>\n<h2>创建任务</h2>\n<ul>\n<li>点击项目管理 -> 项目名称 -> 工作流定义，点击“创建工作流”按钮，进入
DAG 编辑页面；</li>\n<li>拖动工具栏的<img src=\"/img/tasks/icons/datax.png\"
width=\"15\"/>
任务节点到画板中。</li>\n</ul>\n<h2>任务参数</h2>\n<ul>\n<li>节点名称：设置任务节点的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个结点是否能正常调度，如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>\n<li>任务优先级：worker
线程数不足 [...]
+ "__html": "<h1>DATAX 节点</h1>\n<h2>综述</h2>\n<p>DataX 任务类型，用于执行 DataX 程序。对于
DataX 节点，worker 会通过执行 <code>${DATAX_HOME}/bin/datax.py</code> 来解析传入的 json
文件。</p>\n<h2>创建任务</h2>\n<ul>\n<li>点击项目管理 -> 项目名称 -> 工作流定义，点击“创建工作流”按钮，进入
DAG 编辑页面；</li>\n<li>拖动工具栏的<img src=\"/img/tasks/icons/datax.png\"
width=\"15\"/>
任务节点到画板中。</li>\n</ul>\n<h2>任务参数</h2>\n<ul>\n<li>节点名称：设置任务节点的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个结点是否能正常调度，如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>\n<li>任务优先级：worker
线程数不足 [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/datax.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
b/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
index e15720825..5a465b8ac 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/jupyter.html
@@ -34,7 +34,9 @@
<li>Worker分组：任务分配给worker组的机器机执行，选择Default，会随机选择一台worker机执行。</li>
<li>失败重试次数：任务失败重新提交的次数，支持下拉和手填。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。</li>
-<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.</li>
+<li>Cpu 配额:
为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。</li>
+<li>最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
+<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>前置任务：选择当前任务的前置任务，会将被选择的前置任务设置为当前任务的上游。</li>
<li>Conda Env Name: Conda环境名称。</li>
<li>Input Note Path: 输入的jupyter note模板路径。</li>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
b/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
index e21ca7c33..418bf1f85 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/jupyter.json
@@ -1,6 +1,6 @@
{
"filename": "jupyter.md",
- "__html":
"<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p><code>Jupyter</code>任务类型，用于创建并执行<code>Jupyter</code>类型任务。worker
执行该任务的时候，会通过<code>papermill</code>执行<code>jupyter note</code>。\n点击<a
href=\"https://papermill.readthedocs.io/en/latest/\";>这里</a>
获取更多关于<code>papermill</code>的信息。</p>\n<h2>Conda
Configuration</h2>\n<ul>\n<li>在<code>common.properties</code>配置<code>conda.path</code>，将其指向您的<code>conda.sh</code>。这里的<code>conda</code>应该是您用来管理您的
<code>papermill</code>和<code>jupyter</code>所在python [...]
+ "__html":
"<h1>Jupyter</h1>\n<h2>Overview</h2>\n<p><code>Jupyter</code>任务类型，用于创建并执行<code>Jupyter</code>类型任务。worker
执行该任务的时候，会通过<code>papermill</code>执行<code>jupyter note</code>。\n点击<a
href=\"https://papermill.readthedocs.io/en/latest/\";>这里</a>
获取更多关于<code>papermill</code>的信息。</p>\n<h2>Conda
Configuration</h2>\n<ul>\n<li>在<code>common.properties</code>配置<code>conda.path</code>，将其指向您的<code>conda.sh</code>。这里的<code>conda</code>应该是您用来管理您的
<code>papermill</code>和<code>jupyter</code>所在python [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/jupyter.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/python.html
b/zh-cn/docs/dev/user_doc/guide/task/python.html
index b8cf06a4d..0acb098e8 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/python.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/python.html
@@ -29,6 +29,8 @@
<li>环境名称：配置运行脚本的环境。</li>
<li>失败重试次数：任务失败重新提交的次数，支持下拉和手填。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。</li>
+<li>Cpu 配额:
为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
+<li>最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.</li>
<li>脚本：用户开发的PYTHON程序。</li>
<li>资源：是指脚本中需要调用的资源文件列表，资源中心-文件管理上传或创建的文件。</li>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/python.json
b/zh-cn/docs/dev/user_doc/guide/task/python.json
index b1624256f..12edbbacc 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/python.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/python.json
@@ -1,6 +1,6 @@
{
"filename": "python.md",
- "__html": "<h1>Python 节点</h1>\n<h2>Overview</h2>\n<p>Python 任务类型，用于创建 Python
类型的任务并执行一系列的 Python 脚本。worker 执行该任务的时候，会生成一个临时python脚本，\n并使用与租户同名的 linux
用户执行这个脚本。</p>\n<h2>Create
Task</h2>\n<ul>\n<li>点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入DAG编辑页面。</li>\n<li>工具栏中拖动
<img src=\"/img/tasks/icons/python.png\" width=\"15\"/>
到画板中，即可完成创建。</li>\n</ul>\n<h2>Task
Parameter</h2>\n<ul>\n<li>任务名称：设置任务的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>
[...]
+ "__html": "<h1>Python 节点</h1>\n<h2>Overview</h2>\n<p>Python 任务类型，用于创建 Python
类型的任务并执行一系列的 Python 脚本。worker 执行该任务的时候，会生成一个临时python脚本，\n并使用与租户同名的 linux
用户执行这个脚本。</p>\n<h2>Create
Task</h2>\n<ul>\n<li>点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入DAG编辑页面。</li>\n<li>工具栏中拖动
<img src=\"/img/tasks/icons/python.png\" width=\"15\"/>
到画板中，即可完成创建。</li>\n</ul>\n<h2>Task
Parameter</h2>\n<ul>\n<li>任务名称：设置任务的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/python.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/shell.html
b/zh-cn/docs/dev/user_doc/guide/task/shell.html
index d56020ad0..293fb807a 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/shell.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/shell.html
@@ -28,6 +28,8 @@
<li>环境名称：配置运行脚本的环境。</li>
<li>失败重试次数：任务失败重新提交的次数，支持下拉和手填。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。</li>
+<li>Cpu 配额:
为执行的任务分配指定的CPU时间配额，单位百分比，默认-1代表不限制，例如1个核心的CPU满载是100%，16个核心的是1600%。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
+<li>最大内存：为执行的任务分配指定的内存大小，超过会触发OOM被Kill同时不会进行自动重试，单位MB，默认-1代表不限制。这个功能由 <a
href="../../architecture/configuration.md">task.resource.limit.state</a> 控制</li>
<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.</li>
<li>脚本：用户开发的 SHELL 程序。</li>
<li>资源：是指脚本中需要调用的资源文件列表，资源中心-文件管理上传或创建的文件。</li>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/shell.json
b/zh-cn/docs/dev/user_doc/guide/task/shell.json
index 2e00ba07f..992bfd3e5 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/shell.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/shell.json
@@ -1,6 +1,6 @@
{
"filename": "shell.md",
- "__html": "<h1>Shell</h1>\n<h2>综述</h2>\n<p>Shell 任务类型，用于创建 Shell
类型的任务并执行一系列的 Shell 脚本。worker 执行该任务的时候，会生成一个临时 shell 脚本，并使用与租户同名的 linux
用户执行这个脚本。</p>\n<h2>创建任务</h2>\n<ul>\n<li>点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入
DAG 编辑页面。</li>\n<li>工具栏中拖动 <img src=\"/img/tasks/icons/shell.png\"
width=\"15\"/>
到画板中，即可完成创建。</li>\n</ul>\n<h2>任务参数</h2>\n<ul>\n<li>任务名称：设置任务的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>\n<li>任务优先级：worker
线程数不足时，根据优先 [...]
+ "__html": "<h1>Shell</h1>\n<h2>综述</h2>\n<p>Shell 任务类型，用于创建 Shell
类型的任务并执行一系列的 Shell 脚本。worker 执行该任务的时候，会生成一个临时 shell 脚本，并使用与租户同名的 linux
用户执行这个脚本。</p>\n<h2>创建任务</h2>\n<ul>\n<li>点击项目管理-项目名称-工作流定义，点击"创建工作流"按钮，进入
DAG 编辑页面。</li>\n<li>工具栏中拖动 <img src=\"/img/tasks/icons/shell.png\"
width=\"15\"/>
到画板中，即可完成创建。</li>\n</ul>\n<h2>任务参数</h2>\n<ul>\n<li>任务名称：设置任务的名称。一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述：描述该节点的功能。</li>\n<li>任务优先级：worker
线程数不足时，根据优先 [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/shell.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/switch.html
b/zh-cn/docs/dev/user_doc/guide/task/switch.html
index 5b1a24ba6..4d8f264a0 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/switch.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/switch.html
@@ -11,37 +11,45 @@
</head>
<body>
<div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
有关于 Apache DolphinScheduler 的疑问，加入 Slack 频道来讨论他们 <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/zh-cn/index.html"><img class="logo"
src="/img/hlogo_white.svg"/></a><div cla [...]
-<p>Switch是一个条件判断节点，依据<a
href="../parameter/global.md">全局变量</a>的值和用户所编写的表达式判断结果执行对应分支。</p>
+<p>Switch 是一个条件判断节点，依据<a
href="../parameter/global.md">全局变量</a>的值和用户所编写的表达式判断结果执行对应分支。
+<strong>注意</strong>使用 javax.script.ScriptEngine.eval 执行表达式。</p>
<h2>创建任务</h2>
-<p>拖动工具栏中的<img src="/img/switch.png"
width="20"/>任务节点到画板中即能完成任务创建，<strong>注意</strong>switch任务创建后，要先配置上下游，才能配置任务分支的参数</p>
+<p>点击项目管理 -> 项目名称 -> 工作流定义，点击"创建工作流"按钮，进入 DAG 编辑页面。
+拖动工具栏中的 <img src="/img/switch.png" width="20"/> 任务节点到画板中即能完成任务创建。
+<strong>注意</strong> switch 任务创建后，要先配置上下游，才能配置任务分支的参数。</p>
<h2>任务参数</h2>
<ul>
<li>节点名称：一个工作流定义中的节点名称是唯一的。</li>
<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>
<li>描述信息：描述该节点的功能。</li>
-<li>任务优先级：worker线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。</li>
-<li>Worker分组：任务分配给worker组的机器机执行，选择Default，会随机选择一台worker机执行。</li>
+<li>任务优先级：worker 线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。</li>
+<li>Worker 分组：任务分配给 worker 组的机器机执行，选择 Default，会随机选择一台 worker 机执行。</li>
+<li>环境名称：安全中心中配置的环境，不配置则不使用。</li>
+<li>任务组名称：资源中心中配置的任务组，不配置则不使用。</li>
<li>失败重试次数：任务失败重新提交的次数，支持下拉和手填。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。</li>
-<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.</li>
-<li>条件：可以为switch任务配置多个条件，当条件满足时，就会执行指定的分支，可以配置多个不同的条件来满足不同的业务</li>
-<li>分支流转：默认的流转内容，当<strong>条件</strong>中的内容为全部不符合要求时，则运行<strong>分支流转</strong>中指定的分支</li>
+<li>延时执行时间：任务延迟执行的时间。</li>
+<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败。</li>
+<li>条件：可以为 switch
任务配置多个条件，当条件满足时，就会执行指定的分支，可以配置多个不同的条件来满足不同的业务，使用字符串判断时需要使用""。</li>
+<li>分支流转：默认的流转内容，当<strong>条件</strong>中的内容为全部不符合要求时，则运行<strong>分支流转</strong>中指定的分支。</li>
</ul>
-<h2>详细流程</h2>
-<p>假设我们三个任务，其依赖关系是 <code>A -> B -> [C, D]</code>
其中task_a是shell任务，task_b是switch任务</p>
-<ul>
-<li>任务A中通过<a
href="../parameter/global.md">全局变量</a>定义了名为<code>id</code>的全局变量，声明方式为<code>${setValue(id=1)}</code></li>
-<li>任务B增加条件，使用上游声明的全局变量实现条件判断（注意：只要直接、非直接上游在switch运行前对全局变量赋值，switch运行时就可以获取该全局变量）。下面我们想要实现当id为1时，运行任务C，其他运行任务D
-<ul>
-<li>配置当全局变量<code>id=1</code>时，运行任务C。则在任务B的条件中编辑<code>${id} ==
1</code>，分支流转选择<code>C</code></li>
-<li>对于其他任务，在分支流转中选择<code>D</code></li>
-</ul>
-</li>
-</ul>
-<p>最终switch任务的配置如下</p>
-<p><img src="/img/switch_configure.jpg" alt="task-switch-configure"></p>
-<h2>相关任务</h2>
-<p><a href="conditions.md">condition</a>：<a
href="conditions.md">Condition</a>节点主要依据上游节点的执行状态（成功、失败）执行对应分支。Switch节点主要依据全局变量的值和用户所编写的表达式判断结果执行对应分支</p>
+<h2>任务样例</h2>
+<p>这里使用一个 switch 任务以及三个 shell 任务来演示。</p>
+<h3>创建工作流</h3>
+<p>新建 switch 任务，以及下游的三个 shell 任务。shell 任务没有要求。
+switch 任务需要和下游任务连线配置关系后，才可以进行下游任务的选择。</p>
+<p><img src="/img/tasks/demo/switch_01.png" alt="switch_01"></p>
+<h3>设置条件</h3>
+<p>配置条件和默认分支，满足条件会走指定分支，都不满足则走默认分支。
+图中如果变量的值为 "A" 则执行分支 taskA，如果变量的值为 "B" 则执行分支 taskB ，都不满足则执行
default。</p>
+<p><img src="/img/tasks/demo/switch_02.png" alt="switch_02"></p>
+<p>条件使用了全局变量，请参考<a href="../parameter/global.md">全局变量</a>。
+这里配置全局变量的值为 A。</p>
+<p><img src="/img/tasks/demo/switch_03.png" alt="switch_03"></p>
+<p>如果执行正确，那么 taskA 会被正确执行。</p>
+<h3>执行</h3>
+<p>执行，并且查看是否符合预期。可以看到符合预期，执行了指定的下游任务 taskA。</p>
+<p><img src="/img/tasks/demo/switch_04.png" alt="switch_04"></p>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>联系我们</h3><h4>有问题需要反馈？请通过以下方式联系我们。</h4></div><div
class="contact-container"><ul><li><a
href="/zh-cn/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>邮件列表</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png"/><img class="img-change"
src="/img/twitterblue.png"/><p [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/switch.json
b/zh-cn/docs/dev/user_doc/guide/task/switch.json
index ae3f2bfd9..b16cf0e3e 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/switch.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/switch.json
@@ -1,6 +1,6 @@
{
"filename": "switch.md",
- "__html": "<h1>Switch</h1>\n<p>Switch是一个条件判断节点，依据<a
href=\"../parameter/global.md\">全局变量</a>的值和用户所编写的表达式判断结果执行对应分支。</p>\n<h2>创建任务</h2>\n<p>拖动工具栏中的<img
src=\"/img/switch.png\"
width=\"20\"/>任务节点到画板中即能完成任务创建，<strong>注意</strong>switch任务创建后，要先配置上下游，才能配置任务分支的参数</p>\n<h2>任务参数</h2>\n<ul>\n<li>节点名称：一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述信息：描述该节点的功能。</li>\n<li>任务优先级：worker线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。</li>\n<li>Worker分组：任务分配给worker组的机器机执行，选择Defa
[...]
+ "__html": "<h1>Switch</h1>\n<p>Switch 是一个条件判断节点，依据<a
href=\"../parameter/global.md\">全局变量</a>的值和用户所编写的表达式判断结果执行对应分支。\n<strong>注意</strong>使用
javax.script.ScriptEngine.eval 执行表达式。</p>\n<h2>创建任务</h2>\n<p>点击项目管理 -> 项目名称
-> 工作流定义，点击"创建工作流"按钮，进入 DAG 编辑页面。\n拖动工具栏中的 <img
src=\"/img/switch.png\" width=\"20\"/> 任务节点到画板中即能完成任务创建。\n<strong>注意</strong>
switch
任务创建后，要先配置上下游，才能配置任务分支的参数。</p>\n<h2>任务参数</h2>\n<ul>\n<li>节点名称：一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/switch.html",
"meta": {}
}
\ No newline at end of file
