This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new aab4293fc Automated deployment:
599a349d789c45946bc350521dda94150b67d196
aab4293fc is described below
commit aab4293fc1a154bf24dfdc3b09053186bf3bcb13
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Sun Jul 10 05:06:54 2022 +0000
Automated deployment: 599a349d789c45946bc350521dda94150b67d196
---
.../dev/user_doc/guide/expansion-reduction.html | 32 ++++++------
.../dev/user_doc/guide/expansion-reduction.json | 2 +-
.../user_doc/guide/howto/datasource-setting.html | 2 +-
.../user_doc/guide/howto/datasource-setting.json | 2 +-
.../guide/installation/pseudo-cluster.html | 22 ++++-----
.../guide/installation/pseudo-cluster.json | 2 +-
.../user_doc/guide/installation/standalone.html | 6 +--
.../user_doc/guide/installation/standalone.json | 2 +-
en-us/docs/dev/user_doc/guide/task/emr.html | 55 +++++++++++++++++++--
en-us/docs/dev/user_doc/guide/task/emr.json | 2 +-
en-us/docs/dev/user_doc/guide/upgrade.html | 5 +-
en-us/docs/dev/user_doc/guide/upgrade.json | 2 +-
img/tasks/demo/emr_add_job_flow_steps.png | Bin 0 -> 131722 bytes
img/tasks/demo/emr_jobFlowId.png | Bin 0 -> 133514 bytes
img/tasks/demo/emr_run_job_flow.png | Bin 0 -> 139410 bytes
.../dev/user_doc/guide/expansion-reduction.html | 32 ++++++------
.../dev/user_doc/guide/expansion-reduction.json | 2 +-
.../user_doc/guide/howto/datasource-setting.html | 2 +-
.../user_doc/guide/howto/datasource-setting.json | 2 +-
.../guide/installation/pseudo-cluster.html | 22 ++++-----
.../guide/installation/pseudo-cluster.json | 2 +-
.../user_doc/guide/installation/standalone.html | 6 +--
.../user_doc/guide/installation/standalone.json | 2 +-
zh-cn/docs/dev/user_doc/guide/task/emr.html | 48 ++++++++++++++++--
zh-cn/docs/dev/user_doc/guide/task/emr.json | 2 +-
zh-cn/docs/dev/user_doc/guide/upgrade.html | 2 +-
zh-cn/docs/dev/user_doc/guide/upgrade.json | 2 +-
27 files changed, 172 insertions(+), 86 deletions(-)
diff --git a/en-us/docs/dev/user_doc/guide/expansion-reduction.html
b/en-us/docs/dev/user_doc/guide/expansion-reduction.html
index 069d69567..d4560e81f 100644
--- a/en-us/docs/dev/user_doc/guide/expansion-reduction.html
+++ b/en-us/docs/dev/user_doc/guide/expansion-reduction.html
@@ -131,19 +131,19 @@ workers="existing worker01:default,existing
worker02:default,ds3:default,ds
bin/stop-all.sh # stop all services
-sh bin/dolphinscheduler-daemon.sh stop master-server # stop master service
-sh bin/dolphinscheduler-daemon.sh stop worker-server # stop worker service
-sh bin/dolphinscheduler-daemon.sh stop api-server # stop api service
-sh bin/dolphinscheduler-daemon.sh stop alert-server # stop alert service
+bash bin/dolphinscheduler-daemon.sh stop master-server # stop master service
+bash bin/dolphinscheduler-daemon.sh stop worker-server # stop worker service
+bash bin/dolphinscheduler-daemon.sh stop api-server # stop api service
+bash bin/dolphinscheduler-daemon.sh stop alert-server # stop alert service
<span class="hljs-meta">
#</span><span class="bash"> start <span
class="hljs-built_in">command</span>::</span>
bin/start-all.sh # start all services
-sh bin/dolphinscheduler-daemon.sh start master-server # start master service
-sh bin/dolphinscheduler-daemon.sh start worker-server # start worker service
-sh bin/dolphinscheduler-daemon.sh start api-server # start api service
-sh bin/dolphinscheduler-daemon.sh start alert-server # start alert service
+bash bin/dolphinscheduler-daemon.sh start master-server # start master service
+bash bin/dolphinscheduler-daemon.sh start worker-server # start worker service
+bash bin/dolphinscheduler-daemon.sh start api-server # start api service
+bash bin/dolphinscheduler-daemon.sh start alert-server # start alert service
</code></pre>
<pre><code> Attention: When using `stop-all.sh` or `stop-all.sh`, if the
physical machine execute the command is not configured to be ssh-free on all
machines, it will prompt to enter the password
@@ -176,19 +176,19 @@ There are two steps for shrinking. After performing the
following two steps, the
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> stop <span class="hljs-built_in">command</span>:</span>
bin/stop-all.sh # stop all services
-sh bin/dolphinscheduler-daemon.sh stop master-server # stop master service
-sh bin/dolphinscheduler-daemon.sh stop worker-server # stop worker service
-sh bin/dolphinscheduler-daemon.sh stop api-server # stop api service
-sh bin/dolphinscheduler-daemon.sh stop alert-server # stop alert service
+bash bin/dolphinscheduler-daemon.sh stop master-server # stop master service
+bash bin/dolphinscheduler-daemon.sh stop worker-server # stop worker service
+bash bin/dolphinscheduler-daemon.sh stop api-server # stop api service
+bash bin/dolphinscheduler-daemon.sh stop alert-server # stop alert service
<span class="hljs-meta">
#</span><span class="bash"> start <span
class="hljs-built_in">command</span>:</span>
bin/start-all.sh # start all services
-sh bin/dolphinscheduler-daemon.sh start master-server # start master service
-sh bin/dolphinscheduler-daemon.sh start worker-server # start worker service
-sh bin/dolphinscheduler-daemon.sh start api-server # start api service
-sh bin/dolphinscheduler-daemon.sh start alert-server # start alert service
+bash bin/dolphinscheduler-daemon.sh start master-server # start master service
+bash bin/dolphinscheduler-daemon.sh start worker-server # start worker service
+bash bin/dolphinscheduler-daemon.sh start api-server # start api service
+bash bin/dolphinscheduler-daemon.sh start alert-server # start alert service
</code></pre>
<pre><code> Attention: When using `stop-all.sh` or `stop-all.sh`, if the
machine without the command is not configured to be ssh-free for all machines,
it will prompt to enter the password
diff --git a/en-us/docs/dev/user_doc/guide/expansion-reduction.json
b/en-us/docs/dev/user_doc/guide/expansion-reduction.json
index 9ceed575a..831fb2c22 100644
--- a/en-us/docs/dev/user_doc/guide/expansion-reduction.json
+++ b/en-us/docs/dev/user_doc/guide/expansion-reduction.json
@@ -1,6 +1,6 @@
{
"filename": "expansion-reduction.md",
- "__html": "<h1>DolphinScheduler Expansion and
Reduction</h1>\n<h2>Expansion</h2>\n<p>This article describes how to add a new
master service or worker service to an existing DolphinScheduler
cluster.</p>\n<pre><code> Attention: There cannot be more than one master
service process or worker service process on a physical machine.\n If the
physical machine which locate the expansion master or worker node has already
installed the scheduled service, check the [1.4 Modify configuration [...]
+ "__html": "<h1>DolphinScheduler Expansion and
Reduction</h1>\n<h2>Expansion</h2>\n<p>This article describes how to add a new
master service or worker service to an existing DolphinScheduler
cluster.</p>\n<pre><code> Attention: There cannot be more than one master
service process or worker service process on a physical machine.\n If the
physical machine which locate the expansion master or worker node has already
installed the scheduled service, check the [1.4 Modify configuration [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/expansion-reduction.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/howto/datasource-setting.html
b/en-us/docs/dev/user_doc/guide/howto/datasource-setting.html
index 9db6c3b6b..42ab5fcbf 100644
--- a/en-us/docs/dev/user_doc/guide/howto/datasource-setting.html
+++ b/en-us/docs/dev/user_doc/guide/howto/datasource-setting.html
@@ -60,7 +60,7 @@ export SPRING_DATASOURCE_USERNAME={user}
export SPRING_DATASOURCE_PASSWORD={password}
</code></pre>
<p>After the above steps done you would create a new database for
DolphinScheduler, then run the Shell script to init database:</p>
-<pre><code class="language-shell">sh tools/bin/upgrade-schema.sh
+<pre><code class="language-shell">bash tools/bin/upgrade-schema.sh
</code></pre>
<h2>DataSource Center</h2>
<p>DataSource supports MySQL, PostgreSQL, Hive/Impala, Spark, ClickHouse,
Oracle, SQL Server and other DataSources.</p>
diff --git a/en-us/docs/dev/user_doc/guide/howto/datasource-setting.json
b/en-us/docs/dev/user_doc/guide/howto/datasource-setting.json
index 8fd07c066..eecc352e8 100644
--- a/en-us/docs/dev/user_doc/guide/howto/datasource-setting.json
+++ b/en-us/docs/dev/user_doc/guide/howto/datasource-setting.json
@@ -1,6 +1,6 @@
{
"filename": "datasource-setting.md",
- "__html": "<h1>Datasource Setting</h1>\n<h2>Standalone Switching Metadata
Database Configuration</h2>\n<p>We here use MySQL as an example to illustrate
how to configure an external database:</p>\n<ul>\n<li>First of all, follow the
instructions in <a href=\"datasource-setting.md\">datasource-setting</a>
<code>Pseudo-Cluster/Cluster Initialize the Database</code> section to create
and initialize database</li>\n<li>Set the following environment variables in
your terminal or modify the <co [...]
+ "__html": "<h1>Datasource Setting</h1>\n<h2>Standalone Switching Metadata
Database Configuration</h2>\n<p>We here use MySQL as an example to illustrate
how to configure an external database:</p>\n<ul>\n<li>First of all, follow the
instructions in <a href=\"datasource-setting.md\">datasource-setting</a>
<code>Pseudo-Cluster/Cluster Initialize the Database</code> section to create
and initialize database</li>\n<li>Set the following environment variables in
your terminal or modify the <co [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/howto/datasource-setting.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html
b/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html
index 93f42df79..e2beaac06 100644
--- a/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html
+++ b/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html
@@ -136,7 +136,7 @@ export
PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME/bin:
<p>Follow the instructions in <a
href="../howto/datasource-setting.md">datasource-setting</a>
<code>Pseudo-Cluster/Cluster Initialize the Database</code> section to create
and initialize database</p>
<h2>Start DolphinScheduler</h2>
<p>Use <strong>deployment user</strong> you created above, running the
following command to complete the deployment, and the server log will be stored
in the logs folder.</p>
-<pre><code class="language-shell">sh ./bin/install.sh
+<pre><code class="language-shell">bash ./bin/install.sh
</code></pre>
<blockquote>
<p><strong><em>Note:</em></strong> For the first time deployment, there maybe
occur five times of <code>sh: bin/dolphinscheduler-daemon.sh: No such file or
directory</code> in the terminal,
@@ -146,26 +146,26 @@ this is non-important information that you can ignore.</p>
<p>Access address <code>http://localhost:12345/dolphinscheduler/ui</code> and
login DolphinScheduler UI. The default username and password are
<strong>admin/dolphinscheduler123</strong></p>
<h2>Start or Stop Server</h2>
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Stop all DolphinScheduler server</span>
-sh ./bin/stop-all.sh
+bash ./bin/stop-all.sh
<span class="hljs-meta">
#</span><span class="bash"> Start all DolphinScheduler server</span>
-sh ./bin/start-all.sh
+bash ./bin/start-all.sh
<span class="hljs-meta">
#</span><span class="bash"> Start or stop DolphinScheduler Master</span>
-sh ./bin/dolphinscheduler-daemon.sh stop master-server
-sh ./bin/dolphinscheduler-daemon.sh start master-server
+bash ./bin/dolphinscheduler-daemon.sh stop master-server
+bash ./bin/dolphinscheduler-daemon.sh start master-server
<span class="hljs-meta">
#</span><span class="bash"> Start or stop DolphinScheduler Worker</span>
-sh ./bin/dolphinscheduler-daemon.sh start worker-server
-sh ./bin/dolphinscheduler-daemon.sh stop worker-server
+bash ./bin/dolphinscheduler-daemon.sh start worker-server
+bash ./bin/dolphinscheduler-daemon.sh stop worker-server
<span class="hljs-meta">
#</span><span class="bash"> Start or stop DolphinScheduler Api</span>
-sh ./bin/dolphinscheduler-daemon.sh start api-server
-sh ./bin/dolphinscheduler-daemon.sh stop api-server
+bash ./bin/dolphinscheduler-daemon.sh start api-server
+bash ./bin/dolphinscheduler-daemon.sh stop api-server
<span class="hljs-meta">
#</span><span class="bash"> Start or stop Alert</span>
-sh ./bin/dolphinscheduler-daemon.sh start alert-server
-sh ./bin/dolphinscheduler-daemon.sh stop alert-server
+bash ./bin/dolphinscheduler-daemon.sh start alert-server
+bash ./bin/dolphinscheduler-daemon.sh stop alert-server
</code></pre>
<blockquote>
<p><strong><em>Note1:</em></strong>: Each server have
<code>dolphinscheduler_env.sh</code> file in path
<code><server-name>/conf/dolphinscheduler_env.sh</code> which
diff --git a/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.json
b/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.json
index 711fd2945..a2d54331d 100644
--- a/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.json
+++ b/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.json
@@ -1,6 +1,6 @@
{
"filename": "pseudo-cluster.md",
- "__html": "<h1>Pseudo-Cluster Deployment</h1>\n<p>The purpose of the
pseudo-cluster deployment is to deploy the DolphinScheduler service on a single
machine. In this mode, DolphinScheduler's master, worker, API server, are all
on the same machine.</p>\n<p>If you are a new hand and want to experience
DolphinScheduler functions, we recommend you install follow <a
href=\"standalone.md\">Standalone deployment</a>. If you want to experience
more complete functions and schedule massive tasks [...]
+ "__html": "<h1>Pseudo-Cluster Deployment</h1>\n<p>The purpose of the
pseudo-cluster deployment is to deploy the DolphinScheduler service on a single
machine. In this mode, DolphinScheduler's master, worker, API server, are all
on the same machine.</p>\n<p>If you are a new hand and want to experience
DolphinScheduler functions, we recommend you install follow <a
href=\"standalone.md\">Standalone deployment</a>. If you want to experience
more complete functions and schedule massive tasks [...]
"link":
"/dist/en-us/docs/dev/user_doc/guide/installation/pseudo-cluster.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/installation/standalone.html
b/en-us/docs/dev/user_doc/guide/installation/standalone.html
index 3ae5adfd5..6f82e1002 100644
--- a/en-us/docs/dev/user_doc/guide/installation/standalone.html
+++ b/en-us/docs/dev/user_doc/guide/installation/standalone.html
@@ -28,16 +28,16 @@ When Standalone stops or restarts, in-memory H2 database
will clear up. To use S
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Extract and start Standalone Server</span>
tar -xvzf apache-dolphinscheduler-*-bin.tar.gz
cd apache-dolphinscheduler-*-bin
-sh ./bin/dolphinscheduler-daemon.sh start standalone-server
+bash ./bin/dolphinscheduler-daemon.sh start standalone-server
</code></pre>
<h3>Login DolphinScheduler</h3>
<p>Access address <code>http://localhost:12345/dolphinscheduler/ui</code> and
login DolphinScheduler UI. The default username and password are
<strong>admin/dolphinscheduler123</strong></p>
<h3>Start or Stop Server</h3>
<p>The script <code>./bin/dolphinscheduler-daemon.sh</code>can be used not
only quickly start standalone, but also to stop the service operation. The
following are all the commands:</p>
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Start Standalone Server</span>
-sh ./bin/dolphinscheduler-daemon.sh start standalone-server
+bash ./bin/dolphinscheduler-daemon.sh start standalone-server
<span class="hljs-meta">#</span><span class="bash"> Stop Standalone
Server</span>
-sh ./bin/dolphinscheduler-daemon.sh stop standalone-server
+bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
</code></pre>
<blockquote>
<p>Note: Python gateway service is started along with the api-server, and if
you do not want to start Python gateway
diff --git a/en-us/docs/dev/user_doc/guide/installation/standalone.json
b/en-us/docs/dev/user_doc/guide/installation/standalone.json
index 79b048355..06a2831a3 100644
--- a/en-us/docs/dev/user_doc/guide/installation/standalone.json
+++ b/en-us/docs/dev/user_doc/guide/installation/standalone.json
@@ -1,6 +1,6 @@
{
"filename": "standalone.md",
- "__html": "<h1>Standalone</h1>\n<p>Standalone only for quick experience for
DolphinScheduler.</p>\n<p>If you are a new hand and want to experience
DolphinScheduler functions, we recommend you install follow Standalone
deployment. If you want to experience more complete functions and schedule
massive tasks, we recommend you install follow <a
href=\"pseudo-cluster.md\">pseudo-cluster deployment</a>. If you want to deploy
DolphinScheduler in production, we recommend you follow <a href=\"c [...]
+ "__html": "<h1>Standalone</h1>\n<p>Standalone only for quick experience for
DolphinScheduler.</p>\n<p>If you are a new hand and want to experience
DolphinScheduler functions, we recommend you install follow Standalone
deployment. If you want to experience more complete functions and schedule
massive tasks, we recommend you install follow <a
href=\"pseudo-cluster.md\">pseudo-cluster deployment</a>. If you want to deploy
DolphinScheduler in production, we recommend you follow <a href=\"c [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/installation/standalone.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/task/emr.html
b/en-us/docs/dev/user_doc/guide/task/emr.html
index aec01c528..94dc1f980 100644
--- a/en-us/docs/dev/user_doc/guide/task/emr.html
+++ b/en-us/docs/dev/user_doc/guide/task/emr.html
@@ -12,7 +12,12 @@
<body>
<div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
Have queries regarding Apache DolphinScheduler, Join Slack channel to disscuss
them <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/en-us/index.html"><img class="logo"
src=" [...]
<h2>Overview</h2>
-<p>Amazon EMR task type, for creating EMR clusters on AWS and running
computing tasks. Using <a
href="https://aws.amazon.com/cn/sdk-for-java/";>aws-java-sdk</a> in the
background code, to transfer JSON parameters to <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
object and submit to AWS.</p>
+<p>Amazon EMR task type, for operation EMR clusters on AWS and running
computing tasks.
+Using <a href="https://aws.amazon.com/cn/sdk-for-java/";>aws-java-sdk</a> in
the background code, to transfer JSON parameters to task object and submit to
AWS, Two program types are currently supported:</p>
+<ul>
+<li><code>RUN_JOB_FLOW</code> Using <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples";>API_RunJobFlow</a>
submit <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
object</li>
+<li><code>ADD_JOB_FLOW_STEPS</code> Using <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_AddJobFlowSteps.html#API_AddJobFlowSteps_Examples";>API_AddJobFlowSteps</a>
submit <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/AddJobFlowStepsRequest.html";>AddJobFlowStepsRequest</a>
object</li>
+</ul>
<h2>Create Task</h2>
<ul>
<li>Click <code>Project Management -> Project Name -> Workflow
Definition</code>, click the <code>Create Workflow</code> button to enter the
DAG editing page.</li>
@@ -52,20 +57,32 @@
<td>The number of times the task failed to resubmit. You can select from
drop-down or fill-in a number.</td>
</tr>
<tr>
-<td>Failed retry interval: The time interval for resubmitting the task after a
failed task. You can select from drop-down or fill-in a number.</td>
-<td></td>
+<td>Failed retry interval</td>
+<td>The time interval for resubmitting the task after a failed task. You can
select from drop-down or fill-in a number.</td>
</tr>
<tr>
<td>Timeout alarm</td>
<td>Check the timeout alarm and timeout failure. When the task runs exceed the
"timeout", an alarm email will send and the task execution will
fail.</td>
</tr>
<tr>
-<td>JSON</td>
+<td>Program Type</td>
+<td>Select the program type. If it is <code>RUN_JOB_FLOW</code>, you need to
fill in <code>jobFlowDefineJson</code>, if it is
<code>ADD_JOB_FLOW_STEPS</code>, you need to fill in
<code>stepsDefineJson</code>.</td>
+</tr>
+<tr>
+<td>jobFlowDefineJson</td>
<td>JSON corresponding to the <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
object, for details refer to <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples";>API_RunJobFlow_Examples</a>.</td>
</tr>
+<tr>
+<td>stepsDefineJson</td>
+<td>JSON corresponding to the <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/AddJobFlowStepsRequest.html";>AddJobFlowStepsRequest</a>
object, for details refer to <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_AddJobFlowSteps.html#API_AddJobFlowSteps_Examples";>API_AddJobFlowSteps_Examples</a>.</td>
+</tr>
</tbody>
</table>
-<h2>JSON example</h2>
+<h2>Task Example</h2>
+<h3>Create an EMR cluster and run Steps</h3>
+<p>This example shows how to create an <code>EMR</code> task node of type
<code>RUN_JOB_FLOW</code>. Taking the execution of <code>SparkPi</code> as an
example, the task will create an <code>EMR</code> cluster and execute the
<code>SparkPi</code> sample program.
+<img src="/img/tasks/demo/emr_run_job_flow.png" alt="RUN_JOB_FLOW"></p>
+<p>jobFlowDefineJson example</p>
<pre><code class="language-json">{
<span class="hljs-attr">"Name"</span>: <span
class="hljs-string">"SparkPi"</span>,
<span class="hljs-attr">"ReleaseLabel"</span>: <span
class="hljs-string">"emr-5.34.0"</span>,
@@ -104,6 +121,34 @@
<span class="hljs-attr">"ServiceRole"</span>: <span
class="hljs-string">"EMR_DefaultRole"</span>
}
</code></pre>
+<h3>Add a Step to a Running EMR Cluster</h3>
+<p>This example shows how to create an <code>EMR</code> task node of type
<code>ADD_JOB_FLOW_STEPS</code>. Taking the execution of <code>SparkPi</code>
as an example, the task will add a <code>SparkPi</code> sample program to the
running <code>EMR</code> cluster.
+<img src="/img/tasks/demo/emr_add_job_flow_steps.png" alt="ADD_JOB_FLOW_STEPS">
+<img src="/img/tasks/demo/emr_jobFlowId.png" alt="JobFlowId"></p>
+<p>stepsDefineJson example</p>
+<pre><code class="language-json">{
+ <span class="hljs-attr">"JobFlowId"</span>: <span
class="hljs-string">"j-3V628TKAERHP8"</span>,
+ <span class="hljs-attr">"Steps"</span>: [
+ {
+ <span class="hljs-attr">"Name"</span>: <span
class="hljs-string">"calculate_pi"</span>,
+ <span class="hljs-attr">"ActionOnFailure"</span>: <span
class="hljs-string">"CONTINUE"</span>,
+ <span class="hljs-attr">"HadoopJarStep"</span>: {
+ <span class="hljs-attr">"Jar"</span>: <span
class="hljs-string">"command-runner.jar"</span>,
+ <span class="hljs-attr">"Args"</span>: [
+ <span
class="hljs-string">"/usr/lib/spark/bin/run-example"</span>,
+ <span class="hljs-string">"SparkPi"</span>,
+ <span class="hljs-string">"15"</span>
+ ]
+ }
+ }
+ ]
+}
+</code></pre>
+<h2>Notice</h2>
+<ul>
+<li>Failover on EMR Task type has not been implemented. In this time,
DolphinScheduler only supports failover on yarn task type . Other task type,
such as EMR task, k8s task not ready yet.</li>
+<li><code>stepsDefineJson</code> A task definition only supports the
association of a single step, which can better ensure the reliability of the
task state.</li>
+</ul>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>About us</h3><h4>Do you need feedback? Please
contact us through the following ways.</h4></div><div
class="contact-container"><ul><li><a
href="/en-us/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>Email List</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/en-us/docs/dev/user_doc/guide/task/emr.json
b/en-us/docs/dev/user_doc/guide/task/emr.json
index df6ec3e7c..e44015a86 100644
--- a/en-us/docs/dev/user_doc/guide/task/emr.json
+++ b/en-us/docs/dev/user_doc/guide/task/emr.json
@@ -1,6 +1,6 @@
{
"filename": "emr.md",
- "__html": "<h1>Amazon EMR</h1>\n<h2>Overview</h2>\n<p>Amazon EMR task type,
for creating EMR clusters on AWS and running computing tasks. Using <a
href=\"https://aws.amazon.com/cn/sdk-for-java/\";>aws-java-sdk</a> in the
background code, to transfer JSON parameters to <a
href=\"https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html\";>RunJobFlowRequest</a>
object and submit to AWS.</p>\n<h2>Create Task</h2>\n<ul>\n<li>C [...]
+ "__html": "<h1>Amazon EMR</h1>\n<h2>Overview</h2>\n<p>Amazon EMR task type,
for operation EMR clusters on AWS and running computing tasks.\nUsing <a
href=\"https://aws.amazon.com/cn/sdk-for-java/\";>aws-java-sdk</a> in the
background code, to transfer JSON parameters to task object and submit to AWS,
Two program types are currently
supported:</p>\n<ul>\n<li><code>RUN_JOB_FLOW</code> Using <a
href=\"https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Ex
[...]
"link": "/dist/en-us/docs/dev/user_doc/guide/task/emr.html",
"meta": {}
}
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/upgrade.html
b/en-us/docs/dev/user_doc/guide/upgrade.html
index d7c07e516..f4c913f66 100644
--- a/en-us/docs/dev/user_doc/guide/upgrade.html
+++ b/en-us/docs/dev/user_doc/guide/upgrade.html
@@ -27,13 +27,12 @@ jar package and add it to the <code>./tools/libs</code>
directory, then change <
<pre><code>```shell
export DATABASE=${DATABASE:-mysql}
export SPRING_PROFILES_ACTIVE=${DATABASE}
-export
SPRING_DATASOURCE_URL=jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
+export
SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
export SPRING_DATASOURCE_USERNAME={user}
export SPRING_DATASOURCE_PASSWORD={password}
```
</code></pre>
-<p>Execute database upgrade script: <code>sh
./tools/bin/upgrade-schema.sh</code>
-K</p>
+<p>Execute database upgrade script: <code>sh
./tools/bin/upgrade-schema.sh</code></p>
<h3>Upgrade Service</h3>
<h4>Change Configuration <code>bin/env/install_config.conf</code></h4>
<ul>
diff --git a/en-us/docs/dev/user_doc/guide/upgrade.json
b/en-us/docs/dev/user_doc/guide/upgrade.json
index b548e4806..8e25aa7c4 100644
--- a/en-us/docs/dev/user_doc/guide/upgrade.json
+++ b/en-us/docs/dev/user_doc/guide/upgrade.json
@@ -1,6 +1,6 @@
{
"filename": "upgrade.md",
- "__html": "<h1>DolphinScheduler Upgrade</h1>\n<h2>Prepare</h2>\n<h3>Backup
Previous Version's Files and Database</h3>\n<p>To prevent data loss by some
miss-operation, it is recommended to back up data before upgrading. The backup
way according to your environment.</p>\n<h3>Download the Latest Version
Installation Package</h3>\n<p>Download the latest binary distribute package
from <a href=\"/en-us/download/download.html\">download</a> and then put it in
the different\ndirectory where cu [...]
+ "__html": "<h1>DolphinScheduler Upgrade</h1>\n<h2>Prepare</h2>\n<h3>Backup
Previous Version's Files and Database</h3>\n<p>To prevent data loss by some
miss-operation, it is recommended to back up data before upgrading. The backup
way according to your environment.</p>\n<h3>Download the Latest Version
Installation Package</h3>\n<p>Download the latest binary distribute package
from <a href=\"/en-us/download/download.html\">download</a> and then put it in
the different\ndirectory where cu [...]
"link": "/dist/en-us/docs/dev/user_doc/guide/upgrade.html",
"meta": {}
}
\ No newline at end of file
diff --git a/img/tasks/demo/emr_add_job_flow_steps.png
b/img/tasks/demo/emr_add_job_flow_steps.png
new file mode 100644
index 000000000..bc8de30f7
Binary files /dev/null and b/img/tasks/demo/emr_add_job_flow_steps.png differ
diff --git a/img/tasks/demo/emr_jobFlowId.png b/img/tasks/demo/emr_jobFlowId.png
new file mode 100644
index 000000000..b79097afd
Binary files /dev/null and b/img/tasks/demo/emr_jobFlowId.png differ
diff --git a/img/tasks/demo/emr_run_job_flow.png
b/img/tasks/demo/emr_run_job_flow.png
new file mode 100644
index 000000000..ad09c5f7c
Binary files /dev/null and b/img/tasks/demo/emr_run_job_flow.png differ
diff --git a/zh-cn/docs/dev/user_doc/guide/expansion-reduction.html
b/zh-cn/docs/dev/user_doc/guide/expansion-reduction.html
index 9f2471346..5813e65e9 100644
--- a/zh-cn/docs/dev/user_doc/guide/expansion-reduction.html
+++ b/zh-cn/docs/dev/user_doc/guide/expansion-reduction.html
@@ -134,19 +134,19 @@
workers="现有worker01:default,现有worker02:default,ds3:default,ds4:defa
<pre><code class="language-shell">停止命令:
bin/stop-all.sh 停止所有服务
-sh bin/dolphinscheduler-daemon.sh stop master-server 停止 master 服务
-sh bin/dolphinscheduler-daemon.sh stop worker-server 停止 worker 服务
-sh bin/dolphinscheduler-daemon.sh stop api-server 停止 api 服务
-sh bin/dolphinscheduler-daemon.sh stop alert-server 停止 alert 服务
+bash bin/dolphinscheduler-daemon.sh stop master-server 停止 master 服务
+bash bin/dolphinscheduler-daemon.sh stop worker-server 停止 worker 服务
+bash bin/dolphinscheduler-daemon.sh stop api-server 停止 api 服务
+bash bin/dolphinscheduler-daemon.sh stop alert-server 停止 alert 服务
启动命令:
bin/start-all.sh 启动所有服务
-sh bin/dolphinscheduler-daemon.sh start master-server 启动 master 服务
-sh bin/dolphinscheduler-daemon.sh start worker-server 启动 worker 服务
-sh bin/dolphinscheduler-daemon.sh start api-server 启动 api 服务
-sh bin/dolphinscheduler-daemon.sh start alert-server 启动 alert 服务
+bash bin/dolphinscheduler-daemon.sh start master-server 启动 master 服务
+bash bin/dolphinscheduler-daemon.sh start worker-server 启动 worker 服务
+bash bin/dolphinscheduler-daemon.sh start api-server 启动 api 服务
+bash bin/dolphinscheduler-daemon.sh start alert-server 启动 alert 服务
</code></pre>
<pre><code> 注意：
使用stop-all.sh或者stop-all.sh的时候,如果执行该命令的物理机没有配置到所有机器的ssh免登陆的话,会提示输入密码
@@ -179,19 +179,19 @@ sh bin/dolphinscheduler-daemon.sh start alert-server 启动
alert 服务
<pre><code class="language-shell">停止命令:
bin/stop-all.sh 停止所有服务
-sh bin/dolphinscheduler-daemon.sh stop master-server 停止 master 服务
-sh bin/dolphinscheduler-daemon.sh stop worker-server 停止 worker 服务
-sh bin/dolphinscheduler-daemon.sh stop api-server 停止 api 服务
-sh bin/dolphinscheduler-daemon.sh stop alert-server 停止 alert 服务
+bash bin/dolphinscheduler-daemon.sh stop master-server 停止 master 服务
+bash bin/dolphinscheduler-daemon.sh stop worker-server 停止 worker 服务
+bash bin/dolphinscheduler-daemon.sh stop api-server 停止 api 服务
+bash bin/dolphinscheduler-daemon.sh stop alert-server 停止 alert 服务
启动命令:
bin/start-all.sh 启动所有服务
-sh bin/dolphinscheduler-daemon.sh start master-server 启动 master 服务
-sh bin/dolphinscheduler-daemon.sh start worker-server 启动 worker 服务
-sh bin/dolphinscheduler-daemon.sh start api-server 启动 api 服务
-sh bin/dolphinscheduler-daemon.sh start alert-server 启动 alert 服务
+bash bin/dolphinscheduler-daemon.sh start master-server 启动 master 服务
+bash bin/dolphinscheduler-daemon.sh start worker-server 启动 worker 服务
+bash bin/dolphinscheduler-daemon.sh start api-server 启动 api 服务
+bash bin/dolphinscheduler-daemon.sh start alert-server 启动 alert 服务
</code></pre>
<pre><code> 注意：
使用stop-all.sh或者stop-all.sh的时候,如果没有执行该命令的机器没有配置到所有机器的ssh免登陆的话,会提示输入密码
diff --git a/zh-cn/docs/dev/user_doc/guide/expansion-reduction.json
b/zh-cn/docs/dev/user_doc/guide/expansion-reduction.json
index 130fb80c3..69d1b4425 100644
--- a/zh-cn/docs/dev/user_doc/guide/expansion-reduction.json
+++ b/zh-cn/docs/dev/user_doc/guide/expansion-reduction.json
@@ -1,6 +1,6 @@
{
"filename": "expansion-reduction.md",
- "__html": "<h1>DolphinScheduler扩容/缩容 文档</h1>\n<h2>1.
DolphinScheduler扩容文档</h2>\n<p>本文扩容是针对现有的DolphinScheduler集群添加新的master或者worker节点的操作说明.</p>\n<pre><code>
注意： 一台物理机上不能存在多个master服务进程或者worker服务进程.\n
如果扩容master或者worker节点所在的物理机已经安装了调度的服务,请直接跳到 [1.4.修改配置]. 编辑 ** 所有 ** 节点上的配置文件
`conf/config/install_config.conf`.
新增masters或者workers参数,重启调度集群即可.\n</code></pre>\n<h3>1.1.
基础软件安装(必装项请自行安装)</h3>\n<ul>\n<li>[必装] <a
href=\"https://www.oracle.com/technetwork/java/javase/downloads/index.html\";>JD
[...]
+ "__html": "<h1>DolphinScheduler扩容/缩容 文档</h1>\n<h2>1.
DolphinScheduler扩容文档</h2>\n<p>本文扩容是针对现有的DolphinScheduler集群添加新的master或者worker节点的操作说明.</p>\n<pre><code>
注意： 一台物理机上不能存在多个master服务进程或者worker服务进程.\n
如果扩容master或者worker节点所在的物理机已经安装了调度的服务,请直接跳到 [1.4.修改配置]. 编辑 ** 所有 ** 节点上的配置文件
`conf/config/install_config.conf`.
新增masters或者workers参数,重启调度集群即可.\n</code></pre>\n<h3>1.1.
基础软件安装(必装项请自行安装)</h3>\n<ul>\n<li>[必装] <a
href=\"https://www.oracle.com/technetwork/java/javase/downloads/index.html\";>JD
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/expansion-reduction.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.html
b/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.html
index 72ddb05a2..527af3cd5 100644
--- a/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.html
+++ b/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.html
@@ -60,7 +60,7 @@ export SPRING_DATASOURCE_USERNAME={user}
export SPRING_DATASOURCE_PASSWORD={password}
</code></pre>
<p>完成上述步骤后，您已经为 DolphinScheduler 创建一个新数据库，现在你可以通过快速的 Shell 脚本来初始化数据库</p>
-<pre><code class="language-shell">sh tools/bin/upgrade-schema.sh
+<pre><code class="language-shell">bash tools/bin/upgrade-schema.sh
</code></pre>
<h2>数据源中心</h2>
<p>数据源中心支持MySQL、POSTGRESQL、HIVE/IMPALA、SPARK、CLICKHOUSE、ORACLE、SQLSERVER等数据源。</p>
diff --git a/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.json
b/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.json
index ba21bc194..e8c288dce 100644
--- a/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.json
+++ b/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.json
@@ -1,6 +1,6 @@
{
"filename": "datasource-setting.md",
- "__html": "<h1>数据源配置</h1>\n<h2>Standalone 切换元数据库</h2>\n<p>我们这里以 MySQL
为例来说明如何配置外部数据库：</p>\n<ul>\n<li>首先，参照 <a
href=\"datasource-setting.md\">数据源配置</a> <code>伪分布式/分布式安装初始化数据库</code>
创建并初始化数据库</li>\n<li>在你的命令行或者修改 bin/env/dolphinscheduler_env.sh 设定下列环境变量，将
<code>{user}</code> 和 <code>{password}</code>
改为你数据库的用户名和密码</li>\n</ul>\n<pre><code class=\"language-shell\">export
DATABASE=mysql\nexport SPRING_PROFILES_ACTIVE=${DATABASE}\nexport
SPRING_DATASOURCE_USERNAME={user}\nexport SPRING_DATA [...]
+ "__html": "<h1>数据源配置</h1>\n<h2>Standalone 切换元数据库</h2>\n<p>我们这里以 MySQL
为例来说明如何配置外部数据库：</p>\n<ul>\n<li>首先，参照 <a
href=\"datasource-setting.md\">数据源配置</a> <code>伪分布式/分布式安装初始化数据库</code>
创建并初始化数据库</li>\n<li>在你的命令行或者修改 bin/env/dolphinscheduler_env.sh 设定下列环境变量，将
<code>{user}</code> 和 <code>{password}</code>
改为你数据库的用户名和密码</li>\n</ul>\n<pre><code class=\"language-shell\">export
DATABASE=mysql\nexport SPRING_PROFILES_ACTIVE=${DATABASE}\nexport
SPRING_DATASOURCE_USERNAME={user}\nexport SPRING_DATA [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/howto/datasource-setting.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.html
b/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.html
index f2a4fe88c..0bb66994f 100644
--- a/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.html
+++ b/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.html
@@ -133,7 +133,7 @@ deployUser="dolphinscheduler"
<p>请参考 <a href="../howto/datasource-setting.md">数据源配置</a>
<code>伪分布式/分布式安装初始化数据库</code> 创建并初始化数据库</p>
<h2>启动 DolphinScheduler</h2>
<p>使用上面创建的<strong>部署用户</strong>运行以下命令完成部署，部署后的运行日志将存放在 logs 文件夹内</p>
-<pre><code class="language-shell">sh ./bin/install.sh
+<pre><code class="language-shell">bash ./bin/install.sh
</code></pre>
<blockquote>
<p><strong><em>注意:</em></strong> 第一次部署的话，可能出现 5 次<code>sh:
bin/dolphinscheduler-daemon.sh: No such file or
directory</code>相关信息，次为非重要信息直接忽略即可</p>
@@ -142,26 +142,26 @@ deployUser="dolphinscheduler"
<p>浏览器访问地址 <a
href="http://localhost:12345/dolphinscheduler/ui";>http://localhost:12345/dolphinscheduler/ui</a>
即可登录系统UI。默认的用户名和密码是 <strong>admin/dolphinscheduler123</strong></p>
<h2>启停服务</h2>
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> 一键停止集群所有服务</span>
-sh ./bin/stop-all.sh
+bash ./bin/stop-all.sh
<span class="hljs-meta">
#</span><span class="bash"> 一键开启集群所有服务</span>
-sh ./bin/start-all.sh
+bash ./bin/start-all.sh
<span class="hljs-meta">
#</span><span class="bash"> 启停 Master</span>
-sh ./bin/dolphinscheduler-daemon.sh stop master-server
-sh ./bin/dolphinscheduler-daemon.sh start master-server
+bash ./bin/dolphinscheduler-daemon.sh stop master-server
+bash ./bin/dolphinscheduler-daemon.sh start master-server
<span class="hljs-meta">
#</span><span class="bash"> 启停 Worker</span>
-sh ./bin/dolphinscheduler-daemon.sh start worker-server
-sh ./bin/dolphinscheduler-daemon.sh stop worker-server
+bash ./bin/dolphinscheduler-daemon.sh start worker-server
+bash ./bin/dolphinscheduler-daemon.sh stop worker-server
<span class="hljs-meta">
#</span><span class="bash"> 启停 Api</span>
-sh ./bin/dolphinscheduler-daemon.sh start api-server
-sh ./bin/dolphinscheduler-daemon.sh stop api-server
+bash ./bin/dolphinscheduler-daemon.sh start api-server
+bash ./bin/dolphinscheduler-daemon.sh stop api-server
<span class="hljs-meta">
#</span><span class="bash"> 启停 Alert</span>
-sh ./bin/dolphinscheduler-daemon.sh start alert-server
-sh ./bin/dolphinscheduler-daemon.sh stop alert-server
+bash ./bin/dolphinscheduler-daemon.sh start alert-server
+bash ./bin/dolphinscheduler-daemon.sh stop alert-server
</code></pre>
<blockquote>
<p><strong><em>注意1:</em></strong>: 每个服务在路径
<code><server-name>/conf/dolphinscheduler_env.sh</code> 中都有
<code>dolphinscheduler_env.sh</code> 文件，这是可以为微
diff --git a/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.json
b/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.json
index 933e13494..ba8cb1586 100644
--- a/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.json
+++ b/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.json
@@ -1,6 +1,6 @@
{
"filename": "pseudo-cluster.md",
- "__html": "<h1>伪集群部署</h1>\n<p>伪集群部署目的是在单台机器部署 DolphinScheduler
服务，该模式下master、worker、api server 都在同一台机器上</p>\n<p>如果你是新手，想要体验 DolphinScheduler
的功能，推荐使用<a
href=\"standalone.md\">Standalone</a>方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用<a
href=\"pseudo-cluster.md\">伪集群部署</a>。如果你是在生产中使用，推荐使用<a
href=\"cluster.md\">集群部署</a>或者<a
href=\"kubernetes.md\">kubernetes</a></p>\n<h2>前置准备工作</h2>\n<p>伪分布式部署
DolphinScheduler 需要有外部软件的支持</p>\n<ul>\n<li>JDK：下载<a
href=\"https://www.oracle.com/technetwork/java/javase/dow [...]
+ "__html": "<h1>伪集群部署</h1>\n<p>伪集群部署目的是在单台机器部署 DolphinScheduler
服务，该模式下master、worker、api server 都在同一台机器上</p>\n<p>如果你是新手，想要体验 DolphinScheduler
的功能，推荐使用<a
href=\"standalone.md\">Standalone</a>方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用<a
href=\"pseudo-cluster.md\">伪集群部署</a>。如果你是在生产中使用，推荐使用<a
href=\"cluster.md\">集群部署</a>或者<a
href=\"kubernetes.md\">kubernetes</a></p>\n<h2>前置准备工作</h2>\n<p>伪分布式部署
DolphinScheduler 需要有外部软件的支持</p>\n<ul>\n<li>JDK：下载<a
href=\"https://www.oracle.com/technetwork/java/javase/dow [...]
"link":
"/dist/zh-cn/docs/dev/user_doc/guide/installation/pseudo-cluster.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/installation/standalone.html
b/zh-cn/docs/dev/user_doc/guide/installation/standalone.html
index 0709e5d5d..55b7face0 100644
--- a/zh-cn/docs/dev/user_doc/guide/installation/standalone.html
+++ b/zh-cn/docs/dev/user_doc/guide/installation/standalone.html
@@ -28,16 +28,16 @@
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> 解压并运行 Standalone Server</span>
tar -xvzf apache-dolphinscheduler-*-bin.tar.gz
cd apache-dolphinscheduler-*-bin
-sh ./bin/dolphinscheduler-daemon.sh start standalone-server
+bash ./bin/dolphinscheduler-daemon.sh start standalone-server
</code></pre>
<h3>登录 DolphinScheduler</h3>
<p>浏览器访问地址 <a
href="http://localhost:12345/dolphinscheduler/ui";>http://localhost:12345/dolphinscheduler/ui</a>
即可登录系统UI。默认的用户名和密码是 <strong>admin/dolphinscheduler123</strong></p>
<h2>启停服务</h2>
<p>脚本 <code>./bin/dolphinscheduler-daemon.sh</code> 除了可以快捷启动 standalone
外，还能停止服务运行，全部命令如下</p>
<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> 启动 Standalone Server 服务</span>
-sh ./bin/dolphinscheduler-daemon.sh start standalone-server
+bash ./bin/dolphinscheduler-daemon.sh start standalone-server
<span class="hljs-meta">#</span><span class="bash"> 停止 Standalone Server
服务</span>
-sh ./bin/dolphinscheduler-daemon.sh stop standalone-server
+bash ./bin/dolphinscheduler-daemon.sh stop standalone-server
</code></pre>
<h2>配置数据库</h2>
<p>Standalone server 使用 H2
数据库作为其元数据存储数据，这是为了上手简单，用户在启动服务器之前不需要启动数据库。但是如果用户想将元数据库存储在
diff --git a/zh-cn/docs/dev/user_doc/guide/installation/standalone.json
b/zh-cn/docs/dev/user_doc/guide/installation/standalone.json
index 0c6b9bfeb..d345a65a4 100644
--- a/zh-cn/docs/dev/user_doc/guide/installation/standalone.json
+++ b/zh-cn/docs/dev/user_doc/guide/installation/standalone.json
@@ -1,6 +1,6 @@
{
"filename": "standalone.md",
- "__html": "<h1>Standalone极速体验版</h1>\n<p>Standalone 仅适用于 DolphinScheduler
的快速体验.</p>\n<p>如果你是新手，想要体验 DolphinScheduler
的功能，推荐使用Standalone方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用<a
href=\"pseudo-cluster.md\">伪集群部署</a>。如果你是在生产中使用，推荐使用<a
href=\"cluster.md\">集群部署</a>或者<a
href=\"kubernetes.md\">kubernetes</a></p>\n<blockquote>\n<p><strong><em>注意:</em></strong>
Standalone仅建议20个以下工作流使用，因为其采用内存式的H2 Database, Zookeeper Testing
Server，任务过多可能导致不稳定，并且如果重启或者停止standalone-server会导致内存中数据库里的数据清空。\n如果您要连接外部数据库，比如m
[...]
+ "__html": "<h1>Standalone极速体验版</h1>\n<p>Standalone 仅适用于 DolphinScheduler
的快速体验.</p>\n<p>如果你是新手，想要体验 DolphinScheduler
的功能，推荐使用Standalone方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用<a
href=\"pseudo-cluster.md\">伪集群部署</a>。如果你是在生产中使用，推荐使用<a
href=\"cluster.md\">集群部署</a>或者<a
href=\"kubernetes.md\">kubernetes</a></p>\n<blockquote>\n<p><strong><em>注意:</em></strong>
Standalone仅建议20个以下工作流使用，因为其采用内存式的H2 Database, Zookeeper Testing
Server，任务过多可能导致不稳定，并且如果重启或者停止standalone-server会导致内存中数据库里的数据清空。\n如果您要连接外部数据库，比如m
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/installation/standalone.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/task/emr.html
b/zh-cn/docs/dev/user_doc/guide/task/emr.html
index a8d0e1de4..14fa0f439 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/emr.html
+++ b/zh-cn/docs/dev/user_doc/guide/task/emr.html
@@ -12,7 +12,12 @@
<body>
<div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="banner-tips"><div>🤔
有关于 Apache DolphinScheduler 的疑问，加入 Slack 频道来讨论他们 <a class="link-tips"
href="https://s.apache.org/dolphinscheduler-slack";>join #dolphinscheduler
channel</a>! 🌟</div></div><div class="header-body"><span class="mobile-menu-btn
mobile-menu-btn-dark"></span><a href="/zh-cn/index.html"><img class="logo"
src="/img/hlogo_white.svg"/></a><div cla [...]
<h2>综述</h2>
-<p>Amazon EMR任务类型，用于在AWS上创建EMR集群并执行计算任务。 后台使用<a
href="https://aws.amazon.com/cn/sdk-for-java/";>aws-java-sdk</a> 将json参数转换为<a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
对象，提交到AWS</p>
+<p>Amazon EMR 任务类型，用于在AWS上操作EMR集群并执行计算任务。
+后台使用 <a href="https://aws.amazon.com/cn/sdk-for-java/";>aws-java-sdk</a>
将JSON参数转换为任务对象，提交到AWS，目前支持两种程序类型：</p>
+<ul>
+<li><code>RUN_JOB_FLOW</code> 使用 <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples";>API_RunJobFlow</a>
提交 <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
对象</li>
+<li><code>ADD_JOB_FLOW_STEPS</code> 使用 <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_AddJobFlowSteps.html#API_AddJobFlowSteps_Examples";>API_AddJobFlowSteps</a>
提交 <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/AddJobFlowStepsRequest.html";>AddJobFlowStepsRequest</a>
对象</li>
+</ul>
<h2>任务参数</h2>
<ul>
<li>节点名称：一个工作流定义中的节点名称是唯一的。</li>
@@ -23,9 +28,18 @@
<li>失败重试次数：任务失败重新提交的次数，支持下拉和手填。</li>
<li>失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。</li>
<li>超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.</li>
-<li>json: <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
对象对应的json，详细json定义参见 <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples";>API_RunJobFlow_Examples</a></li>
+<li>程序类型：选择程序类型，如果是<code>RUN_JOB_FLOW</code>，则需要填写<code>jobFlowDefineJson</code>，如果是<code>ADD_JOB_FLOW_STEPS</code>，则需要填写<code>stepsDefineJson</code>。
+<ul>
+<li>jobFlowDefineJson: <a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html";>RunJobFlowRequest</a>
对象对应的JSON，详细JSON定义参见 <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples";>API_RunJobFlow_Examples</a></li>
+<li>stepsDefineJson：<a
href="https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/AddJobFlowStepsRequest.html";>AddJobFlowStepsRequest</a>
对象对应的JSON，详细JSON定义参见 <a
href="https://docs.aws.amazon.com/emr/latest/APIReference/API_AddJobFlowSteps.html#API_AddJobFlowSteps_Examples";>API_AddJobFlowSteps_Examples</a></li>
</ul>
-<h2>json参数样例</h2>
+</li>
+</ul>
+<h2>任务样例</h2>
+<h3>创建EMR集群并运行Steps</h3>
+<p>该样例展示了如何创建<code>RUN_JOB_FLOW</code>类型<code>EMR</code>任务节点，以执行<code>SparkPi</code>为例，该任务会创建一个<code>EMR</code>集群，并且执行<code>SparkPi</code>示例程序。
+<img src="/img/tasks/demo/emr_run_job_flow.png" alt="RUN_JOB_FLOW"></p>
+<p>jobFlowDefineJson 参数样例</p>
<pre><code class="language-json">{
<span class="hljs-attr">"Name"</span>: <span
class="hljs-string">"SparkPi"</span>,
<span class="hljs-attr">"ReleaseLabel"</span>: <span
class="hljs-string">"emr-5.34.0"</span>,
@@ -64,6 +78,34 @@
<span class="hljs-attr">"ServiceRole"</span>: <span
class="hljs-string">"EMR_DefaultRole"</span>
}
</code></pre>
+<h3>向运行中的EMR集群添加Step</h3>
+<p>该样例展示了如何创建<code>ADD_JOB_FLOW_STEPS</code>类型<code>EMR</code>任务节点，以执行<code>SparkPi</code>为例，该任务会向运行中的<code>EMR</code>集群添加一个<code>SparkPi</code>示例程序。
+<img src="/img/tasks/demo/emr_add_job_flow_steps.png" alt="ADD_JOB_FLOW_STEPS">
+<img src="/img/tasks/demo/emr_jobFlowId.png" alt="JobFlowId"></p>
+<p>stepsDefineJson 参数样例</p>
+<pre><code class="language-json">{
+ <span class="hljs-attr">"JobFlowId"</span>: <span
class="hljs-string">"j-3V628TKAERHP8"</span>,
+ <span class="hljs-attr">"Steps"</span>: [
+ {
+ <span class="hljs-attr">"Name"</span>: <span
class="hljs-string">"calculate_pi"</span>,
+ <span class="hljs-attr">"ActionOnFailure"</span>: <span
class="hljs-string">"CONTINUE"</span>,
+ <span class="hljs-attr">"HadoopJarStep"</span>: {
+ <span class="hljs-attr">"Jar"</span>: <span
class="hljs-string">"command-runner.jar"</span>,
+ <span class="hljs-attr">"Args"</span>: [
+ <span
class="hljs-string">"/usr/lib/spark/bin/run-example"</span>,
+ <span class="hljs-string">"SparkPi"</span>,
+ <span class="hljs-string">"15"</span>
+ ]
+ }
+ }
+ ]
+}
+</code></pre>
+<h2>注意事项：</h2>
+<ul>
+<li>EMR 任务类型的故障转移尚未实现。目前，DolphinScheduler 仅支持对 yarn task type 进行故障转移。其他任务类型，如
EMR 任务、k8s 任务尚未准备好。</li>
+<li><code>stepsDefineJson</code> 一个任务定义仅支持关联单个step，这样可以更好的保证任务状态的可靠性。</li>
+</ul>
</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>联系我们</h3><h4>有问题需要反馈？请通过以下方式联系我们。</h4></div><div
class="contact-container"><ul><li><a
href="/zh-cn/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>邮件列表</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png"/><img class="img-change"
src="/img/twitterblue.png"/><p [...]
<script src="/asset/js/react/react-with-addons.min.js"></script>
<script src="/asset/js/react/react-dom.min.js"></script>
diff --git a/zh-cn/docs/dev/user_doc/guide/task/emr.json
b/zh-cn/docs/dev/user_doc/guide/task/emr.json
index 0f1fb9b0e..fd171df01 100644
--- a/zh-cn/docs/dev/user_doc/guide/task/emr.json
+++ b/zh-cn/docs/dev/user_doc/guide/task/emr.json
@@ -1,6 +1,6 @@
{
"filename": "emr.md",
- "__html": "<h1>Amazon EMR</h1>\n<h2>综述</h2>\n<p>Amazon
EMR任务类型，用于在AWS上创建EMR集群并执行计算任务。 后台使用<a
href=\"https://aws.amazon.com/cn/sdk-for-java/\";>aws-java-sdk</a> 将json参数转换为<a
href=\"https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html\";>RunJobFlowRequest</a>
对象，提交到AWS</p>\n<h2>任务参数</h2>\n<ul>\n<li>节点名称：一个工作流定义中的节点名称是唯一的。</li>\n<li>运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。</li>\n<li>描述信息：描述该节点的功能。</li>\n<li>任务优先级：worker线程数不
[...]
+ "__html": "<h1>Amazon EMR</h1>\n<h2>综述</h2>\n<p>Amazon EMR
任务类型，用于在AWS上操作EMR集群并执行计算任务。\n后台使用 <a
href=\"https://aws.amazon.com/cn/sdk-for-java/\";>aws-java-sdk</a>
将JSON参数转换为任务对象，提交到AWS，目前支持两种程序类型：</p>\n<ul>\n<li><code>RUN_JOB_FLOW</code> 使用
<a
href=\"https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples\";>API_RunJobFlow</a>
提交 <a
href=\"https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlow
[...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/task/emr.html",
"meta": {}
}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/upgrade.html
b/zh-cn/docs/dev/user_doc/guide/upgrade.html
index a0f61171e..33629756e 100644
--- a/zh-cn/docs/dev/user_doc/guide/upgrade.html
+++ b/zh-cn/docs/dev/user_doc/guide/upgrade.html
@@ -26,7 +26,7 @@ jar 包 并添加到 <code>./tools/libs</code> 目录下，修改 <code>./bin/en
<pre><code>```shell
export DATABASE=${DATABASE:-mysql}
export SPRING_PROFILES_ACTIVE=${DATABASE}
-export
SPRING_DATASOURCE_URL=jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
+export
SPRING_DATASOURCE_URL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
export SPRING_DATASOURCE_USERNAME={user}
export SPRING_DATASOURCE_PASSWORD={password}
```
diff --git a/zh-cn/docs/dev/user_doc/guide/upgrade.json
b/zh-cn/docs/dev/user_doc/guide/upgrade.json
index 70b4820ca..ec208a67d 100644
--- a/zh-cn/docs/dev/user_doc/guide/upgrade.json
+++ b/zh-cn/docs/dev/user_doc/guide/upgrade.json
@@ -1,6 +1,6 @@
{
"filename": "upgrade.md",
- "__html": "<h1>DolphinScheduler
升级</h1>\n<h2>准备工作</h2>\n<h3>备份上一版本文件和数据库</h3>\n<p>为了防止操作错误导致数据丢失，建议升级之前备份数据，备份方法请结合你数据库的情况来定</p>\n<h3>下载新版本的安装包</h3>\n<p>在<a
href=\"/zh-cn/download/download.html\">下载</a>页面下载最新版本的二进制安装包，并将二进制包放到与当前
dolphinscheduler 服务不一样的路径中，以下升级操作都需要在新版本的目录进行。</p>\n<h2>升级步骤</h2>\n<h3>停止
dolphinscheduler 所有服务</h3>\n<p>根据你部署方式停止 dolphinscheduler 的所有服务，如果你是通过 <a
href=\"./installation/cluster.md\">集群部署</a> 来部署你的 dolphinscheduler 的话，可以通过
<code>sh ./script/stop-all.sh</code> [...]
+ "__html": "<h1>DolphinScheduler
升级</h1>\n<h2>准备工作</h2>\n<h3>备份上一版本文件和数据库</h3>\n<p>为了防止操作错误导致数据丢失，建议升级之前备份数据，备份方法请结合你数据库的情况来定</p>\n<h3>下载新版本的安装包</h3>\n<p>在<a
href=\"/zh-cn/download/download.html\">下载</a>页面下载最新版本的二进制安装包，并将二进制包放到与当前
dolphinscheduler 服务不一样的路径中，以下升级操作都需要在新版本的目录进行。</p>\n<h2>升级步骤</h2>\n<h3>停止
dolphinscheduler 所有服务</h3>\n<p>根据你部署方式停止 dolphinscheduler 的所有服务，如果你是通过 <a
href=\"./installation/cluster.md\">集群部署</a> 来部署你的 dolphinscheduler 的话，可以通过
<code>sh ./script/stop-all.sh</code> [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/upgrade.html",
"meta": {}
}
\ No newline at end of file
