This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new 94b3640 [doc] Add task plugin doc of EMR (#702)
94b3640 is described below
commit 94b3640744b467f2c9955d78ec25ef7dd28f4092
Author: ronyang1985 <[email protected]>
AuthorDate: Fri Feb 25 13:26:19 2022 +0800
[doc] Add task plugin doc of EMR (#702)
Co-authored-by: Jiajie Zhong <[email protected]>
---
docs/en-us/dev/user_doc/guide/task/emr.md | 59 +++++++++++++++++++++++++++++++
docs/zh-cn/dev/user_doc/guide/task/emr.md | 58 ++++++++++++++++++++++++++++++
site_config/docsdev.js | 8 +++++
3 files changed, 125 insertions(+)
diff --git a/docs/en-us/dev/user_doc/guide/task/emr.md
b/docs/en-us/dev/user_doc/guide/task/emr.md
new file mode 100644
index 0000000..6bd314b
--- /dev/null
+++ b/docs/en-us/dev/user_doc/guide/task/emr.md
@@ -0,0 +1,59 @@
+# Amazon EMR
+
+## Overview
+
+Amazon EMR task type, for creating EMR clusters on AWS and performing
computing tasks. using [aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/)
in the background, create
[RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html)
object from json，then submit it to AWS.
+
+## Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does
not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are
executed in order from high to low, and when the priority is the same, they are
executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to
execute. If Default is selected, a worker machine will be randomly selected for
execution.
+- Number of failed retry attempts: The number of times the task failed to be
resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a
failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task
exceeds the "timeout period", an alarm email will be sent and the task
execution will fail.
+- json: The json corresponding to the
[RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html)
object，can also refer to
[API_RunJobFlow_Examples](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples)
+
+## json example
+```json
+{
+ "Name": "SparkPi",
+ "ReleaseLabel": "emr-5.34.0",
+ "Applications": [
+ {
+ "Name": "Spark"
+ }
+ ],
+ "Instances": {
+ "InstanceGroups": [
+ {
+ "Name": "Primary node",
+ "InstanceRole": "MASTER",
+ "InstanceType": "m4.xlarge",
+ "InstanceCount": 1
+ }
+ ],
+ "KeepJobFlowAliveWhenNoSteps": false,
+ "TerminationProtected": false
+ },
+ "Steps": [
+ {
+ "Name": "calculate_pi",
+ "ActionOnFailure": "CONTINUE",
+ "HadoopJarStep": {
+ "Jar": "command-runner.jar",
+ "Args": [
+ "/usr/lib/spark/bin/run-example",
+ "SparkPi",
+ "15"
+ ]
+ }
+ }
+ ],
+ "JobFlowRole": "EMR_EC2_DefaultRole",
+ "ServiceRole": "EMR_DefaultRole"
+}
+```
+
diff --git a/docs/zh-cn/dev/user_doc/guide/task/emr.md
b/docs/zh-cn/dev/user_doc/guide/task/emr.md
new file mode 100644
index 0000000..dfa17f6
--- /dev/null
+++ b/docs/zh-cn/dev/user_doc/guide/task/emr.md
@@ -0,0 +1,58 @@
+# Amazon EMR
+
+## 综述
+
+Amazon EMR任务类型，用于在AWS上创建EMR集群并执行计算任务。
后台使用[aws-java-sdk](https://aws.amazon.com/cn/sdk-for-java/)
将json参数转换为[RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html)
对象，提交到AWS
+
+## 任务参数
+- 节点名称：一个工作流定义中的节点名称是唯一的。
+- 运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。
+- 描述信息：描述该节点的功能。
+- 任务优先级：worker线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。
+- Worker分组：任务分配给worker组的机器机执行，选择Default，会随机选择一台worker机执行。
+- 失败重试次数：任务失败重新提交的次数，支持下拉和手填。
+- 失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。
+- 超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.
+- json:
[RunJobFlowRequest](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/RunJobFlowRequest.html)
对象对应的json，详细json定义参见
[API_RunJobFlow_Examples](https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html#API_RunJobFlow_Examples)
+
+## json参数样例
+```json
+{
+ "Name": "SparkPi",
+ "ReleaseLabel": "emr-5.34.0",
+ "Applications": [
+ {
+ "Name": "Spark"
+ }
+ ],
+ "Instances": {
+ "InstanceGroups": [
+ {
+ "Name": "Primary node",
+ "InstanceRole": "MASTER",
+ "InstanceType": "m4.xlarge",
+ "InstanceCount": 1
+ }
+ ],
+ "KeepJobFlowAliveWhenNoSteps": false,
+ "TerminationProtected": false
+ },
+ "Steps": [
+ {
+ "Name": "calculate_pi",
+ "ActionOnFailure": "CONTINUE",
+ "HadoopJarStep": {
+ "Jar": "command-runner.jar",
+ "Args": [
+ "/usr/lib/spark/bin/run-example",
+ "SparkPi",
+ "15"
+ ]
+ }
+ }
+ ],
+ "JobFlowRole": "EMR_EC2_DefaultRole",
+ "ServiceRole": "EMR_DefaultRole"
+}
+```
+
diff --git a/site_config/docsdev.js b/site_config/docsdev.js
index 8f887d1..5b07a23 100644
--- a/site_config/docsdev.js
+++ b/site_config/docsdev.js
@@ -138,6 +138,10 @@ export default {
title: 'Switch',
link:
'/en-us/docs/dev/user_doc/guide/task/switch.html',
},
+ {
+ title: 'Amazon EMR',
+ link:
'/en-us/docs/dev/user_doc/guide/task/emr.html',
+ },
],
},
{
@@ -413,6 +417,10 @@ export default {
title: 'Switch',
link:
'/zh-cn/docs/dev/user_doc/guide/task/switch.html',
},
+ {
+ title: 'Amazon EMR',
+ link:
'/zh-cn/docs/dev/user_doc/guide/task/emr.html',
+ },
],
},
{
