zhongjiajie commented on code in PR #10937:
URL: https://github.com/apache/dolphinscheduler/pull/10937#discussion_r921269344
##########
docs/docs/en/guide/task/chunjun.md:
##########
@@ -0,0 +1,73 @@
+# CHUNJUN
+
+## Overview
+
+CHUNJUN task type for executing CHUNJUN programs. For CHUNJUN nodes, the
worker will execute `${CHUNJUN_HOME}/bin/start-chunjun` to analyze the input
json file.
+
+## Create Task
+
+- Click `Project Management -> Project Name -> Workflow Definition`, and click
the `Create Workflow` button to enter the DAG editing page.
+- Drag the <img src="../../../../img/tasks/icons/chunjun.png" width="15"/>
from the toolbar to the drawing board.
+
+## Task Parameters
+
+| **Parameter** | **Description** |
+| ------- | ---------- |
+| Node name | The node name in a workflow definition is unique. |
+| Run flag | Identifies whether this node schedules normally, if it does not
need to execute, select the prohibition execution. |
+| Task priority | When the number of worker threads is insufficient, execute
in the order of priority from high to low, and tasks with the same priority
will execute in a first-in first-out order. |
+| Description | Describe the function of the node. |
+| Worker group | Assign tasks to the machines of the worker group to execute.
If `Default` is selected, randomly select a worker machine for execution. |
+| Environment Name | Configure the environment name in which run the script. |
+| Number of failed retries | The number of times the task failed to resubmit. |
+| Failed retry interval | The time interval (unit minute) for resubmitting the
task after a failed task. |
+| Task group name | The task group name. |
+| Priority | The task priority. |
+| Delayed execution time | The time, in cents, that a task is delayed in
execution. |
+| Timeout alarm | Check the timeout alarm and timeout failure. When the task
exceeds the "timeout period", an alarm email will be sent and the task
execution will fail. |
+| Custom template | Custom the content of the CHUNJUN node's json profile. |
+| json | json configuration file for CHUNJUN synchronization. |
+| Custom parameters | SQL task type, and stored procedure is a custom
parameter order to set values for the method. The custom parameter type and
data type are the same as the stored procedure task type. The difference is
that the SQL task type custom parameter will replace the \${variable} in the
SQL statement. |
+| Deploy mode | Execute chunjun task mode, eg local standalone. |
+| Option Parameters | Support such as `-confProp
"{\"flink.checkpoint.interval\":60000}"` |
+| Predecessor task | Selecting a predecessor task for the current task will
set the selected predecessor task as upstream of the current task. |
+
+## Task Example
+
+This example demonstrates importing data from Hive into MySQL.
+
+### Configuring the CHUNJUN environment in DolphinScheduler
+
+If you are using the CHUNJUN task type in a production environment, it is
necessary to configure the required environment first. The configuration file
is as follows: `/dolphinscheduler/conf/env/dolphinscheduler_env.sh`.
+
+
+
+After the environment has been configured, DolphinScheduler needs to be
restarted.
+
+### Configuring CHUNJUN Task Node
+
+As the data to be read from Hive, a custom json is required, refer to: the
template json in directory chunjun/chunjun-examples/json/hive.
+
+After writing the required json file, you can configure the node content by
following the steps in the diagram below.
+
+
+
+### View run results
+
+
+
+### Note
+
+Before execute ${CHUNJUN_HOME}/bin/start-chunjun, need to change the shell
${CHUNJUN_HOME}/bin/start-chunjun, remove '&' in order to run in front.
+
+ such as:
+
+```shell
+nohup $JAVA_RUN -cp $JAR_DIR $CLASS_NAME $@ &
+```
+
+update to following:
+
+```shell
+nohup $JAVA_RUN -cp $JAR_DIR $CLASS_NAME $@
Review Comment:
It is seems you are using the different format in this for English and
Chinses docs, could you migrate them to the same format?
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-chunjun/src/main/java/org/apache/dolphinscheduler/plugin/task/chunjun/ChunJunParameters.java:
##########
@@ -0,0 +1,264 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.chunjun;
+
+import org.apache.dolphinscheduler.plugin.task.api.enums.ResourceType;
+import org.apache.dolphinscheduler.plugin.task.api.model.ResourceInfo;
+import
org.apache.dolphinscheduler.plugin.task.api.parameters.AbstractParameters;
+import
org.apache.dolphinscheduler.plugin.task.api.parameters.resource.ResourceParametersHelper;
+import org.apache.dolphinscheduler.spi.enums.Flag;
+import org.apache.dolphinscheduler.spi.utils.StringUtils;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * chunjun parameters
+ */
+public class ChunJunParameters extends AbstractParameters {
Review Comment:
I wonder can we use `@Data` here to getter and setter?
##########
docs/docs/en/guide/task/chunjun.md:
##########
@@ -0,0 +1,73 @@
+# CHUNJUN
+
+## Overview
+
+CHUNJUN task type for executing CHUNJUN programs. For CHUNJUN nodes, the
worker will execute `${CHUNJUN_HOME}/bin/start-chunjun` to analyze the input
json file.
Review Comment:
could you add some hyper link for `CHUNJUN`? and I wonder is it have to use
uppercase for all characters?
##########
docs/docs/en/guide/task/chunjun.md:
##########
@@ -0,0 +1,73 @@
+# CHUNJUN
+
+## Overview
+
+CHUNJUN task type for executing CHUNJUN programs. For CHUNJUN nodes, the
worker will execute `${CHUNJUN_HOME}/bin/start-chunjun` to analyze the input
json file.
Review Comment:
Add you have to add `ChuJun` to
https://github.com/apache/dolphinscheduler/blob/81930e54208f7c0f305c0b9f8846149bfc38f428/docs/configs/docsdev.js#L603
and
https://github.com/apache/dolphinscheduler/blob/81930e54208f7c0f305c0b9f8846149bfc38f428/docs/configs/docsdev.js#L183
to display it in the dolphinscheduler website sidebar
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-chunjun/src/main/java/org/apache/dolphinscheduler/plugin/task/chunjun/ChunJunConstants.java:
##########
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.chunjun;
+
+/**
+ * ChunJun constants
+ */
+public class ChunJunConstants {
+
+ public static final String FLINK_CONF_DIR = "${FLINK_HOME}/conf";
+
+ public static final String FLINK_LIB_DIR = "${FLINK_HOME}/lib";
+
+ public static final String HADOOP_CONF_DIR = "${HADOOP_HOME}/etc/hadoop";
Review Comment:
Does ChuJun have to run base on flink or hadoop? and have to config both
flink and hadoop home? if so please add them to our docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
