This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository
https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new bcd0839 Automated deployment: Tue Mar 3 03:26:22 UTC 2020
b5040cf267c9d784276035cfd27f2de3d43ac0ee
bcd0839 is described below
commit bcd0839ff699f092a67bba336a371ec71965bf80
Author: dailidong <[email protected]>
AuthorDate: Tue Mar 3 03:26:22 2020 +0000
Automated deployment: Tue Mar 3 03:26:22 UTC 2020
b5040cf267c9d784276035cfd27f2de3d43ac0ee
---
build/documentation.js | 2 +-
en-us/docs/1.2.0/user_doc/cluster-deployment.html | 478 +++++++++++++++++++++
en-us/docs/1.2.0/user_doc/cluster-deployment.json | 6 +
.../docs/1.2.0/user_doc/standalone-deployment.html | 438 +++++++++++++++++++
.../docs/1.2.0/user_doc/standalone-deployment.json | 6 +
5 files changed, 929 insertions(+), 1 deletion(-)
diff --git a/build/documentation.js b/build/documentation.js
index 8e7d75e..7aec56f 100644
--- a/build/documentation.js
+++ b/build/documentation.js
@@ -40,4 +40,4 @@ Object.defineProperty(t,"__esModule",{value:!0});var
i="function"==typeof Symbol
* Copyright © 2012-2019 Faisal Salman <[email protected]>
* Licensed under MIT License
*/
-return"string"==typeof e?e.replace(/[^\d\.]/g,"").split(".")[0]:void
0},trim:function(e){return
e.replace(/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,"")}},m={rgx:function(e,t){for(var
n,r,o,i,a,c,s=0;s<t.length&&!a;){var
l=t[s],u=t[s+1];for(n=r=0;n<l.length&&!a;)if(a=l[n++].exec(e))for(o=0;o<u.length;o++)c=a[++r],i=u[o],"object"==typeof
i&&i.length>0?2==i.length?"function"==typeof
i[1]?this[i[0]]=i[1].call(this,c):this[i[0]]=i[1]:3==i.length?"function"!=typeof
i[1]||i[1].exec&&i[1].test?this[i [...]
\ No newline at end of file
+return"string"==typeof e?e.replace(/[^\d\.]/g,"").split(".")[0]:void
0},trim:function(e){return
e.replace(/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g,"")}},m={rgx:function(e,t){for(var
n,r,o,i,a,c,s=0;s<t.length&&!a;){var
l=t[s],u=t[s+1];for(n=r=0;n<l.length&&!a;)if(a=l[n++].exec(e))for(o=0;o<u.length;o++)c=a[++r],i=u[o],"object"==typeof
i&&i.length>0?2==i.length?"function"==typeof
i[1]?this[i[0]]=i[1].call(this,c):this[i[0]]=i[1]:3==i.length?"function"!=typeof
i[1]||i[1].exec&&i[1].test?this[i [...]
\ No newline at end of file
diff --git a/en-us/docs/1.2.0/user_doc/cluster-deployment.html
b/en-us/docs/1.2.0/user_doc/cluster-deployment.html
new file mode 100644
index 0000000..ec8ce16
--- /dev/null
+++ b/en-us/docs/1.2.0/user_doc/cluster-deployment.html
@@ -0,0 +1,478 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=1.0, user-scalable=no">
+ <meta name="keywords" content="cluster-deployment" />
+ <meta name="description" content="cluster-deployment" />
+ <!-- 网页标签标题 -->
+ <title>cluster-deployment</title>
+ <link rel="shortcut icon" href="/img/docsite.ico"/>
+ <link rel="stylesheet" href="/build/documentation.css" />
+</head>
+<body>
+ <div id="root"><div class="documentation-page"
data-reactroot=""><header class="header-container header-container-normal"><div
class="header-body"><a href="/en-us/index.html"><img class="logo"
src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch
language-switch-normal">中</span><div class="header-menu"><img
class="header-menu-toggle" src="/img/system/menu_gray.png"/><div><ul
class="ant-menu blackClass ant [...]
+<p>DolphinScheduler Cluster deployment is divided into two parts: backend
deployment and frontend deployment.</p>
+<h1>1、Backend Deployment</h1>
+<h3>1.1: Before you begin (please install requirement basic software by
yourself)</h3>
+<ul>
+<li>PostgreSQL (8.2.15+) or Mysql (5.7): Choose One</li>
+<li><a
href="https://www.oracle.com/technetwork/java/javase/downloads/index.html";>JDK</a>
(1.8+): Required. Double-check configure JAVA_HOME and PATH environment
variables in /etc/profile</li>
+<li>ZooKeeper (3.4.6+): Required</li>
+<li>Hadoop (2.6+) or MinIO: Optional. If you need to upload a resource
function, you can choose a local file directory as the upload folder for a
single machine (this operation does not need to deploy Hadoop). Of course, you
can also choose to upload to Hadoop or MinIO.</li>
+</ul>
+<pre><code class="language-markdown"> Tips：DolphinScheduler itself does not
rely on Hadoop, Hive, Spark, only use their clients for the corresponding task
of running.
+</code></pre>
+<h3>1.2: Download the backend package.</h3>
+<ul>
+<li>Please download the latest version of the default installation package to
the server deployment directory. For example, use /opt/dolphinscheduler as the
installation and deployment directory. Download address: <a
href="https://dolphinscheduler.apache.org/en-us/docs/release/download.html";>Download</a>
(Take 1.2.0 for an example). Download the package and move to the installation
and deployment directory. Then unzip it.</li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Create the deployment directory. Do not choose a deployment
directory with a high-privilege directory such as / root or / home.</span>
+mkdir -p /opt/dolphinscheduler;
+cd /opt/dolphinscheduler;
+<span class="hljs-meta">#</span><span class="bash"> unzip</span>
+tar -zxvf
apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-backend-bin.tar.gz -C
/opt/dolphinscheduler;
+
+mv apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-backend-bin
dolphinscheduler-backend
+</code></pre>
+<h3>1.3：Create deployment user and hosts mapping</h3>
+<ul>
+<li>Create a deployment user on the ** all ** deployment machines, and be sure
to configure sudo passwordless. If we plan to deploy DolphinScheduler on 4
machines: ds1, ds2, ds3, and ds4, we first need to create a deployment user on
each machine.</li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> To create a user, you need to <span
class="hljs-built_in">log</span> <span class="hljs-keyword">in</span> as root
and <span class="hljs-built_in">set</span> the deployment user name. Please
modify it yourself. The following uses dolphinscheduler as an example.</span>
+useradd dolphinscheduler;
+<span class="hljs-meta">
+#</span><span class="bash"> Set the user password, please modify it yourself.
The following takes dolphinscheduler123 as an example.</span>
+echo "dolphinscheduler123" | passwd --stdin dolphinscheduler
+<span class="hljs-meta">
+#</span><span class="bash"> Configure sudo passwordless</span>
+echo 'dolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' >>
/etc/sudoers
+</code></pre>
+<pre><code> Notes：
+ - Because the task execution service is based on 'sudo -u {linux-user}' to
switch between different Linux users to implement multi-tenant running jobs,
the deployment user needs to have sudo permissions and is passwordless. The
first-time learners who can ignore it if they don't understand.
+ - If find the "Default requiretty" in the "/etc/sudoers"
file, also comment out.
+ - If you need to use resource upload, you need to assign the user of
permission to operate the local file system, HDFS or MinIO.
+</code></pre>
+<h3>1.4 : Configure hosts mapping and ssh access and modify directory
permissions.</h3>
+<ul>
+<li>
+<p>Use the first machine (hostname is ds1) as the deployment machine,
configure the hosts of all machines to be deployed on ds1, and login as root on
ds1.</p>
+<pre><code class="language-shell">vi /etc/hosts
+<span class="hljs-meta">
+#</span><span class="bash"> add ip hostname</span>
+192.168.xxx.xxx ds1
+192.168.xxx.xxx ds2
+192.168.xxx.xxx ds3
+192.168.xxx.xxx ds4
+</code></pre>
+<p><em>Note: Please delete or comment out the line 127.0.0.1</em></p>
+</li>
+<li>
+<p>Sync /etc/hosts on ds1 to all deployment machines</p>
+<pre><code class="language-shell">for ip in ds2 ds3; # Please replace ds2
ds3 here with the hostname of machines you want to deploy
+do
+ sudo scp -r /etc/hosts $ip:/etc/ # Need to enter root password
during operation
+done
+</code></pre>
+<p><em>Note: can use <code>sshpass -p xxx sudo scp -r /etc/hosts
$ip:/etc/</code> to avoid type password.</em></p>
+<blockquote>
+<p>Install sshpass in Centos:</p>
+<ol>
+<li>
+<p>Install epel</p>
+<p>yum install -y epel-release</p>
+<p>yum repolist</p>
+</li>
+<li>
+<p>After installing epel, you can install sshpass</p>
+<p>yum install -y sshpass</p>
+</li>
+</ol>
+</blockquote>
+</li>
+<li>
+<p>On ds1, switch to the deployment user and configure ssh passwordless
login</p>
+<pre><code class="language-shell">su dolphinscheduler;
+
+ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
+cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
+chmod 600 ~/.ssh/authorized_keys
+</code></pre>
+<p>Note: <em>If configure success, the dolphinscheduler user does not need to
enter a password when executing the command <code>ssh localhost</code></em></p>
+</li>
+<li>
+<p>On ds1, configure the deployment user dolphinscheduler ssh to connect to
other machines to be deployed.</p>
+<pre><code class="language-shell">su dolphinscheduler;
+for ip in ds2 ds3; # Please replace ds2 ds3 here with the hostname of the
machine you want to deploy.
+do
+ ssh-copy-id $ip # You need to manually enter the password of the
dolphinscheduler user during the operation.
+done
+<span class="hljs-meta">#</span><span class="bash"> can use `sshpass -p xxx
ssh-copy-id <span class="hljs-variable">$ip</span>` to avoid <span
class="hljs-built_in">type</span> password.</span>
+</code></pre>
+</li>
+<li>
+<p>On ds1, modify the directory permissions so that the deployment user has
operation permissions on the dolphinscheduler-backend directory.</p>
+<pre><code class="language-shell">sudo chown -R
dolphinscheduler:dolphinscheduler dolphinscheduler-backend
+</code></pre>
+</li>
+</ul>
+<h3>1.5: Database initialization</h3>
+<ul>
+<li>Into the database. The default database is PostgreSQL. If you select
Mysql, you need to add the mysql-connector-java driver package to the lib
directory of DolphinScheduler.</li>
+</ul>
+<pre><code>mysql -uroot -p
+</code></pre>
+<ul>
+<li>After entering the database command line window, execute the database
initialization command and set the user and password. <strong>Note: {user} and
{password} need to be replaced with a specific database username and
password</strong></li>
+</ul>
+<pre><code class="language-mysql"> mysql> CREATE DATABASE
dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
+ mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%'
IDENTIFIED BY '{password}';
+ mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO
'{user}'@'localhost' IDENTIFIED BY '{password}';
+ mysql> flush privileges;
+</code></pre>
+<ul>
+<li>
+<p>Create tables and import basic data</p>
+<ul>
+<li>Modify the following configuration in application-dao.properties under the
conf directory</li>
+</ul>
+<pre><code class="language-shell"> vi conf/application-dao.properties
+</code></pre>
+<ul>
+<li>If you choose Mysql, please comment out the relevant configuration of
PostgreSQL (vice versa), you also need to manually add the
[[mysql-connector-java driver jar] (<a
href="https://downloads.mysql.com/archives/c-j/";>https://downloads.mysql.com/archives/c-j/</a>)]
package to lib under the directory, and then configure the database connection
information correctly.</li>
+</ul>
+<pre><code class="language-properties"><span class="hljs-comment">
#postgre</span>
+<span class="hljs-comment">
#spring.datasource.driver-class-name=org.postgresql.Driver</span>
+<span class="hljs-comment">
#spring.datasource.url=jdbc:postgresql://localhost:5432/dolphinscheduler</span>
+<span class="hljs-comment"> # mysql</span>
+ <span class="hljs-meta">spring.datasource.driver-class-name</span>=<span
class="hljs-string">com.mysql.jdbc.Driver</span>
+ <span class="hljs-meta">spring.datasource.url</span>=<span
class="hljs-string">jdbc:mysql://xxx:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
# Replace the correct IP address</span>
+ <span class="hljs-meta">spring.datasource.username</span>=<span
class="hljs-string">xxx # replace
the currect {user} value</span>
+ <span class="hljs-meta">spring.datasource.password</span>=<span
class="hljs-string">xxx # replace
the currect {password} value</span>
+</code></pre>
+<ul>
+<li>After modifying and saving, execute the create table and import data
script in the script directory.</li>
+</ul>
+<pre><code class="language-shell">sh script/create-dolphinscheduler.sh
+</code></pre>
+</li>
+</ul>
+<p> <em>Note: If you execute the above script and report
"/bin/java: No such file or directory" error, please configure
JAVA_HOME and PATH variables in /etc/profile</em></p>
+<h3>1.6: Modify runtime parameters.</h3>
+<ul>
+<li>
+<p>Modify the environment variable in <code>.dolphinscheduler_env.sh</code>
file which on the 'conf/env' directory (take the relevant software installed
under '/opt/soft' as an example)</p>
+<pre><code class="language-shell">export HADOOP_HOME=/opt/soft/hadoop
+export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
+<span class="hljs-meta">#</span><span class="bash"><span
class="hljs-built_in">export</span> SPARK_HOME1=/opt/soft/spark1</span>
+export SPARK_HOME2=/opt/soft/spark2
+export PYTHON_HOME=/opt/soft/python
+export JAVA_HOME=/opt/soft/java
+export HIVE_HOME=/opt/soft/hive
+export FLINK_HOME=/opt/soft/flink
+export
PATH=$HADOOP_HOME/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$PATH
+</code></pre>
+<pre><code>`Note: This step is very important. For example, JAVA_HOME and PATH
must be configured. Those that are not used can be ignored or commented out. If
".dolphinscheduler_env.sh" cannot be found, run "ls -a"`
+</code></pre>
+</li>
+<li>
+<p>Create Soft link jdk to /usr/bin/java (still JAVA_HOME=/opt/soft/java as an
example)</p>
+<pre><code class="language-shell">sudo ln -s /opt/soft/java/bin/java
/usr/bin/java
+</code></pre>
+</li>
+<li>
+<p>Modify the parameters in the one-click deployment script
<code>install.sh</code>, pay special attention to the configuration of the
following parameters.</p>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Choose mysql or postgresql</span>
+dbtype="mysql"
+<span class="hljs-meta">
+#</span><span class="bash"> Database connection address</span>
+dbhost="192.168.xx.xx:3306"
+<span class="hljs-meta">
+#</span><span class="bash"> Database schema name</span>
+dbname="dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> Database username</span>
+username="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> Database password, <span
class="hljs-keyword">if</span> there are special characters, please use <span
class="hljs-string">'\'</span> escape, you need to modify the specific value of
{passowrd} <span class="hljs-built_in">set</span> above</span>
+passowrd="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> The directory <span
class="hljs-built_in">where</span> DS is installed, such as: <span
class="hljs-string">'/opt/soft/dolphinscheduler'</span>, <span
class="hljs-built_in">which</span> is different from the current
directory.</span>
+installPath="/opt/soft/dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> The system user created <span
class="hljs-keyword">in</span> section 1.3.</span>
+deployUser="dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> zookeeper cluster address</span>
+zkQuorum="192.168.xx.xx:2181,192.168.xx.xx:2181,192.168.xx.xx:2181"
+<span class="hljs-meta">
+#</span><span class="bash"> On machines <span
class="hljs-built_in">which</span> the DS service is deployed</span>
+ips="ds1,ds2,ds3,ds4"
+<span class="hljs-meta">
+#</span><span class="bash"> On machines <span
class="hljs-built_in">which</span> the master service is deployed</span>
+masters="ds1,ds2"
+<span class="hljs-meta">
+#</span><span class="bash"> On machines <span
class="hljs-built_in">which</span> the worker service is deployed</span>
+workers="ds3,ds4"
+<span class="hljs-meta">
+#</span><span class="bash"> On machines <span
class="hljs-built_in">which</span> the alert service is deployed</span>
+alertServer="ds2"
+<span class="hljs-meta">
+#</span><span class="bash"> On machines <span
class="hljs-built_in">which</span> the api service is deployed</span>
+apiServers="ds1"
+<span class="hljs-meta">
+
+#</span><span class="bash"> EMail configuration, taking QQ mailbox as an
example</span>
+<span class="hljs-meta">#</span><span class="bash"> EMail protocol</span>
+mailProtocol="SMTP"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail server address</span>
+mailServerHost="smtp.exmail.qq.com"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail server Port</span>
+mailServerPort="25"
+<span class="hljs-meta">
+#</span><span class="bash"> mailSender and mailUser can be the same one.</span>
+<span class="hljs-meta">#</span><span class="bash"> Sender</span>
+mailSender="[email protected]"
+<span class="hljs-meta">
+#</span><span class="bash"> Receiver</span>
+mailUser="[email protected]"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail password</span>
+mailPassword="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> Set <span class="hljs-literal">true</span> <span
class="hljs-keyword">if</span> the mailbox is TLS protocol, otherwise <span
class="hljs-built_in">set</span> to <span
class="hljs-literal">false</span>.</span>
+starttlsEnable="true"
+<span class="hljs-meta">
+#</span><span class="bash"> Mail service address value, refer to
mailServerHost above.</span>
+sslTrust="smtp.exmail.qq.com"
+<span class="hljs-meta">
+#</span><span class="bash"> Set <span class="hljs-literal">true</span> <span
class="hljs-keyword">if</span> the mailbox is SSL protocol, otherwise <span
class="hljs-built_in">set</span> to <span class="hljs-literal">false</span>.
Note: starttlsEnable and sslEnable cannot be <span
class="hljs-literal">true</span> at the same time.</span>
+sslEnable="false"
+<span class="hljs-meta">
+#</span><span class="bash"> Download path of excel</span>
+xlsFilePath="/tmp/xls"
+<span class="hljs-meta">
+#</span><span class="bash"> Where are some sql and other resource files used
<span class="hljs-keyword">for</span> business uploaded. Can be <span
class="hljs-built_in">set</span>: HDFS, S3, NONE. If a standalone wants to use
the <span class="hljs-built_in">local</span> file system, please configure it
as HDFS, because HDFS supports the <span class="hljs-built_in">local</span>
file system; <span class="hljs-keyword">if</span> you <span
class="hljs-keyword">do</span> not need the resource [...]
+resUploadStartupType="HDFS"
+<span class="hljs-meta">
+#</span><span class="bash"> Note: If you want to upload to HDFS and the
NameNode has HA enabled, you need to put core-site.xml and hdfs-site.xml <span
class="hljs-keyword">in</span> the conf directory. In this example, it is
placed under /opt/dolphinscheduler/conf, and Configure the namenode cluster
name; <span class="hljs-keyword">if</span> the NameNode is not HA, modify it to
a specific IP or host name.</span>
+defaultFS="hdfs://mycluster:8020"
+<span class="hljs-meta">
+
+#</span><span class="bash"> If the ResourceManager is HA, configure it as the
active-standby IP or hostname of the ResourceManager node, such as <span
class="hljs-string">"192.168.xx.xx, 192.168.xx.xx"</span>; otherwise, <span
class="hljs-keyword">if</span> it is a single ResourceManager or yarn is not
used at all, please configure yarnHaIps = <span class="hljs-string">""</span>.
That<span class="hljs-string">'s it, I don'</span>t use yarn here, the
configuration is <span class="hljs-str [...]
+yarnHaIps=""
+<span class="hljs-meta">
+#</span><span class="bash"> If it is a single ResourceManager, configure it as
the ResourceManager node ip or hostname, otherwise, keep the default value.
Yarn is not used here, keep the default.</span>
+singleYarnIp="ark1"
+</code></pre>
+<p><em>Attention:</em></p>
+<ul>
+<li>If you need to upload resources to the Hadoop cluster, and the NameNode of
the Hadoop cluster is configured with HA, you need to enable HDFS resource
upload, and you need to copy the core-site.xml and hdfs-site.xml in the Hadoop
cluster to /opt/ dolphinscheduler/conf. Non-NameNode HA skips the next
step.</li>
+</ul>
+</li>
+</ul>
+<h3>1.7: Install python's zookeeper tool kazoo</h3>
+<ul>
+<li>Install python's zookeeper tool. <code>This step is only used for
one-click deployment.</code></li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Install pip</span>
+sudo yum -y install python-pip; # ubuntu: sudo apt-get install python-pip
+sudo pip install kazoo;
+</code></pre>
+<p><em>Note: If yum does not find python-pip, you can also install it by
following commands</em></p>
+<pre><code class="language-shell">sudo curl
https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+sudo python get-pip.py # 如果是python3，使用sudo python3 get-pip.py
+<span class="hljs-meta">#</span><span class="bash"> <span
class="hljs-keyword">then</span></span>
+sudo pip install kazoo;
+</code></pre>
+<ul>
+<li>
+<p>Switch to the deployment user and execute the one-click deployment
script</p>
+<p><code>sh install.sh</code></p>
+</li>
+</ul>
+<pre><code>Note:
+For the first deployment, the following message appears in step 3 of `3, stop
server` during operation. This message can be ignored.
+sh: bin/dolphinscheduler-daemon.sh: No such file or directory
+</code></pre>
+<ul>
+<li>After the script is completed, the following 5 services will be started.
Use the <code>jps</code> command to check whether the services are started
(<code>jps</code> comes with <code>java JDK</code>)</li>
+</ul>
+<pre><code class="language-aidl"> MasterServer ----- master service
+ WorkerServer ----- worker service
+ LoggerServer ----- logger service
+ ApiApplicationServer ----- api service
+ AlertServer ----- alert service
+</code></pre>
+<p>If the above services are started normally, the automatic deployment is
successful.</p>
+<p>After the deployment is successful, you can view the logs. The logs are
stored in the logs folder.</p>
+<pre><code class="language-log"> logs/
+ ├── dolphinscheduler-alert-server.log
+ ├── dolphinscheduler-master-server.log
+ |—— dolphinscheduler-worker-server.log
+ |—— dolphinscheduler-api-server.log
+ |—— dolphinscheduler-logger-server.log
+</code></pre>
+<h1>2. Frontend Deployment</h1>
+<p>Please download the latest version of the frontend installation package to
the server deployment directory, download address: [Download] (<a
href="https://dolphinscheduler.apache.org/en-us/docs/release/download.html";>https://dolphinscheduler.apache.org/en-us/docs/release/download.html</a>)
(Take 1.2.0 version as an example ), Upload the tar.gz package to this
directory after downloading and unzip it.</p>
+<pre><code class="language-shell">cd /opt/dolphinscheduler;
+
+tar -zxvf
apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-front-bin.tar.gz -C
/opt/dolphinscheduler;
+
+mv apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-front-bin
dolphinscheduler-ui
+</code></pre>
+<p><strong>Choose any one of the following methods, automated deployment is
recommended.</strong></p>
+<h3>2.1 Automated Deployment</h3>
+<ul>
+<li>
+<p>Enter the dolphinscheduler-ui directory and execute (<code>Note: Automated
deployment will automatically download nginx</code>)</p>
+<pre><code class="language-shell">cd dolphinscheduler-ui;
+sh ./install-dolphinscheduler-ui.sh;
+</code></pre>
+<ul>
+<li>After the execution, please type the frontend port during operation, the
default port is 8888, if you choose the default, type 'y', or type another
port.</li>
+<li>Then it will let you type the api-server ip that interacts with the
frontend UI.</li>
+<li>Next is the port of the api-server that lets you type to interact with the
frontend UI.</li>
+<li>Next is the operating system selection.</li>
+<li>Wait for deployment to complete.</li>
+</ul>
+</li>
+<li>
+<p>After deployment, in order to prevent too large resources from uploading to
the resource center, it is recommended to modify the nginx upload size
parameters, as follows:</p>
+<ul>
+<li>Add Nginx configuration client_max_body_size 1024m, you can add it in the
http method body.</li>
+</ul>
+<pre><code class="language-shell">vi /etc/nginx/nginx.conf
+<span class="hljs-meta">
+#</span><span class="bash"> add param</span>
+client_max_body_size 1024m;
+</code></pre>
+<ul>
+<li>Then restart Nginx service</li>
+</ul>
+<pre><code class="language-shell">systemctl restart nginx
+</code></pre>
+</li>
+<li>
+<p>Visit the front page address: <a
href="http://localhost:8888";>http://localhost:8888</a>. If the front login page
appears, the front web installation is complete.</p>
+<pre><code class="language-html"><span class="hljs-tag"><<span
class="hljs-name">p</span> <span class="hljs-attr">align</span>=<span
class="hljs-string">"center"</span>></span>
+ <span class="hljs-tag"><<span class="hljs-name">img</span> <span
class="hljs-attr">src</span>=<span class="hljs-string">"/img/login.png"</span>
<span class="hljs-attr">width</span>=<span class="hljs-string">"60%"</span>
/></span>
+ <span class="hljs-tag"></<span class="hljs-name">p</span>></span>
+</code></pre>
+</li>
+</ul>
+<h3>2.2 Manual Deployment</h3>
+<ul>
+<li>
+<p>Install nginx by yourself, download it from the official website: Or
<code>yum install nginx -y</code></p>
+</li>
+<li>
+<p>Modify the nginx configuration file (Note: some place need to be modified
by yourself)</p>
+</li>
+</ul>
+<pre><code class="language-html">vi /etc/nginx/nginx.conf
+
+server {
+ listen 8888; # Your Port
+ server_name localhost;
+ #charset koi8-r;
+ #access_log /var/log/nginx/host.access.log main;
+ location / {
+ root /opt/soft/dolphinscheduler-ui/dist; # Your dist directory
which
+ index index.html index.html;
+ }
+ location /dolphinscheduler {
+ proxy_pass http://localhost:12345; # Your ApiApplicationServer
address
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header x_real_ipP $remote_addr;
+ proxy_set_header remote_addr $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_http_version 1.1;
+ proxy_connect_timeout 4s;
+ proxy_read_timeout 30s;
+ proxy_send_timeout 12s;
+ proxy_set_header Upgrade $http_upgrade;
+ proxy_set_header Connection "upgrade";
+ }
+ #error_page 404 /404.html;
+ # redirect server error pages to the static page /50x.html
+ #
+ error_page 500 502 503 504 /50x.html;
+ location = /50x.html {
+ root /usr/share/nginx/html;
+ }
+}
+</code></pre>
+<ul>
+<li>
+<p>Then restart Nginx service</p>
+<pre><code class="language-shell">systemctl restart nginx
+</code></pre>
+</li>
+<li>
+<p>Visit the front page address: <a
href="http://localhost:8888";>http://localhost:8888</a>. If the front login page
appears, the front web installation is complete.</p>
+<pre><code class="language-html"><span class="hljs-tag"><<span
class="hljs-name">p</span> <span class="hljs-attr">align</span>=<span
class="hljs-string">"center"</span>></span>
+ <span class="hljs-tag"><<span class="hljs-name">img</span> <span
class="hljs-attr">src</span>=<span class="hljs-string">"/img/login.png"</span>
<span class="hljs-attr">width</span>=<span class="hljs-string">"60%"</span>
/></span>
+ <span class="hljs-tag"></<span class="hljs-name">p</span>></span>
+</code></pre>
+</li>
+</ul>
+<h1>3. Start and stop service</h1>
+<ul>
+<li>
+<p>Stop all services</p>
+<p><code>sh ./bin/stop-all.sh</code></p>
+</li>
+<li>
+<p>Start all services</p>
+<p><code>sh ./bin/start-all.sh</code></p>
+</li>
+<li>
+<p>Start and stop master service</p>
+</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
master-server
+sh ./bin/dolphinscheduler-daemon.sh stop master-server
+</code></pre>
+<ul>
+<li>Start and stop worker Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
worker-server
+sh ./bin/dolphinscheduler-daemon.sh stop worker-server
+</code></pre>
+<ul>
+<li>Start and stop api Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
api-server
+sh ./bin/dolphinscheduler-daemon.sh stop api-server
+</code></pre>
+<ul>
+<li>Start and stop logger Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
logger-server
+sh ./bin/dolphinscheduler-daemon.sh stop logger-server
+</code></pre>
+<ul>
+<li>Start and stop alert service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
alert-server
+sh ./bin/dolphinscheduler-daemon.sh stop alert-server
+</code></pre>
+<p><code>Note: Please refer to the "Architecture Design" section for
service usage</code></p>
+</div></section><footer class="footer-container"><div class="footer-body"><img
src="/img/ds_gray.svg"/><div class="cols-container"><div class="col
col-12"><h3>Disclaimer</h3><p>Apache DolphinScheduler (incubating) is an effort
undergoing incubation at The Apache Software Foundation (ASF), sponsored by
Incubator.
+Incubation is required of all newly accepted projects until a further review
indicates
+that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects.
+While incubation status is not necessarily a reflection of the completeness or
stability of the code,
+it does indicate that the project has yet to be fully endorsed by the
ASF.</p></div><div class="col col-6"><dl><dt>Documentation</dt><dd><a
href="/en-us/docs/1.2.0/user_doc/architecture-design.html"
target="_self">Overview</a></dd><dd><a
href="/en-us/docs/1.2.0/user_doc/quick-start.html" target="_self">Quick
start</a></dd><dd><a href="/en-us/docs/1.2.0/user_doc/backend-development.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href=" [...]
+ <script
src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js";></script>
+ <script
src="https://f.alicdn.com/react/15.4.1/react-dom.min.js";></script>
+ <script>
+ window.rootPath = '';
+ </script>
+ <script src="/build/documentation.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/en-us/docs/1.2.0/user_doc/cluster-deployment.json
b/en-us/docs/1.2.0/user_doc/cluster-deployment.json
new file mode 100644
index 0000000..70e120b
--- /dev/null
+++ b/en-us/docs/1.2.0/user_doc/cluster-deployment.json
@@ -0,0 +1,6 @@
+{
+ "filename": "cluster-deployment.md",
+ "__html": "<h1>Cluster Deployment</h1>\n<p>DolphinScheduler Cluster
deployment is divided into two parts: backend deployment and frontend
deployment.</p>\n<h1>1、Backend Deployment</h1>\n<h3>1.1: Before you begin
(please install requirement basic software by
yourself)</h3>\n<ul>\n<li>PostgreSQL (8.2.15+) or Mysql (5.7): Choose
One</li>\n<li><a
href=\"https://www.oracle.com/technetwork/java/javase/downloads/index.html\";>JDK</a>
(1.8+): Required. Double-check configure JAVA_HOME and PATH [...]
+ "link": "/en-us/docs/1.2.0/user_doc/cluster-deployment.html",
+ "meta": {}
+}
\ No newline at end of file
diff --git a/en-us/docs/1.2.0/user_doc/standalone-deployment.html
b/en-us/docs/1.2.0/user_doc/standalone-deployment.html
new file mode 100644
index 0000000..7962f97
--- /dev/null
+++ b/en-us/docs/1.2.0/user_doc/standalone-deployment.html
@@ -0,0 +1,438 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=1.0, user-scalable=no">
+ <meta name="keywords" content="standalone-deployment" />
+ <meta name="description" content="standalone-deployment" />
+ <!-- 网页标签标题 -->
+ <title>standalone-deployment</title>
+ <link rel="shortcut icon" href="/img/docsite.ico"/>
+ <link rel="stylesheet" href="/build/documentation.css" />
+</head>
+<body>
+ <div id="root"><div class="documentation-page"
data-reactroot=""><header class="header-container header-container-normal"><div
class="header-body"><a href="/en-us/index.html"><img class="logo"
src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch
language-switch-normal">中</span><div class="header-menu"><img
class="header-menu-toggle" src="/img/system/menu_gray.png"/><div><ul
class="ant-menu blackClass ant [...]
+<p>DolphinScheduler Standalone deployment is divided into two parts: backend
deployment and frontend deployment.</p>
+<h1>1. Backend Deployment</h1>
+<h3>1.1: Before you begin (please install requirement basic software by
yourself)</h3>
+<ul>
+<li>PostgreSQL (8.2.15+) or Mysql (5.6 or 5.7): Choose One</li>
+<li><a
href="https://www.oracle.com/technetwork/java/javase/downloads/index.html";>JDK</a>
(1.8+): Required. Double-check configure JAVA_HOME and PATH environment
variables in /etc/profile</li>
+<li>ZooKeeper (3.4.6+): Required</li>
+<li>Hadoop (2.6+) or MinIO: Optional. If you need to upload a resource
function, you can choose a local file directory as the upload folder for a
single machine (this operation does not need to deploy Hadoop). Of course, you
can also choose to upload to Hadoop or MinIO.</li>
+</ul>
+<pre><code class="language-markdown"> Tips：DolphinScheduler itself does not
rely on Hadoop, Hive, Spark, only use their clients for the corresponding task
of running.
+</code></pre>
+<h3>1.2: Download the backend package.</h3>
+<ul>
+<li>Please download the latest version of the default installation package to
the server deployment directory. For example, use /opt/dolphinscheduler as the
installation and deployment directory. Download address: <a
href="https://dolphinscheduler.apache.org/en-us/docs/release/download.html";>Download</a>
(Take 1.2.0 for an example). Download the package and move to the installation
and deployment directory. Then unzip it.</li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Create the deployment directory. Do not choose a deployment
directory with a high-privilege directory such as / root or / home.</span>
+mkdir -p /opt/dolphinscheduler;
+cd /opt/dolphinscheduler;
+<span class="hljs-meta">#</span><span class="bash"> unzip</span>
+tar -zxvf
apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-backend-bin.tar.gz -C
/opt/dolphinscheduler;
+
+mv apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-backend-bin
dolphinscheduler-backend
+</code></pre>
+<p>###1.3: Create an individual user for deployment and grant directory
operation permissions</p>
+<ul>
+<li>Create an individual user, and be sure to configure sudo passwordless.
Take creating 'dolphinscheduler' user as an example.</li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> useradd need root permission</span>
+useradd dolphinscheduler;
+<span class="hljs-meta">
+#</span><span class="bash"> setup password</span>
+echo "dolphinscheduler" | passwd --stdin dolphinscheduler
+<span class="hljs-meta">
+#</span><span class="bash"> setup sudo passwordless</span>
+sed -i '$adolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' /etc/sudoers
+<span class="hljs-meta">
+#</span><span class="bash"> Modify the directory permissions so that the
deployment user has operation permissions on the dolphinscheduler-backend
directory </span>
+chown -R dolphinscheduler:dolphinscheduler dolphinscheduler-backend
+</code></pre>
+<pre><code> Notes：
+ - Because the task execution service is based on 'sudo -u {linux-user}' to
switch between different Linux users to implement multi-tenant running jobs,
the deployment user needs to have sudo permissions and is passwordless. The
first-time learners who can ignore it if they don't understand.
+ - If find the "Default requiretty" in the "/etc/sudoers"
file, also comment out.
+ - If you need to use resource upload, you need to assign the user of
permission to operate the local file system, HDFS or MinIO.
+</code></pre>
+<h3>1.4: ssh passwordless configuration</h3>
+<ul>
+<li>Switch to deployment user and configure ssh passwordless login</li>
+</ul>
+<pre><code class="language-shell">su dolphinscheduler;
+
+ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
+cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
+chmod 600 ~/.ssh/authorized_keys
+</code></pre>
+<p>Note: <em>If configure success, the dolphinscheduler user does not need to
enter a password when executing the command <code>ssh localhost</code></em></p>
+<h3>1.5: Database initialization</h3>
+<ul>
+<li>Into the database. The default database is PostgreSQL. If you select
Mysql, you need to add the mysql-connector-java driver package to the lib
directory of DolphinScheduler.</li>
+</ul>
+<pre><code>mysql -uroot -p
+</code></pre>
+<ul>
+<li>
+<p>After entering the database command line window, execute the database
initialization command and set the user and password. <strong>Note: {user} and
{password} need to be replaced with a specific database username and
password</strong></p>
+<pre><code class="language-mysql">mysql> CREATE DATABASE dolphinscheduler
DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
+mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%'
IDENTIFIED BY '{password}';
+mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost'
IDENTIFIED BY '{password}';
+mysql> flush privileges;
+</code></pre>
+</li>
+<li>
+<p>Create tables and import basic data</p>
+<ul>
+<li>
+<p>Modify the following configuration in application-dao.properties under the
conf directory</p>
+<ul>
+<li>
+<pre><code class="language-shell">vi conf/application-dao.properties
+</code></pre>
+</li>
+</ul>
+</li>
+<li>
+<p>If you choose Mysql, please comment out the relevant configuration of
PostgreSQL (vice versa), you also need to manually add the
[[mysql-connector-java driver jar] (<a
href="https://downloads.mysql.com/archives/c-j/";>https://downloads.mysql.com/archives/c-j/</a>)]
package to lib under the directory, and then configure the database connection
information correctly.</p>
+</li>
+</ul>
+<pre><code class="language-properties"><span class="hljs-comment"> #
postgre</span>
+<span class="hljs-comment"> #
spring.datasource.driver-class-name=org.postgresql.Driver</span>
+<span class="hljs-comment"> #
spring.datasource.url=jdbc:postgresql://localhost:5432/dolphinscheduler</span>
+<span class="hljs-comment"> # mysql</span>
+ <span class="hljs-meta">spring.datasource.driver-class-name</span>=<span
class="hljs-string">com.mysql.jdbc.Driver</span>
+ <span class="hljs-meta">spring.datasource.url</span>=<span
class="hljs-string">jdbc:mysql://xxx:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
# Replace the correct IP address</span>
+ <span class="hljs-meta">spring.datasource.username</span>=<span
class="hljs-string">xxx # replace
the currect {user} value</span>
+ <span class="hljs-meta">spring.datasource.password</span>=<span
class="hljs-string">xxx # replace
the currect {password} value</span>
+</code></pre>
+<ul>
+<li>After modifying and saving, execute the create table and import data
script in the script directory.</li>
+</ul>
+<pre><code class="language-shell">sh script/create-dolphinscheduler.sh
+</code></pre>
+</li>
+</ul>
+<p> <em>Note: If you execute the above script and report
"/bin/java: No such file or directory" error, please configure
JAVA_HOME and PATH variables in /etc/profile</em></p>
+<h3>1.6: Modify runtime parameters.</h3>
+<ul>
+<li>
+<p>Modify the environment variable in <code>.dolphinscheduler_env.sh</code>
file which on the 'conf/env' directory (take the relevant software installed
under '/opt/soft' as an example)</p>
+<pre><code class="language-shell">export HADOOP_HOME=/opt/soft/hadoop
+export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
+<span class="hljs-meta">#</span><span class="bash"><span
class="hljs-built_in">export</span> SPARK_HOME1=/opt/soft/spark1</span>
+export SPARK_HOME2=/opt/soft/spark2
+export PYTHON_HOME=/opt/soft/python
+export JAVA_HOME=/opt/soft/java
+export HIVE_HOME=/opt/soft/hive
+export FLINK_HOME=/opt/soft/flink
+export
PATH=$HADOOP_HOME/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$PATH
+</code></pre>
+<p><code>Note: This step is very important. For example, JAVA_HOME and PATH
must be configured. Those that are not used can be ignored or commented out. If
".dolphinscheduler_env.sh" cannot be found, run "ls
-a"</code></p>
+</li>
+<li>
+<p>Create Soft link jdk to /usr/bin/java (still JAVA_HOME=/opt/soft/java as an
example)</p>
+<pre><code class="language-shell">sudo ln -s /opt/soft/java/bin/java
/usr/bin/java
+</code></pre>
+</li>
+<li>
+<p>Modify the parameters in the one-click deployment script
<code>install.sh</code>, pay special attention to the configuration of the
following parameters.</p>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Choose mysql or postgresql</span>
+dbtype="mysql"
+<span class="hljs-meta">
+#</span><span class="bash"> Database connection address</span>
+dbhost="localhost:3306"
+<span class="hljs-meta">
+#</span><span class="bash"> Database schema name</span>
+dbname="dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> Database username</span>
+username="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> Database password, <span
class="hljs-keyword">if</span> there are special characters, please use <span
class="hljs-string">'\'</span> escape, you need to modify the specific value of
{passowrd} <span class="hljs-built_in">set</span> above</span>
+passowrd="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> The directory <span
class="hljs-built_in">where</span> DS is installed, such as: <span
class="hljs-string">'/opt/soft/dolphinscheduler'</span>, <span
class="hljs-built_in">which</span> is different from the current
directory.</span>
+installPath="/opt/soft/dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> The system user created <span
class="hljs-keyword">in</span> section 1.3.</span>
+deployUser="dolphinscheduler"
+<span class="hljs-meta">
+#</span><span class="bash"> zookeeper connection address, standalone machine
is localhost:2181, port must be provided.</span>
+zkQuorum="localhost:2181"
+<span class="hljs-meta">
+#</span><span class="bash"> On machine <span
class="hljs-built_in">which</span> the DS service is deployed, <span
class="hljs-built_in">set</span> localhost</span>
+ips="localhost"
+<span class="hljs-meta">
+#</span><span class="bash"> On machine <span
class="hljs-built_in">which</span> the master service is deployed, <span
class="hljs-built_in">set</span> localhost</span>
+masters="localhost"
+<span class="hljs-meta">
+#</span><span class="bash"> On machine <span
class="hljs-built_in">which</span> the worker service is deployed, <span
class="hljs-built_in">set</span> localhost</span>
+workers="localhost"
+<span class="hljs-meta">
+#</span><span class="bash"> On machine <span
class="hljs-built_in">which</span> the alert service is deployed, <span
class="hljs-built_in">set</span> localhost</span>
+alertServer="localhost"
+<span class="hljs-meta">
+#</span><span class="bash"> On machine <span
class="hljs-built_in">which</span> the api service is deployed, <span
class="hljs-built_in">set</span> localhost</span>
+apiServers="localhost"
+<span class="hljs-meta">
+
+#</span><span class="bash"> EMail configuration, taking QQ mailbox as an
example</span>
+<span class="hljs-meta">#</span><span class="bash"> EMail protocol</span>
+mailProtocol="SMTP"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail server address</span>
+mailServerHost="smtp.exmail.qq.com"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail server Port</span>
+mailServerPort="25"
+<span class="hljs-meta">
+#</span><span class="bash"> mailSender and mailUser can be the same one.</span>
+<span class="hljs-meta">#</span><span class="bash"> Sender</span>
+mailSender="[email protected]"
+<span class="hljs-meta">
+#</span><span class="bash"> Receiver</span>
+mailUser="[email protected]"
+<span class="hljs-meta">
+#</span><span class="bash"> EMail password</span>
+mailPassword="xxx"
+<span class="hljs-meta">
+#</span><span class="bash"> Set <span class="hljs-literal">true</span> <span
class="hljs-keyword">if</span> the mailbox is TLS protocol, otherwise <span
class="hljs-built_in">set</span> to <span
class="hljs-literal">false</span>.</span>
+starttlsEnable="true"
+<span class="hljs-meta">
+#</span><span class="bash"> Mail service address value, refer to
mailServerHost above.</span>
+sslTrust="smtp.exmail.qq.com"
+<span class="hljs-meta">
+#</span><span class="bash"> Set <span class="hljs-literal">true</span> <span
class="hljs-keyword">if</span> the mailbox is SSL protocol, otherwise <span
class="hljs-built_in">set</span> to <span class="hljs-literal">false</span>.
Note: starttlsEnable and sslEnable cannot be <span
class="hljs-literal">true</span> at the same time.</span>
+sslEnable="false"
+<span class="hljs-meta">
+#</span><span class="bash"> Download path of excel</span>
+xlsFilePath="/tmp/xls"
+<span class="hljs-meta">
+#</span><span class="bash"> Where are some sql and other resource files used
<span class="hljs-keyword">for</span> business uploaded. Can be <span
class="hljs-built_in">set</span>: HDFS, S3, NONE. If a standalone wants to use
the <span class="hljs-built_in">local</span> file system, please configure it
as HDFS, because HDFS supports the <span class="hljs-built_in">local</span>
file system; <span class="hljs-keyword">if</span> you <span
class="hljs-keyword">do</span> not need the resource [...]
+resUploadStartupType="HDFS"
+<span class="hljs-meta">
+#</span><span class="bash"> Take the <span class="hljs-built_in">local</span>
file system as an example.</span>
+<span class="hljs-meta">#</span><span class="bash"> Note: If you want to
upload to HDFS and the NameNode has HA enabled, you need to put core-site.xml
and hdfs-site.xml <span class="hljs-keyword">in</span> the conf directory. In
this example, it is placed under /opt/dolphinscheduler/conf, and Configure the
namenode cluster name; <span class="hljs-keyword">if</span> the NameNode is not
HA, modify it to a specific IP or host name.</span>
+defaultFS="file:///data/dolphinscheduler" # hdfs://{ip|hostname}:8020
+<span class="hljs-meta">
+
+#</span><span class="bash"> If the ResourceManager is HA, configure it as the
active-standby IP or hostname of the ResourceManager node, such as <span
class="hljs-string">"192.168.xx.xx, 192.168.xx.xx"</span>; otherwise, <span
class="hljs-keyword">if</span> it is a single ResourceManager or yarn is not
used at all, please configure yarnHaIps = <span class="hljs-string">""</span>.
That<span class="hljs-string">'s it, I don'</span>t use yarn here, the
configuration is <span class="hljs-str [...]
+yarnHaIps=""
+<span class="hljs-meta">
+#</span><span class="bash"> If it is a single ResourceManager, configure it as
the ResourceManager node ip or hostname, otherwise, keep the default value.
Yarn is not used here, keep the default.</span>
+singleYarnIp="ark1"
+<span class="hljs-meta">
+#</span><span class="bash"> Since HDFS supports the <span
class="hljs-built_in">local</span> file system, you need to ensure that the
<span class="hljs-built_in">local</span> folder exists and has <span
class="hljs-built_in">read</span> and write permissions.</span>
+hdfsPath="/data/dolphinscheduler"
+</code></pre>
+<p><em>Note: If you plan to use the <code>Resource Center</code> function,
execute the following command:</em></p>
+<pre><code class="language-shell">sudo mkdir /data/dolphinscheduler
+sudo chown -R dolphinscheduler:dolphinscheduler /data/dolphinscheduler
+</code></pre>
+</li>
+</ul>
+<h3>1.7: Install python's zookeeper tool kazoo</h3>
+<ul>
+<li>Install python's zookeeper tool. <code>This step is only used for
one-click deployment.</code></li>
+</ul>
+<pre><code class="language-shell"><span class="hljs-meta">#</span><span
class="bash"> Install pip</span>
+sudo yum -y install python-pip; # ubuntu: sudo apt-get install python-pip
+sudo pip install kazoo;
+</code></pre>
+<p><em>Note: If yum does not find python-pip, you can also install it by
following commands</em></p>
+<pre><code class="language-shell">sudo curl
https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+sudo python get-pip.py # python3: sudo python3 get-pip.py
+<span class="hljs-meta">#</span><span class="bash"> <span
class="hljs-keyword">then</span></span>
+sudo pip install kazoo;
+</code></pre>
+<ul>
+<li>
+<p>Switch to the deployment user and execute the one-click deployment
script</p>
+<p><code>sh install.sh</code></p>
+</li>
+</ul>
+<pre><code>Note:
+For the first deployment, the following message appears in step 3 of `3, stop
server` during operation. This message can be ignored.
+sh: bin/dolphinscheduler-daemon.sh: No such file or directory
+</code></pre>
+<ul>
+<li>After the script is completed, the following 5 services will be started.
Use the <code>jps</code> command to check whether the services are started
(<code>jps</code> comes with <code>java JDK</code>)</li>
+</ul>
+<pre><code class="language-aidl"> MasterServer ----- master service
+ WorkerServer ----- worker service
+ LoggerServer ----- logger service
+ ApiApplicationServer ----- api service
+ AlertServer ----- alert service
+</code></pre>
+<p>If the above services are started normally, the automatic deployment is
successful.</p>
+<p>After the deployment is successful, you can view the logs. The logs are
stored in the logs folder.</p>
+<pre><code class="language-log"> logs/
+ ├── dolphinscheduler-alert-server.log
+ ├── dolphinscheduler-master-server.log
+ |—— dolphinscheduler-worker-server.log
+ |—— dolphinscheduler-api-server.log
+ |—— dolphinscheduler-logger-server.log
+</code></pre>
+<h1>2. Frontend Deployment</h1>
+<p>Please download the latest version of the frontend installation package to
the server deployment directory, download address: [Download] (<a
href="https://dolphinscheduler.apache.org/en-us/docs/release/download.html";>https://dolphinscheduler.apache.org/en-us/docs/release/download.html</a>)
(Take 1.2.0 version as an example ), Upload the tar.gz package to this
directory after downloading and unzip it.</p>
+<pre><code class="language-shell">cd /opt/dolphinscheduler;
+
+tar -zxvf
apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-front-bin.tar.gz -C
/opt/dolphinscheduler;
+
+mv apache-dolphinscheduler-incubating-1.2.0-dolphinscheduler-front-bin
dolphinscheduler-ui
+</code></pre>
+<p><strong>Choose any one of the following methods, automated deployment is
recommended.</strong></p>
+<h3>2.1 Automated Deployment</h3>
+<ul>
+<li>
+<p>Enter the dolphinscheduler-ui directory and execute (<code>Note: Automated
deployment will automatically download nginx</code>)</p>
+<pre><code class="language-shell">cd dolphinscheduler-ui;
+sh ./install-dolphinscheduler-ui.sh;
+</code></pre>
+<ul>
+<li>After the execution, please type the frontend port during operation, the
default port is 8888, if you choose the default, type 'y', or type another
port.</li>
+<li>Then it will let you type the api-server ip that interacts with the
frontend UI.</li>
+<li>Next is the port of the api-server that lets you type to interact with the
frontend UI.</li>
+<li>Next is the operating system selection.</li>
+<li>Wait for deployment to complete.</li>
+</ul>
+</li>
+<li>
+<p>After deployment, in order to prevent too large resources from uploading to
the resource center, it is recommended to modify the nginx upload size
parameters, as follows:</p>
+<ul>
+<li>Add Nginx configuration client_max_body_size 1024m, you can add it in the
http method body.</li>
+</ul>
+<pre><code class="language-shell">vi /etc/nginx/nginx.conf
+<span class="hljs-meta">
+#</span><span class="bash"> add param</span>
+client_max_body_size 1024m;
+</code></pre>
+<ul>
+<li>Then restart Nginx service</li>
+</ul>
+<pre><code class="language-shell">systemctl restart nginx
+</code></pre>
+</li>
+<li>
+<p>Visit the front page address: <a
href="http://localhost:8888";>http://localhost:8888</a>. If the front login page
appears, the front web installation is complete.</p>
+<pre><code class="language-html"><span class="hljs-tag"><<span
class="hljs-name">p</span> <span class="hljs-attr">align</span>=<span
class="hljs-string">"center"</span>></span>
+ <span class="hljs-tag"><<span class="hljs-name">img</span> <span
class="hljs-attr">src</span>=<span class="hljs-string">"/img/login.png"</span>
<span class="hljs-attr">width</span>=<span class="hljs-string">"60%"</span>
/></span>
+ <span class="hljs-tag"></<span class="hljs-name">p</span>></span>
+</code></pre>
+</li>
+</ul>
+<h3>2.2 Manual Deployment</h3>
+<ul>
+<li>
+<p>Install nginx by yourself, download it from the official website: Or
<code>yum install nginx -y</code></p>
+</li>
+<li>
+<p>Modify the nginx configuration file (Note: some place need to be modified
by yourself)</p>
+</li>
+</ul>
+<pre><code class="language-html">vi /etc/nginx/nginx.conf
+
+server {
+ listen 8888; # Your Port
+ server_name localhost;
+ #charset koi8-r;
+ #access_log /var/log/nginx/host.access.log main;
+ location / {
+ root /opt/soft/dolphinscheduler-ui/dist; # Your dist directory
which you unzip
+ index index.html index.html;
+ }
+ location /dolphinscheduler {
+ proxy_pass http://localhost:12345; # Your ApiApplicationServer
address
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header x_real_ipP $remote_addr;
+ proxy_set_header remote_addr $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_http_version 1.1;
+ proxy_connect_timeout 4s;
+ proxy_read_timeout 30s;
+ proxy_send_timeout 12s;
+ proxy_set_header Upgrade $http_upgrade;
+ proxy_set_header Connection "upgrade";
+ }
+ #error_page 404 /404.html;
+ # redirect server error pages to the static page /50x.html
+ #
+ error_page 500 502 503 504 /50x.html;
+ location = /50x.html {
+ root /usr/share/nginx/html;
+ }
+}
+</code></pre>
+<ul>
+<li>
+<p>Then restart Nginx service</p>
+<pre><code class="language-shell">systemctl restart nginx
+</code></pre>
+</li>
+<li>
+<p>Visit the front page address: <a
href="http://localhost:8888";>http://localhost:8888</a>. If the front login page
appears, the front web installation is complete.</p>
+<pre><code class="language-html"><span class="hljs-tag"><<span
class="hljs-name">p</span> <span class="hljs-attr">align</span>=<span
class="hljs-string">"center"</span>></span>
+ <span class="hljs-tag"><<span class="hljs-name">img</span> <span
class="hljs-attr">src</span>=<span class="hljs-string">"/img/login.png"</span>
<span class="hljs-attr">width</span>=<span class="hljs-string">"60%"</span>
/></span>
+ <span class="hljs-tag"></<span class="hljs-name">p</span>></span>
+</code></pre>
+</li>
+</ul>
+<h1>3. Start and stop service</h1>
+<ul>
+<li>
+<p>Stop all services</p>
+<p><code>sh ./bin/stop-all.sh</code></p>
+</li>
+<li>
+<p>Start all services</p>
+<p><code>sh ./bin/start-all.sh</code></p>
+</li>
+<li>
+<p>Start and stop master service</p>
+</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
master-server
+sh ./bin/dolphinscheduler-daemon.sh stop master-server
+</code></pre>
+<ul>
+<li>Start and stop worker Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
worker-server
+sh ./bin/dolphinscheduler-daemon.sh stop worker-server
+</code></pre>
+<ul>
+<li>Start and stop api Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
api-server
+sh ./bin/dolphinscheduler-daemon.sh stop api-server
+</code></pre>
+<ul>
+<li>Start and stop logger Service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
logger-server
+sh ./bin/dolphinscheduler-daemon.sh stop logger-server
+</code></pre>
+<ul>
+<li>Start and stop alert service</li>
+</ul>
+<pre><code class="language-shell">sh ./bin/dolphinscheduler-daemon.sh start
alert-server
+sh ./bin/dolphinscheduler-daemon.sh stop alert-server
+</code></pre>
+<p><code>Note: Please refer to the "Architecture Design" section for
service usage</code></p>
+</div></section><footer class="footer-container"><div class="footer-body"><img
src="/img/ds_gray.svg"/><div class="cols-container"><div class="col
col-12"><h3>Disclaimer</h3><p>Apache DolphinScheduler (incubating) is an effort
undergoing incubation at The Apache Software Foundation (ASF), sponsored by
Incubator.
+Incubation is required of all newly accepted projects until a further review
indicates
+that the infrastructure, communications, and decision making process have
stabilized in a manner consistent with other successful ASF projects.
+While incubation status is not necessarily a reflection of the completeness or
stability of the code,
+it does indicate that the project has yet to be fully endorsed by the
ASF.</p></div><div class="col col-6"><dl><dt>Documentation</dt><dd><a
href="/en-us/docs/1.2.0/user_doc/architecture-design.html"
target="_self">Overview</a></dd><dd><a
href="/en-us/docs/1.2.0/user_doc/quick-start.html" target="_self">Quick
start</a></dd><dd><a href="/en-us/docs/1.2.0/user_doc/backend-development.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href=" [...]
+ <script
src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js";></script>
+ <script
src="https://f.alicdn.com/react/15.4.1/react-dom.min.js";></script>
+ <script>
+ window.rootPath = '';
+ </script>
+ <script src="/build/documentation.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/en-us/docs/1.2.0/user_doc/standalone-deployment.json
b/en-us/docs/1.2.0/user_doc/standalone-deployment.json
new file mode 100644
index 0000000..e6fde99
--- /dev/null
+++ b/en-us/docs/1.2.0/user_doc/standalone-deployment.json
@@ -0,0 +1,6 @@
+{
+ "filename": "standalone-deployment.md",
+ "__html": "<h1>Standalone Deployment</h1>\n<p>DolphinScheduler Standalone
deployment is divided into two parts: backend deployment and frontend
deployment.</p>\n<h1>1. Backend Deployment</h1>\n<h3>1.1: Before you begin
(please install requirement basic software by
yourself)</h3>\n<ul>\n<li>PostgreSQL (8.2.15+) or Mysql (5.6 or 5.7): Choose
One</li>\n<li><a
href=\"https://www.oracle.com/technetwork/java/javase/downloads/index.html\";>JDK</a>
(1.8+): Required. Double-check configure JAVA [...]
+ "link": "/en-us/docs/1.2.0/user_doc/standalone-deployment.html",
+ "meta": {}
+}
\ No newline at end of file
