This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 88921dd Automated deployment: 9e361dfd9825d12631032bb29a4ce1ee8efe36a1
88921dd is described below
commit 88921ddce01aeeacc2b789b4ba2853d66c71f88a
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Apr 1 14:36:24 2022 +0000
Automated deployment: 9e361dfd9825d12631032bb29a4ce1ee8efe36a1
---
en-us/docs/dev/user_doc/guide/start/docker.html | 917 ++++++++++++++++++++++++
en-us/docs/dev/user_doc/guide/start/docker.json | 6 +
zh-cn/docs/dev/user_doc/guide/start/docker.html | 17 +-
zh-cn/docs/dev/user_doc/guide/start/docker.json | 2 +-
4 files changed, 932 insertions(+), 10 deletions(-)
diff --git a/en-us/docs/dev/user_doc/guide/start/docker.html
b/en-us/docs/dev/user_doc/guide/start/docker.html
new file mode 100644
index 0000000..b2f7ee4
--- /dev/null
+++ b/en-us/docs/dev/user_doc/guide/start/docker.html
@@ -0,0 +1,917 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=1.0, user-scalable=no">
+ <meta name="keywords" content="docker">
+ <meta name="description" content="docker">
+ <title>docker</title>
+ <link rel="shortcut icon" href="/img/favicon.ico">
+ <link rel="stylesheet" href="/build/vendor.23870e5.css">
+</head>
+<body>
+ <div id="root"><div class="md2html docs-page" data-reactroot=""><header
class="header-container header-container-dark"><div class="header-body"><span
class="mobile-menu-btn mobile-menu-btn-dark"></span><a
href="/en-us/index.html"><img class="logo" src="/img/hlogo_white.svg"/></a><div
class="search search-dark"><span class="icon-search"></span></div><span
class="language-switch language-switch-dark">中</span><div
class="header-menu"><div><ul class="ant-menu whiteClass ant-menu-light ant-
[...]
+<h2>Pre-conditions</h2>
+<ul>
+<li><a href="https://docs.docker.com/engine/install/";>Docker</a> 1.13.1+</li>
+<li><a href="https://docs.docker.com/compose/";>Docker Compose</a> 1.11.0+</li>
+</ul>
+<h2>How to use docker image?</h2>
+<p>There are 3 ways to quickly try DolphinScheduler.</p>
+<h3>I. Start DolphinScheduler as docker-compose (recommended)</h3>
+<p>This method requires the installation of <a
href="https://docs.docker.com/compose/";>docker-compose</a>. The installation of
docker-compose is widely available online, so please install it yourself.</p>
+<p>For Windows 7-10 versions, you can install <a
href="https://github.com/docker/toolbox/releases";>Docker Toolbox</a>. For
Windows 10 64-bit, you can install <a
href="https://docs.docker.com/docker-for-windows/install/";>Docker Desktop</a>
and note the [system requirements](<a
href="https://docs.docker.com/";>https://docs.docker.com/</a>
docker-for-windows/install/#system-requirements).</p>
+<h4>0. Please allocate at least 4GB of memory</h4>
+<p>For Mac users, click on <code>Docker Desktop -> Preferences ->
Resources -> Memory</code>.</p>
+<p>For Windows Docker Toolbox users, there are two items that need to be
configured.</p>
+<ul>
+<li><strong>Memory</strong>: Open Oracle VirtualBox Manager and if you double
click on Docker Quickstart Terminal and run Docker Toolbox successfully, you
will see a virtual machine named <code>default</code>. Click on <code>Settings
-> System -> Motherboard -> Memory Size</code>.</li>
+<li><strong>Port Forwarding</strong>: Click <code>Settings -> Network ->
Advanced -> Port Forwarding -> Add</code>. <code>Name</code>, fill in
<code>12345</code> for both <code>Host Port</code> and <code>Subsystem
Port</code>, leave out the <code>Host IP</code> and <code>Subsystem
IP</code>.</li>
+</ul>
+<p>For Windows Docker Desktop users.</p>
+<ul>
+<li><strong>Hyper-V Mode</strong>: Click <code>Docker Desktop -> Settings
-> Resources -> Memory</code></li>
+<li><strong>WSL 2 Mode</strong>: Reference <a
href="https://docs.microsoft.com/zh-cn/windows/wsl/wsl-config#configure-global-options-with-wslconfig";>WSL
2 utility VM</a></li>
+</ul>
+<h4>1.Download the source code package</h4>
+<p>Please download the source package apache-dolphinscheduler-x.x.x-src.tar.gz
from: <a href="/en-us/download/download.html">download</a></p>
+<h4>2.Pull the image and start the service</h4>
+<blockquote>
+<p>For Mac and Linux user, open <strong>Terminal</strong>
+For Windows Docker Toolbox user, open <strong>Docker Quickstart
Terminal</strong>
+For Windows Docker Desktop user, open <strong>Windows PowerShell</strong></p>
+</blockquote>
+<pre><code>$ tar -zxvf apache-dolphinscheduler-1.3.8-src.tar.gz
+$ cd apache-dolphinscheduler-1.3.8-src/docker/docker-swarm
+$ docker pull dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+$ docker tag apache/dolphinscheduler:1.3.8 apache/dolphinscheduler:latest
+$ docker-compose up -d
+</code></pre>
+<blockquote>
+<p>PowerShell should use <code>cd
apache-dolphinscheduler-1.3.8-src\docker\docker-swarm</code></p>
+</blockquote>
+<p><strong>PostgreSQL</strong> (user <code>root</code>, password
<code>root</code>, database <code>dolphinscheduler</code>) and
<strong>ZooKeeper</strong> services will be started by default</p>
+<h4>3.Login system</h4>
+<p>To access the front-end page: <a
href="http://localhost:12345/dolphinscheduler";>http://localhost:12345/dolphinscheduler</a>,
please change to the corresponding IP address if necessary</p>
+<p>The default user is <code>admin</code> and the default password is
<code>dolphinscheduler123</code>.</p>
+<p><img src="/img/new_ui/dev/quick-start/login.png" alt="login"></p>
+<p>Please refer to the user manual chapter [Quick Start] (...
/start/quick-start.md) to see how to use DolphinScheduler.</p>
+<h3>II. By specifying the existing PostgreSQL and ZooKeeper services</h3>
+<p>This method requires the installation of <a
href="https://docs.docker.com/engine/install/";>docker</a>. The installation of
docker is well documented on the web, so please install it yourself.</p>
+<h4>1.Basic software installation (please install it yourself)</h4>
+<ul>
+<li><a href="https://www.postgresql.org/download/";>PostgreSQL</a>
(8.2.15+)</li>
+<li><a href="https://zookeeper.apache.org/releases.html";>ZooKeeper</a>
(3.4.6+)</li>
+<li><a href="https://docs.docker.com/engine/install/";>Docker</a> (1.13.1+)</li>
+</ul>
+<h4>2. Please login to the PostgreSQL database and create a database named
<code>dolphinscheduler</code>.</h4>
+<h4>3. Initialize the database and import
<code>sql/dolphinscheduler_postgre.sql</code> to create tables and import the
base data</h4>
+<h4>4. Download the DolphinScheduler image</h4>
+<p>We have uploaded the DolphinScheduler images for users to the docker
repository. Instead of building the image locally, users can pull the image
from the docker repository by running the following command.</p>
+<pre><code>docker pull
dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+</code></pre>
+<h4>5. Run a DolphinScheduler instance</h4>
+<pre><code>$ docker run -d --name dolphinscheduler \
+-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e
DATABASE_DATABASE="dolphinscheduler" \
+-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
+-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
+-p 12345:12345 \
+apache/dolphinscheduler:1.3.8 all
+</code></pre>
+<p>Note: The database user test and password test need to be replaced with the
actual PostgreSQL user and password. 192.168.x.x needs to be replaced with the
host IP of PostgreSQL and ZooKeeper.</p>
+<h4>6. Login system</h4>
+<p>As above.</p>
+<h3>III. Running a standalone service in DolphinScheduler</h3>
+<p>When the container is started, the following services are automatically
started.</p>
+<pre><code> MasterServer ----- master service
+ WorkerServer ----- worker service
+ ApiApplicationServer ----- api service
+ AlertServer ----- alert service
+</code></pre>
+<p>If you just want to run some of the services in dolphinscheduler. You can
run some of the services in dolphinscheduler by executing the following
command.</p>
+<ul>
+<li>Start a <strong>master server</strong>, as follows:</li>
+</ul>
+<pre><code>$ docker run -d --name dolphinscheduler-master \
+-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e
DATABASE_DATABASE="dolphinscheduler" \
+-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
+-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
+apache/dolphinscheduler:1.3.8 master-server
+</code></pre>
+<ul>
+<li>Start a <strong>worker server</strong>, as follows:</li>
+</ul>
+<pre><code>$ docker run -d --name dolphinscheduler-worker \
+-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e
DATABASE_DATABASE="dolphinscheduler" \
+-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
+-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
+apache/dolphinscheduler:1.3.8 worker-server
+</code></pre>
+<ul>
+<li>Start a <strong>api server</strong>, as follows:</li>
+</ul>
+<pre><code>$ docker run -d --name dolphinscheduler-api \
+-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e
DATABASE_DATABASE="dolphinscheduler" \
+-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
+-e ZOOKEEPER_QUORUM="192.168.x.x:2181" \
+-p 12345:12345 \
+apache/dolphinscheduler:1.3.8 api-server
+</code></pre>
+<ul>
+<li>Start a <strong>alter server</strong>, as follows:</li>
+</ul>
+<pre><code>$ docker run -d --name dolphinscheduler-alert \
+-e DATABASE_HOST="192.168.x.x" -e DATABASE_PORT="5432" -e
DATABASE_DATABASE="dolphinscheduler" \
+-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
+apache/dolphinscheduler:1.3.8 alert-server
+</code></pre>
+<p><strong>NOTE</strong>: When you run some of the services in
dolphinscheduler, you must specify these environment variables
<code>DATABASE_HOST</code>, <code>DATABASE_PORT</code>,
<code>DATABASE_DATABASE</code>, <code>DATABASE_USERNAME</code>, <code>DATABASE_
PASSWORD</code>, <code>ZOOKEEPER_QUORUM</code>.</p>
+<h2>Environment variables</h2>
+<p>Docker containers are configured via environment variables,
appendix-environment-variables lists the configurable environment variables for
DolphinScheduler and their default values <! --
markdown-link-check-disable-line --></p>
+<p>In particular, in Docker Compose and Docker Swarm, this can be configured
via the environment variable configuration file <code>config.env.sh</code>.</p>
+<h2>Support Matrix</h2>
+<table>
+<thead>
+<tr>
+<th>Type</th>
+<th>Support</th>
+<th>Notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>Shell</td>
+<td>Yes</td>
+<td>Yes</td>
+</tr>
+<tr>
+<td>Python2</td>
+<td>Yes</td>
+<td>Yes</td>
+</tr>
+<tr>
+<td>Python3</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Hadoop2</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Hadoop3</td>
+<td>Not yet determined</td>
+<td>Not yet tested</td>
+</tr>
+<tr>
+<td>Spark-Local(client)</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Spark-YARN(cluster)</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Spark-Standalone(cluster)</td>
+<td>not yet</td>
+<td></td>
+</tr>
+<tr>
+<td>Spark-Kubernetes(cluster)</td>
+<td>not yet</td>
+<td></td>
+</tr>
+<tr>
+<td>Flink-Local(local>=1.11)</td>
+<td>Not yet</td>
+<td>Generic CLI mode is not yet supported</td>
+</tr>
+<tr>
+<td>Flink-YARN(yarn-cluster)</td>
+<td>indirectly supported</td>
+<td>see FAQ</td>
+</tr>
+<tr>
+<td>Flink-YARN(yarn-session/yarn-per-job/yarn-application>=1.11)</td>
+<td>Not yet</td>
+<td>Generic CLI mode is not yet supported</td>
+</tr>
+<tr>
+<td>Flink-Standalone(default)</td>
+<td>not yet</td>
+<td></td>
+</tr>
+<tr>
+<td>Flink-Standalone(remote>=1.11)</td>
+<td>Not yet</td>
+<td>Generic CLI mode is not yet supported</td>
+</tr>
+<tr>
+<td>Flink-Kubernetes(default)</td>
+<td>not yet</td>
+<td></td>
+</tr>
+<tr>
+<td>Flink-Kubernetes(remote>=1.11)</td>
+<td>Not yet</td>
+<td>Generic CLI mode is not yet supported</td>
+</tr>
+<tr>
+<td>Flink-NativeKubernetes(kubernetes-session/application>=1.11)</td>
+<td>Not yet</td>
+<td>Generic CLI mode is not yet supported</td>
+</tr>
+<tr>
+<td>MapReduce</td>
+<td>Indirectly supported</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Kerberos</td>
+<td>Indirectly supported</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>HTTP</td>
+<td>Yes</td>
+<td>Yes</td>
+</tr>
+<tr>
+<td>DataX</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>Sqoop</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-MySQL</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-PostgreSQL</td>
+<td>Yes</td>
+<td>Yes</td>
+</tr>
+<tr>
+<td>SQL-Hive</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-Spark</td>
+<td>Indirect support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-ClickHouse</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-Oracle</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-SQLServer</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+<tr>
+<td>SQL-DB2</td>
+<td>Indirect Support</td>
+<td>See FAQ</td>
+</tr>
+</tbody>
+</table>
+<h2>FAQ</h2>
+<h3>How to manage DolphinScheduler via docker-compose?</h3>
+<p>Start, restart, stop or list all containers:</p>
+<pre><code>docker-compose start
+docker-compose restart
+docker-compose stop
+docker-compose ps
+</code></pre>
+<p>Stop all containers and remove all containers, networks:</p>
+<pre><code>docker-compose down
+</code></pre>
+<p>Stop all containers and remove all containers, networks and storage
volumes:</p>
+<pre><code>docker-compose down -v
+</code></pre>
+<h3>How do I check the logs of a container?</h3>
+<p>Lists all running containers:</p>
+<pre><code>docker ps
+docker ps --format "{{.Names}}" # 只打印名字
+</code></pre>
+<p>View the logs of the container named
docker-swarm_dolphinscheduler-api_1:</p>
+<pre><code>docker logs docker-swarm_dolphinscheduler-api_1
+docker logs -f docker-swarm_dolphinscheduler-api_1 # 跟随日志输出
+docker logs --tail 10 docker-swarm_dolphinscheduler-api_1 # 显示倒数10行日志
+</code></pre>
+<h3>How to scale master and worker with docker-compose?</h3>
+<p>Scaling master to 2 instances:</p>
+<pre><code>docker-compose up -d --scale dolphinscheduler-master=2
dolphinscheduler-master
+</code></pre>
+<p>Scaling worker to 3 instances:</p>
+<pre><code>docker-compose up -d --scale dolphinscheduler-worker=3
dolphinscheduler-worker
+</code></pre>
+<h3>How to deploy DolphinScheduler on Docker Swarm?</h3>
+<p>Assuming the Docker Swarm cluster has been deployed (see <a
href="https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/";>create-swarm</a>
if a Docker Swarm cluster has not been created yet).</p>
+<p>Start a stack called dolphinscheduler:</p>
+<pre><code>docker stack deploy -c docker-stack.yml dolphinscheduler
+</code></pre>
+<p>List all services of the stack named dolphinscheduler:</p>
+<pre><code>docker stack services dolphinscheduler
+</code></pre>
+<p>Stop and remove the stack named dolphinscheduler:</p>
+<pre><code>docker stack rm dolphinscheduler
+</code></pre>
+<p>Remove all storage volumes from the stack named dolphinscheduler:</p>
+<pre><code>docker volume rm -f $(docker volume ls --format
"{{.Name}}" | grep -e "^dolphinscheduler")
+</code></pre>
+<h3>How to scale up and down master and worker on Docker Swarm?</h3>
+<p>Scaling up the master of a stack named dolphinscheduler to 2 instances:</p>
+<pre><code>docker service scale dolphinscheduler_dolphinscheduler-master=2
+</code></pre>
+<p>Scaling up the workers of the stack named dolphinscheduler to 3
instances:</p>
+<pre><code>docker service scale dolphinscheduler_dolphinscheduler-worker=3
+</code></pre>
+<h3>How to build a Docker image?</h3>
+<h4>Build from source (requires Maven 3.3+ & JDK 1.8+)</h4>
+<p>In a Unix-like system, execute in Terminal:</p>
+<pre><code class="language-bash">$ bash ./docker/build/hooks/build
+</code></pre>
+<p>In a Windows system, execute in cmd or PowerShell:</p>
+<pre><code class="language-bat"><span class="hljs-function">C:\<span
class="hljs-title">dolphinscheduler</span>-<span
class="hljs-title">src</span>>.\<span class="hljs-title">docker</span>\<span
class="hljs-title">build</span>\<span class="hljs-title">hooks</span>\<span
class="hljs-title">build.bat</span>
+</span></code></pre>
+<p>If you don't understand <code>. /docker/build/hooks/build</code> <code>.
/docker/build/hooks/build.bat</code> these scripts, please read the
contents.</p>
+<h4>Build from binary packages (Maven 3.3+ & JDK 1.8+ not required)</h4>
+<p>Please download the binary package apache-dolphinscheduler-1.3.8-bin.tar.gz
from: <a href="/zh-cn/download/download.html">download</a>. Then put
apache-dolphinscheduler-1.3.8-bin.tar.gz into the
<code>apache-dolphinscheduler-1.3.8-src/docker/build</code> directory and
execute it in Terminal or PowerShell:</p>
+<pre><code>$ cd apache-dolphinscheduler-1.3.8-src/docker/build
+$ docker build --build-arg VERSION=1.3.8 -t apache/dolphinscheduler:1.3.8 .
+</code></pre>
+<blockquote>
+<p>PowerShell should use <code>cd
apache-dolphinscheduler-1.3.8-src/docker/build</code></p>
+</blockquote>
+<h4>Building images for multi-platform architectures</h4>
+<p>Currently supports building images for <code>linux/amd64</code> and
<code>linux/arm64</code> platform architectures, requiring</p>
+<ol>
+<li>support for <a
href="https://docs.docker.com/engine/reference/commandline/buildx/";>docker
buildx</a></li>
+<li>have push permissions for <a
href="https://hub.docker.com/r/apache/dolphinscheduler";>https://hub.docker.com/r/apache/dolphinscheduler</a>
(<strong>Be careful</strong>: the build command automatically pushes
multi-platform architecture images to the apache/dolphinscheduler docker hub by
default)</li>
+</ol>
+<p>Execute :</p>
+<pre><code class="language-bash">$ docker login <span class="hljs-comment">#
login, use to push apache/dolphinscheduler</span>
+$ bash ./docker/build/hooks/build x
+</code></pre>
+<h3>How to add an environment variable to Docker?</h3>
+<p>If you want to add additional actions and environment variables at compile
time or run time, you can do so in the <code>/root/start-init-conf.sh</code>
file, and if you need to change the configuration file, please change the
corresponding configuration file in
<code>/opt/dolphinscheduler/conf/*.tpl</code>. configuration file</p>
+<p>For example, add an environment variable
<code>SECURITY_AUTHENTICATION_TYPE</code> to
<code>/root/start-init-conf.sh</code>.</p>
+<pre><code>export SECURITY_AUTHENTICATION_TYPE=PASSWORD
+</code></pre>
+<p>After adding the above environment variables, you should add this
environment variable configuration to the corresponding template file
<code>application-api.properties.tpl</code>:</p>
+<pre><code>security.authentication.type=${SECURITY_AUTHENTICATION_TYPE}
+</code></pre>
+<p><code>/root/start-init-conf.sh</code> will dynamically generate a
configuration file based on the template file.</p>
+<pre><code class="language-sh"><span class="hljs-built_in">echo</span> <span
class="hljs-string">"generate dolphinscheduler config"</span>
+ls <span class="hljs-variable">${DOLPHINSCHEDULER_HOME}</span>/conf/ | grep
<span class="hljs-string">".tpl"</span> | <span
class="hljs-keyword">while</span> <span class="hljs-built_in">read</span> line;
<span class="hljs-keyword">do</span>
+<span class="hljs-built_in">eval</span> <span class="hljs-string">"cat
<< EOF
+<span class="hljs-subst">$(cat ${DOLPHINSCHEDULER_HOME}/conf/${line})</span>
+EOF
+"</span> > <span
class="hljs-variable">${DOLPHINSCHEDULER_HOME}</span>/conf/<span
class="hljs-variable">${line%.*}</span>
+<span class="hljs-keyword">done</span>
+</code></pre>
+<h3>How to replace PostgreSQL with MySQL as the database for
DolphinScheduler?</h3>
+<blockquote>
+<p>Due to commercial licensing, we cannot use MySQL driver packages
directly.</p>
+<p>If you want to use MySQL, you can build it based on the official image
<code>apache/dolphinscheduler</code>.</p>
+</blockquote>
+<ol>
+<li>
+<p>Download the MySQL driver package <a
href="https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar";>mysql-connector-java-8.0.16.jar</a></p>
+</li>
+<li>
+<p>Create a new <code>Dockerfile</code> to add the MySQL driver package:</p>
+</li>
+</ol>
+<pre><code>FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
+</code></pre>
+<ol start="3">
+<li>Build a new image containing the MySQL driver package:</li>
+</ol>
+<pre><code>docker build -t apache/dolphinscheduler:mysql-driver
+</code></pre>
+<ol start="4">
+<li>Modify all the image fields in the <code>docker-compose.yml</code> file to
<code>apache/dolphinscheduler:mysql-driver</code>.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you need to modify
<code>docker-stack.yml</code>.</p>
+</blockquote>
+<ol start="5">
+<li>
+<p>Comment out the <code>dolphinscheduler-postgresql</code> block in the
<code>docker-compose.yml</code> file.</p>
+</li>
+<li>
+<p>Add the <code>dolphinscheduler-mysql</code> service to the
<code>docker-compose.yml</code> file (<strong>optional</strong>, you can use an
external MySQL database directly).</p>
+</li>
+<li>
+<p>Modify the DATABASE environment variable in the <code>config.env.sh</code>
file.</p>
+</li>
+</ol>
+<pre><code>DATABASE_TYPE=mysql
+DATABASE_DRIVER=com.mysql.jdbc.Driver
+DATABASE_HOST=dolphinscheduler-mysql
+DATABASE_PORT=3306
+DATABASE_USERNAME=root
+DATABASE_PASSWORD=root
+DATABASE_DATABASE=dolphinscheduler
+DATABASE_PARAMS=useUnicode=true&characterEncoding=UTF-8
+</code></pre>
+<blockquote>
+<p>If you have already added the <code>dolphinscheduler-mysql</code> service,
set <code>DATABASE_HOST</code> to <code>dolphinscheduler-mysql</code>.</p>
+</blockquote>
+<ol start="8">
+<li>Run dolphinscheduler (see <strong>How to use a docker image</strong> for
details).</li>
+</ol>
+<h3>How do I support MySQL data sources in the Data Source Centre?</h3>
+<blockquote>
+<p>Due to commercial licensing, we cannot use MySQL's driver packages
directly.</p>
+<p>If you want to add a MySQL datasource, you can build it based on the
official image <code>apache/dolphinscheduler</code>.</p>
+</blockquote>
+<ol>
+<li>
+<p>Download MySQL driver package <a
href="https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar";>mysql-connector-java-8.0.16.jar</a></p>
+</li>
+<li>
+<p>Create a new <code>Dockerfile</code> to add the MySQL driver package:</p>
+</li>
+</ol>
+<pre><code>FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
+</code></pre>
+<ol start="3">
+<li>Build a new image containing the MySQL driver package:</li>
+</ol>
+<pre><code>docker build -t apache/dolphinscheduler:mysql-driver .
+</code></pre>
+<ol start="4">
+<li>Change all <code>image</code> fields in the
<code>docker-compose.yml</code> file to
<code>apache/dolphinscheduler:mysql-driver</code>.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you will need to
modify <code>docker-stack.yml</code>.</p>
+</blockquote>
+<ol start="5">
+<li>
+<p>Run dolphinscheduler (see <strong>How to use a docker image</strong> for
details).</p>
+</li>
+<li>
+<p>Add a MySQL data source to the data source centre.</p>
+</li>
+</ol>
+<h3>How to support Oracle data sources in the Data Source Centre?</h3>
+<blockquote>
+<p>Due to commercial licensing, we cannot use Oracle's driver packages
directly.</p>
+<p>If you want to add an Oracle datasource, you can build it based on the
official image <code>apache/dolphinscheduler</code>.</p>
+</blockquote>
+<ol>
+<li>
+<p>Download the Oracle driver package <a
href="https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/";>ojdbc8.jar</a>
(such as <code>ojdbc8-19.9.0.0.jar</code>).</p>
+</li>
+<li>
+<p>Create a new <code>Dockerfile</code> to add the Oracle driver package:</p>
+</li>
+</ol>
+<pre><code>FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
+</code></pre>
+<ol start="3">
+<li>Build a new image containing the Oracle driver package:</li>
+</ol>
+<pre><code>docker build -t apache/dolphinscheduler:oracle-driver .
+</code></pre>
+<ol start="4">
+<li>Change all <code>image</code> fields in the
<code>docker-compose.yml</code> file to
<code>apache/dolphinscheduler:oracle-driver</code>.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you will need to
modify <code>docker-stack.yml</code>.</p>
+</blockquote>
+<ol start="5">
+<li>
+<p>Run dolphinscheduler (see <strong>How to use a docker image</strong> for
details).</p>
+</li>
+<li>
+<p>Add an Oracle data source to the data source centre.</p>
+</li>
+</ol>
+<h3>How to support Python 2 pip and custom requirements.txt?</h3>
+<ol>
+<li>Create a new <code>Dockerfile</code> for installing pip:</li>
+</ol>
+<pre><code>FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+COPY requirements.txt /tmp
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends python-pip && \
+ pip install --no-cache-dir -r /tmp/requirements.txt && \
+ rm -rf /var/lib/apt/lists/*
+</code></pre>
+<p>This command will install the default <strong>pip 18.1</strong>. If you
want to upgrade pip, just add a line.</p>
+<pre><code> pip install --no-cache-dir -U pip && \
+</code></pre>
+<ol start="2">
+<li>Build a new image containing pip.</li>
+</ol>
+<pre><code>docker build -t apache/dolphinscheduler:pip .
+</code></pre>
+<ol start="3">
+<li>Change all <code>image</code> fields in the
<code>docker-compose.yml</code> file to
<code>apache/dolphinscheduler:pip</code>.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you will need to
modify <code>docker-stack.yml</code>.</p>
+</blockquote>
+<ol start="4">
+<li>
+<p>Run dolphinscheduler (see <strong>How to use docker images</strong> for
details).</p>
+</li>
+<li>
+<p>Verify pip under a new Python task.</p>
+</li>
+</ol>
+<h3>How do I support Python 3?</h3>
+<ol>
+<li>Create a new <code>Dockerfile</code> for installing Python 3:</li>
+</ol>
+<pre><code>FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:1.3.8
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends python3 && \
+ rm -rf /var/lib/apt/lists/*
+</code></pre>
+<p>This command will install the default <strong>Python 3.7.3</strong>. If you
also want to install <strong>pip3</strong>, replace <code>python3</code> with
<code>python3-pip</code> and you're done.</p>
+<pre><code> apt-get install -y --no-install-recommends python3-pip
&& \
+</code></pre>
+<ol start="2">
+<li>Build a new image containing Python 3:</li>
+</ol>
+<pre><code>docker build -t apache/dolphinscheduler:python3 .
+</code></pre>
+<ol start="3">
+<li>Change all <code>image</code> fields in the
<code>docker-compose.yml</code> file to
<code>apache/dolphinscheduler:python3</code>.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you will need to
modify <code>docker-stack.yml</code>.</p>
+</blockquote>
+<ol start="4">
+<li>
+<p>Modify <code>PYTHON_HOME</code> to <code>/usr/bin/python3</code> in the
<code>config.env.sh</code> file.</p>
+</li>
+<li>
+<p>Run dolphinscheduler (see <strong>How to use docker images</strong> for
details).</p>
+</li>
+<li>
+<p>Verify Python 3 under a new Python task.</p>
+</li>
+</ol>
+<h3>How to support Hadoop, Spark, Flink, Hive or DataX?</h3>
+<p>Take Spark 2.4.7 as an example:</p>
+<ol>
+<li>
+<p>Download the Spark 2.4.7 release binary package
<code>spark-2.4.7-bin-hadoop2.7.tgz</code>.</p>
+</li>
+<li>
+<p>Run dolphinscheduler (see <strong>How to use a docker image</strong> for
details).</p>
+</li>
+<li>
+<p>Copy the Spark 2.4.7 binary package to the Docker container.</p>
+</li>
+</ol>
+<pre><code class="language-bash">docker cp spark-2.4.7-bin-hadoop2.7.tgz
docker-swarm_dolphinscheduler-worker_1:/opt/soft
+</code></pre>
+<p>Because the storage volume <code>dolphinscheduler-shared-local</code> is
mounted to <code>/opt/soft</code>, all files in <code>/opt/soft</code> will not
be lost.</p>
+<ol start="4">
+<li>Login to the container and make sure <code>SPARK_HOME2</code> exists.</li>
+</ol>
+<pre><code class="language-bash">docker <span
class="hljs-built_in">exec</span> -it docker-swarm_dolphinscheduler-worker_1
bash
+<span class="hljs-built_in">cd</span> /opt/soft
+tar zxf spark-2.4.7-bin-hadoop2.7.tgz
+rm -f spark-2.4.7-bin-hadoop2.7.tgz
+ln -s spark-2.4.7-bin-hadoop2.7 spark2 <span class="hljs-comment"># or
mv</span>
+<span class="hljs-variable">$SPARK_HOME2</span>/bin/spark-submit --version
+</code></pre>
+<p>If everything executes correctly, the last command will print the Spark
version information.</p>
+<ol start="5">
+<li>Verify Spark in a shell task.</li>
+</ol>
+<pre><code>$SPARK_HOME2/bin/spark-submit --class
org.apache.spark.examples.SparkPi
$SPARK_HOME2/examples/jars/spark-examples_2.11-2.4.7.jar
+</code></pre>
+<p>Check if the task log contains the output <code>Pi is roughly
3.146015</code>.</p>
+<ol start="6">
+<li>Verifying Spark in a Spark Task</li>
+</ol>
+<p>The file <code>spark-examples_2.11-2.4.7.jar</code> needs to be uploaded to
the Resource Center first, then a Spark task created and set up:</p>
+<ul>
+<li>Spark version: <code>SPARK2</code></li>
+<li>Class of main function: <code>org.apache.spark.examples.SparkPi</code></li>
+<li>Main package: <code>spark-examples_2.11-2.4.7.jar</code></li>
+<li>Deployment method: <code>local</code></li>
+</ul>
+<p>Similarly, check if the task log contains output <code>Pi is roughly
3.146015</code></p>
+<ol start="7">
+<li>Verifying Spark on YARN</li>
+</ol>
+<p>Spark on YARN (deployed as a <code>cluster</code> or <code>client</code>)
requires Hadoop support. Similar to Spark support, supporting Hadoop is almost
identical to the previous steps.</p>
+<p>Make sure <code>$HADOOP_HOME</code> and <code>$HADOOP_CONF_DIR</code> are
present.</p>
+<h3>How is Spark 3 supported?</h3>
+<p>In fact, submitting an application using <code>spark-submit</code> is the
same, whether it is Spark 1, 2 or 3. In other words, the semantics of
<code>SPARK_HOME2</code> is a second <code>SPARK_HOME</code>, not the
<code>HOME</code> of <code>SPARK2</code>, so simply setting
<code>SPARK_HOME2=/path/to/ spark3</code>.</p>
+<p>Let's take Spark 3.1.1 as an example:</p>
+<ol>
+<li>
+<p>Download the Spark 3.1.1 release binary package
<code>spark-3.1.1-bin-hadoop2.7.tgz</code>.</p>
+</li>
+<li>
+<p>Run dolphinscheduler (see <strong>How to use a docker image</strong> for
details).</p>
+</li>
+<li>
+<p>Copy the Spark 3.1.1 binary package to the Docker container</p>
+</li>
+</ol>
+<pre><code class="language-bash">docker cp spark-3.1.1-bin-hadoop2.7.tgz
docker-swarm_dolphinscheduler-worker_1:/opt/soft
+</code></pre>
+<ol start="4">
+<li>log in to the container and ensure that <code>SPARK_HOME2</code>
exists</li>
+</ol>
+<pre><code class="language-bash">docker <span
class="hljs-built_in">exec</span> -it docker-swarm_dolphinscheduler-worker_1
bash
+<span class="hljs-built_in">cd</span> /opt/soft
+tar zxf spark-3.1.1-bin-hadoop2.7.tgz
+rm -f spark-3.1.1-bin-hadoop2.7.tgz
+ln -s spark-3.1.1-bin-hadoop2.7 spark2 <span class="hljs-comment"># or
mv</span>
+<span class="hljs-variable">$SPARK_HOME2</span>/bin/spark-submit --version
+</code></pre>
+<p>If everything executes correctly, the last command will print the Spark
version information.</p>
+<ol start="5">
+<li>Verify Spark in a shell task.</li>
+</ol>
+<pre><code>$SPARK_HOME2/bin/spark-submit --class
org.apache.spark.examples.SparkPi
$SPARK_HOME2/examples/jars/spark-examples_2.12-3.1.1.jar
+</code></pre>
+<p>Check if the task log contains the output <code>Pi is roughly
3.146015</code>.</p>
+<h3>How to support shared storage between Master, Worker and Api services?</h3>
+<blockquote>
+<p><strong>Note</strong>: If you are deploying via docker-compose on a single
machine, steps 1 and 2 can be skipped and you can execute commands like `docker
cp hadoop-3.2.2.tar.gz docker-swarm_dolphinscheduler-worker_1:/opt/soft ' Place
Hadoop in the container under the shared directory /opt/soft.</p>
+</blockquote>
+<p>For example, the Master, Worker and Api services may use Hadoop at the same
time.</p>
+<ol>
+<li>Modify the <code>dolphinscheduler-shared-local</code> storage volume in
the <code>docker-compose.yml</code> file to support nfs.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you need to modify
<code>docker-stack.yml</code>.</p>
+</blockquote>
+<pre><code class="language-yaml"><span class="hljs-attr">volumes:</span>
+ <span class="hljs-attr">dolphinscheduler-shared-local:</span>
+ <span class="hljs-attr">driver_opts:</span>
+ <span class="hljs-attr">type:</span> <span
class="hljs-string">"nfs"</span>
+ <span class="hljs-attr">o:</span> <span
class="hljs-string">"addr=10.40.0.199,nolock,soft,rw"</span>
+ <span class="hljs-attr">device:</span> <span
class="hljs-string">":/path/to/shared/dir"</span>
+</code></pre>
+<ol start="2">
+<li>
+<p>Put Hadoop into nfs.</p>
+</li>
+<li>
+<p>Make sure <code>$HADOOP_HOME</code> and <code>$HADOOP_CONF_DIR</code> are
correct.</p>
+</li>
+</ol>
+<h3>How to support local file storage instead of HDFS and S3?</h3>
+<blockquote>
+<p><strong>Note</strong>: If you are deploying on a standalone machine via
docker-compose, you can skip step 2.</p>
+</blockquote>
+<ol>
+<li>modify the following environment variables in the
<code>config.env.sh</code> file:</li>
+</ol>
+<pre><code>RESOURCE_STORAGE_TYPE=HDFS
+FS_DEFAULT_FS=file:///
+</code></pre>
+<ol start="2">
+<li>Modify the <code>dolphinscheduler-resource-local</code> storage volume in
the <code>docker-compose.yml</code> file to support nfs.</li>
+</ol>
+<blockquote>
+<p>If you want to deploy dolphinscheduler on Docker Swarm, you need to modify
<code>docker-stack.yml</code>.</p>
+</blockquote>
+<pre><code class="language-yaml"><span class="hljs-attr">volumes:</span>
+ <span class="hljs-attr">dolphinscheduler-resource-local:</span>
+ <span class="hljs-attr">driver_opts:</span>
+ <span class="hljs-attr">type:</span> <span
class="hljs-string">"nfs"</span>
+ <span class="hljs-attr">o:</span> <span
class="hljs-string">"addr=10.40.0.199,nolock,soft,rw"</span>
+ <span class="hljs-attr">device:</span> <span
class="hljs-string">":/path/to/resource/dir"</span>
+</code></pre>
+<h3>How do I support S3 resource stores such as MinIO?</h3>
+<p>Take MinIO as an example: Modify the following environment variables in the
<code>config.env.sh</code> file.</p>
+<pre><code>RESOURCE_STORAGE_TYPE=S3
+RESOURCE_UPLOAD_PATH=/dolphinscheduler
+FS_DEFAULT_FS=s3a://BUCKET_NAME
+FS_S3A_ENDPOINT=http://MINIO_IP:9000
+FS_S3A_ACCESS_KEY=MINIO_ACCESS_KEY
+FS_S3A_SECRET_KEY=MINIO_SECRET_KEY
+</code></pre>
+<p><code>BUCKET_NAME</code>, <code>MINIO_IP</code>,
<code>MINIO_ACCESS_KEY</code> and <code>MINIO_SECRET_KEY</code> need to be
changed to actual values.</p>
+<blockquote>
+<p><strong>NOTE</strong>: <code>MINIO_IP</code> can only use IPs and not
domain names, as DolphinScheduler does not yet support S3 path style access.</p>
+</blockquote>
+<h3>How to configure SkyWalking?</h3>
+<p>Modify the SKYWALKING environment variables in the
<code>config.env.sh</code> file.</p>
+<pre><code>SKYWALKING_ENABLE=true
+SW_AGENT_COLLECTOR_BACKEND_SERVICES=127.0.0.1:11800
+SW_GRPC_LOG_SERVER_HOST=127.0.0.1
+SW_GRPC_LOG_SERVER_PORT=11800
+</code></pre>
+<h2>Appendix - Environment Variables</h2>
+<h3>Database</h3>
+<p><strong><code>DATABASE_TYPE</code></strong></p>
+<p>Configure the <code>TYPE</code> of the <code>database</code>, default value
<code>postgresql</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running the <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, and <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can build distributed services
better.</p>
+<p><strong><code>DATABASE_DRIVER</code></strong></p>
+<p>Configure <code>DRIVER</code> for <code>database</code>, default value
<code>org.postgresql.Driver</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running the <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, and <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can better build distributed
services.</p>
+<p><strong><code>DATABASE_HOST</code></strong></p>
+<p>Configure the <code>HOST</code> of <code>database</code>, default value
<code>127.0.0.1</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code> services in
<code>dolphinscheduler</code> so that you can build distributed services
better.</p>
+<p><strong><code>DATABASE_PORT</code></strong></p>
+<p>Configure <code>PORT</code> for <code>database</code>, default value
<code>5432</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code> services in
<code>dolphinscheduler</code> so that you can build distributed services
better.</p>
+<p><strong><code>DATABASE_USERNAME</code></strong></p>
+<p>Configure the <code>USERNAME</code> of <code>database</code>, default value
<code>root</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can build distributed services
better.</p>
+<p><strong><code>DATABASE_PASSWORD</code></strong></p>
+<p>Configure <code>PASSWORD</code> for <code>database</code>, default value
<code>root</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can better build distributed
services.</p>
+<p><strong><code>DATABASE_DATABASE</code></strong></p>
+<p>Configure <code>DATABASE</code> for <code>database</code>, default value
<code>dolphinscheduler</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running the <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, and <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can better build distributed
services.</p>
+<p><strong><code>DATABASE_PARAMS</code></strong></p>
+<p>Configure <code>PARAMS</code> for <code>database</code>, default value
<code>characterEncoding=utf8</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code> services in
<code>dolphinscheduler</code>, so that you can build distributed services
better.</p>
+<h3>ZooKeeper</h3>
+<p><strong><code>ZOOKEEPER_QUORUM</code></strong></p>
+<p>Configure the <code>Zookeeper</code> address for
<code>dolphinscheduler</code>, default value <code>127.0.0.1:2181</code>.</p>
+<p><strong>NOTE</strong>: This environment variable must be specified when
running the <code>master-server</code>, <code>worker-server</code>,
<code>api-server</code> services in <code>dolphinscheduler</code>, so that you
can build distributed services better.</p>
+<p><strong><code>ZOOKEEPER_ROOT</code></strong></p>
+<p>Configure <code>dolphinscheduler</code> as the root directory for data
storage in <code>zookeeper</code>, default value
<code>/dolphinscheduler</code>.</p>
+<h3>General</h3>
+<p><strong><code>DOLPHINSCHEDULER_OPTS</code></strong></p>
+<p>Configure <code>jvm options</code> for <code>dolphinscheduler</code>, for
<code>master-server</code>, <code>worker-server</code>,
<code>api-server</code>, <code>alert-server</code>, default
<code>""</code>,</p>
+<p><strong><code>DATA_BASEDIR_PATH</code></strong></p>
+<p>User data directory, user configured, make sure it exists and user
read/write access, default value <code>/tmp/dolphinscheduler</code>.</p>
+<p><strong><code>RESOURCE_STORAGE_TYPE</code></strong></p>
+<p>Configure the resource storage type for <code>dolphinscheduler</code>,
options are <code>HDFS</code>, <code>S3</code>, <code>NONE</code>, default
<code>HDFS</code>.</p>
+<p><strong><code>RESOURCE_UPLOAD_PATH</code></strong></p>
+<p>Configure the resource storage path on <code>HDFS/S3</code>, default value
<code>/dolphinscheduler</code>.</p>
+<p><strong><code>FS_DEFAULT_FS</code></strong></p>
+<p>Configure the file system protocol for the resource store, e.g.
<code>file:///</code>, <code>hdfs://mycluster:8020</code> or
<code>s3a://dolphinscheduler</code>, default value <code>file:///</code>.</p>
+<p><strong><code>FS_S3A_ENDPOINT</code></strong></p>
+<p>When <code>RESOURCE_STORAGE_TYPE=S3</code>, the access path to
<code>S3</code> needs to be configured, default value
<code>s3.xxx.amazonaws.com</code>.</p>
+<p><strong><code>FS_S3A_ACCESS_KEY</code></strong></p>
+<p>When <code>RESOURCE_STORAGE_TYPE=S3</code>, you need to configure the
<code>s3 access key</code> of <code>S3</code>, default value
<code>xxxxxxx</code>.</p>
+<p><strong><code>FS_S3A_SECRET_KEY</code></strong></p>
+<p>When <code>RESOURCE_STORAGE_TYPE=S3</code>, you need to configure <code>s3
secret key</code> for <code>S3</code>, default value <code>xxxxxxx</code>.</p>
+<p><strong><code>HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE</code></strong></p>
+<p>Configure whether <code>dolphinscheduler</code> is kerberos enabled,
default value <code>false</code>.</p>
+<p><strong><code>JAVA_SECURITY_KRB5_CONF_PATH</code></strong></p>
+<p>Configure the path to java.security.krb5.conf for
<code>dolphinscheduler</code>, default value <code>/opt/krb5.conf</code>.</p>
+<p><strong><code>LOGIN_USER_KEYTAB_USERNAME</code></strong></p>
+<p>Configure the keytab username for the <code>dolphinscheduler</code> login
user, default value <code>[email protected]</code>.</p>
+<p><strong><code>LOGIN_USER_KEYTAB_PATH</code></strong></p>
+<p>Configure the keytab path for the <code>dolphinscheduler</code> login user,
default value <code>/opt/hdfs.keytab</code>.</p>
+<p><strong><code>KERBEROS_EXPIRE_TIME</code></strong></p>
+<p>Configure the kerberos expiration time for <code>dolphinscheduler</code>,
in hours, default value <code>2</code>.</p>
+<p><strong><code>HDFS_ROOT_USER</code></strong></p>
+<p>Configure the root user name of hdfs for <code>dolphinscheduler</code> when
<code>RESOURCE_STORAGE_TYPE=HDFS</code>, default value <code>hdfs</code>.</p>
+<p><strong><code>RESOURCE_MANAGER_HTTPADDRESS_PORT</code></strong></p>
+<p>Configure the resource manager httpaddress port for
<code>dolphinscheduler</code>, default value <code>8088</code>.</p>
+<p><strong><code>YARN_RESOURCEMANAGER_HA_RM_IDS</code></strong></p>
+<p>Configure <code>dolphinscheduler</code>'s yarn resourcemanager ha rm ids,
default value <code>null</code>.</p>
+<p><strong><code>YARN_APPLICATION_STATUS_ADDRESS</code></strong></p>
+<p>Configure the yarn application status address for
<code>dolphinscheduler</code>, default value
<code>http://ds1:%s/ws/v1/cluster/apps/%s</code>.</p>
+<p><strong><code>SKYWALKING_ENABLE</code></strong></p>
+<p>Configure whether <code>skywalking</code> is enabled or not. Default value
<code>false</code>.</p>
+<p><strong><code>SW_AGENT_COLLECTOR_BACKEND_SERVICES</code></strong></p>
+<p>Configure the collector back-end address for <code>skywalking</code>.
Default value <code>127.0.0.1:11800</code>.</p>
+<p><strong><code>SW_GRPC_LOG_SERVER_HOST</code></strong></p>
+<p>Configure the grpc service host or IP for <code>skywalking</code>. Default
value <code>127.0.0.1</code>.</p>
+<p><strong><code>SW_GRPC_LOG_SERVER_PORT</code></strong></p>
+<p>Configure the grpc service port for <code>skywalking</code>. Default value
<code>11800</code>.</p>
+<p><strong><code>HADOOP_HOME</code></strong></p>
+<p>Configure <code>HADOOP_HOME</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/hadoop</code>.</p>
+<p><strong><code>HADOOP_CONF_DIR</code></strong></p>
+<p>Configure <code>HADOOP_CONF_DIR</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/hadoop/etc/hadoop</code>.</p>
+<p><strong><code>SPARK_HOME1</code></strong></p>
+<p>Configure <code>SPARK_HOME1</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/spark1</code>.</p>
+<p><strong><code>SPARK_HOME2</code></strong></p>
+<p>Configure <code>SPARK_HOME2</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/spark2</code>.</p>
+<p><strong><code>PYTHON_HOME</code></strong></p>
+<p>Configure <code>PYTHON_HOME</code> for <code>dolphinscheduler</code>,
default value <code>/usr/bin/python</code>.</p>
+<p><strong><code>JAVA_HOME</code></strong></p>
+<p>Configure <code>JAVA_HOME</code> for <code>dolphinscheduler</code>, default
value <code>/usr/local/openjdk-8</code>.</p>
+<p><strong><code>HIVE_HOME</code></strong></p>
+<p>Configure <code>HIVE_HOME</code> for <code>dolphinscheduler</code>, default
value <code>/opt/soft/hive</code>.</p>
+<p><strong><code>FLINK_HOME</code></strong></p>
+<p>Configure <code>FLINK_HOME</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/flink</code>.</p>
+<p><strong><code>DATAX_HOME</code></strong></p>
+<p>Configure <code>DATAX_HOME</code> for <code>dolphinscheduler</code>,
default value <code>/opt/soft/datax</code>.</p>
+<h3>Master Server</h3>
+<p><strong><code>MASTER_SERVER_OPTS</code></strong></p>
+<p>Configure <code>jvm options</code> for <code>master-server</code>, default
value <code>-Xms1g -Xmx1g -Xmn512m</code>.</p>
+<p><strong><code>MASTER_EXEC_THREADS</code></strong></p>
+<p>Configure the number of threads to be executed in
<code>master-server</code>, default value <code>100</code>.</p>
+<p><strong><code>MASTER_EXEC_TASK_NUM</code></strong></p>
+<p>Configure the number of tasks to be executed in <code>master-server</code>,
default value <code>20</code>.</p>
+<p><strong><code>MASTER_DISPATCH_TASK_NUM</code></strong></p>
+<p>Configure the number of tasks to be dispatched in
<code>master-server</code>, default value <code>3</code>.</p>
+<p><strong><code>MASTER_HOST_SELECTOR</code></strong></p>
+<p>Configure the selector for the worker host when dispatching tasks in
<code>master-server</code>, optional values are <code>Random</code>,
<code>RoundRobin</code> and <code>LowerWeight</code>, default value
<code>LowerWeight</code>.</p>
+<p><strong>`MASTER_HEARTBEAT_INTERVAL'</strong></p>
+<p>Configure the heartbeat interaction time in <code>master-server</code>,
default value <code>10</code>.</p>
+<p><strong><code>MASTER_TASK_COMMIT_RETRYTIMES</code></strong></p>
+<p>Configure the number of task commit retries in <code>master-server</code>,
default value <code>5</code>.</p>
+<p><strong><code>MASTER_TASK_COMMIT_INTERVAL</code></strong></p>
+<p>Configure the task commit interaction time in <code>master-server</code>,
default value <code>1</code>.</p>
+<p><strong><code>MASTER_MAX_CPULOAD_AVG</code></strong></p>
+<p>Configure the <code>load average</code> value in the CPU in
<code>master-server</code>, default value <code>-1</code>.</p>
+<p><strong><code>MASTER_RESERVED_MEMORY</code></strong></p>
+<p>Configure the reserved memory in G for <code>master-server</code>, default
value <code>0.3</code>.</p>
+<h3>Worker Server</h3>
+<p><strong><code>WORKER_SERVER_OPTS</code></strong></p>
+<p>Configure <code>jvm options</code> for <code>worker-server</code>, default
value <code>-Xms1g -Xmx1g -Xmn512m</code>.</p>
+<p><strong><code>WORKER_EXEC_THREADS</code></strong></p>
+<p>Configure the number of threads to be executed in
<code>worker-server</code>, default value <code>100</code>.</p>
+<p><strong><code>WORKER_HEARTBEAT_INTERVAL</code></strong></p>
+<p>Configure the heartbeat interaction time in <code>worker-server</code>,
default value <code>10</code>.</p>
+<p><strong><code>WORKER_MAX_CPULOAD_AVG</code></strong></p>
+<p>Configure the maximum <code>load average</code> value in the CPU in
<code>worker-server</code>, default value <code>-1</code>.</p>
+<p><strong><code>WORKER_RESERVED_MEMORY</code></strong></p>
+<p>Configure the reserved memory in G for <code>worker-server</code>, default
value <code>0.3</code>.</p>
+<p><strong><code>WORKER_GROUPS</code></strong></p>
+<p>Configure the grouping of <code>worker-server</code>, default value
<code>default</code>.</p>
+<h3>Alert Server</h3>
+<p><strong><code>ALERT_SERVER_OPTS</code></strong></p>
+<p>Configure <code>jvm options</code> for <code>alert-server</code>, default
value <code>-Xms512m -Xmx512m -Xmn256m</code>.</p>
+<p><strong><code>XLS_FILE_PATH</code></strong></p>
+<p>Configure the path to store <code>XLS</code> files for the
<code>alert-server</code>, default value <code>/tmp/xls</code>.</p>
+<p><strong><code>MAIL_SERVER_HOST</code></strong></p>
+<p>Configure the mail service address for <code>alert-server</code>, default
value <code>empty</code>.</p>
+<p><strong><code>MAIL_SERVER_PORT</code></strong></p>
+<p>Configure the mail service port for <code>alert-server</code>, default
value <code>empty</code>.</p>
+<p><strong><code>MAIL_SENDER</code></strong></p>
+<p>Configure the mail sender for <code>alert-server</code>, default value
<code>empty</code>.</p>
+<p><strong><code>MAIL_USER=</code></strong></p>
+<p>Configure the user name of the mail service for <code>alert-server</code>,
default value <code>empty</code>.</p>
+<p><strong><code>MAIL_PASSWD</code></strong></p>
+<p>Configure the mail service user password for <code>alert-server</code>,
default value <code>empty</code>.</p>
+<p><strong><code>MAIL_SMTP_STARTTLS_ENABLE</code></strong></p>
+<p>Configure whether TLS is enabled for <code>alert-server</code>'s mail
service, default value <code>true</code>.</p>
+<p><strong><code>MAIL_SMTP_SSL_ENABLE</code></strong></p>
+<p>Configure whether the mail service of <code>alert-server</code> is SSL
enabled or not, default value <code>false</code>.</p>
+<p><strong><code>MAIL_SMTP_SSL_TRUST</code></strong></p>
+<p>Configure the trusted address for SSL for <code>alert-server</code>'s mail
service, default value <code>null</code>.</p>
+<p><strong><code>ENTERPRISE_WECHAT_ENABLE</code></strong></p>
+<p>Configure whether the mail service of <code>alert-server</code> has
Enterprise Wechat enabled, default value <code>false</code>.</p>
+<p><strong><code>ENTERPRISE_WECHAT_CORP_ID</code></strong></p>
+<p>Configures the Enterprise Wechat <code>ID</code> of the mail service for
<code>alert-server</code>, default value <code>null</code>.</p>
+<p><strong><code>ENTERPRISE_WECHAT_SECRET</code></strong></p>
+<p>Configure the mail service enterprise wechat <code>SECRET</code> for
<code>alert-server</code>, default value <code>Empty</code>.</p>
+<p><strong><code>ENTERPRISE_WECHAT_AGENT_ID</code></strong></p>
+<p>Configure <code>AGENT_ID</code> of the mail service enterprise wechat for
<code>alert-server</code>, default value <code>Empty</code>.</p>
+<p><strong><code>ENTERPRISE_WECHAT_USERS</code></strong></p>
+<p>Configure <code>USERS</code> for the mail service enterprise microsoft for
<code>alert-server</code>, default value <code>empty</code>.</p>
+<h3>Api Server</h3>
+<p><strong><code>API_SERVER_OPTS</code></strong></p>
+<p>Configure <code>jvm options</code> for <code>api-server</code>, default
value <code>-Xms512m -Xmx512m -Xmn256m</code>.</p>
+</div></section><footer class="footer-container"><div
class="footer-body"><div><h3>About us</h3><h4>Do you need feedback? Please
contact us through the following ways.</h4></div><div
class="contact-container"><ul><li><a
href="/en-us/community/development/subscribe.html"><img class="img-base"
src="/img/emailgray.png"/><img class="img-change"
src="/img/emailblue.png"/><p>Email List</p></a></li><li><a
href="https://twitter.com/dolphinschedule";><img class="img-base"
src="/img/twittergray.png [...]
+ <script
src="//cdn.jsdelivr.net/npm/[email protected]/dist/react-with-addons.min.js"></script>
+ <script
src="//cdn.jsdelivr.net/npm/[email protected]/dist/react-dom.min.js"></script>
+ <script>window.rootPath = '';</script>
+ <script src="/build/vendor.90dcf97.js"></script>
+ <script src="/build/docs.md.8182f21.js"></script>
+ <script>
+ var _hmt = _hmt || [];
+ (function() {
+ var hm = document.createElement("script");
+ hm.src = "https://hm.baidu.com/hm.js?4e7b4b400dd31fa015018a435c64d06f";;
+ var s = document.getElementsByTagName("script")[0];
+ s.parentNode.insertBefore(hm, s);
+ })();
+ </script>
+ <!-- Global site tag (gtag.js) - Google Analytics -->
+ <script async
src="https://www.googletagmanager.com/gtag/js?id=G-899J8PYKJZ";></script>
+ <script>
+ window.dataLayer = window.dataLayer || [];
+ function gtag(){dataLayer.push(arguments);}
+ gtag('js', new Date());
+
+ gtag('config', 'G-899J8PYKJZ');
+ </script>
+</body>
+</html>
\ No newline at end of file
diff --git a/en-us/docs/dev/user_doc/guide/start/docker.json
b/en-us/docs/dev/user_doc/guide/start/docker.json
new file mode 100644
index 0000000..552f1f6
--- /dev/null
+++ b/en-us/docs/dev/user_doc/guide/start/docker.json
@@ -0,0 +1,6 @@
+{
+ "filename": "docker.md",
+ "__html": "<h1>Quick Trial Docker
Deployment</h1>\n<h2>Pre-conditions</h2>\n<ul>\n<li><a
href=\"https://docs.docker.com/engine/install/\";>Docker</a>
1.13.1+</li>\n<li><a href=\"https://docs.docker.com/compose/\";>Docker
Compose</a> 1.11.0+</li>\n</ul>\n<h2>How to use docker image?</h2>\n<p>There
are 3 ways to quickly try DolphinScheduler.</p>\n<h3>I. Start DolphinScheduler
as docker-compose (recommended)</h3>\n<p>This method requires the installation
of <a href=\"https://docs.docker.com [...]
+ "link": "/dist/en-us/docs/dev/user_doc/guide/start/docker.html",
+ "meta": {}
+}
\ No newline at end of file
diff --git a/zh-cn/docs/dev/user_doc/guide/start/docker.html
b/zh-cn/docs/dev/user_doc/guide/start/docker.html
index ca6ae64..ebb5d7b 100644
--- a/zh-cn/docs/dev/user_doc/guide/start/docker.html
+++ b/zh-cn/docs/dev/user_doc/guide/start/docker.html
@@ -83,13 +83,12 @@ apache/dolphinscheduler:1.3.8 all
<p>同上</p>
<h3>三、运行 DolphinScheduler 中的独立服务</h3>
<p>在容器启动时，会自动启动以下服务：</p>
-<pre><code> MasterServer ----- master服务
- WorkerServer ----- worker服务
- ApiApplicationServer ----- api服务
- AlertServer ----- alert服务
+<pre><code> MasterServer ----- master 服务
+ WorkerServer ----- worker 服务
+ ApiApplicationServer ----- api 服务
+ AlertServer ----- alert 服务
</code></pre>
-<p>如果你只是想运行 dolphinscheduler 中的部分服务</p>
-<p>你可以够通执行以下命令来运行dolphinscheduler中的部分服务</p>
+<p>如果你只是想运行 dolphinscheduler 中的部分服务,你可以够通执行以下命令来运行 dolphinscheduler 中的部分服务</p>
<ul>
<li>启动一个 <strong>master server</strong>, 如下:</li>
</ul>
@@ -126,7 +125,7 @@ apache/dolphinscheduler:1.3.8 api-server
-e DATABASE_USERNAME="test" -e DATABASE_PASSWORD="test" \
apache/dolphinscheduler:1.3.8 alert-server
</code></pre>
-<p><strong>注意</strong>: 当你运行dolphinscheduler中的部分服务时，你必须指定这些环境变量
<code>DATABASE_HOST</code>, <code>DATABASE_PORT</code>,
<code>DATABASE_DATABASE</code>, <code>DATABASE_USERNAME</code>,
<code>DATABASE_PASSWORD</code>, <code>ZOOKEEPER_QUORUM</code>。</p>
+<p><strong>注意</strong>: 当你运行 dolphinscheduler 中的部分服务时，你必须指定这些环境变量
<code>DATABASE_HOST</code>, <code>DATABASE_PORT</code>,
<code>DATABASE_DATABASE</code>, <code>DATABASE_USERNAME</code>,
<code>DATABASE_PASSWORD</code>, <code>ZOOKEEPER_QUORUM</code>。</p>
<h2>环境变量</h2>
<p>Docker 容器通过环境变量进行配置，<a href="#appendix-environment-variables">附录-环境变量</a>
列出了 DolphinScheduler 的可配置环境变量及其默认值 <!-- markdown-link-check-disable-line --></p>
<p>特别地，在 Docker Compose 和 Docker Swarm 中，可以通过环境变量配置文件
<code>config.env.sh</code> 进行配置</p>
@@ -379,7 +378,7 @@ $ bash ./docker/build/hooks/build x
<p>当添加以上环境变量后，你应该在相应的模板文件<code>application-api.properties.tpl</code>中添加这个环境变量配置:</p>
<pre><code>security.authentication.type=${SECURITY_AUTHENTICATION_TYPE}
</code></pre>
-<p><code>/root/start-init-conf.sh</code>将根据模板文件动态的生成配置文件：</p>
+<p><code>/root/start-init-conf.sh</code> 将根据模板文件动态的生成配置文件：</p>
<pre><code class="language-sh"><span class="hljs-built_in">echo</span> <span
class="hljs-string">"generate dolphinscheduler config"</span>
ls <span class="hljs-variable">${DOLPHINSCHEDULER_HOME}</span>/conf/ | grep
<span class="hljs-string">".tpl"</span> | <span
class="hljs-keyword">while</span> <span class="hljs-built_in">read</span> line;
<span class="hljs-keyword">do</span>
<span class="hljs-built_in">eval</span> <span class="hljs-string">"cat
<< EOF
@@ -617,7 +616,7 @@ ln -s spark-2.4.7-bin-hadoop2.7 spark2 <span
class="hljs-comment"># 或者 mv</s
<p>文件 <code>spark-examples_2.11-2.4.7.jar</code> 需要先被上传到资源中心，然后创建一个 Spark
任务并设置:</p>
<ul>
<li>Spark版本: <code>SPARK2</code></li>
-<li>主函数的Class: <code>org.apache.spark.examples.SparkPi</code></li>
+<li>主函数的 Class: <code>org.apache.spark.examples.SparkPi</code></li>
<li>主程序包: <code>spark-examples_2.11-2.4.7.jar</code></li>
<li>部署方式: <code>local</code></li>
</ul>
diff --git a/zh-cn/docs/dev/user_doc/guide/start/docker.json
b/zh-cn/docs/dev/user_doc/guide/start/docker.json
index d19b3f1..a7ccfeb 100644
--- a/zh-cn/docs/dev/user_doc/guide/start/docker.json
+++ b/zh-cn/docs/dev/user_doc/guide/start/docker.json
@@ -1,6 +1,6 @@
{
"filename": "docker.md",
- "__html": "<h1>快速试用 Docker 部署</h1>\n<h2>先决条件</h2>\n<ul>\n<li><a
href=\"https://docs.docker.com/engine/install/\";>Docker</a>
1.13.1+</li>\n<li><a href=\"https://docs.docker.com/compose/\";>Docker
Compose</a> 1.11.0+</li>\n</ul>\n<h2>如何使用 Docker 镜像</h2>\n<p>有 3 种方式可以快速试用
DolphinScheduler</p>\n<h3>一、以 docker-compose 的方式启动 DolphinScheduler
(推荐)</h3>\n<p>这种方式需要先安装 <a
href=\"https://docs.docker.com/compose/\";>docker-compose</a>, docker-compose
的安装网上已经有非常多的资料，请自行安装即可</p>\n<p>对于 Windows 7-10，你可 [...]
+ "__html": "<h1>快速试用 Docker 部署</h1>\n<h2>先决条件</h2>\n<ul>\n<li><a
href=\"https://docs.docker.com/engine/install/\";>Docker</a>
1.13.1+</li>\n<li><a href=\"https://docs.docker.com/compose/\";>Docker
Compose</a> 1.11.0+</li>\n</ul>\n<h2>如何使用 Docker 镜像</h2>\n<p>有 3 种方式可以快速试用
DolphinScheduler</p>\n<h3>一、以 docker-compose 的方式启动 DolphinScheduler
(推荐)</h3>\n<p>这种方式需要先安装 <a
href=\"https://docs.docker.com/compose/\";>docker-compose</a>, docker-compose
的安装网上已经有非常多的资料，请自行安装即可</p>\n<p>对于 Windows 7-10，你可 [...]
"link": "/dist/zh-cn/docs/dev/user_doc/guide/start/docker.html",
"meta": {}
}
\ No newline at end of file
