This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository
https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 069096c Automated deployment: 0e5ca02dda846e45453657829a6ff3b5feef189a
069096c is described below
commit 069096ce54b13fba35415f10ed26993195f2e758
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Sun Jan 17 10:44:21 2021 +0000
Automated deployment: 0e5ca02dda846e45453657829a6ff3b5feef189a
---
en-us/docs/1.3.4/user_doc/configuration-file.html | 996 ++++++++++++++++++++++
en-us/docs/1.3.4/user_doc/configuration-file.json | 6 +
2 files changed, 1002 insertions(+)
diff --git a/en-us/docs/1.3.4/user_doc/configuration-file.html
b/en-us/docs/1.3.4/user_doc/configuration-file.html
new file mode 100644
index 0000000..3079636
--- /dev/null
+++ b/en-us/docs/1.3.4/user_doc/configuration-file.html
@@ -0,0 +1,996 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0,
maximum-scale=1.0, user-scalable=no">
+ <meta name="keywords" content="configuration-file" />
+ <meta name="description" content="configuration-file" />
+ <title>configuration-file</title>
+ <link rel="shortcut icon" href="/img/favicon.ico"/>
+ <link rel="stylesheet" href="/build/documentation.md.css" />
+</head>
+<body>
+ <div id="root"><div class="md2html documentation-page"
data-reactroot=""><header class="header-container header-container-normal"><div
class="header-body"><a href="/en-us/index.html"><img class="logo"
src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch
language-switch-normal">中</span><div class="header-menu"><img
class="header-menu-toggle" src="/img/system/menu_gray.png"/><div><ul
class="ant-menu black [...]
+<p>This document explains the DolphinScheduler application configurations
according to DolphinScheduler-1.3.x versions.</p>
+<h1>Directory Structure</h1>
+<p>Currently, all the configuration files are under [conf ] directory. Please
check the following simplified DolphinScheduler installation directories to
have a direct view about the position [conf] directory in and configuration
files inside. This document only describes DolphinScheduler configurations and
other modules are not going into.</p>
+<p>[Note: the DolphinScheduler (hereinafter called the ‘DS’) .]</p>
+<pre><code>
+├─bin DS application commands directory
+│ ├─dolphinscheduler-daemon.sh startup/shutdown DS application
+│ ├─start-all.sh startup all DS services with
configurations
+│ ├─stop-all.sh shutdown all DS services with
configurations
+├─conf configurations directory
+│ ├─application-api.properties API-service config properties
+│ ├─datasource.properties datasource config properties
+│ ├─zookeeper.properties zookeeper config properties
+│ ├─master.properties master config properties
+│ ├─worker.properties worker config properties
+│ ├─quartz.properties quartz config properties
+│ ├─common.properties common-service[storage] config
properties
+│ ├─alert.properties alert-service config properties
+│ ├─config environment variables config directory
+│ ├─install_config.conf DS environment variables
configuration script[install/start DS]
+│ ├─env load environment variables configs
script directory
+│ ├─dolphinscheduler_env.sh load environment variables configs
[eg: JAVA_HOME,HADOOP_HOME, HIVE_HOME ...]
+│ ├─org mybatis mapper files directory
+│ ├─i18n i18n configs directory
+│ ├─logback-api.xml API-service log config
+│ ├─logback-master.xml master-service log config
+│ ├─logback-worker.xml worker-service log config
+│ ├─logback-alert.xml alert-service log config
+├─sql DS metadata to create/upgrade .sql
directory
+│ ├─create create SQL scripts directory
+│ ├─upgrade upgrade SQL scripts directory
+│ ├─dolphinscheduler-postgre.sql postgre database init script
+│ ├─dolphinscheduler_mysql.sql mysql database init script
+│ ├─soft_version current DS version-id file
+├─script DS services deployment, database
create/upgrade scripts directory
+│ ├─create-dolphinscheduler.sh DS database init script
+│ ├─upgrade-dolphinscheduler.sh DS database upgrade script
+│ ├─monitor-server.sh DS monitor-server start script
+│ ├─scp-hosts.sh transfer installation files script
+│ ├─remove-zk-node.sh cleanup zookeeper caches script
+├─ui front-end web resources directory
+├─lib DS .jar dependencies directory
+├─install.sh auto-setup DS services script
+
+
+</code></pre>
+<h1>Configurations in Details</h1>
+<table>
+<thead>
+<tr>
+<th>serial number</th>
+<th>service classification</th>
+<th>config file</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>startup/shutdown DS application</td>
+<td><a
href="http://dolphinscheduler-daemon.sh";>dolphinscheduler-daemon.sh</a></td>
+</tr>
+<tr>
+<td>2</td>
+<td>datasource config properties</td>
+<td>datasource.properties</td>
+</tr>
+<tr>
+<td>3</td>
+<td>zookeeper config properties</td>
+<td>zookeeper.properties</td>
+</tr>
+<tr>
+<td>4</td>
+<td>common-service[storage] config properties</td>
+<td>common.properties</td>
+</tr>
+<tr>
+<td>5</td>
+<td>API-service config properties</td>
+<td>application-api.properties</td>
+</tr>
+<tr>
+<td>6</td>
+<td>master config properties</td>
+<td>master.properties</td>
+</tr>
+<tr>
+<td>7</td>
+<td>worker config properties</td>
+<td>worker.properties</td>
+</tr>
+<tr>
+<td>8</td>
+<td>alert-service config properties</td>
+<td>alert.properties</td>
+</tr>
+<tr>
+<td>9</td>
+<td>quartz config properties</td>
+<td>quartz.properties</td>
+</tr>
+<tr>
+<td>10</td>
+<td>DS environment variables configuration script[install/start DS]</td>
+<td>install_config.conf</td>
+</tr>
+<tr>
+<td>11</td>
+<td>load environment variables configs <br /> [eg: JAVA_HOME,HADOOP_HOME,
HIVE_HOME ...]</td>
+<td>dolphinscheduler_env.sh</td>
+</tr>
+<tr>
+<td>12</td>
+<td>services log config files</td>
+<td>API-service log config : logback-api.xml <br /> master-service log config
: logback-master.xml <br /> worker-service log config : logback-worker.xml
<br /> alert-service log config : logback-alert.xml</td>
+</tr>
+</tbody>
+</table>
+<h2><a
href="http://1.dolphinscheduler-daemon.sh";>1.dolphinscheduler-daemon.sh</a>
[startup/shutdown DS application]</h2>
+<p><a href="http://dolphinscheduler-daemon.sh";>dolphinscheduler-daemon.sh</a>
is responsible for DS startup & shutdown.
+Essentially, <a
href="http://start-all.sh/stop-all.sh";>start-all.sh/stop-all.sh</a>
startup/shutdown the cluster via <a
href="http://dolphinscheduler-daemon.sh";>dolphinscheduler-daemon.sh</a>.
+Currently, DS just makes a basic config, please config further JVＭ options
based on your practical situation of resources.</p>
+<p>Default simplified parameters are:</p>
+<pre><code class="language-bash"><span class="hljs-built_in">export</span>
DOLPHINSCHEDULER_OPTS=<span class="hljs-string">"
+-server
+-Xmx16g
+-Xms1g
+-Xss512k
+-XX:+UseConcMarkSweepGC
+-XX:+CMSParallelRemarkEnabled
+-XX:+UseFastAccessorMethods
+-XX:+UseCMSInitiatingOccupancyOnly
+-XX:CMSInitiatingOccupancyFraction=70
+"</span>
+</code></pre>
+<blockquote>
+<p>"-XX:DisableExplicitGC" is not recommended due to may lead to
memory link (DS dependent on Netty to communicate).</p>
+</blockquote>
+<h2>2.datasource.properties [datasource config properties]</h2>
+<p>DS uses Druid to manage database connections and default simplified configs
are:</p>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>spring.datasource.driver-class-name</td>
+<td></td>
+<td>datasource driver</td>
+</tr>
+<tr>
+<td>spring.datasource.url</td>
+<td></td>
+<td>datasource connection url</td>
+</tr>
+<tr>
+<td>spring.datasource.username</td>
+<td></td>
+<td>datasource username</td>
+</tr>
+<tr>
+<td>spring.datasource.password</td>
+<td></td>
+<td>datasource password</td>
+</tr>
+<tr>
+<td>spring.datasource.initialSize</td>
+<td>5</td>
+<td>initail connection pool size number</td>
+</tr>
+<tr>
+<td>spring.datasource.minIdle</td>
+<td>5</td>
+<td>minimum connection pool size number</td>
+</tr>
+<tr>
+<td>spring.datasource.maxActive</td>
+<td>5</td>
+<td>maximum connection pool size number</td>
+</tr>
+<tr>
+<td>spring.datasource.maxWait</td>
+<td>60000</td>
+<td>max wait mili-seconds</td>
+</tr>
+<tr>
+<td>spring.datasource.timeBetweenEvictionRunsMillis</td>
+<td>60000</td>
+<td>idle connection check interval</td>
+</tr>
+<tr>
+<td>spring.datasource.timeBetweenConnectErrorMillis</td>
+<td>60000</td>
+<td>retry interval</td>
+</tr>
+<tr>
+<td>spring.datasource.minEvictableIdleTimeMillis</td>
+<td>300000</td>
+<td>connections over minEvictableIdleTimeMillis will be collect when idle
check</td>
+</tr>
+<tr>
+<td>spring.datasource.validationQuery</td>
+<td>SELECT 1</td>
+<td>validate connection by running the SQL</td>
+</tr>
+<tr>
+<td>spring.datasource.validationQueryTimeout</td>
+<td>3</td>
+<td>validate connection timeout[seconds]</td>
+</tr>
+<tr>
+<td>spring.datasource.testWhileIdle</td>
+<td>true</td>
+<td>set whether the pool validates the allocated connection when a new
connection request comes</td>
+</tr>
+<tr>
+<td>spring.datasource.testOnBorrow</td>
+<td>true</td>
+<td>validity check when the program requests a new connection</td>
+</tr>
+<tr>
+<td>spring.datasource.testOnReturn</td>
+<td>false</td>
+<td>validity check when the program recalls a connection</td>
+</tr>
+<tr>
+<td>spring.datasource.defaultAutoCommit</td>
+<td>true</td>
+<td>whether auto commit</td>
+</tr>
+<tr>
+<td>spring.datasource.keepAlive</td>
+<td>true</td>
+<td>runs validationQuery SQL to avoid the connection closed by pool when the
connection idles over minEvictableIdleTimeMillis</td>
+</tr>
+<tr>
+<td>spring.datasource.poolPreparedStatements</td>
+<td>true</td>
+<td>Open PSCache</td>
+</tr>
+<tr>
+<td>spring.datasource.maxPoolPreparedStatementPerConnectionSize</td>
+<td>20</td>
+<td>specify the size of PSCache on each connection</td>
+</tr>
+</tbody>
+</table>
+<h2>3.zookeeper.properties [zookeeper config properties]</h2>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>zookeeper.quorum</td>
+<td>localhost:2181</td>
+<td>zookeeper cluster connection info</td>
+</tr>
+<tr>
+<td>zookeeper.dolphinscheduler.root</td>
+<td>/dolphinscheduler</td>
+<td>DS is stored under zookeeper root directory</td>
+</tr>
+<tr>
+<td>zookeeper.session.timeout</td>
+<td>60000</td>
+<td>session timeout</td>
+</tr>
+<tr>
+<td>zookeeper.connection.timeout</td>
+<td>30000</td>
+<td>connection timeout</td>
+</tr>
+<tr>
+<td>zookeeper.retry.base.sleep</td>
+<td>100</td>
+<td>time to wait between subsequent retries</td>
+</tr>
+<tr>
+<td>zookeeper.retry.max.sleep</td>
+<td>30000</td>
+<td>maximum time to wait between subsequent retries</td>
+</tr>
+<tr>
+<td>zookeeper.retry.maxtime</td>
+<td>10</td>
+<td>maximum retry times</td>
+</tr>
+</tbody>
+</table>
+<h2>4.common.properties [hadoop、s3、yarn config properties]</h2>
+<p>Currently, common.properties mainly configures hadoop/s3a related
configurations.</p>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>resource.storage.type</td>
+<td>NONE</td>
+<td>type of resource files: HDFS, S3, NONE</td>
+</tr>
+<tr>
+<td>resource.upload.path</td>
+<td>/dolphinscheduler</td>
+<td>storage path of resource files</td>
+</tr>
+<tr>
+<td>data.basedir.path</td>
+<td>/tmp/dolphinscheduler</td>
+<td>local directory used to store temp files</td>
+</tr>
+<tr>
+<td>hadoop.security.authentication.startup.state</td>
+<td>false</td>
+<td>whether hadoop grant kerberos permission</td>
+</tr>
+<tr>
+<td>java.security.krb5.conf.path</td>
+<td>/opt/krb5.conf</td>
+<td>kerberos config directory</td>
+</tr>
+<tr>
+<td>login.user.keytab.username</td>
+<td><a href="mailto:[email protected]";>[email protected]</a></td>
+<td>kerberos username</td>
+</tr>
+<tr>
+<td>login.user.keytab.path</td>
+<td>/opt/hdfs.headless.keytab</td>
+<td>kerberos user keytab</td>
+</tr>
+<tr>
+<td>resource.view.suffixs</td>
+<td>txt,log,sh,conf,cfg,py,java,sql,hql,xml,properties</td>
+<td>file types supported by resource center</td>
+</tr>
+<tr>
+<td>hdfs.root.user</td>
+<td>hdfs</td>
+<td>configure users with corresponding permissions if storage type is HDFS</td>
+</tr>
+<tr>
+<td>fs.defaultFS</td>
+<td>hdfs://mycluster:8020</td>
+<td>If resource.storage.type=S3, then the request url would be similar to
's3a://dolphinscheduler'. Otherwise if resource.storage.type=HDFS and hadoop
supports HA, please copy core-site.xml and hdfs-site.xml into 'conf'
directory.</td>
+</tr>
+<tr>
+<td>fs.s3a.endpoint</td>
+<td></td>
+<td>s3 endpoint url</td>
+</tr>
+<tr>
+<td>fs.s3a.access.key</td>
+<td></td>
+<td>s3 access key</td>
+</tr>
+<tr>
+<td>fs.s3a.secret.key</td>
+<td></td>
+<td>s3 secret key</td>
+</tr>
+<tr>
+<td>yarn.resourcemanager.ha.rm.ids</td>
+<td></td>
+<td>specify the yarn resourcemanager url. if resourcemanager supports HA,
input HA IP addresses (separated by comma), or input null for standalone</td>
+</tr>
+<tr>
+<td>yarn.application.status.address</td>
+<td><a
href="http://ds1:8088/ws/v1/cluster/apps/%25s";>http://ds1:8088/ws/v1/cluster/apps/%s</a></td>
+<td>keep default if resourcemanager supports HA or not use resourcemanager. Or
replace ds1 with corresponding hostname if resourcemanager in standalone
mode.</td>
+</tr>
+<tr>
+<td>dolphinscheduler.env.path</td>
+<td>env/dolphinscheduler_env.sh</td>
+<td>load environment variables configs [eg: JAVA_HOME,HADOOP_HOME, HIVE_HOME
...]</td>
+</tr>
+<tr>
+<td>development.state</td>
+<td>false</td>
+<td>specify whether in development state</td>
+</tr>
+<tr>
+<td>kerberos.expire.time</td>
+<td>7</td>
+<td>kerberos expire time [hour]</td>
+</tr>
+</tbody>
+</table>
+<h2>5.application-api.properties [API-service log config]</h2>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>server.port</td>
+<td>12345</td>
+<td>api service communication port</td>
+</tr>
+<tr>
+<td>server.servlet.session.timeout</td>
+<td>7200</td>
+<td>session timeout</td>
+</tr>
+<tr>
+<td>server.servlet.context-path</td>
+<td>/dolphinscheduler</td>
+<td>request path</td>
+</tr>
+<tr>
+<td>spring.servlet.multipart.max-file-size</td>
+<td>1024MB</td>
+<td>maximum file size</td>
+</tr>
+<tr>
+<td>spring.servlet.multipart.max-request-size</td>
+<td>1024MB</td>
+<td>maximum request size</td>
+</tr>
+<tr>
+<td>server.jetty.max-http-post-size</td>
+<td>5000000</td>
+<td>jetty maximum post size</td>
+</tr>
+<tr>
+<td>spring.messages.encoding</td>
+<td>UTF-8</td>
+<td>message encoding</td>
+</tr>
+<tr>
+<td>spring.jackson.time-zone</td>
+<td>GMT+8</td>
+<td>time zone</td>
+</tr>
+<tr>
+<td>spring.messages.basename</td>
+<td>i18n/messages</td>
+<td>i18n config</td>
+</tr>
+<tr>
+<td>security.authentication.type</td>
+<td>PASSWORD</td>
+<td>authentication type</td>
+</tr>
+</tbody>
+</table>
+<h2>6.master.properties [master-service log config]</h2>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>master.listen.port</td>
+<td>5678</td>
+<td>master communication port</td>
+</tr>
+<tr>
+<td>master.exec.threads</td>
+<td>100</td>
+<td>work threads count</td>
+</tr>
+<tr>
+<td>master.exec.task.num</td>
+<td>20</td>
+<td>parallel task count</td>
+</tr>
+<tr>
+<td>master.dispatch.task.num</td>
+<td>3</td>
+<td>dispatch task count</td>
+</tr>
+<tr>
+<td>master.heartbeat.interval</td>
+<td>10</td>
+<td>heartbeat interval</td>
+</tr>
+<tr>
+<td>master.task.commit.retryTimes</td>
+<td>5</td>
+<td>task retry times</td>
+</tr>
+<tr>
+<td>master.task.commit.interval</td>
+<td>1000</td>
+<td>task commit interval</td>
+</tr>
+<tr>
+<td>master.max.cpuload.avg</td>
+<td>-1</td>
+<td>master service operates when cpu load less than this number. (default -1:
cpu cores * 2)</td>
+</tr>
+<tr>
+<td>master.reserved.memory</td>
+<td>0.3</td>
+<td>specify memory threshold value, master service operates when available
memory greater than the threshold</td>
+</tr>
+</tbody>
+</table>
+<h2>7.worker.properties [worker-service log config]</h2>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>worker.listen.port</td>
+<td>1234</td>
+<td>worker communication port</td>
+</tr>
+<tr>
+<td>worker.exec.threads</td>
+<td>100</td>
+<td>work threads count</td>
+</tr>
+<tr>
+<td>worker.heartbeat.interval</td>
+<td>10</td>
+<td>heartbeat interval</td>
+</tr>
+<tr>
+<td>worker.max.cpuload.avg</td>
+<td>-1</td>
+<td>worker service operates when CPU load less than this number. (default -1:
CPU cores * 2)</td>
+</tr>
+<tr>
+<td>worker.reserved.memory</td>
+<td>0.3</td>
+<td>specify memory threshold value, worker service operates when available
memory greater than threshold</td>
+</tr>
+<tr>
+<td>worker.group</td>
+<td>default</td>
+<td>workgroup grouping config. <br> worker will join corresponding group
according to this config when startup</td>
+</tr>
+</tbody>
+</table>
+<h2>8.alert.properties [alert-service log config]</h2>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>alert.type</td>
+<td>EMAIL</td>
+<td>alter type</td>
+</tr>
+<tr>
+<td>mail.protocol</td>
+<td>SMTP</td>
+<td>mail server protocol</td>
+</tr>
+<tr>
+<td>mail.server.host</td>
+<td><a href="http://xxx.xxx.com";>xxx.xxx.com</a></td>
+<td>mail server host</td>
+</tr>
+<tr>
+<td>mail.server.port</td>
+<td>25</td>
+<td>mail server port</td>
+</tr>
+<tr>
+<td>mail.sender</td>
+<td><a href="mailto:[email protected]";>[email protected]</a></td>
+<td>mail sender email</td>
+</tr>
+<tr>
+<td>mail.user</td>
+<td><a href="mailto:[email protected]";>[email protected]</a></td>
+<td>mail sender email name</td>
+</tr>
+<tr>
+<td>mail.passwd</td>
+<td>111111</td>
+<td>mail sender email password</td>
+</tr>
+<tr>
+<td>mail.smtp.starttls.enable</td>
+<td>true</td>
+<td>specify mail whether open tls</td>
+</tr>
+<tr>
+<td>mail.smtp.ssl.enable</td>
+<td>false</td>
+<td>specify mail whether open ssl</td>
+</tr>
+<tr>
+<td>mail.smtp.ssl.trust</td>
+<td><a href="http://xxx.xxx.com";>xxx.xxx.com</a></td>
+<td>specify mail ssl trust list</td>
+</tr>
+<tr>
+<td>xls.file.path</td>
+<td>/tmp/xls</td>
+<td>mail attachment temp storage directory</td>
+</tr>
+<tr>
+<td></td>
+<td>following configure WeCom[optional]</td>
+<td></td>
+</tr>
+<tr>
+<td>enterprise.wechat.enable</td>
+<td>false</td>
+<td>specify whether enable WeCom</td>
+</tr>
+<tr>
+<td><a
href="http://enterprise.wechat.corp.id";>enterprise.wechat.corp.id</a></td>
+<td>xxxxxxx</td>
+<td>WeCom corp id</td>
+</tr>
+<tr>
+<td>enterprise.wechat.secret</td>
+<td>xxxxxxx</td>
+<td>WeCom secret</td>
+</tr>
+<tr>
+<td><a
href="http://enterprise.wechat.agent.id";>enterprise.wechat.agent.id</a></td>
+<td>xxxxxxx</td>
+<td>WeCom agent id</td>
+</tr>
+<tr>
+<td>enterprise.wechat.users</td>
+<td>xxxxxxx</td>
+<td>WeCom users</td>
+</tr>
+<tr>
+<td>enterprise.wechat.token.url</td>
+<td><a
href="https://qyapi.weixin.qq.com/cgi-bin/gettoken";>https://qyapi.weixin.qq.com/cgi-bin/gettoken</a>?
<br /> corpid=corpId&corpsecret=secret</td>
+<td>WeCom token url</td>
+</tr>
+<tr>
+<td>enterprise.wechat.push.url</td>
+<td><a
href="https://qyapi.weixin.qq.com/cgi-bin/message/send";>https://qyapi.weixin.qq.com/cgi-bin/message/send</a>?
<br /> access_token=$token</td>
+<td>WeCom push url</td>
+</tr>
+<tr>
+<td>enterprise.wechat.user.send.msg</td>
+<td></td>
+<td>send message format</td>
+</tr>
+<tr>
+<td>enterprise.wechat.team.send.msg</td>
+<td></td>
+<td>group message format</td>
+</tr>
+<tr>
+<td>plugin.dir</td>
+<td>/Users/xx/your/path/to/plugin/dir</td>
+<td>plugin directory</td>
+</tr>
+</tbody>
+</table>
+<h2>9.quartz.properties [quartz config properties]</h2>
+<p>This part describes quartz configs and please configure them based on your
practical situation and resources.</p>
+<table>
+<thead>
+<tr>
+<th>Parameters</th>
+<th>Default value</th>
+<th>Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>org.quartz.jobStore.driverDelegateClass</td>
+<td>org.quartz.impl.jdbcjobstore.StdJDBCDelegate</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.driverDelegateClass</td>
+<td>org.quartz.impl.jdbcjobstore.PostgreSQLDelegate</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.scheduler.instanceName</td>
+<td>DolphinScheduler</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.scheduler.instanceId</td>
+<td>AUTO</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.scheduler.makeSchedulerThreadDaemon</td>
+<td>true</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.useProperties</td>
+<td>false</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.threadPool.class</td>
+<td>org.quartz.simpl.SimpleThreadPool</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.threadPool.makeThreadsDaemons</td>
+<td>true</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.threadPool.threadCount</td>
+<td>25</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.threadPool.threadPriority</td>
+<td>5</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.class</td>
+<td>org.quartz.impl.jdbcjobstore.JobStoreTX</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.tablePrefix</td>
+<td>QRTZ_</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.isClustered</td>
+<td>true</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.misfireThreshold</td>
+<td>60000</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.clusterCheckinInterval</td>
+<td>5000</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.acquireTriggersWithinLock</td>
+<td>true</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.jobStore.dataSource</td>
+<td>myDs</td>
+<td></td>
+</tr>
+<tr>
+<td>org.quartz.dataSource.myDs.connectionProvider.class</td>
+<td>org.apache.dolphinscheduler.service.quartz.DruidConnectionProvider</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<h2>10.install_config.conf [DS environment variables configuration
script[install/start DS]]</h2>
+<p>install_config.conf is a bit complicated and is mainly used in the
following two places.</p>
+<ul>
+<li>1.DS cluster auto installation</li>
+</ul>
+<blockquote>
+<p>System will load configs in the install_config.conf and auto-configure
files below, based on the file content when executing '<a
href="http://install.sh";>install.sh</a>'.
+Files such as <a
href="http://dolphinscheduler-daemon.sh";>dolphinscheduler-daemon.sh</a>、datasource.properties、zookeeper.properties、common.properties、application-api.properties、master.properties、worker.properties、alert.properties、quartz.properties
and etc.</p>
+</blockquote>
+<ul>
+<li>2.Startup/shutdown DS cluster</li>
+</ul>
+<blockquote>
+<p>The system will load masters, workers, alertServer, apiServers and other
parameters inside the file to startup/shutdown DS cluster.</p>
+</blockquote>
+<p>File content as follows:</p>
+<pre><code class="language-bash">
+<span class="hljs-comment"># Note: please escape the character if the file
contains special characters such as `.*[]^${}\+?|()@#&`.</span>
+<span class="hljs-comment"># eg: `[` escape to `\[`</span>
+
+<span class="hljs-comment"># Database type (DS currently only supports
postgresql and mysql)</span>
+dbtype=<span class="hljs-string">"mysql"</span>
+
+<span class="hljs-comment"># Database url & port</span>
+dbhost=<span class="hljs-string">"192.168.xx.xx:3306"</span>
+
+<span class="hljs-comment"># Database name</span>
+dbname=<span class="hljs-string">"dolphinscheduler"</span>
+
+
+<span class="hljs-comment"># Database username</span>
+username=<span class="hljs-string">"xx"</span>
+
+<span class="hljs-comment"># Database password</span>
+password=<span class="hljs-string">"xx"</span>
+
+<span class="hljs-comment"># Zookeeper url</span>
+zkQuorum=<span
class="hljs-string">"192.168.xx.xx:2181,192.168.xx.xx:2181,192.168.xx.xx:2181"</span>
+
+<span class="hljs-comment"># DS installation path, such as
'/data1_1T/dolphinscheduler'</span>
+installPath=<span class="hljs-string">"/data1_1T/dolphinscheduler"</span>
+
+<span class="hljs-comment"># Deployment user</span>
+<span class="hljs-comment"># Note: Deployment user needs 'sudo' privilege and
has rights to operate HDFS</span>
+<span class="hljs-comment"># Root directory must be created by the same
user if using HDFS, otherwise permission related issues will be raised.</span>
+deployUser=<span class="hljs-string">"dolphinscheduler"</span>
+
+
+<span class="hljs-comment"># Followings are alert-service configs</span>
+<span class="hljs-comment"># Mail server host</span>
+mailServerHost=<span class="hljs-string">"smtp.exmail.qq.com"</span>
+
+<span class="hljs-comment"># Mail server port</span>
+mailServerPort=<span class="hljs-string">"25"</span>
+
+<span class="hljs-comment"># Mail sender</span>
+mailSender=<span class="hljs-string">"xxxxxxxxxx"</span>
+
+<span class="hljs-comment"># Mail user</span>
+mailUser=<span class="hljs-string">"xxxxxxxxxx"</span>
+
+<span class="hljs-comment"># Mail password</span>
+mailPassword=<span class="hljs-string">"xxxxxxxxxx"</span>
+
+<span class="hljs-comment"># Mail supports TLS set true if not set false</span>
+starttlsEnable=<span class="hljs-string">"true"</span>
+
+<span class="hljs-comment"># Mail supports SSL set true if not set false.
Note: starttlsEnable and sslEnable cannot both set true</span>
+sslEnable=<span class="hljs-string">"false"</span>
+
+<span class="hljs-comment"># Mail server host, same as mailServerHost</span>
+sslTrust=<span class="hljs-string">"smtp.exmail.qq.com"</span>
+
+<span class="hljs-comment"># Specify which resource upload function to use for
resources storage such as sql files. And supported options are HDFS, S3 and
NONE. HDFS for upload to HDFS and NONE for not using this function.</span>
+resourceStorageType=<span class="hljs-string">"NONE"</span>
+
+<span class="hljs-comment"># if S3, write S3 address. HA, for example:
s3a://dolphinscheduler，</span>
+<span class="hljs-comment"># Note: s3 make sure to create the root directory
/dolphinscheduler</span>
+defaultFS=<span class="hljs-string">"hdfs://mycluster:8020"</span>
+
+<span class="hljs-comment"># If parameter 'resourceStorageType' is S3,
following configs are needed:</span>
+s3Endpoint=<span class="hljs-string">"http://192.168.xx.xx:9010";</span>
+s3AccessKey=<span class="hljs-string">"xxxxxxxxxx"</span>
+s3SecretKey=<span class="hljs-string">"xxxxxxxxxx"</span>
+
+<span class="hljs-comment"># If ResourceManager supports HA, then input master
and standby node IP or hostname, eg: '192.168.xx.xx,192.168.xx.xx'. Or else
ResourceManager run in standalone mode, please set yarnHaIps="" and "" for not
using yarn.</span>
+yarnHaIps=<span class="hljs-string">"192.168.xx.xx,192.168.xx.xx"</span>
+
+
+<span class="hljs-comment"># If ResourceManager runs in standalone, then set
ResourceManager node ip or hostname, or else remain default.</span>
+singleYarnIp=<span class="hljs-string">"yarnIp1"</span>
+
+<span class="hljs-comment"># Storage path when using HDFS/S3</span>
+resourceUploadPath=<span class="hljs-string">"/dolphinscheduler"</span>
+
+
+<span class="hljs-comment"># HDFS/S3 root user</span>
+hdfsRootUser=<span class="hljs-string">"hdfs"</span>
+
+<span class="hljs-comment"># Followings are kerberos configs</span>
+
+<span class="hljs-comment"># Spicify kerberos enable or not</span>
+kerberosStartUp=<span class="hljs-string">"false"</span>
+
+<span class="hljs-comment"># Kdc krb5 config file path</span>
+krb5ConfPath=<span class="hljs-string">"<span
class="hljs-variable">$installPath</span>/conf/krb5.conf"</span>
+
+<span class="hljs-comment"># Keytab username</span>
+keytabUserName=<span class="hljs-string">"[email protected]"</span>
+
+<span class="hljs-comment"># Username keytab path</span>
+keytabPath=<span class="hljs-string">"<span
class="hljs-variable">$installPath</span>/conf/hdfs.headless.keytab"</span>
+
+
+<span class="hljs-comment"># API-service port</span>
+apiServerPort=<span class="hljs-string">"12345"</span>
+
+
+<span class="hljs-comment"># All hosts deploy DS</span>
+ips=<span class="hljs-string">"ds1,ds2,ds3,ds4,ds5"</span>
+
+<span class="hljs-comment"># Ssh port, default 22</span>
+sshPort=<span class="hljs-string">"22"</span>
+
+<span class="hljs-comment"># Master service hosts</span>
+masters=<span class="hljs-string">"ds1,ds2"</span>
+
+<span class="hljs-comment"># All hosts deploy worker service</span>
+<span class="hljs-comment"># Note: Each worker needs to set a worker group
name and default name is "default"</span>
+workers=<span
class="hljs-string">"ds1:default,ds2:default,ds3:default,ds4:default,ds5:default"</span>
+
+<span class="hljs-comment"># Host deploy alert-service</span>
+alertServer=<span class="hljs-string">"ds3"</span>
+
+<span class="hljs-comment"># Host deploy API-service</span>
+apiServers=<span class="hljs-string">"ds1"</span>
+</code></pre>
+<h2>11.dolphinscheduler_env.sh [load environment variables configs]</h2>
+<p>When using shell to commit tasks, DS will load environment variables inside
dolphinscheduler_env.sh into the host.
+Types of tasks involved are: Shell task、Python task、Spark task、Flink
task、Datax task and etc.</p>
+<pre><code class="language-bash"><span class="hljs-built_in">export</span>
HADOOP_HOME=/opt/soft/hadoop
+<span class="hljs-built_in">export</span>
HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
+<span class="hljs-built_in">export</span> SPARK_HOME1=/opt/soft/spark1
+<span class="hljs-built_in">export</span> SPARK_HOME2=/opt/soft/spark2
+<span class="hljs-built_in">export</span> PYTHON_HOME=/opt/soft/python
+<span class="hljs-built_in">export</span> JAVA_HOME=/opt/soft/java
+<span class="hljs-built_in">export</span> HIVE_HOME=/opt/soft/hive
+<span class="hljs-built_in">export</span> FLINK_HOME=/opt/soft/flink
+<span class="hljs-built_in">export</span>
DATAX_HOME=/opt/soft/datax/bin/datax.py
+
+<span class="hljs-built_in">export</span> PATH=<span
class="hljs-variable">$HADOOP_HOME</span>/bin:<span
class="hljs-variable">$SPARK_HOME1</span>/bin:<span
class="hljs-variable">$SPARK_HOME2</span>/bin:<span
class="hljs-variable">$PYTHON_HOME</span>:<span
class="hljs-variable">$JAVA_HOME</span>/bin:<span
class="hljs-variable">$HIVE_HOME</span>/bin:<span
class="hljs-variable">$PATH</span>:<span
class="hljs-variable">$FLINK_HOME</span>/bin:<span
class="hljs-variable">$DATAX_HOME</span>:<s [...]
+
+</code></pre>
+<h2>12. Services logback configs</h2>
+<table>
+<thead>
+<tr>
+<th>Services name</th>
+<th>logback config name</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>API-service logback config</td>
+<td>logback-api.xml</td>
+</tr>
+<tr>
+<td>master-service logback config</td>
+<td>logback-master.xml</td>
+</tr>
+<tr>
+<td>worker-service logback config</td>
+<td>logback-worker.xml</td>
+</tr>
+<tr>
+<td>alert-service logback config</td>
+<td>logback-alert.xml</td>
+</tr>
+</tbody>
+</table>
+</div></section><footer class="footer-container"><div class="footer-body"><img
src="/img/ds_gray.svg"/><div class="cols-container"><div class="col
col-12"><h3>Disclaimer</h3><p>Apache DolphinScheduler is an effort undergoing
incubation at The Apache Software Foundation (ASF), sponsored by the Apache
Incubator. Incubation is required of all newly accepted projects until a
further review indicates that the infrastructure, communications, and decision
making process have stabilized in a man [...]
+ <script
src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js";></script>
+ <script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js";></script>
+ <script>window.rootPath = '';</script>
+ <script src="/build/documentation.md.js"></script>
+ <script>
+ var _hmt = _hmt || [];
+ (function() {
+ var hm = document.createElement("script");
+ hm.src = "https://hm.baidu.com/hm.js?4e7b4b400dd31fa015018a435c64d06f";;
+ var s = document.getElementsByTagName("script")[0];
+ s.parentNode.insertBefore(hm, s);
+ })();
+ </script>
+</body>
+</html>
\ No newline at end of file
diff --git a/en-us/docs/1.3.4/user_doc/configuration-file.json
b/en-us/docs/1.3.4/user_doc/configuration-file.json
new file mode 100644
index 0000000..43488a4
--- /dev/null
+++ b/en-us/docs/1.3.4/user_doc/configuration-file.json
@@ -0,0 +1,6 @@
+{
+ "filename": "configuration-file.md",
+ "__html": "<h1>Preface</h1>\n<p>This document explains the DolphinScheduler
application configurations according to DolphinScheduler-1.3.x
versions.</p>\n<h1>Directory Structure</h1>\n<p>Currently, all the
configuration files are under [conf ] directory. Please check the following
simplified DolphinScheduler installation directories to have a direct view
about the position [conf] directory in and configuration files inside. This
document only describes DolphinScheduler configurations a [...]
+ "link": "/en-us/docs/1.3.4/user_doc/configuration-file.html",
+ "meta": {}
+}
\ No newline at end of file
