This is an automated email from the ASF dual-hosted git repository.
liuxun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 6a3b701 SUBMARINE-289. Add the distributed applications case about
Kaldi on Submarine
6a3b701 is described below
commit 6a3b701e992dca4722edc112b4320952f0114266
Author: huiyangjian <[email protected]>
AuthorDate: Fri Nov 29 20:51:31 2019 +0800
SUBMARINE-289. Add the distributed applications case about Kaldi on
Submarine
### What is this PR for?
Used submarine running distributed Kaldi, used thchs30 case, the basic
environment needs hadoop 3.1 or above version, need configure Registry DNS
Server, prepare Dockerfile and script, and operate according to the document.
### What type of PR is it?
[Documentation]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-289
### How should this be tested?
[CI Pass](https://travis-ci.org/apache/submarine/builds/618588567)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? Yes/No
* Is there breaking changes for older versions? Yes/No
* Does this needs documentation? Yes/No
Author: huiyangjian <[email protected]>
Closes #115 from huiyangjian/SUBMARINE-289 and squashes the following
commits:
122cb9c [huiyangjian] SUBMARINE-289.Add the distributed applications case
about Kaldi on Submarine
---
docs/README.md | 4 +
.../kaldi/RunningDistributedThchs30KaldiJobs.md | 678 +++++++++++++++++++++
docs/ecosystem/kaldi/WriteDockerfileKaldi.md | 112 ++++
.../base/ubuntu-16.04/Dockerfile.gpu.kaldi_latest | 74 +++
docs/ecosystem/kaldi/build-all.sh | 26 +
docs/ecosystem/kaldi/pictures/sge_cluster.png | Bin 0 -> 149752 bytes
docs/ecosystem/kaldi/pictures/sge_stat.png | Bin 0 -> 123705 bytes
docs/ecosystem/kaldi/sge/gencfs.sh | 154 +++++
docs/ecosystem/kaldi/sge/group | 66 ++
docs/ecosystem/kaldi/sge/passwd | 42 ++
docs/ecosystem/kaldi/sge/resolv.conf | 23 +
docs/ecosystem/kaldi/sge/sge_run.sh | 84 +++
12 files changed, 1263 insertions(+)
diff --git a/docs/README.md b/docs/README.md
index 5ff0512..0331d49 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -40,6 +40,8 @@ Here're some examples about Submarine usage.
[Running Standalone CIFAR 10 PyTorch
Job](./helper/RunningSingleNodeCifar10PTJobs.md)
+[Running Distributed thchs30 Kaldi
Job](./ecosystem/kaldi/RunningDistributedThchs30KaldiJobs.md)
+
## Development
[Submarine Project Development Guide](./development/README.md)
@@ -52,6 +54,8 @@ Here're some examples about Submarine usage.
[How to write Dockerfile for Submarine PyTorch
jobs](./helper/WriteDockerfilePT.md)
+[How to write Dockerfile for Submarine Kaldi
jobs](./ecosystem/kaldi/WriteDockerfileKaldi.md)
+
## Install Dependencies
**Note: You need to install dependencies when using Hadoop YARN 3.1.x + or
above.**
diff --git a/docs/ecosystem/kaldi/RunningDistributedThchs30KaldiJobs.md
b/docs/ecosystem/kaldi/RunningDistributedThchs30KaldiJobs.md
new file mode 100644
index 0000000..cafe371
--- /dev/null
+++ b/docs/ecosystem/kaldi/RunningDistributedThchs30KaldiJobs.md
@@ -0,0 +1,678 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Thchs30 Kaldi Example With YARN Service
+
+## Prepare data for training
+
+Thchs30 is a common benchmark in machine learning for speech data and
transcripts. Below example is based on Thchs30 dataset.
+
+1) download gz file:
+```
+THCHS30_PATH=/data/hdfs1/nfs/aisearch/kaldi/thchs30
+mkdir $THCHS30_PATH/data && cd $THCHS30_PATH/data
+wget http://www.openslr.org/resources/18/data_thchs30.tgz
+wget http://www.openslr.org/resources/18/test-noise.tgz
+wget http://www.openslr.org/resources/18/resource.tgz
+```
+
+2) Checkout https://github.com/apache/submarine.git:
+```
+git clone https://github.com/apache/submarine.git
+```
+
+3) Go to `submarine/docker/ecosystem/`
+```
+cp -r ./kaldi/sge $THCHS30_PATH/sge
+```
+
+4) optional，Modify `/opt/kaldi/egs/thchs30/s5/cmd.sh` in the Container,This
queue is used by default
+```
+export train_cmd="queue.pl -q all.q"
+```
+
+**Warning:**
+
+Please note that YARN service doesn't allow multiple services with the same
name, so please run following command
+```
+yarn application -destroy <service-name>
+```
+to delete services if you want to reuse the same service name.
+
+## Prepare Docker images
+
+Refer to [Write Dockerfile](WriteDockerfileKaldi.md) to build a Docker image
or use prebuilt one:
+
+- hadoopsubmarine/kaldi-latest-gpu-base:0.0.1
+
+## Run Kaldi jobs
+
+### Run distributed training
+
+```
+# Change the variables according to your needs
+SUBMARINE_VERSION=3.3.0-SNAPSHOT
+WORKER_NUM=2
+SGE_CFG_PATH=/cfg
+THCHS30_PATH=/data/hdfs1/nfs/aisearch/kaldi/thchs30
+DOCKER_HADOOP_HDFS_HOME=/app/${SUBMARINE_VERSION}
+
+# Dependent on registrydns, you must fill in < your RegistryDNSIP> in
resolv.conf
+yarn jar /usr/local/matrix/share/hadoop/yarn/${SUBMARINE_VERSION}.jar \
+job run --name kaldi-thchs30-distributed \
+--env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/ \
+--env DOCKER_HADOOP_HDFS_HOME=$DOCKER_HADOOP_HDFS_HOME \
+--env YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK=calico-network \
+--env PYTHONUNBUFFERED="0" \
+--env TZ="Asia/Shanghai" \
+--env
YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS=${THCHS30_PATH}/sge/resolv.conf:/etc/resolv.conf,\
+${THCHS30_PATH}/sge/passwd:/etc/passwd:rw,\
+${THCHS30_PATH}/sge/group:/etc/group:rw,\
+${THCHS30_PATH}/sge:$SGE_CFG_PATH,\
+${THCHS30_PATH}/data:/opt/kaldi/egs/thchs30,\
+${THCHS30_PATH}/mul/s5:/opt/kaldi/egs/mul-thchs30/s5 \
+--input_path /opt/kaldi/egs/thchs30/data \
+--docker_image hadoopsubmarine/kaldi-latest-gpu-base:0.0.1 \
+--num_workers $WORKER_NUM \
+--worker_resources memory=64G,vcores=32,gpu=1 \
+--worker_launch_cmd "sudo mkdir -p /opt/kaldi/egs/mul-thchs30/s5 && \
+sudo cp /opt/kaldi/egs/thchs30/s5/* /opt/kaldi/egs/mul-thchs30/s5 -r && \
+cluster_user=`whoami` domain_suffix="ml.com" && \
+cd /cfg && bash sge_run.sh $WORKER_NUM $SGE_CFG_PATH && \
+if [ $(echo $HOST_NAME |grep "^master-") ] then sleep 2m && cd
/opt/kaldi/egs/mul-thchs30/s5 && ./run.sh fi" \
+--verbose
+```
+
+Explanations:
+
+- `>1` num_workers indicates it is a distributed training.
+- Parameters / resources / Docker image of parameter server can be specified
separately. For many cases, parameter server doesn't require GPU.We don't need
parameter server here
+
+For the meaning of the individual parameters, see the
[QuickStart](../../helper/QuickStart.md) page!
+
+*Outputs of distributed training*
+
+Sample output of master:
+```
+...
+Reading package lists...
+Building dependency tree...
+Reading state information...
+The following additional packages will be installed:
+ bsd-mailx cpio gridengine-common ifupdown iproute2 isc-dhcp-client
+ isc-dhcp-common libatm1 libdns-export162 libisc-export160 liblockfile-bin
+ liblockfile1 libmnl0 libxmuu1 libxtables11 ncurses-term netbase
+ openssh-client openssh-server openssh-sftp-server postfix python3-chardet
+ python3-pkg-resources python3-requests python3-six python3-urllib3
+ ssh-import-id ssl-cert tcsh xauth
+Suggested packages:
+ libarchive1 gridengine-qmon ppp rdnssd iproute2-doc resolvconf avahi-autoipd
+ isc-dhcp-client-ddns apparmor ssh-askpass libpam-ssh keychain monkeysphere
+ rssh molly-guard ufw procmail postfix-mysql postfix-pgsql postfix-ldap
+ postfix-pcre sasl2-bin libsasl2-modules dovecot-common postfix-cdb
+ postfix-doc python3-setuptools python3-ndg-httpsclient python3-openssl
+ python3-pyasn1 openssl-blacklist
+The following NEW packages will be installed:
+ bsd-mailx cpio gridengine-client gridengine-common gridengine-exec
+ gridengine-master ifupdown iproute2 isc-dhcp-client isc-dhcp-common libatm1
+ libdns-export162 libisc-export160 liblockfile-bin liblockfile1 libmnl0
+ libxmuu1 libxtables11 ncurses-term netbase openssh-client openssh-server
+ openssh-sftp-server postfix python3-chardet python3-pkg-resources
+ python3-requests python3-six python3-urllib3 ssh-import-id ssl-cert tcsh
+ xauth
+0 upgraded, 33 newly installed, 0 to remove and 30 not upgraded.
+Need to get 12.1 MB of archives.
+After this operation, 65.8 MB of additional disk space will be used.
+Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libatm1 amd64
1:2.5.1-1.5 [24.2 kB]
+Get:2 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmnl0 amd64 1.0.3-5
[12.0 kB]
+Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 liblockfile-bin amd64
1.09-6ubuntu1 [10.8 kB]
+Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 liblockfile1 amd64
1.09-6ubuntu1 [8056 B]
+Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 cpio amd64
2.11+dfsg-5ubuntu1 [74.8 kB]
+Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 iproute2
amd64 4.3.0-1ubuntu3.16.04.5 [523 kB]
+Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 ifupdown
amd64 0.8.10ubuntu1.4 [54.9 kB]
+Get:8 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
libisc-export160 amd64 1:9.10.3.dfsg.P4-8ubuntu1.15 [153 kB]
+Get:9 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
libdns-export162 amd64 1:9.10.3.dfsg.P4-8ubuntu1.15 [665 kB]
+Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
isc-dhcp-client amd64 4.3.3-5ubuntu12.10 [224 kB]
+Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
isc-dhcp-common amd64 4.3.3-5ubuntu12.10 [105 kB]
+Get:12 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxtables11 amd64
1.6.0-2ubuntu3 [27.2 kB]
+Get:13 http://archive.ubuntu.com/ubuntu xenial/main amd64 netbase all 5.3
[12.9 kB]
+Get:14 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxmuu1 amd64
2:1.1.2-2 [9674 B]
+Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-client amd64 1:7.2p2-4ubuntu2.8 [590 kB]
+Get:16 http://archive.ubuntu.com/ubuntu xenial/main amd64 xauth amd64
1:1.0.9-1ubuntu2 [22.7 kB]
+Get:17 http://archive.ubuntu.com/ubuntu xenial/main amd64 ssl-cert all 1.0.37
[16.9 kB]
+Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 postfix
amd64 3.1.0-3ubuntu0.3 [1152 kB]
+Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 bsd-mailx amd64
8.1.2-0.20160123cvs-2 [63.7 kB]
+Get:20 http://archive.ubuntu.com/ubuntu xenial/universe amd64
gridengine-common all 6.2u5-7.4 [156 kB]
+Get:21 http://archive.ubuntu.com/ubuntu xenial/universe amd64
gridengine-client amd64 6.2u5-7.4 [3394 kB]
+Get:22 http://archive.ubuntu.com/ubuntu xenial/universe amd64 tcsh amd64
6.18.01-5 [410 kB]
+Get:23 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gridengine-exec
amd64 6.2u5-7.4 [990 kB]
+Get:24 http://archive.ubuntu.com/ubuntu xenial/universe amd64
gridengine-master amd64 6.2u5-7.4 [2429 kB]
+Get:25 http://archive.ubuntu.com/ubuntu xenial/main amd64 ncurses-term all
6.0+20160213-1ubuntu1 [249 kB]
+Get:26 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-sftp-server amd64 1:7.2p2-4ubuntu2.8 [38.9 kB]
+Get:27 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-server amd64 1:7.2p2-4ubuntu2.8 [335 kB]
+Get:28 http://archive.ubuntu.com/ubuntu xenial/main amd64
python3-pkg-resources all 20.7.0-1 [79.0 kB]
+Get:29 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-chardet all
2.3.0-2 [96.2 kB]
+Get:30 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-six all
1.10.0-3 [11.0 kB]
+Get:31 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
python3-urllib3 all 1.13.1-2ubuntu0.16.04.3 [58.5 kB]
+Get:32 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
python3-requests all 2.9.1-3ubuntu0.1 [55.8 kB]
+Get:33 http://archive.ubuntu.com/ubuntu xenial/main amd64 ssh-import-id all
5.5-0ubuntu1 [10.2 kB]
+Fetched 12.1 MB in 0s (15.0 MB/s)
+Selecting previously unselected package libatm1:amd64.
+(Reading database ...
+(Reading database ... 5%
+(Reading database ... 10%
+(Reading database ... 15%
+(Reading database ... 20%
+(Reading database ... 25%
+(Reading database ... 30%
+(Reading database ... 35%
+(Reading database ... 40%
+(Reading database ... 45%
+(Reading database ... 50%
+(Reading database ... 55%
+(Reading database ... 60%
+(Reading database ... 65%
+(Reading database ... 70%
+(Reading database ... 75%
+(Reading database ... 80%
+(Reading database ... 85%
+(Reading database ... 90%
+(Reading database ... 95%
+(Reading database ... 100%
+(Reading database ... 21398 files and directories currently installed.)
+Preparing to unpack .../libatm1_1%3a2.5.1-1.5_amd64.deb ...
+Unpacking libatm1:amd64 (1:2.5.1-1.5) ...
+Selecting previously unselected package libmnl0:amd64.
+Preparing to unpack .../libmnl0_1.0.3-5_amd64.deb ...
+Unpacking libmnl0:amd64 (1.0.3-5) ...
+Selecting previously unselected package liblockfile-bin.
+Preparing to unpack .../liblockfile-bin_1.09-6ubuntu1_amd64.deb ...
+Unpacking liblockfile-bin (1.09-6ubuntu1) ...
+Selecting previously unselected package liblockfile1:amd64.
+Preparing to unpack .../liblockfile1_1.09-6ubuntu1_amd64.deb ...
+Unpacking liblockfile1:amd64 (1.09-6ubuntu1) ...
+Selecting previously unselected package cpio.
+Preparing to unpack .../cpio_2.11+dfsg-5ubuntu1_amd64.deb ...
+Unpacking cpio (2.11+dfsg-5ubuntu1) ...
+Selecting previously unselected package iproute2.
+Preparing to unpack .../iproute2_4.3.0-1ubuntu3.16.04.5_amd64.deb ...
+Unpacking iproute2 (4.3.0-1ubuntu3.16.04.5) ...
+Selecting previously unselected package ifupdown.
+Preparing to unpack .../ifupdown_0.8.10ubuntu1.4_amd64.deb ...
+Unpacking ifupdown (0.8.10ubuntu1.4) ...
+Selecting previously unselected package libisc-export160.
+Preparing to unpack
.../libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.15_amd64.deb ...
+Unpacking libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Selecting previously unselected package libdns-export162.
+Preparing to unpack
.../libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.15_amd64.deb ...
+Unpacking libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Selecting previously unselected package isc-dhcp-client.
+Preparing to unpack .../isc-dhcp-client_4.3.3-5ubuntu12.10_amd64.deb ...
+Unpacking isc-dhcp-client (4.3.3-5ubuntu12.10) ...
+Selecting previously unselected package isc-dhcp-common.
+Preparing to unpack .../isc-dhcp-common_4.3.3-5ubuntu12.10_amd64.deb ...
+Unpacking isc-dhcp-common (4.3.3-5ubuntu12.10) ...
+Selecting previously unselected package libxtables11:amd64.
+Preparing to unpack .../libxtables11_1.6.0-2ubuntu3_amd64.deb ...
+Unpacking libxtables11:amd64 (1.6.0-2ubuntu3) ...
+Selecting previously unselected package netbase.
+Preparing to unpack .../archives/netbase_5.3_all.deb ...
+Unpacking netbase (5.3) ...
+Selecting previously unselected package libxmuu1:amd64.
+Preparing to unpack .../libxmuu1_2%3a1.1.2-2_amd64.deb ...
+Unpacking libxmuu1:amd64 (2:1.1.2-2) ...
+Selecting previously unselected package openssh-client.
+Preparing to unpack .../openssh-client_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-client (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package xauth.
+Preparing to unpack .../xauth_1%3a1.0.9-1ubuntu2_amd64.deb ...
+Unpacking xauth (1:1.0.9-1ubuntu2) ...
+Selecting previously unselected package ssl-cert.
+Preparing to unpack .../ssl-cert_1.0.37_all.deb ...
+Unpacking ssl-cert (1.0.37) ...
+Selecting previously unselected package postfix.
+Preparing to unpack .../postfix_3.1.0-3ubuntu0.3_amd64.deb ...
+Unpacking postfix (3.1.0-3ubuntu0.3) ...
+Selecting previously unselected package bsd-mailx.
+Preparing to unpack .../bsd-mailx_8.1.2-0.20160123cvs-2_amd64.deb ...
+Unpacking bsd-mailx (8.1.2-0.20160123cvs-2) ...
+Selecting previously unselected package gridengine-common.
+Preparing to unpack .../gridengine-common_6.2u5-7.4_all.deb ...
+Unpacking gridengine-common (6.2u5-7.4) ...
+Selecting previously unselected package gridengine-client.
+Preparing to unpack .../gridengine-client_6.2u5-7.4_amd64.deb ...
+Unpacking gridengine-client (6.2u5-7.4) ...
+Selecting previously unselected package tcsh.
+Preparing to unpack .../tcsh_6.18.01-5_amd64.deb ...
+Unpacking tcsh (6.18.01-5) ...
+Selecting previously unselected package gridengine-exec.
+Preparing to unpack .../gridengine-exec_6.2u5-7.4_amd64.deb ...
+Unpacking gridengine-exec (6.2u5-7.4) ...
+Selecting previously unselected package gridengine-master.
+Preparing to unpack .../gridengine-master_6.2u5-7.4_amd64.deb ...
+Unpacking gridengine-master (6.2u5-7.4) ...
+Selecting previously unselected package ncurses-term.
+Preparing to unpack .../ncurses-term_6.0+20160213-1ubuntu1_all.deb ...
+Unpacking ncurses-term (6.0+20160213-1ubuntu1) ...
+Selecting previously unselected package openssh-sftp-server.
+Preparing to unpack .../openssh-sftp-server_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-sftp-server (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package openssh-server.
+Preparing to unpack .../openssh-server_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-server (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package python3-pkg-resources.
+Preparing to unpack .../python3-pkg-resources_20.7.0-1_all.deb ...
+Unpacking python3-pkg-resources (20.7.0-1) ...
+Selecting previously unselected package python3-chardet.
+Preparing to unpack .../python3-chardet_2.3.0-2_all.deb ...
+Unpacking python3-chardet (2.3.0-2) ...
+Selecting previously unselected package python3-six.
+Preparing to unpack .../python3-six_1.10.0-3_all.deb ...
+Unpacking python3-six (1.10.0-3) ...
+Selecting previously unselected package python3-urllib3.
+Preparing to unpack .../python3-urllib3_1.13.1-2ubuntu0.16.04.3_all.deb ...
+Unpacking python3-urllib3 (1.13.1-2ubuntu0.16.04.3) ...
+Selecting previously unselected package python3-requests.
+Preparing to unpack .../python3-requests_2.9.1-3ubuntu0.1_all.deb ...
+Unpacking python3-requests (2.9.1-3ubuntu0.1) ...
+Selecting previously unselected package ssh-import-id.
+Preparing to unpack .../ssh-import-id_5.5-0ubuntu1_all.deb ...
+Unpacking ssh-import-id (5.5-0ubuntu1) ...
+Processing triggers for systemd (229-4ubuntu21.22) ...
+Processing triggers for libc-bin (2.23-0ubuntu11) ...
+Setting up libatm1:amd64 (1:2.5.1-1.5) ...
+Setting up libmnl0:amd64 (1.0.3-5) ...
+Setting up liblockfile-bin (1.09-6ubuntu1) ...
+Setting up liblockfile1:amd64 (1.09-6ubuntu1) ...
+Setting up cpio (2.11+dfsg-5ubuntu1) ...
+update-alternatives: using /bin/mt-gnu to provide /bin/mt (mt) in auto mode
+Setting up iproute2 (4.3.0-1ubuntu3.16.04.5) ...
+Setting up ifupdown (0.8.10ubuntu1.4) ...
+Creating /etc/network/interfaces.
+Setting up libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Setting up libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Setting up isc-dhcp-client (4.3.3-5ubuntu12.10) ...
+Setting up isc-dhcp-common (4.3.3-5ubuntu12.10) ...
+Setting up libxtables11:amd64 (1.6.0-2ubuntu3) ...
+Setting up netbase (5.3) ...
+Setting up libxmuu1:amd64 (2:1.1.2-2) ...
+Setting up openssh-client (1:7.2p2-4ubuntu2.8) ...
+Setting up xauth (1:1.0.9-1ubuntu2) ...
+Setting up ssl-cert (1.0.37) ...
+Setting up postfix (3.1.0-3ubuntu0.3) ...
+Creating /etc/postfix/dynamicmaps.cf
+setting myhostname: master-0.XXX
+setting alias maps
+setting alias database
+changing /etc/mailname to master-0.XXX
+setting myorigin
+setting destinations: $myhostname, master-0.XXX, localhost.XXX, , localhost
+setting relayhost:
+setting mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
+setting mailbox_size_limit: 0
+setting recipient_delimiter: +
+setting inet_interfaces: all
+setting inet_protocols: all
+/etc/aliases does not exist, creating it.
+WARNING: /etc/aliases exists, but does not have a root alias.
+
+Postfix is now set up with a default configuration. If you need to make
+changes, edit
+/etc/postfix/main.cf (and others) as needed. To view Postfix configuration
+values, see postconf(1).
+
+After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.
+
+Running newaliases
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of restart.
+Setting up bsd-mailx (8.1.2-0.20160123cvs-2) ...
+update-alternatives: using /usr/bin/bsd-mailx to provide /usr/bin/mailx
(mailx) in auto mode
+Setting up gridengine-common (6.2u5-7.4) ...
+
+Creating config file /etc/default/gridengine with new version
+Setting up gridengine-client (6.2u5-7.4) ...
+Setting up tcsh (6.18.01-5) ...
+update-alternatives: using /bin/tcsh to provide /bin/csh (csh) in auto mode
+Setting up gridengine-exec (6.2u5-7.4) ...
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of start.
+Setting up gridengine-master (6.2u5-7.4) ...
+su: Authentication failure
+(Ignored)
+Initializing cluster with the following parameters:
+ => SGE_ROOT: /var/lib/gridengine
+ => SGE_CELL: default
+ => Spool directory: /var/spool/gridengine/spooldb
+ => Initial manager user: sgeadmin
+Initializing spool (/var/spool/gridengine/spooldb)
+Initializing global configuration based on
/usr/share/gridengine/default-configuration
+Initializing complexes based on /usr/share/gridengine/centry
+Initializing usersets based on /usr/share/gridengine/usersets
+Adding user sgeadmin as a manager
+Cluster creation complete
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of start.
+Setting up ncurses-term (6.0+20160213-1ubuntu1) ...
+Setting up openssh-sftp-server (1:7.2p2-4ubuntu2.8) ...
+Setting up openssh-server (1:7.2p2-4ubuntu2.8) ...
+Creating SSH2 RSA key; this may take some time ...
+2048 SHA256:hfQpES1aS4cjF8AOCIParZR6342vdwutoyITru0wtuE [email protected] (RSA)
+Creating SSH2 DSA key; this may take some time ...
+1024 SHA256:gOsPMVgwXBHJzixN/gtJAG+hVCHqw8t7Fhy4nsx8od0 [email protected] (DSA)
+Creating SSH2 ECDSA key; this may take some time ...
+256 SHA256:3D5SNniUb4z+/BuqXheFgG+DfjsxXqTT/zwWAqdX4jM [email protected]
(ECDSA)
+Creating SSH2 ED25519 key; this may take some time ...
+256 SHA256:SwyeV9iSqOW4TKLi4Wvc0zD8lWtupHCJpDu8oWBwbfU [email protected]
(ED25519)
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of start.
+Setting up python3-pkg-resources (20.7.0-1) ...
+Setting up python3-chardet (2.3.0-2) ...
+Setting up python3-six (1.10.0-3) ...
+Setting up python3-urllib3 (1.13.1-2ubuntu0.16.04.3) ...
+Setting up python3-requests (2.9.1-3ubuntu0.1) ...
+Setting up ssh-import-id (5.5-0ubuntu1) ...
+Processing triggers for libc-bin (2.23-0ubuntu11) ...
+Processing triggers for systemd (229-4ubuntu21.22) ...
+Reading package lists...
+Building dependency tree...
+Reading state information...
+0 upgraded, 0 newly installed, 0 to remove and 30 not upgraded.
+```
+
+cat $SGE_CFG_PATH/setcfg.log
+```
+finish master
+add worker node worker-0.XXX
+```
+
+Sample output of worker:
+```
+please wait
+Reading package lists...
+Building dependency tree...
+Reading state information...
+The following additional packages will be installed:
+ bsd-mailx cpio gridengine-common ifupdown iproute2 isc-dhcp-client
+ isc-dhcp-common libatm1 libdns-export162 libisc-export160 liblockfile-bin
+ liblockfile1 libmnl0 libxmuu1 libxtables11 ncurses-term netbase
+ openssh-client openssh-server openssh-sftp-server postfix python3-chardet
+ python3-pkg-resources python3-requests python3-six python3-urllib3
+ ssh-import-id ssl-cert tcsh xauth
+Suggested packages:
+ libarchive1 gridengine-qmon ppp rdnssd iproute2-doc resolvconf avahi-autoipd
+ isc-dhcp-client-ddns apparmor ssh-askpass libpam-ssh keychain monkeysphere
+ rssh molly-guard ufw procmail postfix-mysql postfix-pgsql postfix-ldap
+ postfix-pcre sasl2-bin libsasl2-modules dovecot-common postfix-cdb
+ postfix-doc python3-setuptools python3-ndg-httpsclient python3-openssl
+ python3-pyasn1 openssl-blacklist
+The following NEW packages will be installed:
+ bsd-mailx cpio gridengine-client gridengine-common gridengine-exec ifupdown
+ iproute2 isc-dhcp-client isc-dhcp-common libatm1 libdns-export162
+ libisc-export160 liblockfile-bin liblockfile1 libmnl0 libxmuu1 libxtables11
+ ncurses-term netbase openssh-client openssh-server openssh-sftp-server
+ postfix python3-chardet python3-pkg-resources python3-requests python3-six
+ python3-urllib3 ssh-import-id ssl-cert tcsh xauth
+0 upgraded, 32 newly installed, 0 to remove and 30 not upgraded.
+Need to get 9633 kB of archives.
+After this operation, 51.2 MB of additional disk space will be used.
+Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libatm1 amd64
1:2.5.1-1.5 [24.2 kB]
+Get:2 http://archive.ubuntu.com/ubuntu xenial/main amd64 libmnl0 amd64 1.0.3-5
[12.0 kB]
+Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 liblockfile-bin amd64
1.09-6ubuntu1 [10.8 kB]
+Get:4 http://archive.ubuntu.com/ubuntu xenial/main amd64 liblockfile1 amd64
1.09-6ubuntu1 [8056 B]
+Get:5 http://archive.ubuntu.com/ubuntu xenial/main amd64 cpio amd64
2.11+dfsg-5ubuntu1 [74.8 kB]
+Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 iproute2
amd64 4.3.0-1ubuntu3.16.04.5 [523 kB]
+Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 ifupdown
amd64 0.8.10ubuntu1.4 [54.9 kB]
+Get:8 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
libisc-export160 amd64 1:9.10.3.dfsg.P4-8ubuntu1.15 [153 kB]
+Get:9 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
libdns-export162 amd64 1:9.10.3.dfsg.P4-8ubuntu1.15 [665 kB]
+Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
isc-dhcp-client amd64 4.3.3-5ubuntu12.10 [224 kB]
+Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
isc-dhcp-common amd64 4.3.3-5ubuntu12.10 [105 kB]
+Get:12 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxtables11 amd64
1.6.0-2ubuntu3 [27.2 kB]
+Get:13 http://archive.ubuntu.com/ubuntu xenial/main amd64 netbase all 5.3
[12.9 kB]
+Get:14 http://archive.ubuntu.com/ubuntu xenial/main amd64 libxmuu1 amd64
2:1.1.2-2 [9674 B]
+Get:15 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-client amd64 1:7.2p2-4ubuntu2.8 [590 kB]
+Get:16 http://archive.ubuntu.com/ubuntu xenial/main amd64 xauth amd64
1:1.0.9-1ubuntu2 [22.7 kB]
+Get:17 http://archive.ubuntu.com/ubuntu xenial/main amd64 ssl-cert all 1.0.37
[16.9 kB]
+Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 postfix
amd64 3.1.0-3ubuntu0.3 [1152 kB]
+Get:19 http://archive.ubuntu.com/ubuntu xenial/main amd64 bsd-mailx amd64
8.1.2-0.20160123cvs-2 [63.7 kB]
+Get:20 http://archive.ubuntu.com/ubuntu xenial/universe amd64
gridengine-common all 6.2u5-7.4 [156 kB]
+Get:21 http://archive.ubuntu.com/ubuntu xenial/universe amd64
gridengine-client amd64 6.2u5-7.4 [3394 kB]
+Get:22 http://archive.ubuntu.com/ubuntu xenial/universe amd64 tcsh amd64
6.18.01-5 [410 kB]
+Get:23 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gridengine-exec
amd64 6.2u5-7.4 [990 kB]
+Get:24 http://archive.ubuntu.com/ubuntu xenial/main amd64 ncurses-term all
6.0+20160213-1ubuntu1 [249 kB]
+Get:25 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-sftp-server amd64 1:7.2p2-4ubuntu2.8 [38.9 kB]
+Get:26 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
openssh-server amd64 1:7.2p2-4ubuntu2.8 [335 kB]
+Get:27 http://archive.ubuntu.com/ubuntu xenial/main amd64
python3-pkg-resources all 20.7.0-1 [79.0 kB]
+Get:28 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-chardet all
2.3.0-2 [96.2 kB]
+Get:29 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-six all
1.10.0-3 [11.0 kB]
+Get:30 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
python3-urllib3 all 1.13.1-2ubuntu0.16.04.3 [58.5 kB]
+Get:31 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64
python3-requests all 2.9.1-3ubuntu0.1 [55.8 kB]
+Get:32 http://archive.ubuntu.com/ubuntu xenial/main amd64 ssh-import-id all
5.5-0ubuntu1 [10.2 kB]
+Fetched 9633 kB in 2s (4496 kB/s)
+Selecting previously unselected package libatm1:amd64.
+(Reading database ...
+(Reading database ... 5%
+(Reading database ... 10%
+(Reading database ... 15%
+(Reading database ... 20%
+(Reading database ... 25%
+(Reading database ... 30%
+(Reading database ... 35%
+(Reading database ... 40%
+(Reading database ... 45%
+(Reading database ... 50%
+(Reading database ... 55%
+(Reading database ... 60%
+(Reading database ... 65%
+(Reading database ... 70%
+(Reading database ... 75%
+(Reading database ... 80%
+(Reading database ... 85%
+(Reading database ... 90%
+(Reading database ... 95%
+(Reading database ... 100%
+(Reading database ... 21398 files and directories currently installed.)
+Preparing to unpack .../libatm1_1%3a2.5.1-1.5_amd64.deb ...
+Unpacking libatm1:amd64 (1:2.5.1-1.5) ...
+Selecting previously unselected package libmnl0:amd64.
+Preparing to unpack .../libmnl0_1.0.3-5_amd64.deb ...
+Unpacking libmnl0:amd64 (1.0.3-5) ...
+Selecting previously unselected package liblockfile-bin.
+Preparing to unpack .../liblockfile-bin_1.09-6ubuntu1_amd64.deb ...
+Unpacking liblockfile-bin (1.09-6ubuntu1) ...
+Selecting previously unselected package liblockfile1:amd64.
+Preparing to unpack .../liblockfile1_1.09-6ubuntu1_amd64.deb ...
+Unpacking liblockfile1:amd64 (1.09-6ubuntu1) ...
+Selecting previously unselected package cpio.
+Preparing to unpack .../cpio_2.11+dfsg-5ubuntu1_amd64.deb ...
+Unpacking cpio (2.11+dfsg-5ubuntu1) ...
+Selecting previously unselected package iproute2.
+Preparing to unpack .../iproute2_4.3.0-1ubuntu3.16.04.5_amd64.deb ...
+Unpacking iproute2 (4.3.0-1ubuntu3.16.04.5) ...
+Selecting previously unselected package ifupdown.
+Preparing to unpack .../ifupdown_0.8.10ubuntu1.4_amd64.deb ...
+Unpacking ifupdown (0.8.10ubuntu1.4) ...
+Selecting previously unselected package libisc-export160.
+Preparing to unpack
.../libisc-export160_1%3a9.10.3.dfsg.P4-8ubuntu1.15_amd64.deb ...
+Unpacking libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Selecting previously unselected package libdns-export162.
+Preparing to unpack
.../libdns-export162_1%3a9.10.3.dfsg.P4-8ubuntu1.15_amd64.deb ...
+Unpacking libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Selecting previously unselected package isc-dhcp-client.
+Preparing to unpack .../isc-dhcp-client_4.3.3-5ubuntu12.10_amd64.deb ...
+Unpacking isc-dhcp-client (4.3.3-5ubuntu12.10) ...
+Selecting previously unselected package isc-dhcp-common.
+Preparing to unpack .../isc-dhcp-common_4.3.3-5ubuntu12.10_amd64.deb ...
+Unpacking isc-dhcp-common (4.3.3-5ubuntu12.10) ...
+Selecting previously unselected package libxtables11:amd64.
+Preparing to unpack .../libxtables11_1.6.0-2ubuntu3_amd64.deb ...
+Unpacking libxtables11:amd64 (1.6.0-2ubuntu3) ...
+Selecting previously unselected package netbase.
+Preparing to unpack .../archives/netbase_5.3_all.deb ...
+Unpacking netbase (5.3) ...
+Selecting previously unselected package libxmuu1:amd64.
+Preparing to unpack .../libxmuu1_2%3a1.1.2-2_amd64.deb ...
+Unpacking libxmuu1:amd64 (2:1.1.2-2) ...
+Selecting previously unselected package openssh-client.
+Preparing to unpack .../openssh-client_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-client (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package xauth.
+Preparing to unpack .../xauth_1%3a1.0.9-1ubuntu2_amd64.deb ...
+Unpacking xauth (1:1.0.9-1ubuntu2) ...
+Selecting previously unselected package ssl-cert.
+Preparing to unpack .../ssl-cert_1.0.37_all.deb ...
+Unpacking ssl-cert (1.0.37) ...
+Selecting previously unselected package postfix.
+Preparing to unpack .../postfix_3.1.0-3ubuntu0.3_amd64.deb ...
+Unpacking postfix (3.1.0-3ubuntu0.3) ...
+Selecting previously unselected package bsd-mailx.
+Preparing to unpack .../bsd-mailx_8.1.2-0.20160123cvs-2_amd64.deb ...
+Unpacking bsd-mailx (8.1.2-0.20160123cvs-2) ...
+Selecting previously unselected package gridengine-common.
+Preparing to unpack .../gridengine-common_6.2u5-7.4_all.deb ...
+Unpacking gridengine-common (6.2u5-7.4) ...
+Selecting previously unselected package gridengine-client.
+Preparing to unpack .../gridengine-client_6.2u5-7.4_amd64.deb ...
+Unpacking gridengine-client (6.2u5-7.4) ...
+Selecting previously unselected package tcsh.
+Preparing to unpack .../tcsh_6.18.01-5_amd64.deb ...
+Unpacking tcsh (6.18.01-5) ...
+Selecting previously unselected package gridengine-exec.
+Preparing to unpack .../gridengine-exec_6.2u5-7.4_amd64.deb ...
+Unpacking gridengine-exec (6.2u5-7.4) ...
+Selecting previously unselected package ncurses-term.
+Preparing to unpack .../ncurses-term_6.0+20160213-1ubuntu1_all.deb ...
+Unpacking ncurses-term (6.0+20160213-1ubuntu1) ...
+Selecting previously unselected package openssh-sftp-server.
+Preparing to unpack .../openssh-sftp-server_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-sftp-server (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package openssh-server.
+Preparing to unpack .../openssh-server_1%3a7.2p2-4ubuntu2.8_amd64.deb ...
+Unpacking openssh-server (1:7.2p2-4ubuntu2.8) ...
+Selecting previously unselected package python3-pkg-resources.
+Preparing to unpack .../python3-pkg-resources_20.7.0-1_all.deb ...
+Unpacking python3-pkg-resources (20.7.0-1) ...
+Selecting previously unselected package python3-chardet.
+Preparing to unpack .../python3-chardet_2.3.0-2_all.deb ...
+Unpacking python3-chardet (2.3.0-2) ...
+Selecting previously unselected package python3-six.
+Preparing to unpack .../python3-six_1.10.0-3_all.deb ...
+Unpacking python3-six (1.10.0-3) ...
+Selecting previously unselected package python3-urllib3.
+Preparing to unpack .../python3-urllib3_1.13.1-2ubuntu0.16.04.3_all.deb ...
+Unpacking python3-urllib3 (1.13.1-2ubuntu0.16.04.3) ...
+Selecting previously unselected package python3-requests.
+Preparing to unpack .../python3-requests_2.9.1-3ubuntu0.1_all.deb ...
+Unpacking python3-requests (2.9.1-3ubuntu0.1) ...
+Selecting previously unselected package ssh-import-id.
+Preparing to unpack .../ssh-import-id_5.5-0ubuntu1_all.deb ...
+Unpacking ssh-import-id (5.5-0ubuntu1) ...
+Processing triggers for systemd (229-4ubuntu21.22) ...
+Processing triggers for libc-bin (2.23-0ubuntu11) ...
+Setting up libatm1:amd64 (1:2.5.1-1.5) ...
+Setting up libmnl0:amd64 (1.0.3-5) ...
+Setting up liblockfile-bin (1.09-6ubuntu1) ...
+Setting up liblockfile1:amd64 (1.09-6ubuntu1) ...
+Setting up cpio (2.11+dfsg-5ubuntu1) ...
+update-alternatives: using /bin/mt-gnu to provide /bin/mt (mt) in auto mode
+Setting up iproute2 (4.3.0-1ubuntu3.16.04.5) ...
+Setting up ifupdown (0.8.10ubuntu1.4) ...
+Creating /etc/network/interfaces.
+Setting up libisc-export160 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Setting up libdns-export162 (1:9.10.3.dfsg.P4-8ubuntu1.15) ...
+Setting up isc-dhcp-client (4.3.3-5ubuntu12.10) ...
+Setting up isc-dhcp-common (4.3.3-5ubuntu12.10) ...
+Setting up libxtables11:amd64 (1.6.0-2ubuntu3) ...
+Setting up netbase (5.3) ...
+Setting up libxmuu1:amd64 (2:1.1.2-2) ...
+Setting up openssh-client (1:7.2p2-4ubuntu2.8) ...
+Setting up xauth (1:1.0.9-1ubuntu2) ...
+Setting up ssl-cert (1.0.37) ...
+Setting up postfix (3.1.0-3ubuntu0.3) ...
+Creating /etc/postfix/dynamicmaps.cf
+setting myhostname: worker-0.XXX
+setting alias maps
+setting alias database
+changing /etc/mailname to worker-0.XXX
+setting myorigin
+setting destinations: $myhostname, worker-0.XXX, localhost.XXX, , localhost
+setting relayhost:
+setting mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
+setting mailbox_size_limit: 0
+setting recipient_delimiter: +
+setting inet_interfaces: all
+setting inet_protocols: all
+/etc/aliases does not exist, creating it.
+WARNING: /etc/aliases exists, but does not have a root alias.
+
+Postfix is now set up with a default configuration. If you need to make
+changes, edit
+/etc/postfix/main.cf (and others) as needed. To view Postfix configuration
+values, see postconf(1).
+
+After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.
+
+Running newaliases
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of restart.
+Setting up bsd-mailx (8.1.2-0.20160123cvs-2) ...
+update-alternatives: using /usr/bin/bsd-mailx to provide /usr/bin/mailx
(mailx) in auto mode
+Setting up gridengine-common (6.2u5-7.4) ...
+
+Creating config file /etc/default/gridengine with new version
+Setting up gridengine-client (6.2u5-7.4) ...
+Setting up tcsh (6.18.01-5) ...
+update-alternatives: using /bin/tcsh to provide /bin/csh (csh) in auto mode
+Setting up gridengine-exec (6.2u5-7.4) ...
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of start.
+Setting up ncurses-term (6.0+20160213-1ubuntu1) ...
+Setting up openssh-sftp-server (1:7.2p2-4ubuntu2.8) ...
+Setting up openssh-server (1:7.2p2-4ubuntu2.8) ...
+Creating SSH2 RSA key; this may take some time ...
+2048 SHA256:ok/TxzwtF5W8I55sDxrt4Agy4fuWn39BiSovvDObhVE [email protected] (RSA)
+Creating SSH2 DSA key; this may take some time ...
+1024 SHA256:4y48kVYt3mS3q1KgZzEoYMnS/2d/tA8TJUK5uNSaxZY [email protected] (DSA)
+Creating SSH2 ECDSA key; this may take some time ...
+256 SHA256:4D7zm4cD2IbDnHoXnzcIo3FISbvOW8eOstGBNf1/bvo [email protected]
(ECDSA)
+Creating SSH2 ED25519 key; this may take some time ...
+256 SHA256:/HrA3xiZiH5CZkXwtcfE6GwcMM+hEhZzTdFHxj4PzDg [email protected]
(ED25519)
+invoke-rc.d: could not determine current runlevel
+invoke-rc.d: policy-rc.d denied execution of start.
+Setting up python3-pkg-resources (20.7.0-1) ...
+Setting up python3-chardet (2.3.0-2) ...
+Setting up python3-six (1.10.0-3) ...
+Setting up python3-urllib3 (1.13.1-2ubuntu0.16.04.3) ...
+Setting up python3-requests (2.9.1-3ubuntu0.1) ...
+Setting up ssh-import-id (5.5-0ubuntu1) ...
+Processing triggers for libc-bin (2.23-0ubuntu11) ...
+Processing triggers for systemd (229-4ubuntu21.22) ...
+Reading package lists...
+Building dependency tree...
+Reading state information...
+0 upgraded, 0 newly installed, 0 to remove and 30 not upgraded.
+```
+
+cat $SGE_CFG_PATH/setcfg.log
+```
+please wait
+Start SGE for worker is finished
+done for worker-0.XXX worker.
+```
+
+Sample output of sge:
+
+
+
\ No newline at end of file
diff --git a/docs/ecosystem/kaldi/WriteDockerfileKaldi.md
b/docs/ecosystem/kaldi/WriteDockerfileKaldi.md
new file mode 100644
index 0000000..c05328a
--- /dev/null
+++ b/docs/ecosystem/kaldi/WriteDockerfileKaldi.md
@@ -0,0 +1,112 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+## Creating Docker Images for Running Kaldi on YARN
+
+### How to create docker images to run Kaldi on YARN
+
+Dockerfile to run Kaldi on YARN need two part:
+
+**Base libraries which Kaldi depends on**
+
+1) OS base image, for example ```nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04```
+
+2) Kaldi depended libraries and packages. For example ```python```, ```g++```,
```make```. For GPU support, need ```cuda```, ```cudnn```, etc.
+
+3) Kaldi compile.
+
+**Libraries to access HDFS**
+
+1) JDK
+
+2) Hadoop
+
+Here's an example of a base image (w/o GPU support) to install Kaldi:
+```shell
+FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
+
+RUN apt-get clean && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ sudo \
+ openjdk-8-jdk \
+ iputils-ping \
+ g++ \
+ make \
+ automake \
+ autoconf \
+ bzip2 \
+ unzip \
+ wget \
+ sox \
+ libtool \
+ git \
+ subversion \
+ python2.7 \
+ python3 \
+ zlib1g-dev \
+ ca-certificates \
+ patch \
+ ffmpeg \
+ vim && \
+ rm -rf /var/lib/apt/lists/* && \
+ ln -s /usr/bin/python2.7 /usr/bin/python
+
+RUN git clone --depth 1 https://github.com/kaldi-asr/kaldi.git /opt/kaldi && \
+ cd /opt/kaldi && \
+ cd /opt/kaldi/tools && \
+ ./extras/install_mkl.sh && \
+ make -j $(nproc) && \
+ cd /opt/kaldi/src && \
+ ./configure --shared --use-cuda && \
+ make depend -j $(nproc) && \
+ make -j $(nproc)
+```
+
+On top of above image, add files, install packages to access HDFS
+```shell
+RUN apt-get update && apt-get install -y openjdk-8-jdk wget
+# Install hadoop
+ENV HADOOP_VERSION="3.2.1"
+ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+RUN wget
https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz
&& \
+ tar zxf hadoop-${HADOOP_VERSION}.tar.gz && \
+ ln -s hadoop-${HADOOP_VERSION} hadoop-current && \
+ rm hadoop-${HADOOP_VERSION}.tar.gz
+```
+
+Build and push to your own docker registry: Use ```docker build ... ``` and
```docker push ...``` to finish this step.
+
+### Use examples to build your own Kaldi docker images
+
+We provided following examples for you to build kaldi docker images.
+
+For latest Kaldi
+
+- *base/ubuntu-16.04/Dockerfile.gpu.kaldi_latest: Latest Kaldi that supports
GPU, which is prebuilt to CUDA10, with models.
+
+### Build Docker images
+
+#### Manually build Docker image:
+
+Under `docker/` directory,The CLUSTER_NAME can be modified in build-all.sh to
have installation permissions, run `build-all.sh` to build Docker images. It
will build following images:
+
+- `kaldi-latest-gpu-base:0.0.1` for base Docker image which includes Hadoop,
Kaldi, GPU base libraries, which includes thchs30 model.
+
+#### Use prebuilt images
+
+(No liability)
+You can also use prebuilt images for convenience in the docker hub:
+- hadoopsubmarine/kaldi-latest-gpu-base:0.0.1
diff --git a/docs/ecosystem/kaldi/base/ubuntu-16.04/Dockerfile.gpu.kaldi_latest
b/docs/ecosystem/kaldi/base/ubuntu-16.04/Dockerfile.gpu.kaldi_latest
new file mode 100644
index 0000000..47d0390
--- /dev/null
+++ b/docs/ecosystem/kaldi/base/ubuntu-16.04/Dockerfile.gpu.kaldi_latest
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu16.04
+
+RUN apt-get clean && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ sudo \
+ openjdk-8-jdk \
+ iputils-ping \
+ g++ \
+ make \
+ automake \
+ autoconf \
+ bzip2 \
+ unzip \
+ wget \
+ sox \
+ libtool \
+ git \
+ subversion \
+ python2.7 \
+ python3 \
+ zlib1g-dev \
+ ca-certificates \
+ patch \
+ ffmpeg \
+ vim && \
+ rm -rf /var/lib/apt/lists/* && \
+ ln -s /usr/bin/python2.7 /usr/bin/python
+
+RUN git clone --depth 1 https://github.com/kaldi-asr/kaldi.git /opt/kaldi && \
+ cd /opt/kaldi && \
+ cd /opt/kaldi/tools && \
+ ./extras/install_mkl.sh && \
+ make -j $(nproc) && \
+ cd /opt/kaldi/src && \
+ ./configure --shared --use-cuda && \
+ make depend -j $(nproc) && \
+ make -j $(nproc)
+
+WORKDIR /
+# Install Hadoop
+ENV HADOOP_VERSION="3.2.1"
+ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ARG CLUSTER_NAME=submarine
+RUN wget
https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz
&& \
+ tar zxf hadoop-${HADOOP_VERSION}.tar.gz && \
+ ln -s hadoop-${HADOOP_VERSION} hadoop-current && \
+ rm hadoop-${HADOOP_VERSION}.tar.gz
+
+RUN echo "Install python related packages" && \
+ pip --no-cache-dir install Pillow h5py ipykernel jupyter matplotlib numpy
pandas scipy sklearn && \
+ python -m ipykernel.kernelspec
+
+ENV CLUSTER_NAME="admin" # Your cluster user sets root privileges
+RUN echo "## Allow root to run any commands anywhere" >> /etc/sudoers && \
+ echo "User_Alias USER_ROOT = tf-docker, $CLUSTER_NAME" >> /etc/sudoers
&& \
+ echo "root ALL=(ALL) ALL" >> /etc/sudoers && \
+ echo "USER_ROOT ALL=(ALL) ALL" >> /etc/sudoers && \
+ echo "USER_ROOT ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
\ No newline at end of file
diff --git a/docs/ecosystem/kaldi/build-all.sh
b/docs/ecosystem/kaldi/build-all.sh
new file mode 100755
index 0000000..041bbda
--- /dev/null
+++ b/docs/ecosystem/kaldi/build-all.sh
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+echo "Building base images"
+
+set -e
+
+cd base/ubuntu-16.04
+
+docker build --build-arg CLUSTER_NAME=test . -f Dockerfile.gpu.kaldi_latest -t
kaldi-latest-gpu-base:0.0.1
+
+echo "Finished building base images"
diff --git a/docs/ecosystem/kaldi/pictures/sge_cluster.png
b/docs/ecosystem/kaldi/pictures/sge_cluster.png
new file mode 100644
index 0000000..9db22ee
Binary files /dev/null and b/docs/ecosystem/kaldi/pictures/sge_cluster.png
differ
diff --git a/docs/ecosystem/kaldi/pictures/sge_stat.png
b/docs/ecosystem/kaldi/pictures/sge_stat.png
new file mode 100644
index 0000000..1e1c92b
Binary files /dev/null and b/docs/ecosystem/kaldi/pictures/sge_stat.png differ
diff --git a/docs/ecosystem/kaldi/sge/gencfs.sh
b/docs/ecosystem/kaldi/sge/gencfs.sh
new file mode 100644
index 0000000..36e4630
--- /dev/null
+++ b/docs/ecosystem/kaldi/sge/gencfs.sh
@@ -0,0 +1,154 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+WORKER_HOST_STRS=$1
+SLOTS=$2
+
+cat << EOF > /tmp/qconf-mc.txt
+arch a RESTRING == YES NO NONE
0
+calendar c RESTRING == YES NO NONE
0
+cpu cpu DOUBLE >= YES NO 0
0
+display_win_gui dwg BOOL == YES NO 0
0
+gpu g INT <= YES YES 0
10000
+h_core h_core MEMORY <= YES NO 0
0
+h_cpu h_cpu TIME <= YES NO 0:0:0
0
+h_data h_data MEMORY <= YES NO 0
0
+h_fsize h_fsize MEMORY <= YES NO 0
0
+h_rss h_rss MEMORY <= YES NO 0
0
+h_rt h_rt TIME <= YES NO 0:0:0
0
+h_stack h_stack MEMORY <= YES NO 0
0
+h_vmem h_vmem MEMORY <= YES NO 0
0
+hostname h HOST == YES NO NONE
0
+load_avg la DOUBLE >= NO NO 0
0
+load_long ll DOUBLE >= NO NO 0
0
+load_medium lm DOUBLE >= NO NO 0
0
+load_short ls DOUBLE >= NO NO 0
0
+m_core core INT <= YES NO 0
0
+m_socket socket INT <= YES NO 0
0
+m_topology topo RESTRING == YES NO NONE
0
+m_topology_inuse utopo RESTRING == YES NO NONE
0
+mem_free mf MEMORY <= YES NO 0
0
+mem_total mt MEMORY <= YES NO 0
0
+mem_used mu MEMORY >= YES NO 0
0
+min_cpu_interval mci TIME <= NO NO 0:0:0
0
+np_load_avg nla DOUBLE >= NO NO 0
0
+np_load_long nll DOUBLE >= NO NO 0
0
+np_load_medium nlm DOUBLE >= NO NO 0
0
+np_load_short nls DOUBLE >= NO NO 0
0
+num_proc p INT == YES NO 0
0
+qname q RESTRING == YES NO NONE
0
+ram_free ram_free MEMORY <= YES JOB 0
0
+rerun re BOOL == NO NO 0
0
+s_core s_core MEMORY <= YES NO 0
0
+s_cpu s_cpu TIME <= YES NO 0:0:0
0
+s_data s_data MEMORY <= YES NO 0
0
+s_fsize s_fsize MEMORY <= YES NO 0
0
+s_rss s_rss MEMORY <= YES NO 0
0
+s_rt s_rt TIME <= YES NO 0:0:0
0
+s_stack s_stack MEMORY <= YES NO 0
0
+s_vmem s_vmem MEMORY <= YES NO 0
0
+seq_no seq INT == NO NO 0
0
+slots s INT <= YES YES 1
1000
+swap_free sf MEMORY <= YES NO 0
0
+swap_rate sr MEMORY >= YES NO 0
0
+swap_rsvd srsv MEMORY >= YES NO 0
0
+swap_total st MEMORY <= YES NO 0
0
+swap_used su MEMORY >= YES NO 0
0
+tmpdir tmp RESTRING == NO NO NONE
0
+virtual_free vf MEMORY <= YES NO 0
0
+virtual_total vt MEMORY <= YES NO 0
0
+virtual_used vu MEMORY >= YES NO 0
0
+EOF
+
+cat << EOF > /tmp/qconf-ae.txt
+hostname $(hostname)
+load_scaling NONE
+complex_values ram_free=160G,gpu=2
+user_lists NONE
+xuser_lists NONE
+projects NONE
+xprojects NONE
+usage_scaling NONE
+report_variables NONE
+EOF
+
+cat << EOS > /tmp/qconf-ap.txt
+pe_name smp
+slots 32
+user_lists NONE
+xuser_lists NONE
+start_proc_args /bin/true
+stop_proc_args /bin/true
+allocation_rule \$pe_slots
+control_slaves FALSE
+job_is_first_task TRUE
+urgency_slots min
+accounting_summary FALSE
+EOS
+
+cat << EOF > /tmp/qconf-aq.txt
+qname all.q
+hostlist $WORKER_HOST_STRS
+seq_no 0
+load_thresholds np_load_avg=1.75
+suspend_thresholds NONE
+nsuspend 1
+suspend_interval 00:05:00
+priority 0
+min_cpu_interval 00:05:00
+processors UNDEFINED
+qtype BATCH INTERACTIVE
+ckpt_list NONE
+pe_list make smp
+rerun FALSE
+slots $SLOTS
+tmpdir /tmp
+shell /bin/bash
+prolog NONE
+epilog NONE
+shell_start_mode posix_compliant
+starter_method NONE
+suspend_method NONE
+resume_method NONE
+terminate_method NONE
+notify 00:00:60
+owner_list NONE
+user_lists NONE
+xuser_lists NONE
+subordinate_list NONE
+complex_values NONE
+projects NONE
+xprojects NONE
+calendar NONE
+initial_state default
+s_rt INFINITY
+h_rt INFINITY
+s_cpu INFINITY
+h_cpu INFINITY
+s_fsize INFINITY
+h_fsize INFINITY
+s_data INFINITY
+h_data INFINITY
+s_stack INFINITY
+h_stack INFINITY
+s_core INFINITY
+h_core INFINITY
+s_rss INFINITY
+h_rss INFINITY
+s_vmem INFINITY
+h_vmem INFINITY
+EOF
diff --git a/docs/ecosystem/kaldi/sge/group b/docs/ecosystem/kaldi/sge/group
new file mode 100644
index 0000000..88c6ba8
--- /dev/null
+++ b/docs/ecosystem/kaldi/sge/group
@@ -0,0 +1,66 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+root:x:0:
+daemon:x:1:
+bin:x:2:
+sys:x:3:
+adm:x:4:
+tty:x:5:
+disk:x:6:
+lp:x:7:
+mail:x:8:
+news:x:9:
+uucp:x:10:
+man:x:12:
+proxy:x:13:
+kmem:x:15:
+dialout:x:20:
+fax:x:21:
+voice:x:22:
+cdrom:x:24:
+floppy:x:25:
+tape:x:26:
+sudo:x:27:
+audio:x:29:
+dip:x:30:
+www-data:x:33:
+backup:x:34:
+operator:x:37:
+list:x:38:
+irc:x:39:
+src:x:40:
+gnats:x:41:
+shadow:x:42:
+utmp:x:43:
+video:x:44:
+sasl:x:45:
+plugdev:x:46:
+staff:x:50:
+games:x:60:
+users:x:100:
+nogroup:x:65534:
+systemd-journal:x:101:
+systemd-timesync:x:102:
+systemd-network:x:103:
+systemd-resolve:x:104:
+systemd-bus-proxy:x:106:
+etcd:x:1099:
+netdev:x:105:
+ssh:x:107:
+ssl-cert:x:108:
+postfix:x:109:
+postdrop:x:110:
+sgeadmin:x:111:
diff --git a/docs/ecosystem/kaldi/sge/passwd b/docs/ecosystem/kaldi/sge/passwd
new file mode 100644
index 0000000..a39047d
--- /dev/null
+++ b/docs/ecosystem/kaldi/sge/passwd
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+root:x:0:0:root:/root:/bin/bash
+daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
+bin:x:2:2:bin:/bin:/usr/sbin/nologin
+sys:x:3:3:sys:/dev:/usr/sbin/nologin
+sync:x:4:65534:sync:/bin:/bin/sync
+games:x:5:60:games:/usr/games:/usr/sbin/nologin
+man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
+lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
+mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
+news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
+uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
+proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
+www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
+backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
+list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
+irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
+gnats:x:41:41:Gnats Bug-Reporting System
(admin):/var/lib/gnats:/usr/sbin/nologin
+nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
+systemd-timesync:x:100:102:systemd Time
Synchronization,,,:/run/systemd:/bin/false
+systemd-network:x:101:103:systemd Network
Management,,,:/run/systemd/netif:/bin/false
+systemd-resolve:x:102:104:systemd Resolver,,,:/run/systemd/resolve:/bin/false
+systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/false
+_apt:x:104:65534::/nonexistent:/bin/false
+etcd:x:1099:1099::/home/etcd:
+postfix:x:105:109::/var/spool/postfix:/bin/false
+sgeadmin:x:106:111::/var/lib/gridengine:/bin/false
+sshd:x:107:65534::/var/run/sshd:/usr/sbin/nologin
diff --git a/docs/ecosystem/kaldi/sge/resolv.conf
b/docs/ecosystem/kaldi/sge/resolv.conf
new file mode 100644
index 0000000..e51b279
--- /dev/null
+++ b/docs/ecosystem/kaldi/sge/resolv.conf
@@ -0,0 +1,23 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Default system properties included when running spark-submit.
+# This is useful for setting default environmental settings.
+
+nameserver 127.0.0.11
+nameserver < your RegistryDNSIP>
+options timeout:1 ndots:0
diff --git a/docs/ecosystem/kaldi/sge/sge_run.sh
b/docs/ecosystem/kaldi/sge/sge_run.sh
new file mode 100644
index 0000000..7b5e166
--- /dev/null
+++ b/docs/ecosystem/kaldi/sge/sge_run.sh
@@ -0,0 +1,84 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Execute command format : sge_run.sh %num_workers% %input_path%
+if [ $# -ne 2 ]
+then
+ echo "Parameter setting error!"
+ exit 1
+fi
+
+WORKER_NUMBER=$1
+CFG_PATH=$2
+
+HOST_NAME=$(hostname)
+JOB_NAME=`echo $HOST_NAME |awk -F "." '{print($2)}'`
+CLUSTER_USER=`echo $HOST_NAME |awk -F "." '{print($3)}'`
+DOMAIN_SUFFIX=`echo $HOST_NAME |awk -F "." '{for(i=4;i<=NF;i++){if(tostr ==
""){tostr=$i}else{tostr=tostr"."$i};if(i==NF)print tostr}}'`
+
+MASTER_HOST=""
+WORKER_HOST_STRS=""
+declare -a WORKER_HOST_LIST
+SLOTS="30"
+
+for ((num=0; num<$WORKER_NUMBER; num++))
+do
+ if [ $num -eq 0 ]
+ then
+ MASTER_HOST="master-0."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX
+ WORKER_HOST_STRS="master-0."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX
+
SLOTS=$SLOTS",[master-0."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX"=48]"
+ else
+ let tmpnum=$num-1
+
WORKER_HOST_STRS=$WORKER_HOST_STRS",worker-"$tmpnum"."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX
+
WORKER_HOST_LIST+=("worker-"$tmpnum"."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX)
+
SLOTS=$SLOTS",[worker-"$tmpnum"."$JOB_NAME"."$CLUSTER_USER"."$DOMAIN_SUFFIX"=48]"
+ fi
+done
+
+if [ $(echo $HOST_NAME |grep "^master-") ]
+then
+ sudo su - -c "sleep 30s && export DEBIAN_FRONTEND=noninteractive &&
apt-get update && \
+ apt-get install -y gridengine-master gridengine-exec
gridengine-client;apt-get autoremove -y && apt-get clean && \
+ /etc/init.d/gridengine-master start && /etc/init.d/gridengine-exec start"
+
+ sudo su - -s /bin/bash -c ". ${CFG_PATH}/gencfs.sh $WORKER_HOST_STRS
$SLOTS"
+ sudo su -c " qconf -Mc /tmp/qconf-mc.txt && qconf -Ae /tmp/qconf-ae.txt &&
qconf -as \`hostname\` &&
+ qconf -Ap /tmp/qconf-ap.txt && qconf -Aq /tmp/qconf-aq.txt && qconf -am
$CLUSTER_USER &&
+ echo finish master >> ${CFG_PATH}/setcfg.log "
+ for worker_num in ${WORKER_HOST_LIST[@]}
+ do
+ echo add $worker_num
+ sudo su -c " qconf -ah $worker_num && echo add worker node $worker_num
>> ${CFG_PATH}/setcfg.log "
+ done
+
+elif [ $(echo $HOST_NAME |grep "^worker-") ]
+then
+ sudo su - -s /bin/bash -c "sleep 2m && echo please wait && echo please
wait >> ${CFG_PATH}/setcfg.log"
+ sudo su - -c "export DEBIAN_FRONTEND=noninteractive && apt-get update && \
+ apt-get install -y gridengine-client gridengine-exec; apt-get autoremove
-y && apt-get clean"
+ sudo su - -c "echo $MASTER_HOST >
/var/lib/gridengine/default/common/act_qmaster"
+ sudo su - -c "/etc/init.d/gridengine-exec start && echo Start SGE for
worker is finished >> ${CFG_PATH}/setcfg.log"
+
+ sudo su - -s /bin/bash -c ". ${CFG_PATH}/gencfs.sh $WORKER_HOST_STRS
$SLOTS"
+ sudo su - -c "qconf -Me /tmp/qconf-ae.txt && echo done for $HOST_NAME
worker. >> ${CFG_PATH}/setcfg.log"
+
+else
+ echo "hostname doesn't match! should start with master or worker!" 1>&2
exit 1
+fi
+
+#sudo su - -s /bin/bash -c "echo sge start to finish >> ${CFG_PATH}/setcfg.log"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
