Dear Ricardo, As you recommended, I tested the command `sudo ./elasticluster.sh -vvv start slurm-on-gce` on Ubuntu 16.04. Output and config file are attached. I think I am missing something as SLURM was not installed on Ubuntu machine either. It is supposed to be auto installed, not by me, right?
Orhan On Sat, Feb 3, 2018 at 11:07 PM, Riccardo Murri <[email protected]> wrote: > Dear Orxan, > > I have just committed to branch "master" a fix for building SLURM clusters > on Debian 9. Your config shows that you are using a custom image, though, > which I cannot access; if a SLURM cluster does not build there, please > post the complete output of `elasticluster -vvv start slurm-on-gce`. > > Ciao, > R > -- You received this message because you are subscribed to the Google Groups "elasticluster" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
orhan@orhan-MS-7850:~$ sudo ./elasticluster.sh -vvv start slurm-on-gce
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`cluster/slurm-on-gce` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`cluster/gridengine-on-gce` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`login/google` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`setup/gridengine` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`setup/slurm` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`setup/pbs` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Checking section
`cloud/google` ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Using class <class
'elasticluster.providers.gce.GoogleCloudProvider'> from module <module
'elasticluster.providers.gce' from 'elasticluster/providers/gce.py'> to
instanciate provider 'google'
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Using class <class
'elasticluster.providers.ansible_provider.AnsibleSetupProvider'> from module
<module 'elasticluster.providers.ansible_provider' from
'elasticluster/providers/ansible_provider.py'> to instanciate provider 'ansible'
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG setting variable
multiuser_cluster=yes for node kind compute
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG setting variable
multiuser_cluster=yes for node kind frontend
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG setting variable
multiuser_cluster=yes for node kind submit
Starting cluster `slurm-on-gce` with:
* 1 frontend nodes.
* 2 compute nodes.
(This may take a while...)
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] INFO Starting cluster
nodes ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG Note: starting 3
nodes concurrently.
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG _start_node:
working on node `frontend001`
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG _start_node:
working on node `compute001`
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] DEBUG _start_node:
working on node `compute002`
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] INFO Starting node
`frontend001` from image `ubuntu-1604-xenial-v20180126` with flavor
n1-standard-1 ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] INFO Starting node
`compute001` from image `ubuntu-1604-xenial-v20180126` with flavor
n1-standard-1 ...
2018-02-04 13:10:11 42c5751151a1 gc3.elasticluster[1] INFO Starting node
`compute002` from image `ubuntu-1604-xenial-v20180126` with flavor
n1-standard-1 ...
2018-02-04 13:10:23 42c5751151a1 gc3.elasticluster[1] DEBUG Node `compute001`
has instance ID `slurm-on-gce-compute001`
2018-02-04 13:10:23 42c5751151a1 gc3.elasticluster[1] INFO Node `compute001`
has been started.
2018-02-04 13:10:26 42c5751151a1 gc3.elasticluster[1] DEBUG Node `frontend001`
has instance ID `slurm-on-gce-frontend001`
2018-02-04 13:10:26 42c5751151a1 gc3.elasticluster[1] INFO Node `frontend001`
has been started.
2018-02-04 13:10:43 42c5751151a1 gc3.elasticluster[1] DEBUG Node `compute002`
has instance ID `slurm-on-gce-compute002`
2018-02-04 13:10:43 42c5751151a1 gc3.elasticluster[1] INFO Node `compute002`
has been started.
2018-02-04 13:10:43 42c5751151a1 gc3.elasticluster[1] DEBUG Getting information
for instance slurm-on-gce-frontend001
2018-02-04 13:10:43 42c5751151a1 gc3.elasticluster[1] DEBUG node `frontend001`
(instance id slurm-on-gce-frontend001) is up.
2018-02-04 13:10:44 42c5751151a1 gc3.elasticluster[1] DEBUG Getting information
for instance slurm-on-gce-compute002
2018-02-04 13:10:44 42c5751151a1 gc3.elasticluster[1] DEBUG node `compute002`
(instance id slurm-on-gce-compute002) is up.
2018-02-04 13:10:44 42c5751151a1 gc3.elasticluster[1] DEBUG Getting information
for instance slurm-on-gce-compute001
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG node `compute001`
(instance id slurm-on-gce-compute001) is up.
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] INFO Checking SSH
connection to nodes ...
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host frontend001 (35.229.16.10)
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG Host frontend001
(35.229.16.10) not reachable: [Errno None] Unable to connect to port 22 on
35.229.16.10.
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host compute002 (35.196.87.44)
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG Host compute002
(35.196.87.44) not reachable: [Errno None] Unable to connect to port 22 on
35.196.87.44.
2018-02-04 13:10:45 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host compute001 (35.227.66.128)
2018-02-04 13:10:46 42c5751151a1 gc3.elasticluster[1] DEBUG Host compute001
(35.227.66.128) not reachable: [Errno None] Unable to connect to port 22 on
35.227.66.128.
2018-02-04 13:10:56 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host frontend001 (35.229.16.10)
2018-02-04 13:10:57 42c5751151a1 gc3.elasticluster[1] DEBUG Connection to
35.229.16.10 succeeded on port 22!
2018-02-04 13:10:57 42c5751151a1 gc3.elasticluster[1] DEBUG Setting
`preferred_ip` to 35.229.16.10
2018-02-04 13:10:57 42c5751151a1 gc3.elasticluster[1] INFO Connection to node
`frontend001` successful, using IP address 35.229.16.10 to connect.
2018-02-04 13:10:57 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host compute002 (35.196.87.44)
2018-02-04 13:10:59 42c5751151a1 gc3.elasticluster[1] DEBUG Connection to
35.196.87.44 succeeded on port 22!
2018-02-04 13:10:59 42c5751151a1 gc3.elasticluster[1] DEBUG Setting
`preferred_ip` to 35.196.87.44
2018-02-04 13:10:59 42c5751151a1 gc3.elasticluster[1] INFO Connection to node
`compute002` successful, using IP address 35.196.87.44 to connect.
2018-02-04 13:10:59 42c5751151a1 gc3.elasticluster[1] DEBUG Trying to connect
to host compute001 (35.227.66.128)
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Connection to
35.227.66.128 succeeded on port 22!
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Setting
`preferred_ip` to 35.227.66.128
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] INFO Connection to node
`compute001` successful, using IP address 35.227.66.128 to connect.
Configuring the cluster.
(this too may take a while...)
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Writing Ansible
inventory to file `/home/orhan/.elasticluster/storage/slurm-on-gce.inventory`
...
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Calling
`ansible-playbook` with the following environment:
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - ANSIBLE_FORKS='10'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
ANSIBLE_HOST_KEY_CHECKING='no'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
ANSIBLE_PRIVATE_KEY_FILE='/home/orhan/.ssh/google_compute_engine'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
ANSIBLE_ROLES_PATH='elasticluster/share/playbooks/roles:elasticluster/share/playbooks:/etc/ansible/roles'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
ANSIBLE_SSH_PIPELINING='yes'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
ANSIBLE_TIMEOUT='120'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - DISPLAY=':0'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
GPG_KEY='C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - HOME='/home/orhan'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
HOSTNAME='42c5751151a1'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - LANG='en_US.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - LANGUAGE='en_US'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_ADDRESS='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_IDENTIFICATION='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_MEASUREMENT='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_MONETARY='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_NAME='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_NUMERIC='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_PAPER='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_TELEPHONE='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LC_TIME='tr_TR.UTF-8'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - LOGNAME='root'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
MAIL='/var/mail/root'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - PWD='/home/orhan'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
PYTHON_PIP_VERSION='9.0.1'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
PYTHON_VERSION='2.7.14'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - SHELL='/bin/bash'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
SSH_AUTH_SOCK='/home/.ssh-agent.sock'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
SUDO_COMMAND='./elasticluster.sh -vvv start slurm-on-gce'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - SUDO_GID='1000'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - SUDO_UID='1000'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - SUDO_USER='orhan'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
TERM='xterm-256color'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - USER='root'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG - USERNAME='root'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG -
XAUTHORITY='/home/orhan/.Xauthority'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Using playbook file
elasticluster/share/playbooks/site.yml.
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] DEBUG Running Ansible
command `ansible-playbook /home/elasticluster/share/playbooks/site.yml
--inventory=/home/orhan/.elasticluster/storage/slurm-on-gce.inventory --become
--become-user=root -vv` ...
'import sitecustomize' failed; use -v for traceback
Traceback (most recent call last):
File "/usr/local/bin/ansible-playbook", line 43, in <module>
import ansible.constants as C
File "/usr/local/lib/python2.7/site-packages/ansible/constants.py", line 202,
in <module>
DEFAULT_LOCAL_TMP = get_config(p, DEFAULTS, 'local_tmp',
'ANSIBLE_LOCAL_TEMP', '~/.ansible/tmp', value_type='tmppath')
File "/usr/local/lib/python2.7/site-packages/ansible/constants.py", line 109,
in get_config
makedirs_safe(value, 0o700)
File "/usr/local/lib/python2.7/site-packages/ansible/utils/path.py", line 71,
in makedirs_safe
raise AnsibleError("Unable to create local directories(%s): %s" %
(to_native(rpath), to_native(e)))
ansible.errors.AnsibleError: Unable to create local
directories(/home/.ansible/tmp): [Errno 13] Permission denied: '/home/.ansible'
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] ERROR Command
`ansible-playbook /home/elasticluster/share/playbooks/site.yml
--inventory=/home/orhan/.elasticluster/storage/slurm-on-gce.inventory --become
--become-user=root -vv` failed with exit code 1.
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] ERROR Check the output
lines above for additional information on this error.
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] ERROR The cluster has
likely *not* been configured correctly. You may need to re-run `elasticluster
setup` or fix the playbooks.
2018-02-04 13:11:01 42c5751151a1 gc3.elasticluster[1] WARNING Cluster
`slurm-on-gce` not yet configured. Please, re-run `elasticluster setup
slurm-on-gce` and/or check your configuration
WARNING: YOUR CLUSTER IS NOT READY YET!
Cluster name: slurm-on-gce
Cluster template: slurm-on-gce
Default ssh to node: frontend001
- frontend nodes: 1
- compute nodes: 2
To login on the frontend node, run the command:
elasticluster ssh slurm-on-gce
To upload or download files to the cluster, use the command:
elasticluster sftp slurm-on-gce
config
Description: Binary data
