Riccardo,
I think there might still be a bug, however it might be caused by this new update. I pulled in the latest commit, and it seems to be reading the HostName entry from ~/.ssh/config now. However, I think there is a type conversion issue when reading in the string from ~/.ssh/config. I tried to track it down myself, but it was late and I got a bit lost =). Here is the traceback (note this happens for IPs and FQDNs, I’ve swapped all the ones in the traceback with placeholders for privacy reasons): gservers: [2017-03-29 21:28:57] INFO : Starting gservers at Wed Mar 29 21:28:57 2017; invoked as '/Users/Kyle/gc3pie/bin/gservers -vv' gservers: [2017-03-29 21:28:57] DEBUG : Configuration.load(): File '/etc/gc3/gc3pie.conf' does not exist, ignoring. gservers: [2017-03-29 21:28:57] DEBUG : Configuration.load(): File '/Users/Kyle/gc3pie/etc/gc3/gc3pie.conf' does not exist, ignoring. gservers: [2017-03-29 21:28:57] DEBUG : Configuration.merge_file(): Reading file '/Users/Kyle/.gc3/gc3pie.conf' ... gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Read configuration stanza for auth 'none' gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Read configuration stanza for auth 'workstations' gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Read configuration stanza for resource 'localhost'. gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Resource 'localhost' defined by: architecture=set(['x86_64']), auth='none', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(4, unit=hour), name='localhost', override='true', transport='local', type='shellcmd'. gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Read configuration stanza for resource 'dell'. gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Resource 'dell' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, frontend='dell', max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='dell', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Read configuration stanza for resource 'lenovo'. gservers: [2017-03-29 21:28:57] DEBUG : Config._parse(): Resource 'lenovo' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, frontend='lenovo', max_cores=6, max_cores_per_job=6, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='lenovo', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-29 21:28:57] DEBUG : Creating resource 'lenovo' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, frontend='lenovo', max_cores=6, max_cores_per_job=6, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='lenovo', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-29 21:28:57] DEBUG : Using class <class 'gc3libs.backends.shellcmd.ShellcmdLrms'> from module <module 'gc3libs.backends.shellcmd' from '/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.pyc'> to instanciate resources of type shellcmd gservers: [2017-03-29 21:28:57] INFO : Computational resource 'lenovo' initialized successfully. gservers: [2017-03-29 21:28:57] DEBUG : Creating resource 'dell' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, frontend='dell', max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='dell', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-29 21:28:57] INFO : Computational resource 'dell' initialized successfully. gservers: [2017-03-29 21:28:57] DEBUG : Creating resource 'localhost' defined by: architecture=set(['x86_64']), auth='none', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(4, unit=hour), name='localhost', override='true', transport='local', type='shellcmd'. gservers: [2017-03-29 21:28:57] INFO : Computational resource 'localhost' initialized successfully. gservers: [2017-03-29 21:28:57] DEBUG : Opening SshTransport... gservers: [2017-03-29 21:28:58] DEBUG : Using no ProxyCommand for SSH connections. gservers: [2017-03-29 21:28:58] DEBUG : Connecting to host ‘ BLAHBLAH.something.org <http://blahblah.something.org/>' (port 22) as user ‘A_PERSON' via SSH (timeout 30s)... gservers: [2017-03-29 21:28:59] ERROR : Could not create ssh connection to BLAHBLAH.something.org <http://blahblah.something.org/>: TypeError: %d format: a number is required, not str gservers: [2017-03-29 21:28:59] ERROR : Ignoring error updating resource 'lenovo': Failed connecting to remote host 'BLAHBLAH.something.org <http://blahblah.something.org/>': %d format: a number is required, not str. gservers: [2017-03-29 21:28:59] DEBUG : Got error 'TransportError' in updating resource 'lenovo'; printing full traceback. Traceback (most recent call last): File "/Users/Kyle/gc3pie/src/gc3libs/core.py", line 831, in update_resources lrms.get_resource_status() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 700, in get_resource_status self._gather_machine_specs() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 499, in _gather_machine_specs self.transport.connect() File "/Users/Kyle/gc3pie/src/gc3libs/backends/transport.py", line 626, in connect .format(hostname=self.remote_frontend, msg=ex)) TransportError: Failed connecting to remote host 'BLAHBLAH.something.org <http://blahblah.something.org/>': %d format: a number is required, not str gservers: [2017-03-29 21:28:59] DEBUG : Opening SshTransport... gservers: [2017-03-29 21:28:59] DEBUG : Using no ProxyCommand for SSH connections. gservers: [2017-03-29 21:28:59] DEBUG : Connecting to host '1.2.3.4' (port 22) as user 'A_PERSON' via SSH (timeout 30s)... gservers: [2017-03-29 21:29:00] ERROR : Could not create ssh connection to 1.2.3.4: TypeError: %d format: a number is required, not str gservers: [2017-03-29 21:29:00] ERROR : Ignoring error updating resource 'dell': Failed connecting to remote host '1.2.3.4': %d format: a number is required, not str. gservers: [2017-03-29 21:29:00] DEBUG : Got error 'TransportError' in updating resource 'dell'; printing full traceback. Traceback (most recent call last): File "/Users/Kyle/gc3pie/src/gc3libs/core.py", line 831, in update_resources lrms.get_resource_status() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 700, in get_resource_status self._gather_machine_specs() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 499, in _gather_machine_specs self.transport.connect() File "/Users/Kyle/gc3pie/src/gc3libs/backends/transport.py", line 626, in connect .format(hostname=self.remote_frontend, msg=ex)) TransportError: Failed connecting to remote host '1.2.3.4': %d format: a number is required, not str gservers: [2017-03-29 21:29:00] DEBUG : Opening LocalTransport... gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'echo "$HOME/.gc3/shellcmd.d"', got exit status: 0 gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'uname -m', got exit status: 0 gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'uname -s', got exit status: 0 gservers: [2017-03-29 21:29:00] DEBUG : Checking if GNU time is available as command `time` gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'command time --version 2>&1 | grep GNU', got exit status: 0 gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'sysctl hw.ncpu', got exit status: 0 gservers: [2017-03-29 21:29:00] DEBUG : Executed local command 'sysctl hw.memsize', got exit status: 0 gservers: [2017-03-29 21:29:00] INFO : Mismatch of value `max_cores` on resource 'localhost': configuration file says `max_cores=4` while it's actually `8`. Updating current value. gservers: [2017-03-29 21:29:00] INFO : Mismatch of value `max_memory_per_core` on resource localhost: configuration file says `max_memory_per_core=4GB` while it's actually `17179MB`. Updating current value. If I try completely commenting out the `frontend` entry in ~/.gc3/gc3pie.conf for one of my remote hosts, I get the following traceback: gservers: [2017-03-30 08:35:06] INFO : Starting gservers at Thu Mar 30 08:35:06 2017; invoked as '/Users/Kyle/gc3pie/bin/gservers -vvv' gservers: [2017-03-30 08:35:06] DEBUG : Configuration.load(): File '/etc/gc3/gc3pie.conf' does not exist, ignoring. gservers: [2017-03-30 08:35:06] DEBUG : Configuration.load(): File '/Users/Kyle/gc3pie/etc/gc3/gc3pie.conf' does not exist, ignoring. gservers: [2017-03-30 08:35:06] DEBUG : Configuration.merge_file(): Reading file '/Users/Kyle/.gc3/gc3pie.conf' ... gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Read configuration stanza for auth 'none' gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Read configuration stanza for auth 'workstations' gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Read configuration stanza for resource 'localhost'. gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Resource 'localhost' defined by: architecture=set(['x86_64']), auth='none', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(4, unit=hour), name='localhost', override='true', transport='local', type='shellcmd'. gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Read configuration stanza for resource 'dell'. gservers: [2017-03-30 08:35:06] DEBUG : Config._parse(): Resource 'dell' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='dell', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-30 08:35:06] DEBUG : Creating resource 'dell' defined by: architecture=set(['x86_64']), auth='workstations', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(48, unit=hour), name='dell', override='true', ssh_config='~/.ssh/config', transport='ssh', type='shellcmd'. gservers: [2017-03-30 08:35:07] DEBUG : Using class <class 'gc3libs.backends.shellcmd.ShellcmdLrms'> from module <module 'gc3libs.backends.shellcmd' from '/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.pyc'> to instanciate resources of type shellcmd gservers: [2017-03-30 08:35:07] INFO : Computational resource 'dell' initialized successfully. gservers: [2017-03-30 08:35:07] DEBUG : Creating resource 'localhost' defined by: architecture=set(['x86_64']), auth='none', enabled=True, max_cores=4, max_cores_per_job=4, max_memory_per_core=Memory(4, unit=GB), max_walltime=Duration(4, unit=hour), name='localhost', override='true', transport='local', type='shellcmd'. gservers: [2017-03-30 08:35:07] INFO : Computational resource 'localhost' initialized successfully. gservers: [2017-03-30 08:35:07] DEBUG : Opening SshTransport... gservers: [2017-03-30 08:35:07] DEBUG : Using no ProxyCommand for SSH connections. gservers: [2017-03-30 08:35:07] DEBUG : Connecting to host 'localhost' (port 22) as user 'None' via SSH (timeout 30s)... gservers: [2017-03-30 08:35:07] ERROR : Could not create ssh connection to localhost: AuthenticationException: Authentication failed. gservers: [2017-03-30 08:35:07] ERROR : ssh-agent is running but no key has been added. Please add a key with `ssh-add` command. gservers: [2017-03-30 08:35:07] ERROR : Ignoring error updating resource 'dell': Failed connecting to remote host 'localhost': Authentication failed.. gservers: [2017-03-30 08:35:07] DEBUG : Got error 'TransportError' in updating resource 'dell'; printing full traceback. Traceback (most recent call last): File "/Users/Kyle/gc3pie/src/gc3libs/core.py", line 831, in update_resources lrms.get_resource_status() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 700, in get_resource_status self._gather_machine_specs() File "/Users/Kyle/gc3pie/src/gc3libs/backends/shellcmd.py", line 499, in _gather_machine_specs self.transport.connect() File "/Users/Kyle/gc3pie/src/gc3libs/backends/transport.py", line 626, in connect .format(hostname=self.remote_frontend, msg=ex)) TransportError: Failed connecting to remote host 'localhost': Authentication failed. Some weirdness is happening here. Perhaps the error messages are just mixing up names, but its seems to be simultaneously referring to "localhost" and "dell" when it fails with the AuthenticationError. 90% sure this isn't a misconfiguration on my part, as auth=None and transport=local in [resource/localhost]. Best, Kyle On Wednesday, March 29, 2017 at 4:46:51 PM UTC-4, Riccardo Murri wrote: > > Hi Kyle, > > the bug was indeed GC3Pie's and should be fixed in commit d1e3aa9, > latest on the "master" branch. > Can you please try it out and tell if it works for you? > > Ciao, > R > -- You received this message because you are subscribed to the Google Groups "gc3pie" group. To unsubscribe from this group and stop receiving emails from it, send an email to gc3pie+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
