Hi All,
I am trying to use amazon memory optimized (R3) instances in the Oregon
region. I am getting 'Connection refused' during the SSH setup phase. I
tried using both root and ec2-user as user ids.
sh: connect to host ec2-54-185-57-74.us-west-2.compute.amazonaws.com port
22: Connection refused
Error executing remote command, retrying after 30 seconds: Command '['ssh',
'-o', 'StrictHostKeyChecking=no', '-i',
'/Users/usman/.ssh/ughani_imp_us_west_2.pem', '-t', '-t',
u'r...@ec2-54-185-57-74.us-west-2.compute.amazonaws.com', "\n [ -f
~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa
&&\n cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys)\n "]'
returned non-zero exit status 255
ssh: connect to host ec2-54-185-57-74.us-west-2.compute.amazonaws.com port
22: Connection refused
Generating cluster's SSH key on master...
ssh: connect to host ec2-54-185-58-135.us-west-2.compute.amazonaws.com port
22: Connection refused
Error executing remote command, retrying after 30 seconds: Command '['ssh',
'-o', 'StrictHostKeyChecking=no', '-i',
'/Users/usman/.ssh/ughani_imp_us_west_2.pem', '-t', '-t',
u'ec2-u...@ec2-54-185-58-135.us-west-2.compute.amazonaws.com', "\n [
-f ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa
&&\n cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys)\n "]'
returned non-zero exit status 255
ssh: connect to host ec2-54-185-58-135.us-west-2.compute.amazonaws.com port
22: Connection refused
Error executing remote command, retrying after 30 seconds: Command '['ssh',
'-o', 'StrictHostKeyChecking=no', '-i',
'/Users/usman/.ssh/ughani_imp_us_west_2.pem', '-t', '-t',
u'ec2-u...@ec2-54-185-58-135.us-west-2.compute.amazonaws.com', "\n [
-f ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa
&&\n cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys)\n "]'
returned non-zero exit status 255
ssh: connect to host ec2-54-185-58-135.us-west-2.compute.amazonaws.com port
22: Connection refused
Error executing remote command, retrying after 30 seconds: Command '['ssh',
'-o', 'StrictHostKeyChecking=no', '-i',
'/Users/usman/.ssh/ughani_imp_us_west_2.pem', '-t', '-t',
u'ec2-u...@ec2-54-185-58-135.us-west-2.compute.amazonaws.com', "\n [
-f ~/.ssh/id_rsa ] ||\n (ssh-keygen -q -t rsa -N '' -f ~/.ssh/id_rsa
&&\n cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys)\n "]'
returned non-zero exit status 255
Warning: Permanently added '
ec2-54-185-58-135.us-west-2.compute.amazonaws.com,54.185.58.135' (RSA) to
the list of known hosts.
Connection to ec2-54-185-58-135.us-west-2.compute.amazonaws.com closed.
Transferring cluster's SSH key to slaves...
ec2-54-185-57-126.us-west-2.compute.amazonaws.com
ssh: connect to host ec2-54-185-57-126.us-west-2.compute.amazonaws.com port
22: Connection refused
Error 255 while executing remote command, retrying after 30 seconds
ssh: connect to host ec2-54-185-57-126.us-west-2.compute.amazonaws.com port
22: Connection refused
Error 255 while executing remote command, retrying after 30 seconds
ssh: connect to host ec2-54-185-57-126.us-west-2.compute.amazonaws.com port
22: Connection refused
Error 255 while executing remote command, retrying after 30 seconds
Warning: Permanently added '
ec2-54-185-57-126.us-west-2.compute.amazonaws.com,54.185.57.126' (RSA) to
the list of known hosts.
ec2-54-188-56-91.us-west-2.compute.amazonaws.com
Warning: Permanently added
'ec2-54-188-56-91.us-west-2.compute.amazonaws.com,54.188.56.91'
(RSA) to the list of known hosts.
ec2-54-188-61-59.us-west-2.compute.amazonaws.com
Warning: Permanently added
'ec2-54-188-61-59.us-west-2.compute.amazonaws.com,54.188.61.59'
(RSA) to the list of known hosts.
ec2-54-188-21-245.us-west-2.compute.amazonaws.com
Warning: Permanently added '
ec2-54-188-21-245.us-west-2.compute.amazonaws.com,54.188.21.245' (RSA) to
the list of known hosts.
Cloning into 'spark-ec2'...
remote: Counting objects: 1407, done.
remote: Compressing objects: 100% (673/673), done.
remote: Total 1407 (delta 457), reused 1398 (delta 454)
Receiving objects: 100% (1407/1407), 219.00 KiB | 340 KiB/s, done.
Resolving deltas: 100% (457/457), done.
Connection to ec2-54-185-58-135.us-west-2.compute.amazonaws.com closed.
Deploying files to master...
WARNING: Don't know number of disks on instance type r3.large; assuming 1
building file list ... done
root/spark-ec2/
rsync: recv_generator: mkdir "/root/spark-ec2" failed: Permission denied
(13)
*** Skipping any contents from this failed directory ***
sent 101 bytes received 26 bytes 254.00 bytes/sec
total size is 1617 speedup is 12.73
rsync error: some files could not be transferred (code 23) at
/SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
Traceback (most recent call last):
File "./spark_ec2.py", line 822, in <module>
main()
File "./spark_ec2.py", line 814, in main
real_main()
File "./spark_ec2.py", line 699, in real_main
setup_cluster(conn, master_nodes, slave_nodes, opts, True)
File "./spark_ec2.py", line 449, in setup_cluster
deploy_files(conn, "deploy.generic", opts, master_nodes, slave_nodes,
modules)
File "./spark_ec2.py", line 592, in deploy_files
subprocess.check_call(command)
File
"/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py",
line 540, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['rsync', '-rv', '-e', 'ssh -o
StrictHostKeyChecking=no -i /Users/usman/.ssh/ughani_imp_us_west_2.pem',
'/var/folders/4f/k2fj1nbs7_17h2dnvttprjw00000gp/T/tmpyXTWlv/',
u'ec2-u...@ec2-54-185-58-135.us-west-2.compute.amazonaws.com:/']' returned
non-zero exit status 23
