Hello,
I'm trying elasticluster and, after figuring out a few things, thought to
be ready to deploy a tiny gce cluster. However, the deployment command
(elasticluster -c /home/odyhpc/elasticluster/ody.d/config start odycluster)
is returning the following chain of errors:
(elasticluster) odyhpc@deployator0002:~/elasticluster$ elasticluster -c
/home/odyhpc/elasticluster/ody.d/config start odycluster
Starting cluster `odycluster` with:
* 1 frontend nodes.
* 2 compute nodes.
(This may take a while...)
2020-08-10 18:08:10 deployator0002 elasticluster[3768] WARNING
DeprecationWarning: Using or importing the ABCs from 'collections' instead
of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it
will stop working
2020-08-10 18:08:10 deployator0002 elasticluster[3768] WARNING UserWarning:
Cannot access
/home/odyhpc/.elasticluster/storage/423015455368-58j4u4st0f38a5ambe6q5bamq6k8d7ha.apps.googleusercontent.com.oauth.dat:
No such file or directory
2020-08-10 18:08:10 deployator0002 elasticluster[3768] WARNING
ResourceWarning: unclosed <socket.socket fd=6,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6,
laddr=('10.128.0.36', 52210), raddr=('169.254.169.254', 80)>
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Error creating
instance `<HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Could not
start node `compute002`: Error creating instance `<HttpError 403 when
requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">` -- <class
'elasticluster.exceptions.InstanceError'>
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 525, in start_instance
response = self._execute_request(request)
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 208, in _execute_request
return request.execute(http=self._auth_http)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/_helpers.py",
line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/http.py",
line 907, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line 580,
in _start_node
node.start()
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line
1312, in start
vm_data = self._cloud_provider.start_instance(
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 531, in start_instance
raise InstanceError("Error creating instance `%s`" % e)
elasticluster.exceptions.InstanceError: Error creating instance `<HttpError
403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Error creating
instance `<HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Could not
start node `frontend001`: Error creating instance `<HttpError 403 when
requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">` -- <class
'elasticluster.exceptions.InstanceError'>
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 525, in start_instance
response = self._execute_request(request)
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 208, in _execute_request
return request.execute(http=self._auth_http)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/_helpers.py",
line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/http.py",
line 907, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line 580,
in _start_node
node.start()
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line
1312, in start
vm_data = self._cloud_provider.start_instance(
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 531, in start_instance
raise InstanceError("Error creating instance `%s`" % e)
elasticluster.exceptions.InstanceError: Error creating instance `<HttpError
403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Error creating
instance `<HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Could not
start node `compute001`: Error creating instance `<HttpError 403 when
requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">` -- <class
'elasticluster.exceptions.InstanceError'>
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 525, in start_instance
response = self._execute_request(request)
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 208, in _execute_request
return request.execute(http=self._auth_http)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/_helpers.py",
line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File
"/home/odyhpc/elasticluster/lib/python3.8/site-packages/googleapiclient/http.py",
line 907, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line 580,
in _start_node
node.start()
File "/home/odyhpc/elasticluster/src/elasticluster/cluster.py", line
1312, in start
vm_data = self._cloud_provider.start_instance(
File "/home/odyhpc/elasticluster/src/elasticluster/providers/gce.py",
line 531, in start_instance
raise InstanceError("Error creating instance `%s`" % e)
elasticluster.exceptions.InstanceError: Error creating instance `<HttpError
403 when requesting
https://compute.googleapis.com/compute/v1/projects/watchful-origin-244417/zones/us-central1-a/instances?alt=json
returned
"Request had insufficient authentication scopes.">`
2020-08-10 18:08:11 deployator0002 elasticluster[3768] WARNING
ResourceWarning: unclosed running multiprocessing pool
<multiprocessing.pool.ThreadPool state=RUN pool_size=3>
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Could not
start cluster `odycluster`: No nodes could be started!
2020-08-10 18:08:11 deployator0002 elasticluster[3768] ERROR Error: No
nodes could be started!
Aborting because of errors: No nodes could be started!.
2020-08-10 18:08:11 deployator0002 elasticluster[3768] WARNING
ResourceWarning: unclosed <ssl.SSLSocket fd=7,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6,
laddr=('10.128.0.36', 54746), raddr=('173.194.197.95', 443)>
2020-08-10 18:08:11 deployator0002 elasticluster[3768] WARNING
ResourceWarning: unclosed <socket.socket fd=6,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6,
laddr=('10.128.0.36', 52212), raddr=('169.254.169.254', 80)>
2020-08-10 18:08:11 deployator0002 elasticluster[3768] WARNING
ResourceWarning: unclosed <ssl.SSLSocket fd=8,
family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6,
laddr=('10.128.0.36', 55886), raddr=('108.177.112.95', 443)>
The error (Request had insufficient authentication scopes) seems to be
relatively general but it's unclear (to me) which is the seed problem
before everything cascades down. One question that cropped up even before
this was about the keys: I had not previously generated the gce_client_id
or gce_client_secret keys; the thing is that, once on the OAuth 2.0 Client
ID section, the system offered me 7 choices for application type but none
of them fitted elasticluster's profile. I chose desktop app (as it was
probably be the closest) but I am still wondering if elasticluster requires
a specific type or if this could be in any way related to the said errors.
I'm also attaching a redacted copy of my config file in case it's useful.
Thanks
--
You received this message because you are subscribed to the Google Groups
"elasticluster" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticluster/6d9cecb2-b083-4ac7-bac0-d5818c2b1b27o%40googlegroups.com.
# GridEngine software to be configured by Ansible
#
# (There is nothing Google-specific in the "setup" section; in fact, it can be
# re-used verbatim with *any* cloud provider or base image)
[setup/gridengine]
provider=ansible
frontend_groups=gridengine_master
compute_groups=gridengine_worker
# Create a cloud provider (call it "google-cloud")
[cloud/google-cloud]
provider=google
gce_project_id=##########
gce_client_id=##########
gce_client_secret=##########
# Create a login (call it "google-login")
#
# In contrast to other cloud providers, GCE creates a personal account on each
# VM so you effectively re-use the same `[login/google]` section across
# different VM images.
[login/ody-login]
image_user=##########
image_user_sudo=root
image_sudo=True
user_key_name=id_rsa
# You can generate the keypair with the command: `gcloud compute config-ssh`
user_key_private=~/.ssh/id_rsa
user_key_public=~/.ssh/id_rsa.pub
# Bring all of the elements together to define a cluster called "odycluster"
[cluster/odycluster]
cloud=google-cloud
login=ody-login
setup=gridengine
security_group=default
image_id=https://console.cloud.google.com/compute/imagesDetail/projects/##########
flavor=n1-standard-1
ssh_to=frontend
frontend_nodes=1
# adjust size of cluster to taste
compute_nodes=2
