[Openstack] VlanManager issues...
I'm setting up a modest cloud for internal use. Currently I have the following
configuration:
Server1 (nova) is running all nova services EXCEPT nova compute.
Server 2 (nova1) is running only nova-compute.
Servers3-12 are idle, but will eventually run nova-compute
Server 1 is configured with public_interface eth0 and vlan_interface eth1. The
nova.conf file is as follows
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--s3_host=173.23.181.1
--rabbit_host=173.23.181.1
--cc_host=173.23.181.1
--ec2_url=http://173.23.181.1:8773/services/Cloud
--nova_url=http://173.23.181.1:8774/v1.1/
--fixed_range=192.168.64.0/18
--network_size=8
--FAKE_subdomain=cloud
--routing_source_ip=173.23.181.1
--sql_connection=mysql://nova:nova@173.23.181.1/nova
--glance_api_servers=192.168.10.1:9292
--public_interface=eth0
--vlan_interface=eth1
--use_deprecated_auth
--iscsi_ip_prefix=192.168.10.
Server 2 is configured with vlan_interface eth3. The nova.conf file is
practically identical to Server1 and is as follows
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--s3_host=173.23.181.1
--rabbit_host=173.23.181.1
--cc_host=173.23.181.1
--ec2_url=http://173.23.181.1:8773/services/Cloud
--nova_url=http://173.23.181.1:8774/v1.1/
--fixed_range=192.168.64.0/18
--network_size=8
--FAKE_subdomain=cloud
--routing_source_ip=173.23.181.1
--sql_connection=mysql://nova:nova@173.23.181.1/nova
--glance_api_servers=192.168.10.1:9292
--vlan_interface=eth3
--use_deprecated_auth
--iscsi_ip_prefix=192.168.10.
I've setup 1 private network 192.168.64.0/19.
I'm seeing the following behavior and wonder if this is a bug. When starting
an instance, if the bridge interface does not exist on Server2 (br100), the
instance fails to start and the nova-compute.log file shows the following error:
2012-01-25 15:59:26,644 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/exception.py, line 98, in w
rapped
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py,
line 672, in spawn
(nova.exception): TRACE: block_device_info=block_device_info)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py,
line 1272, in to_xml
(nova.exception): TRACE: block_device_info)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py,
line 1170, in _prepare_xml_info
(nova.exception): TRACE: nics.append(self.vif_driver.plug(instance,
network, mapping))
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py, line 8
6, in plug
(nova.exception): TRACE: network['bridge_interface'])
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/network/linux_net.py, line
889, in ensure_vlan_bridge
(nova.exception): TRACE: bridge_interface, mac_address)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/utils.py, line 688, in inne
r
(nova.exception): TRACE: retval = f(*args, **kwargs)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/network/linux_net.py, line
909, in ensure_vlan
(nova.exception): TRACE: _execute('ip', 'link', 'set', interface, 'up',
run_as_root=True)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/network/linux_net.py, line
745, in _execute
(nova.exception): TRACE: return utils.execute(*cmd, **kwargs)
(nova.exception): TRACE: File
/usr/lib/python2.7/dist-packages/nova/utils.py, line 191, in exec
ute
(nova.exception): TRACE: cmd=' '.join(cmd))
(nova.exception): TRACE: ProcessExecutionError: Unexpected error while running
command.
(nova.exception): TRACE: Command: sudo ip link set vlan100 up
(nova.exception): TRACE: Exit code: 2
(nova.exception): TRACE: Stdout: ''
(nova.exception): TRACE: Stderr: 'RTNETLINK answers: Network is down\n'
(nova.exception): TRACE:
2012-01-25 15:59:26,645 ERROR nova.compute.manager [-] Instance '11' failed to
spawn. Is virtualiza
tion enabled in the BIOS? Details: Unexpected error while running command.
Command: sudo ip link set vlan100 up
Exit code: 2
Stdout: ''
Stderr: 'RTNETLINK answers: Network is down\n'
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File
/usr/lib/python2.7/dist-packages/nova/compute/manager.py, l
ine 424, in _run_instance
(nova.compute.manager): TRACE: network_info, block_device_info)
(nova.compute.manager): TRACE: File
/usr/lib/python2.7/dist-packages/nova/exception.py, line 12
9, in wrapped
(nova.compute.manager): TRACE: raise Error(str(e))
(nova.compute.manager):
Re: [Openstack] Glance authentication with Keystone woes...
Hi Jay,
Yes, this confused me, however I get the same error using the token I generated
and added (via the keystone-manage command). To wit:
root@nova:~# keystone-manage token list
token userexpiration tenant
---
101112131415161718191 2022-01-01 00:00:00 2
fa89fb9a-60d2-4921-b12d-6aee1c1778231 2012-02-01 15:24:33 1
b06c5e4e-5e59-4293-aa54-ce6879f113712 2012-02-01 15:26:41 1
where the first token is the long-lived one I supplied during installation.
Running the glance command yields identical results:
root@nova:~# glance -v -A 10111213141516171819 details
Failed to show details. Got error:
Internal Server error: Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/eventlet/wsgi.py, line 336, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 208, in call_func
return self.func(req, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/glance/common/wsgi.py, line 113, in
__call__
response = req.get_response(self.application)
File /usr/lib/python2.7/dist-packages/webob/request.py, line 1053, in
get_response
application, catch_exc_info=False)
File /usr/lib/python2.7/dist-packages/webob/request.py, line 1022, in
call_application
app_iter = application(self.environ, start_response)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 208, in call_func
return self.func(req, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/glance/common/wsgi.py, line 110, in
__call__
response = self.process_request(req)
File /usr/lib/python2.7/dist-packages/glance/common/context.py, line 104,
in process_request
raise exception.NotAuthorized()
NotAuthorized: None
Completed in 0.0031 sec.
Interestingly (perhaps) I see nothing in the keystone.log file. In fact, I
don't even see the keystone log file. Keystone opens to log files named
'admin.log' and 'keystone_legacy_auth.log'. Is this right? Also, if I run
keystone interactively (keystone -v -d) then issue the glance command, I see
nothing in the keystone window. This doesn't seem right to me, but I'm just
getting started with keystone integration.
Thanks in advance for any insight...
Regards,
Ross
On Jan 31, 2012, at 6:48 PM, Jay Pipes wrote:
On 01/31/2012 06:28 PM, Lillie Ross-CDSR11 wrote:
I'm reinstalling the various Openstack services from packages in the
ManagedIT PPA to pull in the latest Diablo bug fixes. I'm following the
latest directions in the newly release installation guide as I perform
these upgrades
(http://docs.openstack.org/diablo/openstack-compute/install/content/index.html).
However, I'm having trouble getting Glance to authenticate with
Keystone. All config files have been copied from the examples posted in
the installation guide (and modified accordingly for my admin token, IP
addresses, etc.). Regardless, I continually get the following error
message and stack dump when trying to verify the Glance/Keystone
integration:
Step 1: Grab a token
# curl -d '{auth: {tenantName: default,
passwordCredentials:{username: admin, password: admin}}}' -H
Content-type: application/json http://173.23.181.1:35357/v2.0/tokens |
python -mjson.tool
...
token: {
expires: 2012-02-01T15:24:33,
id: fa89fb9a-60d2-4921-b12d-6aee1c177823,
tenant: {
id: 1,
name: default
}
}
You're going to want to grab a long-lived token (sometimes called a service
token) to use for the Glance API - Glance Registry connection. This service
token should be used in the glance-registry.conf file.
In glance-registry.conf, you'll see a section looking like this:
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_token = 999888777666
Replace admin_token = 999888777666 with the relevant long-lived service token.
Cheers!
-jay
Step 2: Try a Glance command
# glance details -A fa89fb9a-60d2-4921-b12d-6aee1c177823
Failed to show details. Got error:
Internal Server error: Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/eventlet/wsgi.py, line 336, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 208, in call_func
return self.func(req, *args
[Openstack] Keystone installation with ManagedIT packages...
Still having issues.
I found a config bug in my Keystone.conf file in the filter pipeline for the
d5_compat modules. However, when I fix the config to match the installation
guide and restart keystone, I get an error message that the d5_compat module
can't be found. My keystone installation is from the ManagedIT PPA. Is this a
bug in the documentation? In the ManagedIT packages? The pipeline for the
ManageIT package is configured as follows:
[pipeline:admin]
pipeline =
urlrewritefilter
admin_api
[pipeline:keystone-legacy-auth]
pipeline =
urlrewritefilter
legacy_auth
RAX-KEY-extension
service_api
versus what is specified in the installation guide. Does this difference
matter? What are the differences?
Regards,
Ross
On Feb 1, 2012, at 10:31 AM, Lillie Ross-CDSR11 wrote:
Hi Jay,
Yes, this confused me, however I get the same error using the token I
generated and added (via the keystone-manage command). To wit:
root@nova:~# keystone-manage token list
token userexpiration tenant
---
10111213141516171819 1 2022-01-01 00:00:00 2
fa89fb9a-60d2-4921-b12d-6aee1c177823 1 2012-02-01 15:24:33 1
b06c5e4e-5e59-4293-aa54-ce6879f11371 2 2012-02-01 15:26:41 1
where the first token is the long-lived one I supplied during installation.
Running the glance command yields identical results:
root@nova:~# glance -v -A 10111213141516171819 details
Failed to show details. Got error:
Internal Server error: Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/eventlet/wsgi.py, line 336, in
handle_one_response
result = self.application(self.environ, start_response)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 208, in call_func
return self.func(req, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/glance/common/wsgi.py, line 113, in
__call__
response = req.get_response(self.application)
File /usr/lib/python2.7/dist-packages/webob/request.py, line 1053, in
get_response
application, catch_exc_info=False)
File /usr/lib/python2.7/dist-packages/webob/request.py, line 1022, in
call_application
app_iter = application(self.environ, start_response)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 147, in __call__
resp = self.call_func(req, *args, **self.kwargs)
File /usr/lib/python2.7/dist-packages/webob/dec.py, line 208, in call_func
return self.func(req, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/glance/common/wsgi.py, line 110, in
__call__
response = self.process_request(req)
File /usr/lib/python2.7/dist-packages/glance/common/context.py, line 104,
in process_request
raise exception.NotAuthorized()
NotAuthorized: None
Completed in 0.0031 sec.
Interestingly (perhaps) I see nothing in the keystone.log file. In fact, I
don't even see the keystone log file. Keystone opens to log files named
'admin.log' and 'keystone_legacy_auth.log'. Is this right? Also, if I run
keystone interactively (keystone -v -d) then issue the glance command, I see
nothing in the keystone window. This doesn't seem right to me, but I'm just
getting started with keystone integration.
Thanks in advance for any insight...
Regards,
Ross
On Jan 31, 2012, at 6:48 PM, Jay Pipes wrote:
On 01/31/2012 06:28 PM, Lillie Ross-CDSR11 wrote:
I'm reinstalling the various Openstack services from packages in the
ManagedIT PPA to pull in the latest Diablo bug fixes. I'm following the
latest directions in the newly release installation guide as I perform
these upgrades
(http://docs.openstack.org/diablo/openstack-compute/install/content/index.html).
However, I'm having trouble getting Glance to authenticate with
Keystone. All config files have been copied from the examples posted in
the installation guide (and modified accordingly for my admin token, IP
addresses, etc.). Regardless, I continually get the following error
message and stack dump when trying to verify the Glance/Keystone
integration:
Step 1: Grab a token
# curl -d '{auth: {tenantName: default,
passwordCredentials:{username: admin, password: admin}}}' -H
Content-type: application/json http://173.23.181.1:35357/v2.0/tokens |
python -mjson.tool
...
token: {
expires: 2012-02-01T15:24:33,
id: fa89fb9a-60d2-4921-b12d-6aee1c177823,
tenant: {
id: 1,
name: default
}
}
You're going to want to grab a long-lived token (sometimes called a service
token) to use for the Glance API - Glance Registry connection. This
service token should be used in the glance-registry.conf file.
In glance-registry.conf, you'll see a section looking like this:
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
[Openstack] Nova command line versus Euca2ools
I currently have OpenStack installed (using the ManagedIT PPA) to use Keystone
for authentication. However I'm still receiving a number of Malformed request
URL messages, both in Dashboard as well as when using the Nova command line
client. Also, some of the Euca2ools command run OK, others (such as
euca-describe-images) don't. I'm aware that not all the Euca commands are
supported in Diablo, but currently I don't have any commands that let me launch
instances.
For example: euca-describe-availability-zones verbose yields
root@nova:~# euca-describe-availability-zones verbose
AVAILABILITYZONE nova available
AVAILABILITYZONE |- nova
AVAILABILITYZONE | |- nova-network enabled :-) 2012-02-06 22:15:16
AVAILABILITYZONE | |- nova-scheduler enabled :-) 2012-02-06 22:15:15
AVAILABILITYZONE | |- nova-vncproxy enabled :-) 2012-02-06 22:15:14
AVAILABILITYZONE | |- nova-compute enabled :-) 2012-02-06 22:15:07
AVAILABILITYZONE |- nova1
AVAILABILITYZONE | |- nova-compute enabled :-) 2012-02-06 22:15:08
however, euca-describe-images yields (with debug enabled)
root@nova:~# euca-describe-images --debug
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Method: POST
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Path: /services/Cloud/
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Data:
2012-02-06 16:16:28,289 euca2ools [DEBUG]:Headers: {}
2012-02-06 16:16:28,290 euca2ools [DEBUG]:Host: 173.23.181.1:8773
2012-02-06 16:16:28,290 euca2ools [DEBUG]:establishing HTTP connection:
kwargs={}
2012-02-06 16:16:28,290 euca2ools [DEBUG]:using _calc_signature_2
2012-02-06 16:16:28,290 euca2ools [DEBUG]:query string:
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
2012-02-06 16:16:28,290 euca2ools [DEBUG]:string_to_sign: POST
173.23.181.1:8773
/services/Cloud/
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
2012-02-06 16:16:28,290 euca2ools [DEBUG]:len(b64)=44
2012-02-06 16:16:28,290 euca2ools [DEBUG]:base64 encoded digest:
vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI=
2012-02-06 16:16:28,290 euca2ools [DEBUG]:query_string:
AWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31
Signature: vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI=
send: 'POST /services/Cloud/ HTTP/1.1\r\nHost:
173.23.181.1:8773\r\nAccept-Encoding: identity\r\nContent-Length:
207\r\nContent-Type: application/x-www-form-urlencoded;
charset=UTF-8\r\nUser-Agent: Boto/2.0
(linux2)\r\n\r\nAWSAccessKeyId=admin%3AadminAction=DescribeImagesSignatureMethod=HmacSHA256SignatureVersion=2Timestamp=2012-02-06T22%3A16%3A28ZVersion=2010-08-31Signature=vSajubq/uXVIsFyMiUjxViprJ1zYHPpIONPcW5cN5yI%3D'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: text/xml
header: Content-Length: 239
header: Date: Mon, 06 Feb 2012 22:16:28 GMT
2012-02-06 16:16:28,299 euca2ools [DEBUG]:?xml version=1.0?
ResponseErrorsErrorCodeUnknownError/CodeMessageAn unknown error has
occurred. Please try your request
again./Message/Error/ErrorsRequestIDb7f94f66-d309-4dcf-bc5f-c6aa5a09a83a/RequestID/Response
2012-02-06 16:16:28,299 euca2ools [ERROR]:400 Bad Request
2012-02-06 16:16:28,300 euca2ools [ERROR]:?xml version=1.0?
ResponseErrorsErrorCodeUnknownError/CodeMessageAn unknown error has
occurred. Please try your request
again./Message/Error/ErrorsRequestIDb7f94f66-d309-4dcf-bc5f-c6aa5a09a83a/RequestID/Response
UnknownError: An unknown error has occurred. Please try your request again.
The corresponding nova command yields the following (again with debug enabled)
root@nova:~# nova --debug image-list
connect: (173.23.181.1, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: 173.23.181.1:5000\r\nContent-Length:
100\r\ncontent-type: application/json\r\naccept-encoding: gzip,
deflate\r\nuser-agent: python-novaclient\r\n\r\n{auth: {tenantName:
admin, passwordCredentials: {username: admin, password: admin}}}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=UTF-8
header: Content-Length: 1007
header: Date: Mon, 06 Feb 2012 22:19:52 GMT
connect: (173.23.181.1, 8774)
send: u'GET /v1.1/1/images/detail HTTP/1.1\r\nHost:
173.23.181.1:8774\r\nx-auth-project-id: admin\r\nx-auth-token:
10111213141516171819\r\naccept-encoding: gzip, deflate\r\nuser-agent:
python-novaclient\r\n\r\n'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Length: 65
header: Content-Type: application/json; charset=UTF-8
header: Date: Mon, 06 Feb 2012 22:19:52 GMT
Traceback (most recent call last):
File /usr/bin/nova, line 9, in module
load_entry_point('python-novaclient==2012.1', 'console_scripts', 'nova')()
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 353, in main
OpenStackComputeShell().main(sys.argv[1:])
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 304, in main
[Openstack] Swift/Keystone authorization question
I've successfully installed all OpenStack components with Keystone
authorization (well, mostly at least), but am now seeing an interesting problem
for new accounts (created in Dashboard).
Using my admin account, I issue a swift stat command and get the expected
response back from swift-proxy:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U admin
-K admin stat
StorageURL: http://173.23.181.2:8080/v1/AUTH_1
Auth Token: 10111213141516171819
Account: AUTH_1
Containers: 5
Objects: 20
Bytes: 6335748
Accept-Ranges: bytes
X-Trans-Id: tx6ffec7207a5c41329e53dbab6a6e2c37
Looking at the keystone admin.log file (with debugging enabled) I see the
following:
2012-02-22 14:26:38DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:26:38DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:26:38DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin', 'All administrative tasks are to be grouped underneath this tenancy.
Users are not to be associated with this tenant unless they have been granted
admin roles.', 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.tenant_id = %s AND users.id = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] (1L, 1L)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 . . .
However, when I issue the same command with a newly created user account I get
a 401 not authorized command back from swift-proxy. For example:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U
lillie -K changeme stat
Auth GET failed: http://173.23.181.1:5000/v2.0/tokens 401 Unauthorized
and the keystone admin.log file shows the following:
2012-02-22 14:30:40DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:40DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:30:40DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'lillie',)
2012-02-22 14:30:40DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:30:40DEBUG [eventlet.wsgi.server] 173.23.181.2 - -
[22/Feb/2012 14:30:40] POST /v2.0/tokens HTTP/1.1 401 197 0.004990
2012-02-22 14:30:41DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:41DEBUG
Re: [Openstack] Swift/Keystone authorization question
OK.
Reading through the swiftkeystone2 (module that I'm using to support v2
authentication in swift's proxy configuration) source and documentation, I've
figured out the necessary roles that need to be applied to user's accounts and
ACLs to project containers to allow all combinations of access to swift
storage. Works great.
/ross
On Feb 22, 2012, at 3:26 PM, Lillie Ross-CDSR11 wrote:
As a followup, additional info…
Both the admin and glance accounts, that successfully authenticate against
keystone, were created via the command line. Both accounts also have a tenant
of the same name as the user (probably irrelevant). All other user accounts
that have been created for general users won't authenticate agains keystone,
and exhibit the same error pattern as described below.
Interestingly, if I create a tenant with the same name as a user account, then
I get a 403 unable to get HEAD message when issuing a stat command as described
below.
/ross
On Feb 22, 2012, at 2:52 PM, Lillie Ross-CDSR11 wrote:
I've successfully installed all OpenStack components with Keystone
authorization (well, mostly at least), but am now seeing an interesting problem
for new accounts (created in Dashboard).
Using my admin account, I issue a swift stat command and get the expected
response back from swift-proxy:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U admin
-K admin stat
StorageURL: http://173.23.181.2:8080/v1/AUTH_1
Auth Token: 10111213141516171819
Account: AUTH_1
Containers: 5
Objects: 20
Bytes: 6335748
Accept-Ranges: bytes
X-Trans-Id: tx6ffec7207a5c41329e53dbab6a6e2c37
Looking at the keystone admin.log file (with debugging enabled) I see the
following:
2012-02-22 14:26:38DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:26:38DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:26:38DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin', 'All administrative tasks are to be grouped underneath this tenancy.
Users are not to be associated with this tenant unless they have been granted
admin roles.', 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.tenant_id = %s AND users.id = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] (1L, 1L)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 . . .
However, when I issue the same command with a newly created user account I get
a 401 not authorized command back from swift-proxy. For example:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U
lillie -K changeme stat
Auth GET failed: http://173.23.181.1:5000/v2.0/tokens 401 Unauthorized
and the keystone admin.log file shows the following:
2012-02-22 14:30:40DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:40DEBUG
[Openstack] Swift container ACLs and container visibility question
I'm setting up Swift storage for an internal project. For the project's use of Swift, I want all members of the project to be able to see what's stored in Swift. Applying suitable ACLs, it's possible for user's to see the contents of the projects container. However, is there any way to allow users to see a list of containers used by the project? Or must I create an additional container to store this type of project meta data? May be a dumb question and more of a architecture convention issue, but I'm just getting started with Swift and OpenStack in general and was wondering what other's have done. Thanks and regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift container ACLs and container visibility question
Sorry, I should have mentioned my setup. I'm using Keystone from the managedit repository combined with a swift keystone plugin to allow the proxy to use version 2 authentication. Ross (finger tapped on my iPhone) On Feb 23, 2012, at 4:38 PM, Chmouel Boudjnah chmo...@chmouel.com wrote: On Thu, Feb 23, 2012 at 10:25 PM, John Dickinson m...@not.mn wrote: It all depends on the auth system you are using. This is about the same for keystone but to be a .admin like in tempauth or swauth for keystone middleware you need to have one of the role specified in the configuration variable operator_roles[1] which is by default admin and SwiftOperator. Below is for swauth and tempauth: Chmouel. [1] https://github.com/openstack/keystone/blob/master/keystone/middleware/swift_auth.py#L80 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift/Keystone authorization question
As a followup, additional info…
Both the admin and glance accounts, that successfully authenticate against
keystone, were created via the command line. Both accounts also have a tenant
of the same name as the user (probably irrelevant). All other user accounts
that have been created for general users won't authenticate agains keystone,
and exhibit the same error pattern as described below.
Interestingly, if I create a tenant with the same name as a user account, then
I get a 403 unable to get HEAD message when issuing a stat command as described
below.
/ross
On Feb 22, 2012, at 2:52 PM, Lillie Ross-CDSR11 wrote:
I've successfully installed all OpenStack components with Keystone
authorization (well, mostly at least), but am now seeing an interesting problem
for new accounts (created in Dashboard).
Using my admin account, I issue a swift stat command and get the expected
response back from swift-proxy:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U admin
-K admin stat
StorageURL: http://173.23.181.2:8080/v1/AUTH_1
Auth Token: 10111213141516171819
Account: AUTH_1
Containers: 5
Objects: 20
Bytes: 6335748
Accept-Ranges: bytes
X-Trans-Id: tx6ffec7207a5c41329e53dbab6a6e2c37
Looking at the keystone admin.log file (with debugging enabled) I see the
following:
2012-02-22 14:26:38DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:26:38DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:26:38DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin', 'All administrative tasks are to be grouped underneath this tenancy.
Users are not to be associated with this tenant unless they have been granted
admin roles.', 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.tenant_id = %s AND users.id = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] (1L, 1L)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 . . .
However, when I issue the same command with a newly created user account I get
a 401 not authorized command back from swift-proxy. For example:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U
lillie -K changeme stat
Auth GET failed: http://173.23.181.1:5000/v2.0/tokens 401 Unauthorized
and the keystone admin.log file shows the following:
2012-02-22 14:30:40DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:40DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:30:40DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
Re: [Openstack] Swift/Keystone authorization question
I think I'm figuring this out, then again maybe not. For general users, via
the command line, you need to specify your user id as tenant:username after
reading through the source (my Python is really rusty). So, when I try this I
now get a 403 Forbidden error. I had high hopes.
Just another data point.
Ross
On Feb 22, 2012, at 2:52 PM, Lillie Ross-CDSR11 wrote:
I've successfully installed all OpenStack components with Keystone
authorization (well, mostly at least), but am now seeing an interesting problem
for new accounts (created in Dashboard).
Using my admin account, I issue a swift stat command and get the expected
response back from swift-proxy:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U admin
-K admin stat
StorageURL: http://173.23.181.2:8080/v1/AUTH_1
Auth Token: 10111213141516171819
Account: AUTH_1
Containers: 5
Objects: 20
Bytes: 6335748
Accept-Ranges: bytes
X-Trans-Id: tx6ffec7207a5c41329e53dbab6a6e2c37
Looking at the keystone admin.log file (with debugging enabled) I see the
following:
2012-02-22 14:26:38DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:26:38DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:26:38DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin', 'All administrative tasks are to be grouped underneath this tenancy.
Users are not to be associated with this tenant unless they have been granted
admin roles.', 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.tenant_id = %s AND users.id = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] (1L, 1L)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 . . .
However, when I issue the same command with a newly created user account I get
a 401 not authorized command back from swift-proxy. For example:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U
lillie -K changeme stat
Auth GET failed: http://173.23.181.1:5000/v2.0/tokens 401 Unauthorized
and the keystone admin.log file shows the following:
2012-02-22 14:30:40DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:40DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:30:40DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'lillie',)
2012-02
Re: [Openstack] Swift/Keystone authorization question
OK, is this a 'role' grant issue?
/ross
On Feb 22, 2012, at 2:52 PM, Lillie Ross-CDSR11 wrote:
I've successfully installed all OpenStack components with Keystone
authorization (well, mostly at least), but am now seeing an interesting problem
for new accounts (created in Dashboard).
Using my admin account, I issue a swift stat command and get the expected
response back from swift-proxy:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U admin
-K admin stat
StorageURL: http://173.23.181.2:8080/v1/AUTH_1
Auth Token: 10111213141516171819
Account: AUTH_1
Containers: 5
Objects: 20
Bytes: 6335748
Accept-Ranges: bytes
X-Trans-Id: tx6ffec7207a5c41329e53dbab6a6e2c37
Looking at the keystone admin.log file (with debugging enabled) I see the
following:
2012-02-22 14:26:38DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:26:38DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:26:38DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin', 'All administrative tasks are to be grouped underneath this tenancy.
Users are not to be associated with this tenant unless they have been granted
admin roles.', 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.name = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'admin',)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
users.id AS users_id, users.name AS users_name, users.password AS
users_password, users.email AS users_email, users.enabled AS users_enabled,
users.tenant_id AS users_tenant_id
FROM users
WHERE users.tenant_id = %s AND users.id = %s
LIMIT 0, 1
2012-02-22 14:26:38 INFO [sqlalchemy.engine.base.Engine.0x...14d0] (1L, 1L)
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('users_id', 'users_name', 'users_password', 'users_email', 'users_enabled',
'users_tenant_id')
2012-02-22 14:26:38DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Row (1L,
'admin',
'$6$rounds=4$k5f0Zd1lOK3AVXbx$awVYhvdu1CI33hRhugjURheVePZYh60EjWSUa4Zwq0Ha48eNH3SQXSFVQeEYv4ffwUzlRVVkoUbr6C4Ai63WC.',
None, 1L, 1L)
2012-02-22 14:26:38 . . .
However, when I issue the same command with a newly created user account I get
a 401 not authorized command back from swift-proxy. For example:
root@swift:/etc/swift# swift -v -V 2 -A http://173.23.181.1:5000/v2.0/ -U
lillie -K changeme stat
Auth GET failed: http://173.23.181.1:5000/v2.0/tokens 401 Unauthorized
and the keystone admin.log file shows the following:
2012-02-22 14:30:40DEBUG [routes.middleware] Matched POST /tokens
2012-02-22 14:30:40DEBUG [routes.middleware] Route path: '/tokens',
defaults: {'action': u'authenticate', 'controller':
keystone.controllers.auth.AuthController object at 0x170da10}
2012-02-22 14:30:40DEBUG [routes.middleware] Match dict: {'action':
u'authenticate', 'controller': keystone.controllers.auth.AuthController object
at 0x170da10}
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0] SELECT
tenants.id AS tenants_id, tenants.name AS tenants_name, tenants.`desc` AS
tenants_desc, tenants.enabled AS tenants_enabled
FROM tenants
WHERE tenants.name = %s
LIMIT 0, 1
2012-02-22 14:30:40 INFO [sqlalchemy.engine.base.Engine.0x...14d0]
(u'lillie',)
2012-02-22 14:30:40DEBUG [sqlalchemy.engine.base.Engine.0x...14d0] Col
('tenants_id', 'tenants_name', 'tenants_desc', 'tenants_enabled')
2012-02-22 14:30:40DEBUG [eventlet.wsgi.server] 173.23.181.2 - -
[22/Feb/2012 14:30:40] POST /v2.0/tokens HTTP/1.1 401 197 0.004990
[Openstack] Nova instance and cloud-init
All, I've built a custom image according to the various directions available in the nova docs. The instance is loaded into glance and boots fine. When building the image, openssh-server and cloud-init packages are installed. So far so good. However, when I start the image, cloud-init does not 'see' the EC2 datasource (i.e. the meta-data server), and reports back that it's only looking for the NoCloudNet and SourceOVFNet datasources when running in the 'cloud-init start' phase (versus start-local). When I check the '/var/lib/cloud' directory, cloud-init is not setting up the instance link or populating the instance subdirectory under /var/lib/cloud/instances. I'm able to connect to the meta-data service with no problem when logged into the running instance. Obviously there's something I'm missing when configuring my custom image and installing the cloud-init package. UEC images execute flawlessly (downloaded from Ubuntu). I feel like a broken record. What am I missing? Any pointers would be appreciated. Regards, ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Swift, Keystone, and S3 pipeline configuration
Hi Tom, I'm taking the easy way out for the near term. I've reconfigured Swift to use the tempauth middleware, and am replicating the user ids and tenant ids that I have in Keystone. This allows everyone on my project team to have a uniform runtime configuration setup. In the meantime, I'm setting up a devstack setup to allow me to track Essex development and stability. Once Essex is locked down, I'll likely upgrade our entire cloud, once I'm confident that the ubuntu packages are fully baked. This seemed like the best path, as my team members also have to make progress on their project goals, and my Openstack setup is quite stable at present. Regardless, thanks for your guidance and help, Ross -- Ross Lillie ross.lil...@motorolasolutions.com On Mar 28, 2012, at 6:10 PM, Tom Fifield wrote: No problem. Please note: there is a bug with these patches (fixed in Essex) that means that data uploaded using S3 is in a different namespace to data uploaded with the OpenStack API. Regards, Tom On 03/28/2012 07:37 PM, Lillie Ross-CDSR11 wrote: Tom, Thanks! I'll give these a try. Regards Ross Sent from my iPad On Mar 27, 2012, at 6:19 PM, Tom Fifieldfifie...@unimelb.edu.au wrote: Doesn't the s3token patch by Yoshiyama san fix this for Diablo/Keystone ... http://www.debian.or.jp/~yosshy/openstack-diablo/s3token.txt ? Regards, Tom On 03/28/2012 08:36 AM, Chmouel Boudjnah wrote: Hi, On Tue, Mar 27, 2012 at 10:32 PM, Lillie Ross-CDSR11 ross.lil...@motorolasolutions.com mailto:ross.lil...@motorolasolutions.com wrote: Then, if I want to use S3 binding with Swift (Diablo), I need to used the simpler 'swauth' middleware for authenticatoin? Just wondering. Yes this is correct. Cheers, Chmouel. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Mixing Swift authentication (tempauth) with Keystone = Dashboard flakiness
I'm not sure if you can do what I'm trying. I've deployed Openstack Diable (ManagedIT) with Keystone for all service authentication. However to restore S3 API compatibility with Swift, I've modified my proxy server config to use 'tempauth' instead of Keystone. Within the proxy-server.conf, I've replicated the user/tenant/passwords for all accounts that I have in Keystone. This allows a user's .openrc file to work with only one change: the authentication URL for Swift now has to point at the tempauth service versus Keystone. I've verified that S3 is now working (using a small python/boto script). Via the command line, glance is able to load images into the image store. Using the Swift command, I can verify that the image is being loaded into the correct bucket. So far so good, however I'm noticing slight flakiness now with Dashboard. In particular, Dashboard is no longer able to display a list of running instances (it hangs). Also, the 'storage' service appears as though it's not running. Using the 'euca-describe-instances' command works fine. Any thoughts or suggestions would be helpful. Regards, Ross -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Mixing Swift authentication (tempauth) with Keystone = Dashboard flakiness
As a followup. In Dashboard, the overview panel correctly displays the list of running instance. Only when selecting the 'instances' panel do I see problems. -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com On Mar 30, 2012, at 1:18 PM, Lillie Ross-CDSR11 wrote: I'm not sure if you can do what I'm trying. I've deployed Openstack Diable (ManagedIT) with Keystone for all service authentication. However to restore S3 API compatibility with Swift, I've modified my proxy server config to use 'tempauth' instead of Keystone. Within the proxy-server.conf, I've replicated the user/tenant/passwords for all accounts that I have in Keystone. This allows a user's .openrc file to work with only one change: the authentication URL for Swift now has to point at the tempauth service versus Keystone. I've verified that S3 is now working (using a small python/boto script). Via the command line, glance is able to load images into the image store. Using the Swift command, I can verify that the image is being loaded into the correct bucket. So far so good, however I'm noticing slight flakiness now with Dashboard. In particular, Dashboard is no longer able to display a list of running instances (it hangs). Also, the 'storage' service appears as though it's not running. Using the 'euca-describe-instances' command works fine. Any thoughts or suggestions would be helpful. Regards, Ross -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Mixing Swift authentication (tempauth) with Keystone = Dashboard flakiness
Also... unable to start a new instance from dashboard. /ross -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com On Mar 30, 2012, at 1:33 PM, Lillie Ross-CDSR11 wrote: As a followup. In Dashboard, the overview panel correctly displays the list of running instance. Only when selecting the 'instances' panel do I see problems. -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com On Mar 30, 2012, at 1:18 PM, Lillie Ross-CDSR11 wrote: I'm not sure if you can do what I'm trying. I've deployed Openstack Diable (ManagedIT) with Keystone for all service authentication. However to restore S3 API compatibility with Swift, I've modified my proxy server config to use 'tempauth' instead of Keystone. Within the proxy-server.conf, I've replicated the user/tenant/passwords for all accounts that I have in Keystone. This allows a user's .openrc file to work with only one change: the authentication URL for Swift now has to point at the tempauth service versus Keystone. I've verified that S3 is now working (using a small python/boto script). Via the command line, glance is able to load images into the image store. Using the Swift command, I can verify that the image is being loaded into the correct bucket. So far so good, however I'm noticing slight flakiness now with Dashboard. In particular, Dashboard is no longer able to display a list of running instances (it hangs). Also, the 'storage' service appears as though it's not running. Using the 'euca-describe-instances' command works fine. Any thoughts or suggestions would be helpful. Regards, Ross -- Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Best approach for deploying Essex?
With the pending release of Essex, I'm making plans to upgrade our internal cloud infrastructure. My question is what will be the best approach? Our cloud is being used to support internal research activities and thus needs to be 'relatively' stable, however as new features become available (major features), it would be nice to be able to role them into our operational cloud relatively painlessly (wishful thinking perhaps). My question is, should I base our new installation directly off the Essex branch in the git repository, or use the packages that will be deployed as part of the associated Ubuntu 12.04LTS release? With Diablo, I was forced to use packages from the ManagedIT PPA with additional Keystone patches to get a consistent, stable platform up and running. Obviously, some of these problems were due to confusion caused by various documents describing different incarnations of Openstack, and not really knowing what was current and stable. Especially the packages shipped with Ubuntu made assumptions about how Openstack was to be deployed that wasn't really apparent. Just wondering. Any thoughts appreciated. Regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Best approach for deploying Essex?
Hi Adam, Thanks for the update. Actually, I'm in the process of reading about your testing and integration framework for Openstack (http://ubuntuserver.wordpress.com/2012/02/08/704/) as I write this. Yes, Keystone integration seemed to be the big bugaboo in the Ubuntu/Diablo release. I've successfully got everything authenticating with keystone in our current deployment, but as you well know, this precludes using S3 bindings with Swift, and you must use the Openstack glance client to bundle/upload images. This had me pulling my hair out in the early stages. Today I've spun up 3 generic 11.10 servers that I'm planning on testing the next Ubuntu release and Openstack packages. Should I start my testing with the beta1 release of 12.04LTS? In particular I'm interested in seeing and understanding the process of migrating my existing installation and configs to the new release. Once I'm satisfied that I understand everything (not possible) in the new release, I can migrate our operational cloud in a couple of days. Also, what's the best place to keep abreast on the Ubuntu/Canonical integration of openstack? The Ubuntu Wiki? Mailing list? Thanks again, Ross On Apr 3, 2012, at 1:21 PM, Adam Gandelman wrote: On 04/03/2012 08:20 AM, Lillie Ross-CDSR11 wrote: My question is, should I base our new installation directly off the Essex branch in the git repository, or use the packages that will be deployed as part of the associated Ubuntu 12.04LTS release? With Diablo, I was forced to use packages from the ManagedIT PPA with additional Keystone patches to get a consistent, stable platform up and running. Obviously, some of these problems were due to confusion caused by various documents describing different incarnations of Openstack, and not really knowing what was current and stable. Especially the packages shipped with Ubuntu made assumptions about how Openstack was to be deployed that wasn't really apparent. Hey Ross- I can say that the Ubuntu precise packages have been kept relatively in-sync with each components' trunk git repository this cycle. We've made a concerted effort to do weekly snapshot uploads of all Openstack components into the Precise archive starting from the beginning of the Essex+Precise dev cycles. We've also maintained our own trunk PPA (updated continously) around which we center our testing efforts. Now that we're nearing the release of Essex, we've been ensuring the release candidates hit our archive as soon as they are released. As soon as Essex final hits, it'll be uploaded into Ubuntu and give any users who care the remainder of the Ubuntu dev cycle (~1 month) to test and identify issues before LTS ships. Re: deployment assumptions. Last cycle, we were caught off-guard by Keystone's last-minute inclusion into Openstack core and the dependencies this introduced (dashboard especially) It's not that we were making assumptions about how Openstack Diablo should be deployed, just that there was no way we could shoe-horn a new component into the release so late in the game. This time around, a similar curve ball was thrown our way with the Keystone Lite rewrite, but we were able to get this sorted on our end relatively quickly to ensure pending security reviews and main inclusion processes for Keystone weren't blocked. We're making very few assumptions going into LTS and hope to provide as-close-to-pure Essex experience as any. I can only think of a few patches we're carrying, and there are only two default configuration files we ship that differ from those you'd find in the git repos [2]. Perhaps when we release Essex into Precise this/next week, we'll put some notes somewhere outlining any Ubuntu-specific changes for those who are interested. Hope that helps, and of course we welcome your testing and bug reports! -Adam [1] We ship a default nova.conf that configures some Ubuntu defaults: defaults to libvirt compute, uses nova-rootwrap for sudo shell execution (requested by our security team), uses tgt iscsi initiator instead of ietd (tgt is supported in our main archive, ietd is not). Our default Keystone config defaults to the SQL catalog backend instead of the default templated file, though I think SQL catalog is the new default in folsom. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Best approach for deploying Essex?
Hi Kevin, I'm in the process of looking at the Ubuntu packages. I've booted up 3 12.04 instances in our existing cloud and have installed the Ubuntu packages. I'm playing with the new keystone 'lite' service as I write this. In fact, I'm looking at your script to bootstrap keystone versus the script installed via the package in /usr/share/keystone. So far so good, other than some iptable hacks that will be needed (I believe, but haven't looked closely) to let these instances to see their underlying public ip addresses (no problem, obviously seeing their private vlan address). Thanks for your inputs. Regards, Ross On Apr 4, 2012, at 4:20 AM, Kevin Jackson wrote: +1 on the Ubuntu 12.04 Precise and Essex. Whilst there are other methods and OS of choice, given you've been using 11.10 - 12.04 is the natural home for it. The betas of Precise are rock-solid and can vouch for the Ubuntu packaging following quite quickly behind the devs. I highly recommend this approach. Kev On 3 April 2012 20:02, Lillie Ross-CDSR11 ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com wrote: Hi Adam, Thanks for the update. Actually, I'm in the process of reading about your testing and integration framework for Openstack (http://ubuntuserver.wordpress.com/2012/02/08/704/) as I write this. Yes, Keystone integration seemed to be the big bugaboo in the Ubuntu/Diablo release. I've successfully got everything authenticating with keystone in our current deployment, but as you well know, this precludes using S3 bindings with Swift, and you must use the Openstack glance client to bundle/upload images. This had me pulling my hair out in the early stages. Today I've spun up 3 generic 11.10 servers that I'm planning on testing the next Ubuntu release and Openstack packages. Should I start my testing with the beta1 release of 12.04LTS? In particular I'm interested in seeing and understanding the process of migrating my existing installation and configs to the new release. Once I'm satisfied that I understand everything (not possible) in the new release, I can migrate our operational cloud in a couple of days. Also, what's the best place to keep abreast on the Ubuntu/Canonical integration of openstack? The Ubuntu Wiki? Mailing list? Thanks again, Ross On Apr 3, 2012, at 1:21 PM, Adam Gandelman wrote: On 04/03/2012 08:20 AM, Lillie Ross-CDSR11 wrote: My question is, should I base our new installation directly off the Essex branch in the git repository, or use the packages that will be deployed as part of the associated Ubuntu 12.04LTS release? With Diablo, I was forced to use packages from the ManagedIT PPA with additional Keystone patches to get a consistent, stable platform up and running. Obviously, some of these problems were due to confusion caused by various documents describing different incarnations of Openstack, and not really knowing what was current and stable. Especially the packages shipped with Ubuntu made assumptions about how Openstack was to be deployed that wasn't really apparent. Hey Ross- I can say that the Ubuntu precise packages have been kept relatively in-sync with each components' trunk git repository this cycle. We've made a concerted effort to do weekly snapshot uploads of all Openstack components into the Precise archive starting from the beginning of the Essex+Precise dev cycles. We've also maintained our own trunk PPA (updated continously) around which we center our testing efforts. Now that we're nearing the release of Essex, we've been ensuring the release candidates hit our archive as soon as they are released. As soon as Essex final hits, it'll be uploaded into Ubuntu and give any users who care the remainder of the Ubuntu dev cycle (~1 month) to test and identify issues before LTS ships. Re: deployment assumptions. Last cycle, we were caught off-guard by Keystone's last-minute inclusion into Openstack core and the dependencies this introduced (dashboard especially) It's not that we were making assumptions about how Openstack Diablo should be deployed, just that there was no way we could shoe-horn a new component into the release so late in the game. This time around, a similar curve ball was thrown our way with the Keystone Lite rewrite, but we were able to get this sorted on our end relatively quickly to ensure pending security reviews and main inclusion processes for Keystone weren't blocked. We're making very few assumptions going into LTS and hope to provide as-close-to-pure Essex experience as any. I can only think of a few patches we're carrying, and there are only two default configuration files we ship that differ from those you'd find in the git repos [2]. Perhaps when we release Essex into Precise this/next week, we'll put some notes somewhere outlining any Ubuntu-specific changes for those who are interested. Hope that helps, and of course we
[Openstack] Swift Keystone configuration in Essex release and possible logging issue
When configuring the paste pipeline for Swift's proxy, it appears that you must explicitly state the service and authentication protocol as http if you're not using an SSL connection, as this is the default. The current configuration section of the keystone documentation doesn't make this clear. My sample proxy-server.conf file for the Essex packages in the Ubuntu 12.04LTS beta thus looks as follows: DEFAULT] bind_port = 8080 bind_ip = 172.16.1.5 user = swift log_name = SWIFT_PROXY log_level = DEBUG log_headers = True [pipeline:main] pipeline = catch_errors healthcheck cache swift3 s3token authtoken keystone prox y-server #pipeline = catch_errors healthcheck cache authtoken keystone proxy-server [app:proxy-server] use = egg:swift#proxy allow_account_management = true account_autocreate = true [filter:swift3] use = egg:swift#swift3 [filter:s3token] paste.filter_factory = keystone.middleware.s3_token:filter_factory auth_port = 5000 auth_host = essex1 auth_protocol = http [filter:keystone] paste.filter_factory = keystone.middleware.swift_auth:filter_factory operator_roles = admin, swiftoperator, Admin, SwiftOperator [filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory delay_auth_decision = 1 service_protocol = http service_port = 5000 service_host = essex1 auth_protocol = http auth_port = 35357 auth_host = essex1 auth_token = ADMIN8475760012 admin_token = ADMIN8475760012 [filter:catch_errors] use = egg:swift#catch_errors [filter:healthcheck] use = egg:swift#healthcheck [filter:cache] use = egg:swift#memcache memcache_server = 127.0.0.1:11211 Also, setting the log_level in the proxy's configuration file does not enable logging in the keystone authentication modules included in the paste pipeline. To discover this configuration bug, I needed to hack the auth_token.py module to get logging to work (copied from swift's catch_errors.py module) and discover that it was trying to connect via an SSL connection. Is this a bug? Otherwise auth_token.py will report a no handler exception. Regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: Unable to download images using Glance+Keystone+Swift
Begin forwarded message: From: Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com Subject: Re: [Openstack] Unable to download images using Glance+Keystone+Swift Date: April 26, 2012 1:37:45 PM CDT To: Jay Pipes jaypi...@gmail.commailto:jaypi...@gmail.com Cc: Ross Lillie ross.lil...@motorolasolutions.commailto:ross.lil...@motorolasolutions.com Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross On Apr 26, 2012, at 11:55 AM, Jay Pipes wrote: On 04/26/2012 11:54 AM, Lillie Ross-CDSR11 wrote: 4. However, when I try to download the same image, I receive the following error: curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' -H 'Content-type: application/json' http://essex3:9292/v1/images/6720c572-12b7-4cc8-a8c5-95b92998671a | python -mjson.tool You need to remove the | python -mjson.tool :) Don't really want to be piping an image file into that module... Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to download images using Glance+Keystone+Swift
Jay, These are the Ubuntu 12.04 packages from the beta with all known updates. I'm configuring another set of instances with the Ubuntu Precise final packages just to make sure I didn't miss a patch. However, this error seems fundamental to me. I don't see how a glance POST can work but the corresponding GET fails. All calls that just hit the backend DB work fine. Also I can access the bucket and objects directly via swift w no problem. I'll post my results with the final Ubuntu release sometime tomorrow hopefully. (finger tapped on my iPhone) On Apr 26, 2012, at 1:40 PM, Jay Pipes jaypi...@gmail.com wrote: On 04/26/2012 02:37 PM, Lillie Ross-CDSR11 wrote: Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross, what version of Glance and Swift are you using? Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unable to download images using Glance+Keystone+Swift
DOH! (boy I say this a lot…) As usual, a stupid configuration error. In my old Diablo setup, the administrative role was specified via the keyword Admin. I imported the existing user accounts, tenants, and roles into my Essex test setup. However I neglected to set the Glance 'admin_role' configuration flag, so it defaulted to the keyword admin. Once I fixed this, image downloads are working fine and instances boot with no problem. Congrats to everyone that pulled Essex together. Overall stability and functionality is much improved. We'll be migrating our Diablo cloud to Essex over the next week. Ross On Apr 26, 2012, at 7:32 PM, Sam Morrison wrote: Hi Ross, I had the same issue. Could upload images to swift but not download them getting a 404. I needed to apply the patch outlined in this bug to fix it: https://bugs.launchpad.net/glance/+bug/979745 Cheers, Sam On Fri, Apr 27, 2012 at 9:53 AM, Lillie Ross-CDSR11 ross.lil...@motorolasolutions.com wrote: Jay, These are the Ubuntu 12.04 packages from the beta with all known updates. I'm configuring another set of instances with the Ubuntu Precise final packages just to make sure I didn't miss a patch. However, this error seems fundamental to me. I don't see how a glance POST can work but the corresponding GET fails. All calls that just hit the backend DB work fine. Also I can access the bucket and objects directly via swift w no problem. I'll post my results with the final Ubuntu release sometime tomorrow hopefully. (finger tapped on my iPhone) On Apr 26, 2012, at 1:40 PM, Jay Pipes jaypi...@gmail.com wrote: On 04/26/2012 02:37 PM, Lillie Ross-CDSR11 wrote: Hi Jay, Cut and paste error. It still doesn't work. If I issue the simple command (without the pipe or content-type header) I get the following root@essex1:/etc/keystone# curl -v -H 'X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8' http://essex3:9292/v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 * About to connect() to essex3 port 9292 (#0) * Trying 172.16.1.5... connected GET /v1/images/423b0ecc-5ca1-44d8-8e85-5a245ce620e2 HTTP/1.1 User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: essex3:9292 Accept: */* X-Auth-Token: 45d01460a0e04bff967eb954e7f4fee8 HTTP/1.1 404 Not Found Content-Length: 315 Content-Type: text/html; charset=UTF-8 Date: Thu, 26 Apr 2012 18:35:21 GMT html head title404 Not Found/title /head body h1404 Not Found/h1 An object with the specified identifier was not found. Details: Swift could not find image at uri swift+http://service:glance:glance@essex1:5000/v2.0/glance/423b0ecc-5ca1-44d8-8e85-5a245ce620e2br /br / /body * Connection #0 to host essex3 left intact * Closing connection #0 /html root@essex1:/etc/keystone# Now, I can access the image directly via the Swift CLI using my glance tenant, username, and password. However, the Glance REST call fails. All other REST calls work fine. I'm stumped. Ross, what version of Glance and Swift are you using? Best, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Nova volume service and the nova client command
I've configured the nova_volume service using the the Ubuntu 12.04LTS packages,
and am able to create/delete volumes using the euca2ools package. However,
dashboard is not able to retrieve volume info or perform volume operations with
the nova client. If I issue the 'nova volume-list' command, I receive a 400
error response. For example
root@essex1:/etc/nova# nova --debug volume-list
connect: (essex1, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: essex1:5000\r\nContent-Length:
100\r\ncontent-type: application/json\r\naccept-encoding: gzip,
deflate\r\naccept: application/json\r\nuser-agent:
python-novaclient\r\n\r\n{auth: {tenantName: admin,
passwordCredentials: {username: admin, password: admin}}}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Vary: X-Auth-Token
header: Date: Tue, 01 May 2012 20:06:39 GMT
header: Transfer-Encoding: chunked
connect: (essex4, 8776)
connect fail: (u'essex4', 8776)
DEBUG (shell:416) n/a (HTTP 400)
Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 413, in main
OpenStackComputeShell().main(sys.argv[1:])
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 364, in main
args.func(self.cs, args)
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/shell.py, line 858,
in do_volume_list
volumes = cs.volumes.list()
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/volumes.py, line 79,
in list
return self._list(/volumes/detail, volumes)
File /usr/lib/python2.7/dist-packages/novaclient/base.py, line 71, in _list
resp, body = self.api.client.get(url)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 136, in get
return self._cs_request(url, 'GET', **kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 124, in
_cs_request
**kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 107, in
request
raise exceptions.from_response(resp, body)
BadRequest: n/a (HTTP 400)
ERROR: n/a (HTTP 400)
root@essex1:/etc/nova#
Node essex1 is the cloud controller. essex4 is the volume controller. To
verify the connection failure, if I try to telnet to port 8776 I also get a
connect refused. My nova-volume endpoint is specified in keystone as
'http://essex4:8776/v1/$(tenant_id)s'. With the above connection failure,
nothing is written to the nova-volume.log (which makes sense if the server
isn't listening to port 8776). Obviously, none of the other nova volume
commands work.
Anything obvious that I'm missing? Thanks in advance
Regards,
Ross
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova volume service and the nova client command
OK, I'm an idiot.
If I understand correctly, nova-volume provides the 'backend' service. So to
run nova-volume on a separate node, I apparently have to also install the
nova-api (and associated python-keystone modules)? Is this correct? Or will
the backend messaging (rabbitmq) allow the nova-api service running on one node
to correctly control the nova-volume backend on another node?
Thanks in advance!
Ross
On May 1, 2012, at 3:15 PM, Lillie Ross-CDSR11 wrote:
I've configured the nova_volume service using the the Ubuntu 12.04LTS
packages, and am able to create/delete volumes using the euca2ools package.
However, dashboard is not able to retrieve volume info or perform volume
operations with the nova client. If I issue the 'nova volume-list' command,
I receive a 400 error response. For example
root@essex1:/etc/nova# nova --debug volume-list
connect: (essex1, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: essex1:5000\r\nContent-Length:
100\r\ncontent-type: application/json\r\naccept-encoding: gzip,
deflate\r\naccept: application/json\r\nuser-agent:
python-novaclient\r\n\r\n{auth: {tenantName: admin,
passwordCredentials: {username: admin, password: admin}}}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Vary: X-Auth-Token
header: Date: Tue, 01 May 2012 20:06:39 GMT
header: Transfer-Encoding: chunked
connect: (essex4, 8776)
connect fail: (u'essex4', 8776)
DEBUG (shell:416) n/a (HTTP 400)
Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 413, in
main
OpenStackComputeShell().main(sys.argv[1:])
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 364, in
main
args.func(self.cs, args)
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/shell.py, line 858,
in do_volume_list
volumes = cs.volumes.list()
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/volumes.py, line 79,
in list
return self._list(/volumes/detail, volumes)
File /usr/lib/python2.7/dist-packages/novaclient/base.py, line 71, in _list
resp, body = self.api.client.get(url)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 136, in
get
return self._cs_request(url, 'GET', **kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 124, in
_cs_request
**kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 107, in
request
raise exceptions.from_response(resp, body)
BadRequest: n/a (HTTP 400)
ERROR: n/a (HTTP 400)
root@essex1:/etc/nova#
Node essex1 is the cloud controller. essex4 is the volume controller. To
verify the connection failure, if I try to telnet to port 8776 I also get a
connect refused. My nova-volume endpoint is specified in keystone as
'http://essex4:8776/v1/$(tenant_id)s'. With the above connection failure,
nothing is written to the nova-volume.log (which makes sense if the server
isn't listening to port 8776). Obviously, none of the other nova volume
commands work.
Anything obvious that I'm missing? Thanks in advance
Regards,
Ross
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova volume service and the nova client command
Once again, I think I'm answering my own question.
Nova_volume works in conjunction with nova-api to talk to any number of iscsi
targets that you might have configured in your cloud. Each target runs an
instance of nova-volume. The URL for the volume service should point at the
host(s) running the nova-api front end. Nova-volume listens to the appropriate
AMQP channel to perform the necessary LVM commands.
Am I missing anything?
Sorry for all questions. I just needed to think things through a bit more…
/ross
On May 1, 2012, at 3:15 PM, Lillie Ross-CDSR11 wrote:
I've configured the nova_volume service using the the Ubuntu 12.04LTS
packages, and am able to create/delete volumes using the euca2ools package.
However, dashboard is not able to retrieve volume info or perform volume
operations with the nova client. If I issue the 'nova volume-list' command,
I receive a 400 error response. For example
root@essex1:/etc/nova# nova --debug volume-list
connect: (essex1, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost: essex1:5000\r\nContent-Length:
100\r\ncontent-type: application/json\r\naccept-encoding: gzip,
deflate\r\naccept: application/json\r\nuser-agent:
python-novaclient\r\n\r\n{auth: {tenantName: admin,
passwordCredentials: {username: admin, password: admin}}}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Vary: X-Auth-Token
header: Date: Tue, 01 May 2012 20:06:39 GMT
header: Transfer-Encoding: chunked
connect: (essex4, 8776)
connect fail: (u'essex4', 8776)
DEBUG (shell:416) n/a (HTTP 400)
Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 413, in
main
OpenStackComputeShell().main(sys.argv[1:])
File /usr/lib/python2.7/dist-packages/novaclient/shell.py, line 364, in
main
args.func(self.cs, args)
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/shell.py, line 858,
in do_volume_list
volumes = cs.volumes.list()
File /usr/lib/python2.7/dist-packages/novaclient/v1_1/volumes.py, line 79,
in list
return self._list(/volumes/detail, volumes)
File /usr/lib/python2.7/dist-packages/novaclient/base.py, line 71, in _list
resp, body = self.api.client.get(url)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 136, in
get
return self._cs_request(url, 'GET', **kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 124, in
_cs_request
**kwargs)
File /usr/lib/python2.7/dist-packages/novaclient/client.py, line 107, in
request
raise exceptions.from_response(resp, body)
BadRequest: n/a (HTTP 400)
ERROR: n/a (HTTP 400)
root@essex1:/etc/nova#
Node essex1 is the cloud controller. essex4 is the volume controller. To
verify the connection failure, if I try to telnet to port 8776 I also get a
connect refused. My nova-volume endpoint is specified in keystone as
'http://essex4:8776/v1/$(tenant_id)s'. With the above connection failure,
nothing is written to the nova-volume.log (which makes sense if the server
isn't listening to port 8776). Obviously, none of the other nova volume
commands work.
Anything obvious that I'm missing? Thanks in advance
Regards,
Ross
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Configuring Openstack (Essex) to use existing Dynamic DNS servers
All, I'm finally starting the upgrade to our internal cloud to the Essex release that's part of Ubuntu 12.04LTS. I'd like to setup the cloud to register instance names with our existing internal DNS servers. While there's some mention of this searching the Web, I haven't stumbled across anything definitive. My guess is that I need to replace the default dnsmasq server with an alternate backend such as nsupdate, however I'm unsure how the overall DHCP/DNS plumbing is configured. Are there any pointers and/or design documents that I could be pointed towards to give me a better understanding to help me implement this functionality? Also, I see that there appears to be a nova-compute extension in the current development branch that appears to provide this functionality if I've read things correctly. However I'd rather rely upon released packages for our essex deployment, rather than pulling in code from the development trunk. Anyways, any pointers would be appreciated. Thanks in advance, and regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Dashboard (Ubuntu 12.04/Essex)
All, I've upgraded our cloud, but am stymied on one last configuration issue. From the dashboard, I'm unable to display images and/or snapshots. However, I can display loaded images using the nova and glance command line tools with no problems. For example, 'glance index' displays the following: ID Name Disk Format Container Format Size -- -- f4c861aa-ded5-4cb8-9338-c0551a11a5d5 tty-linux ami ami25165824 bfc26a24-8cd5-4259-a57a-7071d909de8c tty-linux-ramdisk ari ari 5882349 df43feb9-aa30-4a2a-a03e-0357df5d4249 tty-linux-kernel aki aki 4404752 and the corresponding command through the nova-api service, 'nova image-list' displays: +--+---+++ | ID |Name | Status | Server | +--+---+++ | bfc26a24-8cd5-4259-a57a-7071d909de8c | tty-linux-ramdisk | ACTIVE || | df43feb9-aa30-4a2a-a03e-0357df5d4249 | tty-linux-kernel | ACTIVE || | f4c861aa-ded5-4cb8-9338-c0551a11a5d5 | tty-linux | ACTIVE || +--+---+++ However, when accessing the Images Snapshots panel in Horizon, I receive 2 error messages that it's unable to retrieve images and snapshots. Looking at the nova-api.log file, I see the following: 2012-06-18 11:13:30 INFO nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1] GET http://173.23.181.1:8776/v1/1/snapshots/detail 2012-06-18 11:13:30 DEBUG nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1] Unrecognized Content-Type provided in request from (pid=9586) get_body /u sr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:697 2012-06-18 11:13:30 INFO nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1] http://173.23.181.1:8776/v1/1/snapshots/detail returned with HTTP 200 which indicates that the request complete OK. From the Apache access logs I see 127.0.0.1:80 173.17.11.184 - - [18/Jun/2012:11:13:24 -0500] GET /nova/images_and_snapshots/ HTTP/1.1 200 49213 http://cloud.dtl.mot-solutions.com/nova/images_and_snaps hots/ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2 which also seems to indicate that all is well with 49K data returned, however nothing displays in the dashboard. Any suggestions? Thanks to all in advance! Regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Dashboard (Ubuntu 12.04/Essex)
thanks Gabriel, No, as usual, it was a typo in my internal URL for glance in the service catalog. I'd reconfigured the internal network and failed to update the internal URL value for the image service. However, I DID have the dashboard's local_settings.py file configured to use public URLs, and dashboard is apparently ignoring this flag. I may file a bug report. Thanks again, Ross On Jun 18, 2012, at 1:56 PM, Gabriel Hurley wrote: It may have to do with the container type set on the images. There is some filtering happening in the Project dashboard that hides the AKI and ARI images that are associated with AMIs. So if you’ve only got AKI/ARI images those would be hidden. You can see (and manage) those images as an administrator in the Admin dashboard. Without further information that’d be my best guess. You can also try setting the log level to DEBUG in your local_settings.py file to get more information about exactly what is returned from the API. - Gabriel From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.netmailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.netmailto:nebula@lists.launchpad.net] On Behalf Of Lillie Ross-CDSR11 Sent: Monday, June 18, 2012 9:32 AM To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Dashboard (Ubuntu 12.04/Essex) All, I've upgraded our cloud, but am stymied on one last configuration issue. From the dashboard, I'm unable to display images and/or snapshots. However, I can display loaded images using the nova and glance command line tools with no problems. For example, 'glance index' displays the following: ID Name Disk Format Container Format Size -- -- f4c861aa-ded5-4cb8-9338-c0551a11a5d5 tty-linux ami ami25165824 bfc26a24-8cd5-4259-a57a-7071d909de8c tty-linux-ramdisk ari ari 5882349 df43feb9-aa30-4a2a-a03e-0357df5d4249 tty-linux-kernel aki aki 4404752 and the corresponding command through the nova-api service, 'nova image-list' displays: +--+---+++ | ID |Name | Status | Server | +--+---+++ | bfc26a24-8cd5-4259-a57a-7071d909de8c | tty-linux-ramdisk | ACTIVE || | df43feb9-aa30-4a2a-a03e-0357df5d4249 | tty-linux-kernel | ACTIVE || | f4c861aa-ded5-4cb8-9338-c0551a11a5d5 | tty-linux | ACTIVE || +--+---+++ However, when accessing the Images Snapshots panel in Horizon, I receive 2 error messages that it's unable to retrieve images and snapshots. Looking at the nova-api.log file, I see the following: 2012-06-18 11:13:30 INFO nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1] GEThttp://173.23.181.1:8776/v1/1/snapshots/detail 2012-06-18 11:13:30 DEBUG nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1] Unrecognized Content-Type provided in request from (pid=9586) get_body /u sr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:697 2012-06-18 11:13:30 INFO nova.api.openstack.wsgi [req-65ef5523-b1ff-47a1-8bd6-4a896df47958 1 1]http://173.23.181.1:8776/v1/1/snapshots/detail returned with HTTP 200 which indicates that the request complete OK. From the Apache access logs I see 127.0.0.1:80 173.17.11.184 - - [18/Jun/2012:11:13:24 -0500] GET /nova/images_and_snapshots/ HTTP/1.1 200 49213 http://cloud.dtl.mot-solutions.com/nova/images_and_snaps hots/ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2 which also seems to indicate that all is well with 49K data returned, however nothing displays in the dashboard. Any suggestions? Thanks to all in advance! Regards, Ross ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Timeout during image build (task Networking)
Vish, Jay,
OK, this looks promising. A couple of questions…
I'm seeing this RPC timeout on the Essex 2012.1 packages released with Ubuntu
12.04. I'm assuming these packages are affected by this bug?
Why would something this fundamental not show up during Essex RC.X testing?
How best to 'fix' this for our production environment (thank god it's only the
research organization!)
My previous testing of Essex (running on Diablo) didn't exhibit this problem.
However during testing, I was configured using FlatDHCP networking versus our
production cloud using VLAN networking. This was what lead me to believe that
it might be a network configuration issue. Apparently not.
As it stands right now, we're dead in the water, so I hope some easy fix for
the Ubuntu Essex release is possible.
Thanks for everyone looking this. Hope to hear a resolution soon.
Regards,
Ross
On Jun 19, 2012, at 2:13 PM, Vishvananda Ishaya wrote:
Sorry, paste fail on the last message.
This seems like a likely culprit:
https://review.openstack.org/#/c/8339/
I'm guessing it only happens on concurrent builds? We probably need a
synchronized somewhere.
Vish
On Jun 19, 2012, at 12:03 PM, Jay Pipes wrote:
cc'ing Vish on this, as this is now occurring on every single devstack +
Tempest run, for multiple servers.
Vish, I am seeing the exact same issue as shown below. Instances end up in
ERROR state and looking into the nova-network log, I find *no* errors at all,
and yet looking at the nova-compute log, I see multiple timeout errors -- all
of them trying to RPC while in the allocate_network method. Always the same
method, always the same error, and no errors in nova-network or nova-api (other
than just reporting a failed build)
Any idea on something that may have crept in recently? This wasn't happening a
week or so ago, AFAICT.
Best,
-jay
On 06/18/2012 06:03 PM, Lillie Ross-CDSR11 wrote:
I'm receiving RPC timeouts when trying to launch an instance. My
installation is the Essex release running on Ubuntu 12.04.
When I launch a test image, the launch fails. In my setup, Nova network
runs on a controller node, and all compute instances run on separate,
dedicated server nodes. The failure is repeatable. Upon examining the
various logs, I see the following (see below). Any insight would be welcome.
Regards,
Ross
From 'nova show instance name' I read the following:
root@cirrus1:~# nova show test
+-+-+
| Property | Value |
+-+-+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-SRV-ATTR:host | nova8 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | instance-0005 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | networking |
| OS-EXT-STS:vm_state | error |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2012-06-18T20:42:56Z |
| fault | {u'message': u'Timeout', u'code': 500, u'created':
u'2012-06-18T20:43:58Z'} |
| flavor | m1.tiny |
| hostId | 50272989300483e2b5e5236cd572fef3f9149ae60faa5f5660f8da54 |
| id | d569b16f-10a8-4cb8-90a3-d5b664c2322d |
| image | tty-linux |
| key_name | admin |
| metadata | {} |
| name | test |
| private_0 network | |
| status | ERROR |
| tenant_id | 1 |
| updated | 2012-06-18T20:43:57Z |
| user_id | 1 |
+-+-+
From the nova-network.log I see the following:
2012-06-18 15:43:36 DEBUG nova.manager [-] Running periodic task
VlanManager._disassociate_stale_fixed_ips from (pid=1381) periodic_tasks
/usr/lib/python2.7/dist-packages
/nova/manager.py:152
2012-06-18 15:43:57 ERROR nova.rpc.common [-] Timed out waiting for RPC
response: timed out
2012-06-18 15:43:57 TRACE nova.rpc.common Traceback (most recent call last):
2012-06-18 15:43:57 TRACE nova.rpc.common File
/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py, line 490, in
ensure
2012-06-18 15:43:57 TRACE nova.rpc.common return method(*args, **kwargs)
2012-06-18 15:43:57 TRACE nova.rpc.common File
/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py, line 567, in
_consume
2012-06-18 15:43:57 TRACE nova.rpc.common return
self.connection.drain_events(timeout=timeout)
2012-06-18 15:43:57 TRACE nova.rpc.common File
/usr/lib/python2.7/dist-packages/kombu/connection.py, line 175, in
drain_events
2012-06-18 15:43:57 TRACE nova.rpc.common return
self.transport.drain_events(self.connection, **kwargs)
2012-06-18 15:43:57 TRACE nova.rpc.common File
/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py, line
238, in drain_events
2012-06-18 15:43:57 TRACE nova.rpc.common return
connection.drain_events(**kwargs)
2012-06-18 15:43:57 TRACE nova.rpc.common File
/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py, line
57, in drain_events
2012-06-18
