Re: [Openstack] Cinder Storage Server Statistics
When I said, we, I meant the ceilometer team. If the auditing app isn't finding any volumes, it's not going to notify us. If you just want to know how much data is being used by cinder, there may be a way to get that from their admin API, but I'm not sure. On Mon, Jul 15, 2013 at 7:08 PM, Ray Sun xiaoq...@gmail.com wrote: D oug, Thanks. I tried it in grizzly, here's the return: sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit Starting volume usage audit Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00 Found 0 volumes Volume usage audit completed Actually, I want to get some data like this: Total Cinder Storage on Physical Machine: 100G Used Cinder Storage on Physical Machine: 10G Is there any way to get this? Best Regards -- Ray On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: We rely on a similar audit program to get the exists notifications about cinder volumes. Look for cinder-volume-usage-audit. Doug On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun xiaoq...@gmail.com wrote: Yes, it should be, but seems not at least in grizzly. Any update of Ceilometer? Best Regards -- Ray On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang hao...@unitedstack.comwrote: I think Statistics should be find in Ceilometer. Ceilometer may provide with enough information you need. Best regards, Haomai Wang, UnitedStack Inc. 在 2013-7-14，上午8:09，Ray Sun xiaoq...@gmail.com 写道： In nova, we have a period task to report the usage of the physical server, including CPU, Memory and Local Disk, but I don't think I can find the same strategy in cinder service. Is there any way to do this or is there any blueprint for this? Thanks. Best Regards -- Ray ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
We rely on a similar audit program to get the exists notifications about cinder volumes. Look for cinder-volume-usage-audit. Doug On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun xiaoq...@gmail.com wrote: Yes, it should be, but seems not at least in grizzly. Any update of Ceilometer? Best Regards -- Ray On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang hao...@unitedstack.comwrote: I think Statistics should be find in Ceilometer. Ceilometer may provide with enough information you need. Best regards, Haomai Wang, UnitedStack Inc. 在 2013-7-14，上午8:09，Ray Sun xiaoq...@gmail.com 写道： In nova, we have a period task to report the usage of the physical server, including CPU, Memory and Local Disk, but I don't think I can find the same strategy in cinder service. Is there any way to do this or is there any blueprint for this? Thanks. Best Regards -- Ray ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer][Healthnmon]
The work to merge ceilometer and healthnmon hasn't made much progress, yet. The ceilometer team is working on collecting information about CPU and RAM utilization for triggering alarms for Heat. That work is schedule for the havana release, and is moving along nicely. We could use some help with code reviews, but I think the overall design is worked out. Doug On Mon, Jul 1, 2013 at 5:51 AM, Emanuel Marzini jema...@gmail.com wrote: Hi, I want to retrive Vm information from Openstack. I am interested of CPU RAM utilization. I known that someone use collectd, libvirt ecc.. or product like Ceilometer or Healthnmon. I am also interested to receive alarm information from the cloud provider if the CPU or RAM value exceeds a threshold. I read that from Havana, Ceilometer and Healthnmon will be merged, it's true? Might be a good solution to my problems? Thanks in advance Emanuel ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer-api Auth Error
On Thu, Jun 6, 2013 at 7:22 AM, Claudio Marques clau...@onesource.ptwrote: Hi Stackers Hi have a problem with ceilometer-api. I want access it via curl or http and every time i try to do it i simple get the same errors. This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. My ceilometer.conf file is like this: [DEFAULT] os_username=admin os_password=admin_pass os_tenant_name=admin os_auth_url=http://10.0.1.167:5000/v2.0/ signing_dirname = /tmp/keystone-signing-ceilometer metering_api_port=8777 auth_strategy=keystone nova_control_exchange=nova hypervisor_inspector=libvirt libvirt_type=qemu glance_control_exchange=glance quantum_control_exchange=quantum debug=true verbose=true log_dir=/var/log/ceilometer rpc_backend=ceilometer.openstack.common.rpc.impl_kombu rabbit_host=localhost rabbit_port=5672 rabbit_userid=guest rabbit_password=guest rabbit_retry_backoff=2 rabbit_max_retries=0 rabbit_use_ssl=False database_connection=mongodb://10.0.1.25:27017/ceilometer sql_connection_debug=0 cinder_control_exchange=cinder enable_v1_api=true [keystone_authtoken] auth_host = localhost auth_port = 5000 admin_user = admin admin_password = admin_pass admin_tenant_name = admin auth_uri = http://10.0.1.167:5000/v2.0/ What auth chould i pass in order to get metrics form ceilometer? The ceilometer API uses keystone authentication, just like the other OpenStack services. If you pass credentials for a regular user, you can see data about the tenant/project you're authenticating with. If you pass credentials for an admin user, you can see all data. Doug Thank's for any reply ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer-api Auth Error
On Thu, Jun 6, 2013 at 10:43 AM, claudio marques mrqss_...@hotmail.comwrote:
Hi Doug
I send authentications from the admin user. In order for me to explain you
better the issue i will paste here the phases that i am trying to do:
I use curl and send the admin user/password and ask for a token:
curl -d '{auth:{passwordCredentials:{username: admin, password:
admin_pass}}}' -H Content-type: application/json
http://localhost:35357/v2.0/tokens
It returns this
{access: {token: {issued_at: 2013-06-06T14:11:26.005501,
expires: 2013-06-07T14:11:26Z, id:
MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE=},
serviceCatalog: [], user: {username: admin, roles_links: [],
id: 0b14e1644ffa4336967087458cd85c56, roles: [], name: admin},
metadata: {is_admin: 0, roles: []}}}
Then, i use curl again with the token that i just received with the
following command
curl -k -D -H X-Auth-Token:
MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE=
-X 'GET' -v http://localhost:8777/v2/meters
The return value is this
** getaddrinfo(3) failed for X-Auth-Token:80*
** Couldn't resolve host 'X-Auth-Token'*
** Closing connection 0*
*curl: (6) Couldn't resolve host 'X-Auth-Token'*
It looks like curl does not like the way you are passing some of the
parameters. It is interpreting the header name as a host name.
Doug
** About to connect() to localhost port 8777 (#1)*
** Trying 127.0.0.1...*
** Connected to localhost (127.0.0.1) port 8777 (#1)*
* GET /v2/meters HTTP/1.1*
* User-Agent: curl/7.29.0*
* Host: localhost:8777*
* Accept: */**
**
** HTTP 1.0, assume close after body*
* HTTP/1.0 401 Unauthorized*
* Date: Thu, 06 Jun 2013 14:14:06 GMT*
* Server: WSGIServer/0.1 Python/2.7.4*
* WWW-Authenticate: Keystone uri='http://127.0.0.1:35357'*
* Content-Length: 381*
* Content-Type: text/html; charset=UTF-8*
**
*html*
* head*
* title401 Unauthorized/title*
* /head*
* body*
* h1401 Unauthorized/h1*
* This server could not verify that you are authorized to access the
document you requested. Either you supplied the wrong credentials (e.g.,
bad password), or your browser does not understand how to supply the
credentials required.br /br /*
*Authentication required*
*
*
*
*
* /body*
** Closing connection 1*
No data is returned in any case.
I manually installed 3 node openStack and ceilometer , so i am not using
devStack.
Even when i try to send manually credentials of the admin user:
*ceilometer --os-username admin --os-password password --os-tenant-name
admin --os-auth-url http://localhost:5000/v2.0 resource-list*
the result is this:
*No handlers could be found for logger ceilometerclient.common.http*
*Invalid OpenStack Identity credentials.*
Is there any possibility that keystone is not validating all the Tokens?
Claudio Marques
--
Date: Thu, 6 Jun 2013 09:42:38 -0400
From: doug.hellm...@dreamhost.com
To: clau...@onesource.pt
CC: openstack@lists.launchpad.net
Subject: Re: [Openstack] Ceilometer-api Auth Error
On Thu, Jun 6, 2013 at 7:22 AM, Claudio Marques clau...@onesource.ptwrote:
Hi Stackers
Hi have a problem with ceilometer-api. I want access it via curl or http
and every time i try to do it i simple get the same errors.
This server could not verify that you are authorized to access the
document you requested. Either you supplied the wrong credentials (e.g.,
bad
Re: [Openstack] Ceilometer-api Auth Error
This is also a good point. I think you're getting an unscoped token, which
won't be useful for authenticating to ceilometer.
Doug
On Thu, Jun 6, 2013 at 12:33 PM, Guangyu Suo guan...@unitedstack.comwrote:
Hello, claudio
I don't think you get the real token, because you did't specify the
tenantName or tenantID in curl command, even if you used admin and
admin_pass. Please read this document
http://docs.openstack.org/api/quick-start/content/ for details.
2013/6/6 claudio marques mrqss_...@hotmail.com
Hi Doug
I send authentications from the admin user. In order for me to explain
you better the issue i will paste here the phases that i am trying to do:
I use curl and send the admin user/password and ask for a token:
curl -d '{auth:{passwordCredentials:{username: admin, password:
admin_pass}}}' -H Content-type: application/json
http://localhost:35357/v2.0/tokens
It returns this
{access: {token: {issued_at: 2013-06-06T14:11:26.005501,
expires: 2013-06-07T14:11:26Z, id:
MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE=},
serviceCatalog: [], user: {username: admin, roles_links: [],
id: 0b14e1644ffa4336967087458cd85c56, roles: [], name: admin},
metadata: {is_admin: 0, roles: []}}}
Then, i use curl again with the token that i just received with the
following command
curl -k -D -H X-Auth-Token:
MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE=
-X 'GET' -v http://localhost:8777/v2/meters
The return value is this
** getaddrinfo(3) failed for X-Auth-Token:80*
** Couldn't resolve host 'X-Auth-Token'*
** Closing connection 0*
*curl: (6) Couldn't resolve host 'X-Auth-Token'*
** About to connect() to localhost port 8777 (#1)*
** Trying 127.0.0.1...*
** Connected to localhost (127.0.0.1) port 8777 (#1)*
* GET /v2/meters HTTP/1.1*
* User-Agent: curl/7.29.0*
* Host: localhost:8777*
* Accept: */**
**
** HTTP 1.0, assume close after body*
* HTTP/1.0 401 Unauthorized*
* Date: Thu, 06 Jun 2013 14:14:06 GMT*
* Server: WSGIServer/0.1 Python/2.7.4*
* WWW-Authenticate: Keystone uri='http://127.0.0.1:35357'*
* Content-Length: 381*
* Content-Type: text/html; charset=UTF-8*
**
*html*
* head*
* title401 Unauthorized/title*
* /head*
* body*
* h1401 Unauthorized/h1*
* This server could not verify that you are authorized to access the
document you requested. Either you supplied the wrong credentials (e.g.,
bad password), or your browser does not understand how to supply the
credentials required.br /br /*
*Authentication required*
*
*
*
*
* /body*
** Closing connection 1*
No data is returned in any case.
I manually installed 3 node openStack and ceilometer , so i am not using
devStack.
Even when i try to send manually credentials of the admin user:
*ceilometer --os-username admin --os-password password --os-tenant-name
admin --os-auth-url http://localhost:5000/v2.0 resource-list*
the result is this:
*No handlers could be found for logger ceilometerclient.common.http*
*Invalid OpenStack Identity credentials.*
Is there any possibility that keystone is not validating all the Tokens?
Claudio Marques
--
Date: Thu, 6 Jun 2013 09:42:38 -0400
From: doug.hellm...@dreamhost.com
To: clau...@onesource.pt
CC: openstack@lists.launchpad.net
Subject: Re: [Openstack] Ceilometer-api Auth Error
On Thu, Jun 6, 2013 at 7:22 AM, Claudio Marques clau...@onesource.ptwrote:
Re: [Openstack] [ceilometer] support for older versions of ceilometer
On Fri, May 31, 2013 at 4:11 AM, Julien Danjou jul...@danjou.info wrote: On Thu, May 30 2013, Tim Bell wrote: I hope that ceilometer can also include this within the Havana timeframe as it becomes a key component of production, large scale clouds. I think we all agree in principle. We'd be happy to enhance Ceilometer and fix potential bugs in this direction, but we'll require some help testing the upgrade path *before* Havana is released. Right, and upgrades are a different issue than running production systems long-term on mixed versions. Most of the problems between grizzly or trunk ceilometer and folsom have had to do with proof-of-concept deployments where people were trying to install a modern ceilometer on the same server as another component that was older. Because the services require incompatible libraries and were running on the same host using the same site-packages directories, there are conflicts and things broke (either ceilometer, or the other service). Deploying into virtualenvs or using separate hosts will reduce (if not eliminate) these problems. The protocol ceilometer uses to talk over the message bus is compatible between versions, AFAIK, so should not have issues with rolling upgrades. Doug -- Julien Danjou # Free Software hacker # freelance consultant # http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] support for older versions of ceilometer
The ceilometer team has had a few requests for help with older versions of ceilometer or running the grizzly version of ceilometer with older versions of other OpenStack components lately. We appreciate the level of interest in the project but, as much as we would like to, unfortunately we are not always able to help everyone with these mixed configurations. During the early phases of our development we tried to maintain compatibility with folsom, even well into the grizzly development cycle. We are no longer doing that, however, and so to make clear what support we can offer, the ceilometer team has put together the statement below. It has been added to our documentation, and we are posting here to make sure everyone has a chance to see it. Regards, Doug The ceilometer team has limited capacity to provide support for older versions of the project. Because the project graduated from incubation around the time of the grizzly release, that is the first version for which we will provide regular ongoing support following the standard deprecation cycle for OpenStack [1]. The grizzly version of ceilometer cannot be installed on the same server with earlier versions of OpenStack because of conflicting package requirements, but is API compatible with the folsom release if installed separately. [1] https://wiki.openstack.org/wiki/StableBranch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] support for older versions of ceilometer
From what I understand, it is unusual to support mixing components from different releases like that. Am I wrong? Doug On Thu, May 30, 2013 at 3:29 PM, Tim Bell tim.b...@cern.ch wrote: ** ** Doug, ** ** Can you advise on what the plan/policy will be for Havana ? ** ** **- **Will I be able to run Havana core components such as Nova with Grizzly ceilometer ? **- **Will I be able to run Havana ceilometer with Grizzly core components (with reduced functionality compared to the Havana core components) ? ** ** Tim ** ** *From:* Openstack [mailto:openstack-bounces+tim.bell= cern...@lists.launchpad.net] *On Behalf Of *Doug Hellmann *Sent:* 30 May 2013 18:59 *To:* openstack@lists.launchpad.net *Subject:* [Openstack] [ceilometer] support for older versions of ceilometer ** ** The ceilometer team has had a few requests for help with older versions of ceilometer or running the grizzly version of ceilometer with older versions of other OpenStack components lately. We appreciate the level of interest in the project but, as much as we would like to, unfortunately we are not always able to help everyone with these mixed configurations. During the early phases of our development we tried to maintain compatibility with folsom, even well into the grizzly development cycle. We are no longer doing that, however, and so to make clear what support we can offer, the ceilometer team has put together the statement below. It has been added to our documentation, and we are posting here to make sure everyone has a chance to see it. ** ** Regards, Doug ** ** ** ** The ceilometer team has limited capacity to provide support for older versions of the project. Because the project graduated from incubation around the time of the grizzly release, that is the first version for which we will provide regular ongoing support following the standard deprecation cycle for OpenStack [1]. The grizzly version of ceilometer cannot be installed on the same server with earlier versions of OpenStack because of conflicting package requirements, but is API compatible with the folsom release if installed separately. ** ** [1] https://wiki.openstack.org/wiki/StableBranch ** ** ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer][Ceilometer-API] Ceilometer-API Error 401 Unauthorized
Right now we only have a web API, which you can use via the client library, curl, or the command line tool. There are a few people working on integrating ceilometer data with horizon, but I don't know the status of that project. Doug On Tue, May 28, 2013 at 3:49 AM, Ildiko Vancsa ildiko.van...@gmail.comwrote: Hi All, I'm new to OpenStack and Ceilometer as well, so I have a few questions. :) Does Ceilometer supports a Web UI or it is available via command line and curl only? I installed the environment with DevStack as I wanted to check how it works and looks like and it sets ceilometer in the apache2 serveice as an enabled site, but I could not find any information about how to set it up correctly. Thanks and Best Regards, Ildiko ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New code name for networks
On Sat, May 11, 2013 at 5:48 PM, Anne Gentle a...@openstack.org wrote: On Sat, May 11, 2013 at 3:12 PM, Monty Taylor mord...@inaugust.comwrote: On 05/11/2013 04:07 PM, Asher Newcomer wrote: Or even better, just continue to call it openstack networking. The code names only serve to confuse the uninitiated. They needlessly steepen the learning curve and slow uptake. The problem with OpenStack Networking (or getting rid of codenames) is seen with pre-incubation-incubation-integrated cycle. A project cannot call itself OpenStack Foo until it actually _is_ openstack foo. So any new project by necessity has to start off as something else. But - if we then require them to drop that name and become openstack foo when they become incubated or integrated, then we've got what's become a stable project with a decent amount of contributors renaming itself. Every. Time. The code names aren't just cute. I kind of wish they were, because it would make several things much easier if we could just ditch them and do a pure openstack. code namespace. But the reality is that it gets really really tricky to deal with a bunch of things if they go away. I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I find this trickiness a weak argument in the face of the invented names that are getting to be as bad as U.S. pharmaceuticals. Plus it forces us to put a lookup table in the docs forever. [1] Let's find a process for naming that meets a few reqs: - describes the service - goes through a legal review - enables new eyes to understanding it by reading the English word that the service represents (that can be translated into a meaningful word in other languages) If we have to preface with Stackforge to indicate pre-incubation, that's fine. Use Incubating or some such to indicate incubation. Meaningful words have meaning. +1 Use a namespace package openstack then each project has a unique package under that for their meaningfully named package (compute, networking, etc.). We only have to rename projects if they start out with bad names to begin with. Projects that want to be integrated should be developing on stackforge and using the openstack namespace package. I acknowledge we still have to indicate what commands, daemon names, and so on are related to a particular service. That issue is also fixable with some resources and effort and pays off in easier adoption and understanding. The unified command line project will resolve the command issue because all commands will be openstack $noun $verb (end users shouldn't have to know which OpenStack component owns a resource in order to operate on it via the command line). Daemon startup scripts could use a similar framework, or just a naming convention (openstack-compute-api, openstack-network-api, etc.). Doug Anne 1. http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/terminology-codenames.html On May 11, 2013 3:59 PM, Davanum Srinivas dava...@gmail.com mailto:dava...@gmail.com wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.net mailto:m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.com mailto:jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather than just refer to it as networking let's come up with a new code name! Thoughts? Thanks, -js ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Davanum Srinivas :: http://davanum.wordpress.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing
Re: [Openstack] New code name for networks
On Mon, May 13, 2013 at 11:30 AM, Sean Dague s...@dague.net wrote: On 05/13/2013 11:03 AM, Doug Hellmann wrote: +1 Use a namespace package openstack then each project has a unique package under that for their meaningfully named package (compute, networking, etc.). Sounds great and all, except when you try to do something like quantum, or even cinder, where you break out function into another project. from openstack import network - wait, is that what used to be nova network, or quantum, or some abstraction? Fair point, and those specific examples of names may not be good. :-) On the other hand, there's no reason each team needs to be limited to deploying a single package. If nova network was openstack.network to begin with, then perhaps quantum could have just been an evolution of that. from openstack.compute import network as compute_network ? Code names are actually incredibly useful in developing this stuff, because it lets us think about an implementation separate from a concept, and work with non native english speakers a lot easier where concept vs. implementation. Honestly, there is already incredible confusion when you talk with people about compute. Where you have to be really paying attention to nuance to figure out if people meant Nova as a whole, the Nova compute daemon, A nova compute node, which might also have nova-network on it. The number of times we had to explain no-db-compute blueprint because of that speaks to the fact that generic names do not make anything easier, they generate more confusion quite often. Code names are useful because it gives us a whole other namespace of words to work with to be very specific about what we mean, that can't be confused for the generic concepts of networking or computing. Yes, it's inside baseball, but when you are dealing with code as complicated as OpenStack, not having that inside baseball really slows things down. This seems like a stronger argument than it's hard to rename things (which I think is at least a large part of the argument against moving away from code names during/after incubation). FWIW, I'm in favor of not branding every little thing we do openstack, especially when components can be reused by other projects (I would love to see us acquire a few downstream users of some of the Oslo libraries, for example). But for the big core stuff, it feels weird to have separate commands, separate package namespaces, etc. Just look at the regular confusion new people have about the 2 uses of the term migrations in Nova, one for the database schema, and one for moving guests around. :) I'm not sure we're going to eliminate confusion entirely. I'm not even sure changing anything will reduce it. But it does seem like we can minimize legal issues by avoiding using code names in the code, which is where this thread started. Doug -Sean -- Sean Dague http://dague.net ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer Install
The database schema isn't part of the formal API, so if you're OK with code
breaking as we make changes to that schema then it would be fine to run
without the API. If you want to ensure your app continues to work across
those changes, it should be straightforward to set up the API server in a
virtualenv.
Doug
On Tue, May 7, 2013 at 7:13 AM, Riki Arslan riki.ars...@cloudturk.netwrote:
Thank you for the through explanation. I have no problem running the
Collector, Central and Compute Agents. So, I believe only the API Server is
trying to use the old oslo-incubator version.
I am still weighing the options.
Just a quick question; since the only thing that does not work in my
environment is the API Server, I believe -as long as I can query MongoDB
directly-, I think I wouldn't need it anyway. Would you say this is correct?
On Mon, May 6, 2013 at 6:08 PM, Doug Hellmann doug.hellm...@dreamhost.com
wrote:
It looks like you still have incompatible versions of things installed.
The configuration library changed during grizzly. The old version and new
version cannot be used together in the same program because they both try
to modify different copies of a global variable. The exception you're
getting is, I think, due to the fact that the API service loads the
keystone middleware to handle authentication. You have a version of the
middleware that uses oslo.config, and a version of ceilometer that uses the
older oslo-incubator version of the configuration library.
The ceilometer team is small, so we have limited capacity to support
old versions (especially pre-incubated versions). We do intend to support
grizzly, but can only offer moderate help with folsom. The g2 release
tarballs *should* be compatible at the communication layer with folsom
versions of the other components, but it looks like you can't install them
into the same Python installation as the other services.
You can separate ceilometer code from the other services a couple of
different ways. The simplest would be to use a separate VM to run
ceilometer. That would let you follow all of the normal instructions, and
ensure that you don't have mismatched versions of libraries. The other way
is to install ceilometer into a virtualenv. That would take more care,
since you need to ensure that the virtualenv does not look at the globally
installed site-packages. I haven't tried doing this, so I can't provide
more detailed steps, and you will likely need to experiment a bit to get it
right.
The one piece of ceilometer that does *need* to be installed in the same
location as the other services is the plugin for the nova compute agent. We
spent a fair amount of time making sure there was a version of that plugin
compatible with folsom, so we believe it should work. However, if you are
just testing ceilometer, or not using it for billing instance-hours, you
could skip deploying that piece entirely.
Doug
On Mon, May 6, 2013 at 9:43 AM, Riki Arslan riki.ars...@cloudturk.netwrote:
I have also installed ceilometer-2013.1~g2~20130107.449.tar.gz from the
tarballs list and still getting the same error:
Traceback (most recent call last):
File /usr/local/bin/ceilometer-api, line 5, in module
pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 499, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1235,
in run_script
execfile(script_filename, namespace, namespace)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api,
line 37, in module
cfg.CONF(sys.argv[1:])
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1024, in __call__
self._cli_values, leftovers = self._parse_cli_opts(args)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1527, in _parse_cli_opts
opt._add_to_cli(self._oparser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 591, in _add_to_cli
container = self._get_argparse_container(parser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 633, in _get_argparse_container
return group._get_argparse_group(parser)
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
On Mon, May 6, 2013 at 3:56 PM, Riki Arslan
riki.ars...@cloudturk.netwrote:
Hi Doug,
I actually got it from a link on your website:
http://doughellmann.com/2013/01/ceilometer-grizzly-2-milestone-available.html
So, do you think this one is not good?
On Thu, May 2, 2013 at 7:33 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Mon, Apr 29, 2013 at 6:42 PM, Riki Arslan
riki.ars...@cloudturk.net
Re: [Openstack] Ceilometer Install
It looks like you still have incompatible versions of things installed.
The configuration library changed during grizzly. The old version and new
version cannot be used together in the same program because they both try
to modify different copies of a global variable. The exception you're
getting is, I think, due to the fact that the API service loads the
keystone middleware to handle authentication. You have a version of the
middleware that uses oslo.config, and a version of ceilometer that uses the
older oslo-incubator version of the configuration library.
The ceilometer team is small, so we have limited capacity to support old
versions (especially pre-incubated versions). We do intend to support
grizzly, but can only offer moderate help with folsom. The g2 release
tarballs *should* be compatible at the communication layer with folsom
versions of the other components, but it looks like you can't install them
into the same Python installation as the other services.
You can separate ceilometer code from the other services a couple of
different ways. The simplest would be to use a separate VM to run
ceilometer. That would let you follow all of the normal instructions, and
ensure that you don't have mismatched versions of libraries. The other way
is to install ceilometer into a virtualenv. That would take more care,
since you need to ensure that the virtualenv does not look at the globally
installed site-packages. I haven't tried doing this, so I can't provide
more detailed steps, and you will likely need to experiment a bit to get it
right.
The one piece of ceilometer that does *need* to be installed in the same
location as the other services is the plugin for the nova compute agent. We
spent a fair amount of time making sure there was a version of that plugin
compatible with folsom, so we believe it should work. However, if you are
just testing ceilometer, or not using it for billing instance-hours, you
could skip deploying that piece entirely.
Doug
On Mon, May 6, 2013 at 9:43 AM, Riki Arslan riki.ars...@cloudturk.netwrote:
I have also installed ceilometer-2013.1~g2~20130107.449.tar.gz from the
tarballs list and still getting the same error:
Traceback (most recent call last):
File /usr/local/bin/ceilometer-api, line 5, in module
pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 499, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1235, in
run_script
execfile(script_filename, namespace, namespace)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api,
line 37, in module
cfg.CONF(sys.argv[1:])
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1024, in __call__
self._cli_values, leftovers = self._parse_cli_opts(args)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1527, in _parse_cli_opts
opt._add_to_cli(self._oparser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 591, in _add_to_cli
container = self._get_argparse_container(parser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 633, in _get_argparse_container
return group._get_argparse_group(parser)
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
On Mon, May 6, 2013 at 3:56 PM, Riki Arslan riki.ars...@cloudturk.netwrote:
Hi Doug,
I actually got it from a link on your website:
http://doughellmann.com/2013/01/ceilometer-grizzly-2-milestone-available.html
So, do you think this one is not good?
On Thu, May 2, 2013 at 7:33 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Mon, Apr 29, 2013 at 6:42 PM, Riki Arslan
riki.ars...@cloudturk.netwrote:
I thought it might help if mentioned little more:
/etc/ceilometer.conf file has the following parameters added:
os_username=ceilometer
os_password=$PASSWORD
os_tenant_name=service
os_auth_url=http://localhost:5000/v2.0/
I checked CLI_OPTIONS in service.py and it looks allright:
CLI_OPTIONS = [
cfg.StrOpt('os-username',
default=os.environ.get('OS_USERNAME', 'ceilometer'),
help='Username to use for openstack service access'),
cfg.StrOpt('os-password',
default=os.environ.get('OS_PASSWORD', 'admin'),
help='Password to use for openstack service access'),
cfg.StrOpt('os-tenant-id',
default=os.environ.get('OS_TENANT_ID', ''),
help='Tenant ID to use for openstack service access'),
cfg.StrOpt('os-tenant-name',
default=os.environ.get('OS_TENANT_NAME', 'admin'),
help='Tenant name to use
Re: [Openstack] Ceilometer Install
On Mon, Apr 29, 2013 at 6:42 PM, Riki Arslan riki.ars...@cloudturk.netwrote:
I thought it might help if mentioned little more:
/etc/ceilometer.conf file has the following parameters added:
os_username=ceilometer
os_password=$PASSWORD
os_tenant_name=service
os_auth_url=http://localhost:5000/v2.0/
I checked CLI_OPTIONS in service.py and it looks allright:
CLI_OPTIONS = [
cfg.StrOpt('os-username',
default=os.environ.get('OS_USERNAME', 'ceilometer'),
help='Username to use for openstack service access'),
cfg.StrOpt('os-password',
default=os.environ.get('OS_PASSWORD', 'admin'),
help='Password to use for openstack service access'),
cfg.StrOpt('os-tenant-id',
default=os.environ.get('OS_TENANT_ID', ''),
help='Tenant ID to use for openstack service access'),
cfg.StrOpt('os-tenant-name',
default=os.environ.get('OS_TENANT_NAME', 'admin'),
help='Tenant name to use for openstack service access'),
cfg.StrOpt('os-auth-url',
default=os.environ.get('OS_AUTH_URL',
'http://localhost:5000/v2.0'),
help='Auth URL to use for openstack service access'),
]
But still, according to the error I am getting, it can not parse
_parse_cli_opts:
Traceback (most recent call last):
File /usr/local/bin/ceilometer-api, line 5, in module
pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 499, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1235, in
run_script
execfile(script_filename, namespace, namespace)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api,
line 38, in module
service.prepare_service()
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py,
line 80, in prepare_service
cfg.CONF(argv[1:], project='ceilometer')
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1024, in __call__
self._cli_values, leftovers = self._parse_cli_opts(args)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1527, in _parse_cli_opts
opt._add_to_cli(self._oparser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 591, in _add_to_cli
container = self._get_argparse_container(parser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 633, in _get_argparse_container
return group._get_argparse_group(parser)
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
I am really puzzled as Collector, Computer Agent and Central Agent are
working fine and Api Server is not.
I don't see a 2013.1~g2.tar.gz tarball listed under
http://tarballs.openstack.org/ceilometer/. Where did you get the source you
are working with?
You may have a bad snapshot, since it is trying to combine
ceilometer/openstack/common/cfg.py with oslo.config.
Doug
On Tue, Apr 30, 2013 at 12:56 AM, Riki Arslan
riki.ars...@cloudturk.netwrote:
Hi Doug,
I have followed the document. The only thing that is different from the
docs is that I did not copy the yaml file (it does not exist in tarball):
cp etc/ceilometer/*.yaml /etc/ceilometer
However, the tarball is the g2 version, which is the last version that
was supposed to work with Folsom.
It seems like Collector, Computer Agent and Central Agent are working. I
only can't get the Api Server working.
On Fri, Apr 26, 2013 at 6:19 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
It sounds like you haven't completed the installation instructions. I
don't know if the manual steps listed at
http://docs.openstack.org/developer/ceilometer/install/manual.html work
with the tarball, but they should be close.
Doug
On Fri, Apr 26, 2013 at 3:46 AM, Riki Arslan
riki.ars...@cloudturk.netwrote:
The command line I am using is: sudo /usr/local/bin/ceilometer-api.
However, the ceilometer.ini file is missing. The version of Ceilometer
I am using is ceilometer-2013.1~g2.tar.gz. And, I only have the
following configuration files:
/etc/ceilometer/ceilometer.conf
/etc/ceilometer/policy.json
/etc/ceilometer/sources.json
Thanks.
On Fri, Apr 26, 2013 at 1:10 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Apr 25, 2013 at 8:37 AM, Riki Arslan
riki.ars...@cloudturk.net wrote:
I thought Ceilometer did not set a dependency on any DB drivers. I
have installed the driver Mongo using sudo pip install pymongo.
Ceilometer does use a database. You have to install the right driver.
If you want
Re: [Openstack] Ceilometer Install
It sounds like you haven't completed the installation instructions. I don't
know if the manual steps listed at
http://docs.openstack.org/developer/ceilometer/install/manual.html work
with the tarball, but they should be close.
Doug
On Fri, Apr 26, 2013 at 3:46 AM, Riki Arslan riki.ars...@cloudturk.netwrote:
The command line I am using is: sudo /usr/local/bin/ceilometer-api.
However, the ceilometer.ini file is missing. The version of Ceilometer I
am using is ceilometer-2013.1~g2.tar.gz. And, I only have the following
configuration files:
/etc/ceilometer/ceilometer.conf
/etc/ceilometer/policy.json
/etc/ceilometer/sources.json
Thanks.
On Fri, Apr 26, 2013 at 1:10 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Apr 25, 2013 at 8:37 AM, Riki Arslan
riki.ars...@cloudturk.netwrote:
I thought Ceilometer did not set a dependency on any DB drivers. I have
installed the driver Mongo using sudo pip install pymongo.
Ceilometer does use a database. You have to install the right driver. If
you want Mongo, then it sounds like you've done the right thing. It's
possible mako is also being used somewhere else, I'm not sure.
Regarding the current problem; the traceback is as follows:
Traceback (most recent call last):
File /usr/local/bin/ceilometer-api, line 5, in module
pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 499, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1235,
in run_script
execfile(script_filename, namespace, namespace)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api,
line 38, in module
service.prepare_service()
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py,
line 80, in prepare_service
cfg.CONF(argv[1:], project='ceilometer')
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1024, in __call__
self._cli_values, leftovers = self._parse_cli_opts(args)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1527, in _parse_cli_opts
opt._add_to_cli(self._oparser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 591, in _add_to_cli
container = self._get_argparse_container(parser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 633, in _get_argparse_container
return group._get_argparse_group(parser)
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
That is coming from oslo.config. Can you post the ceilometer.ini file and
command line you are using to start the service?
Doug
Thank for the help.
On Thu, Apr 25, 2013 at 3:27 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thursday, April 25, 2013, Riki Arslan wrote:
I have encountered other problems too.
First of all, when starting the Central Agent I have had Glance
endpoint 404 not found errors. As, Julien pointed out (
https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed
the v1 from the Glance URLs and it worked well.
Secondly, when starting the API Server, I have received ImportError:
No module named mako.template error. Thus, I have installed python-mako
module (sudo apt-get install python-mako), and the error disappeared.
Mako is a dependency do sqlalchemy, I think. Are you using the
sqlalchemy storage driver for ceilometer?
Now, I am receiving another error within the API Server. The error is
as follows:
AttributeError: 'OptGroup' object has no attribute
'_get_argparse_group'
That sounds like a problem with the config module. Was there a full
traceback? If not, try adding the --debug option when starting the service.
Doug
Do you think it has something to do with mod_wsgi (
http://docs.openstack.org/developer/ceilometer/install/mod_wsgi.html)?
I would appreciate your help on this.
Thanks.
On Thu, Apr 25, 2013 at 12:27 AM, Riki Arslan
riki.ars...@cloudturk.net wrote:
Hi Doug,
Your email helped me. It was actually glanceclient version 0.5.1 that
was causing the conflict. After updating it, the conflict error
disappeared.
I hope this would help someone else too.
Thanks again.
On Wed, Apr 24, 2013 at 11:49 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan
riki.ars...@cloudturk.net wrote:
Hi,
We are trying to install ceilometer-2013.1~g2.tar.gz which
presumably has Folsom compatibility.
The requirment is python-keystoneclient=0.2,0.3 and we have
the version 2.3.
But, still, setup quits with the following message:
error: Installed distribution python
Re: [Openstack] Ceilometer Install
On Thursday, April 25, 2013, Riki Arslan wrote:
I have encountered other problems too.
First of all, when starting the Central Agent I have had Glance endpoint
404 not found errors. As, Julien pointed out (
https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed the
v1 from the Glance URLs and it worked well.
Secondly, when starting the API Server, I have received ImportError: No
module named mako.template error. Thus, I have installed python-mako
module (sudo apt-get install python-mako), and the error disappeared.
Mako is a dependency do sqlalchemy, I think. Are you using the sqlalchemy
storage driver for ceilometer?
Now, I am receiving another error within the API Server. The error is as
follows:
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
That sounds like a problem with the config module. Was there a full
traceback? If not, try adding the --debug option when starting the service.
Doug
Do you think it has something to do with mod_wsgi (
http://docs.openstack.org/developer/ceilometer/install/mod_wsgi.html)?
I would appreciate your help on this.
Thanks.
On Thu, Apr 25, 2013 at 12:27 AM, Riki Arslan
riki.ars...@cloudturk.netjavascript:_e({}, 'cvml',
'riki.ars...@cloudturk.net');
wrote:
Hi Doug,
Your email helped me. It was actually glanceclient version 0.5.1 that was
causing the conflict. After updating it, the conflict error disappeared.
I hope this would help someone else too.
Thanks again.
On Wed, Apr 24, 2013 at 11:49 PM, Doug Hellmann
doug.hellm...@dreamhost.com javascript:_e({}, 'cvml',
'doug.hellm...@dreamhost.com'); wrote:
On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan
riki.ars...@cloudturk.netjavascript:_e({}, 'cvml',
'riki.ars...@cloudturk.net');
wrote:
Hi,
We are trying to install ceilometer-2013.1~g2.tar.gz which presumably
has Folsom compatibility.
The requirment is python-keystoneclient=0.2,0.3 and we have
the version 2.3.
But, still, setup quits with the following message:
error: Installed distribution python-keystoneclient 0.2.3
conflicts with requirement python-keystoneclient=0.1.2,0.2
The funny thing is, although pip-requires states
python-keystoneclient=0.2,0.3, the error message complains that it is
not python-keystoneclient=0.1.2,0.2.
Something else you have installed already wants an older version of the
keystone client, so the installation of ceilometer is not able to upgrade
to the version we need.
Doug
Your help is greatly appreciated.
Thank you in advance.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net javascript:_e({}, 'cvml',
'openstack@lists.launchpad.net');
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer Install
On Thu, Apr 25, 2013 at 8:37 AM, Riki Arslan riki.ars...@cloudturk.netwrote:
I thought Ceilometer did not set a dependency on any DB drivers. I have
installed the driver Mongo using sudo pip install pymongo.
Ceilometer does use a database. You have to install the right driver. If
you want Mongo, then it sounds like you've done the right thing. It's
possible mako is also being used somewhere else, I'm not sure.
Regarding the current problem; the traceback is as follows:
Traceback (most recent call last):
File /usr/local/bin/ceilometer-api, line 5, in module
pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 499, in
run_script
self.require(requires)[0].run_script(script_name, ns)
File /usr/lib/python2.7/dist-packages/pkg_resources.py, line 1235, in
run_script
execfile(script_filename, namespace, namespace)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api,
line 38, in module
service.prepare_service()
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py,
line 80, in prepare_service
cfg.CONF(argv[1:], project='ceilometer')
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1024, in __call__
self._cli_values, leftovers = self._parse_cli_opts(args)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py,
line 1527, in _parse_cli_opts
opt._add_to_cli(self._oparser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 591, in _add_to_cli
container = self._get_argparse_container(parser, group)
File
/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py,
line 633, in _get_argparse_container
return group._get_argparse_group(parser)
AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
That is coming from oslo.config. Can you post the ceilometer.ini file and
command line you are using to start the service?
Doug
Thank for the help.
On Thu, Apr 25, 2013 at 3:27 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thursday, April 25, 2013, Riki Arslan wrote:
I have encountered other problems too.
First of all, when starting the Central Agent I have had Glance endpoint
404 not found errors. As, Julien pointed out (
https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed the
v1 from the Glance URLs and it worked well.
Secondly, when starting the API Server, I have received ImportError: No
module named mako.template error. Thus, I have installed python-mako
module (sudo apt-get install python-mako), and the error disappeared.
Mako is a dependency do sqlalchemy, I think. Are you using the sqlalchemy
storage driver for ceilometer?
Now, I am receiving another error within the API Server. The error is as
follows:
AttributeError: 'OptGroup' object has no attribute
'_get_argparse_group'
That sounds like a problem with the config module. Was there a full
traceback? If not, try adding the --debug option when starting the service.
Doug
Do you think it has something to do with mod_wsgi (
http://docs.openstack.org/developer/ceilometer/install/mod_wsgi.html)?
I would appreciate your help on this.
Thanks.
On Thu, Apr 25, 2013 at 12:27 AM, Riki Arslan riki.ars...@cloudturk.net
wrote:
Hi Doug,
Your email helped me. It was actually glanceclient version 0.5.1 that
was causing the conflict. After updating it, the conflict error
disappeared.
I hope this would help someone else too.
Thanks again.
On Wed, Apr 24, 2013 at 11:49 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan
riki.ars...@cloudturk.net wrote:
Hi,
We are trying to install ceilometer-2013.1~g2.tar.gz which
presumably has Folsom compatibility.
The requirment is python-keystoneclient=0.2,0.3 and we have
the version 2.3.
But, still, setup quits with the following message:
error: Installed distribution python-keystoneclient 0.2.3
conflicts with requirement python-keystoneclient=0.1.2,0.2
The funny thing is, although pip-requires states
python-keystoneclient=0.2,0.3, the error message complains that it is
not python-keystoneclient=0.1.2,0.2.
Something else you have installed already wants an older version of
the keystone client, so the installation of ceilometer is not able to
upgrade to the version we need.
Doug
Your help is greatly appreciated.
Thank you in advance.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer Install
On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan riki.ars...@cloudturk.netwrote: Hi, We are trying to install ceilometer-2013.1~g2.tar.gz which presumably has Folsom compatibility. The requirment is python-keystoneclient=0.2,0.3 and we have the version 2.3. But, still, setup quits with the following message: error: Installed distribution python-keystoneclient 0.2.3 conflicts with requirement python-keystoneclient=0.1.2,0.2 The funny thing is, although pip-requires states python-keystoneclient=0.2,0.3, the error message complains that it is not python-keystoneclient=0.1.2,0.2. Something else you have installed already wants an older version of the keystone client, so the installation of ceilometer is not able to upgrade to the version we need. Doug Your help is greatly appreciated. Thank you in advance. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ceilometer-agent-central starting fail
On Thu, Apr 11, 2013 at 11:34 PM, Liu Wenmao marvel...@gmail.com wrote:
Thanks, the ceilometer seems to lack some default options in configuration
files and the official guidance. （
http://docs.openstack.org/developer/ceilometer/configuration.html）
I have opened a bug to address the missing details in the configuration
documentation. https://bugs.launchpad.net/ceilometer/+bug/1168375
Doug
So maybe it is not ready for users yet?
On Wed, Apr 10, 2013 at 8:28 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Wed, Apr 10, 2013 at 6:10 AM, Liu Wenmao marvel...@gmail.com wrote:
Actually this is not over.
The main reason of service failure is that central/manager.py
and service.py use different vairables:
central/manager.py
70 def interval_task(self, task):
71 self.keystone = ksclient.Client(
72 username=cfg.CONF.*os_username*,
73 password=cfg.CONF.os_password,
74 tenant_id=cfg.CONF.os_tenant_id,
75 tenant_name=cfg.CONF.os_tenant_name,
76 auth_url=cfg.CONF.os_auth_url)
44 CLI_OPTIONS = [
45 cfg.StrOpt('*os-username*',
46default=os.environ.get('OS_USERNAME', 'ceilometer'),
47help='Username to use for openstack service access'),
48 cfg.StrOpt('os-password',
49default=os.environ.get('OS_PASSWORD', 'admin'),
50help='Password to use for openstack service access'),
51 cfg.StrOpt('os-tenant-id',
52default=os.environ.get('OS_TENANT_ID', ''),
53help='Tenant ID to use for openstack service access'),
54 cfg.StrOpt('os-tenant-name',
55default=os.environ.get('OS_TENANT_NAME', 'admin'),
56help='Tenant name to use for openstack service
access'),
57 cfg.StrOpt('os_auth_url',
58default=os.environ.get('OS_AUTH_URL',
59 'http://localhost:5000/v2.0'),
So after I change all - to _ and modify all options in
/etc/ceilometer/ceilometer.conf, the service starts OK.
The thing that fixed it was changing - to _ in your configuration
file. The options library allows option names to have - in them so they
look nice as command line switches, but the option name uses the _.
Doug
On Wed, Apr 10, 2013 at 2:02 PM, Liu Wenmao marvel...@gmail.com wrote:
I solve this problem by two steps:
1 modify /etc/init/ceilometer-agent-central.conf
exec start-stop-daemon --start --chuid ceilometer --exec
/usr/local/bin/ceilometer-agent-central --
--config-file=/etc/ceilometer/ceilometer.conf
2 add some lines to /etc/ceilometer/ceilometer.conf:
os-username=ceilometer
os-password=nsfocus
os-tenant-name=service
os-auth-url=http://controller:5000/v2.0
On Wed, Apr 10, 2013 at 1:36 PM, Liu Wenmao marvel...@gmail.comwrote:
Hi all:
I have just install ceilometer grizzly github version, but fail to
start ceilometer-agent-central service. I think it is due to that I didn't
set up the keystone user/password like other projects. but I follow the
instructions(
http://docs.openstack.org/developer/ceilometer/install/manual.html#configuring-keystone-to-work-with-api)
but it does not include the ceilometer configuration.
# service ceilometer-agent-central start
ceilometer-agent-central start/running, process 5679
# cat /etc/init/ceilometer-agent-central.conf
description ceilometer-agent-compute
author Chuck Short zul...@ubuntu.com
start on runlevel [2345]
stop on runlelvel [!2345]
chdir /var/run
pre-start script
mkdir -p /var/run/ceilometer
chown ceilometer:ceilometer /var/run/ceilometer
mkdir -p /var/lock/ceilometer
chown ceilometer:ceilometer /var/lock/ceilometer
end script
exec start-stop-daemon --start --chuid ceilometer --exec
/usr/local/bin/ceilometer-agent-central
/var/log/ceilometer/ceilometer-agent-central.log
2013-04-10 13:01:39ERROR [ceilometer.openstack.common.loopingcall]
in looping call
Traceback (most recent call last):
File
/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/openstack/common/loopingcall.py,
line 67, in _inner
self.f(*self.args, **self.kw)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/central/manager.py,
line 76, in interval_task
auth_url=cfg.CONF.os_auth_url)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py,
line 134, in __init__
self.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/client.py,
line 205, in authenticate
token)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py,
line 174, in get_raw_token_from_identity_servicetoken=token)
File
/usr/local/lib/python2.7/dist-packages
Re: [Openstack] ceilometer-agent-central starting fail
On Wed, Apr 10, 2013 at 6:10 AM, Liu Wenmao marvel...@gmail.com wrote:
Actually this is not over.
The main reason of service failure is that central/manager.py
and service.py use different vairables:
central/manager.py
70 def interval_task(self, task):
71 self.keystone = ksclient.Client(
72 username=cfg.CONF.*os_username*,
73 password=cfg.CONF.os_password,
74 tenant_id=cfg.CONF.os_tenant_id,
75 tenant_name=cfg.CONF.os_tenant_name,
76 auth_url=cfg.CONF.os_auth_url)
44 CLI_OPTIONS = [
45 cfg.StrOpt('*os-username*',
46default=os.environ.get('OS_USERNAME', 'ceilometer'),
47help='Username to use for openstack service access'),
48 cfg.StrOpt('os-password',
49default=os.environ.get('OS_PASSWORD', 'admin'),
50help='Password to use for openstack service access'),
51 cfg.StrOpt('os-tenant-id',
52default=os.environ.get('OS_TENANT_ID', ''),
53help='Tenant ID to use for openstack service access'),
54 cfg.StrOpt('os-tenant-name',
55default=os.environ.get('OS_TENANT_NAME', 'admin'),
56help='Tenant name to use for openstack service access'),
57 cfg.StrOpt('os_auth_url',
58default=os.environ.get('OS_AUTH_URL',
59 'http://localhost:5000/v2.0'),
So after I change all - to _ and modify all options in
/etc/ceilometer/ceilometer.conf, the service starts OK.
The thing that fixed it was changing - to _ in your configuration file.
The options library allows option names to have - in them so they look
nice as command line switches, but the option name uses the _.
Doug
On Wed, Apr 10, 2013 at 2:02 PM, Liu Wenmao marvel...@gmail.com wrote:
I solve this problem by two steps:
1 modify /etc/init/ceilometer-agent-central.conf
exec start-stop-daemon --start --chuid ceilometer --exec
/usr/local/bin/ceilometer-agent-central --
--config-file=/etc/ceilometer/ceilometer.conf
2 add some lines to /etc/ceilometer/ceilometer.conf:
os-username=ceilometer
os-password=nsfocus
os-tenant-name=service
os-auth-url=http://controller:5000/v2.0
On Wed, Apr 10, 2013 at 1:36 PM, Liu Wenmao marvel...@gmail.com wrote:
Hi all:
I have just install ceilometer grizzly github version, but fail to
start ceilometer-agent-central service. I think it is due to that I didn't
set up the keystone user/password like other projects. but I follow the
instructions(
http://docs.openstack.org/developer/ceilometer/install/manual.html#configuring-keystone-to-work-with-api)
but it does not include the ceilometer configuration.
# service ceilometer-agent-central start
ceilometer-agent-central start/running, process 5679
# cat /etc/init/ceilometer-agent-central.conf
description ceilometer-agent-compute
author Chuck Short zul...@ubuntu.com
start on runlevel [2345]
stop on runlelvel [!2345]
chdir /var/run
pre-start script
mkdir -p /var/run/ceilometer
chown ceilometer:ceilometer /var/run/ceilometer
mkdir -p /var/lock/ceilometer
chown ceilometer:ceilometer /var/lock/ceilometer
end script
exec start-stop-daemon --start --chuid ceilometer --exec
/usr/local/bin/ceilometer-agent-central
/var/log/ceilometer/ceilometer-agent-central.log
2013-04-10 13:01:39ERROR [ceilometer.openstack.common.loopingcall]
in looping call
Traceback (most recent call last):
File
/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/openstack/common/loopingcall.py,
line 67, in _inner
self.f(*self.args, **self.kw)
File
/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/central/manager.py,
line 76, in interval_task
auth_url=cfg.CONF.os_auth_url)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py,
line 134, in __init__
self.authenticate()
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/client.py,
line 205, in authenticate
token)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py,
line 174, in get_raw_token_from_identity_servicetoken=token)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py,
line 202, in _base_authN
resp, body = self.request(url, 'POST', body=params, headers=headers)
File
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/client.py,
line 366, in request
raise exceptions.from_response(resp, resp.text)
Unauthorized: Unable to communicate with identity service: {error:
{message: Invalid user / password, code: 401, title: Not
Authorized}}. (HTTP 401)
2013-04-10 13:01:39
Re: [Openstack] How to purge old meter data of Ceilometer
I created a blueprint for this: https://blueprints.launchpad.net/ceilometer/+spec/purge-data On Thu, Apr 4, 2013 at 4:06 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Apr 04 2013, Harri Pyy wrote: With the default 1 minute interval, Ceilometer collects quite large amounts of meter data. Does Ceilometer provide a TTL configuration option for the meter data, or some other functionality or API for purging old meter data? Not yet unfortunately. -- Julien Danjou ;; Free Software hacker ; freelance consultant ;; http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer question
You want the statistics command from the v2 API. ceilometer --ceilometer-api-version 2 help statistics The CLI needs a little work to make the options clear, but the API documentation may help explain: http://docs.openstack.org/developer/ceilometer/webapi/v2.html Doug On Tue, Mar 26, 2013 at 12:22 PM, Wyllys Ingersoll wyllys.ingers...@evault.com wrote: Perhaps this is documented but I can't seem to find it… I want to use ceilometer to show usage from the swift objectstore. What would the correct command be to extract that information? Currently, the various -list commands just spit out various meter types along with resource and project id values. Thanks, Wyllys Ingersoll EVault On Mar 26, 2013, at 11:47 AM, Julien Danjou jul...@danjou.info wrote: Hi, We're pleased to announce that the RC1 version for the Grizzly release of Ceilometer is available! https://launchpad.net/ceilometer/grizzly/grizzly-rc1 Unless release-critical issues are found that warrant a release candidate respin, this version will be formally released as the Ceilometer 2013.1 version on April 4. You are therefore strongly encouraged to test and validate this tarball. If you find an issue that could be considered release-critical, please file it at: https://bugs.launchpad.net/ceilometer/+filebug Note that the master branch for Ceilometer is now open for Havana development, and feature freeze restriction no longer apply. -- Julien Danjou -- Free Software hacker - freelance consultant -- http://julien.danjou.info ATT1___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Regarding openstack metering
Hi, Arumon, You will find the ceilometer documentation at http://docs.openstack.org/developer/ceilometer/ There are some basic installation and configuration instructions, as well as an architectural overview. Please do not hesitate to ask questions, so we can expand and improve what is documented so far based on your feedback. Doug On Fri, Mar 8, 2013 at 5:36 AM, Aru s arumo...@gmail.com wrote: Hi, I am new to openstack and looking for the metering tools. I read about the ceilometer, but not found any proper documentation for this. Please help me understand regarding the available metering tools. Regards, Arumon ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] UK Trademark issue with Python
The Python Software Foundation, maintainers of the IP behind the Python language, are currently fighting a trademark claim against a company in the UK that is trying to trademark the use of the term Python for all software, services, and servers apparently as part of branding their cloud service. If you work for a company with an office in an EU Community member state, the PSF would appreciate your help documenting the community's prior claim to the name Python. For details about where to send supporting evidence, please see the PSF blog post [1]. Thanks, Doug [1] http://pyfound.blogspot.com/2013/02/python-trademark-at-risk-in-europe-we.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Name it Hood!
Will we have hoodies instead of t-shirts for ODS attendees? Doug On Thu, Jan 24, 2013 at 2:50 PM, Monty Taylor mord...@inaugust.com wrote: Hey all! Here's my pitch for Hood: a) It's the tallest mountain in Oregon, and honestly, it's a pretty kick-ass mountain in general b) Being in the pacific northwest, the mountain itself is quite regularly in the clouds. That's gotta count for something. c) It's actually a volcano. d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in Cuba. (We should have a design summit in cuba!!!) e) Harbor is super-problematic because of the US/UK clash in spelling. Half of us will spell it wrong no matter what. f) Hood is only 4 letters. Think about that when you think about typing hatfield a lot. Also, if we name it hatfield, we're going to have to have the M summit somewhere that has a town called McCoy. g) I'll buy you a beer at the summit if you vote for Hood. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Calling all user group and meetup organizers
Someone on the call today offered meeting space in Alpharetta, GA but I couldn't hear your name clearly at the time. Could you drop me a note off list, please, so we can talk about those arrangements? Thanks, Doug On Tue, Jan 8, 2013 at 7:54 PM, Sean Roberts sean...@yahoo-inc.com wrote: We are going to have a planning meeting on Tuesday, 15 Jan 2012 11:00am to 1:00pm PST. RSVP via http://www.meetup.com/openstack/events/93593062/ Connect remotely via webex https://yahoomeetings.webex.com/yahoomeetings/j.php?ED=160663792UID=492396097RT=MiM0 If this time doesn't work for you, get ahold of me directly via skype:seanroberts66, email, mobile, irc:sarob, twitter:sarob, or carrier pigeon. Stefano and Thierry will be joining us. I want to get input from people that run user groups and meetups. See you then! Sean Roberts Infrastructure Strategy sean...@yahoo-inc.com Direct (408) 349-5234 Mobile (925) 980-4729 701 First Avenue, Sunnyvale, CA, 94089-0703, US Phone (408) 349-3300 Fax (408) 349-3301 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Is python-openstackclient dead?
On Sun, Dec 23, 2012 at 9:39 AM, Alessio Ababilov aababi...@griddynamics.com wrote: Hi everyone! There is a nice project in OpenStack - python-openstackclient. Its purpose is to provide a convenient command line interface to all OpenStack services, including nova. glance, and keystone, according to http://wiki.openstack.org/UnifiedCLI. I noticed that the last commit that added functionality to this project was on August 20, 2012. In current state, python-openstackclient implements only operations with nova instances and several keystone commands, so, the most of required functionality is not ready. In the same time, python-novaclient and python-keystoneclient still extend their command line interfaces - the latest commits were accepted in December, 2012. So, instead of moving CLI to python-openstackclient, it is developed in separate python-*client projects. Is python-openstackclient considered to be dead? Please, do not kill this project! Now it is really convenient, and all it needs is just more attention from the community that will add lacked functionality! Lets us drop CLI support from miscellaneous python-PROJECTclients and add it to python-openstackclient! It's not dead, but most of the contributors haven't had time to prioritize working on it during this cycle. If you are interested in helping out, I'll be happy to do code reviews for you. Doug -- Alessio Ababilov Software Engineer Grid Dynamics ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
+1 It will be good to see if the storage API design works for other backends like HBase. Doug On Thu, Nov 8, 2012 at 5:50 AM, shengjie_...@dell.com wrote: Thanks Julien, I will write up a blueprint soon. Shengjie -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: 07 November 2012 18:12 To: Min, Shengjie Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint? On Wed, Nov 07 2012, shengjie_...@dell.com wrote: Hi Shengjie, I am looking for a way to propose a blueprint to the ceilometer team - Adding Hbase storage backend for ceilometer. Currently we have only MongoDB and SQLalchemy as the possible ceilometer storage backend. Given the storage interface is clearly designed and allows us to have different backend, it would be nice to have more options which make Ceilometer more adoptive to some providers' existing architectures. As a cloud IAAS provider, Ceilometer might not be just considered as a usage db, also it's being seen as a centralized place to store all the usage data cross platforms. Additionally, all the historical usage data stored can be used for historical usage analysis as well, this is where MapReduce comes into play. With Hadoop Hbase, it allows usage data to be stored in HDFS, also it gives us the ability to run large scale massive MapRed analysis jobs in the future. This is indeed a good idea, and as you pointed out, Ceilometer collector has been designed so this is being possible. I think you can create a blueprint here: https://blueprints.launchpad.net/ceilometer And write in what should be done. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Thu, Nov 8, 2012 at 9:33 AM, shengjie_...@dell.com wrote: Hi Doug, ** ** +1 It will be good to see if the storage API design works for other backends like HBase. ** ** The first glance I had on ceilometer/storage/base.py, the design of basic storage class does look ok to work with. ** ** The only challenge is that the Hbase schema is not as flexible as traditional RDBMS or MongoDB, so the efforts would be more on the Hbase implementation side. Eg. the design of row key, column family etc. I hope that the API provides enough encapsulation to let you implement that in a reasonable way. ** ** We can discuss more once I had a blueprint draft registered J I'm looking forward to it! Doug ** ** -- Thanks, Shengjie ** ** ** ** ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
On Wed, Nov 7, 2012 at 10:21 PM, Sandy Walsh sandy.wa...@rackspace.comwrote: Hey! (sorry for the top-posting, crappy web client) There is a periodic task already in the compute manager that can handle this: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3021 There seems to be some recent (to me) changes in the manager now wrt the resource_tracker.py and stats.py files about how this information gets relayed. Now it seems it only goes to the db, but previously it was sent to a fanout queue that the schedulers could use. Regardless, this is done at a high enough level that it doesn't really care about the underlying virt layer, so long at the virt layer supports the get_available_resource() method. https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L152 https://github.com/openstack/nova/blob/master/nova/virt/xenapi/driver.py#L392 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2209 I'd add a hook in there do what we want with this data. Write it to the db, send it over the wire, whatever. If there is additional information required, it should go in this dictionary (or we should define a format for extensions to it). It looks like that is collecting resource data, but not usage data. For example, there's no disk I/O information, just disk space. Is that what you mean by adding extra information to the dictionary? Doug The --periodic_interval value is meant to be the fastest tick approach and the individual methods have to deal with how many multiples of the base tick it should use. So you can have different data reported at different intervals. Now, the question of polling vs. pushing shouldn't really matter if the sampling rate is predetermined. We can push when the sample is taken or we can read from some other store from an external process ... but the sampling should only be done in one place, once. Hope I answered your question? If not, just repeat it in another way and I'll try again :) -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh= rackspace@lists.launchpad.net] on behalf of Eoghan Glynn [ egl...@redhat.com] Sent: Wednesday, November 07, 2012 4:32 PM To: OpenStack Development Mailing List Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ... Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks for putting this together Sandy, We were debating on IRC (#heat) earlier the merits of moving the ceilometer emission logic into the services, e.g. directly into the nova-compute node. At first sight, this seemed to be what you were getting at with the suggestion: Remove the Compute service that Ceilometer uses and integrate the existing fanout compute notifications into the data collected by the workers. There's no need for yet-another-worker. While this could be feasible for measurements driven directly by notifications, I'm struggling with the idea of moving say the libvirt polling out of the ceilometer compute agent, as this seems to leak too many monitoring-related concerns directly into nova (cadence of polling, semantics of libvirt stats reported etc.). So I just wanted to clarify whether the type of low level unification you're proposing includes both push pull (i.e. notification polling) or whether you mainly had just former in mind when it comes to ceilometer. Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Nov 6, 2012, at 6:45 AM, Tim Bell tim.b...@cern.ch wrote: There is a use case for base metal hardware metering in the private cloud where the user allocated the machine does not have root access to kill the metering. How can we detect that special case? Being able to create a single metering infrastructure for the entire private cloud, virtual or bare-metal allocation, is a need technically, it is not clear how to guarantee it but it is worth exploring. I agree, it would be good to have an answer. Ceilometer can already hold the data, even if the agent to collect it is a custom solution. Doug Tim -Original Message- From: openstack-bounces+tim.bell=cern...@lists.launchpad.net [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf Of Robert Collins Sent: 06 November 2012 11:00 To: Graf Lucas (graflu0); Zehnder Toni (zehndton); openstack@lists.launchpad.net; Doug Hellmann Subject: Re: [Openstack] [ceilometer] Monitoring physical devices On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Fri, Nov 2, 2012 at 4:42 PM, Dan Dyer dan.dye...@gmail.com wrote: Yes, I am assuming the service controller provides a different stream of data from the lower level VM events. So the question is how to represent and store this additional meta data in ceilometer. Note that there doesn't necessarily need to be a linkage/grouping between the resources since the association is what is actually contained in the metadata that is provided by the service controller. As a summary Nova provides its normal events for usage Service controller provides a mapping of nova instances to service type and actual end user So the problem isn't necessarily that you want to measure something different, but that the ownership in the existing data is not correct from the perspective of the billing system. We have a similar issue at DreamHost. Our existing user database has account ids that need to be mapped to tenant ids from keystone. Rather than putting that information in keystone, or ceilometer, we decided to store it in our system and have the DreamHost billing system drive the ceilometer API. Does it make sense to do something similar here? If we definitely want ceilometer to hold the metadata, then I could also see adding an API to let an outside system add metadata to a resource. That would let the PaaS code, which knows about each VM, store extra data that would be returned with the VM metadata when a caller visits /resources/resourceid. Would you expect to be able to query using the metadata? For example, provide the total instance hours for all instances with paas_tag=foo? Doug Dan On 11/1/2012 11:25 AM, Doug Hellmann wrote: On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer dan.dye...@gmail.com wrote: In some cases, the service controller is actually running inside a VM. It would not have access to the internals of the VM's. It maintains its metadata separately from the Nova infrastructure. It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or join it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug DD On 10/25/2012 2:25 AM, Nick Barcet wrote: Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each Unit of the service such as constant:uuid If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specified so that we can ignore the proper VM's. There has also been talk of hybrid billing models that charge for some part of the VM usage as well as other metrics. Once again we need a way to characterize the relationships so that processing can associate and filter correctly. Dan On 10/24/2012 3:35 PM, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about
Re: [Openstack] [ceilometer] Monitoring physical devices
On Fri, Nov 2, 2012 at 3:07 AM, Patrick Petit patrick.michel.pe...@gmail.com wrote: Folks, I'd like to add to this that physical server metering shouldn't be treated differently in Ceilometer now that bare metal provisioning framework enters into Grizzly. Physical servers will just become billable resources much like VMs. I am not speaking of physical server monitoring here. Just extending Ceilometer agent to also report usage data out of the physical box. Thanks for posting this, Patrick. I understood physical devices as host server not as bare-metal server. I suspect we could use the same agent framework, but almost all of the pollsters would need to be different because they would be running inside the guest OS rather than on a host VM, so the APIs they will use to collect the same data will be different. Doug Cheers Patrick Envoyé de mon iPad Le 1 nov. 2012 à 19:13, Julien Danjou jul...@danjou.info a écrit : On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 1:39 PM, Julien Danjou jul...@danjou.info wrote: On Wed, Oct 31 2012, Eoghan Glynn wrote: Would we have also have some 'misses' with the cumulative approach when the ceilometer agent was down? No, unless the counter resets several times while your agent is down. But delta has the same issue. If I understood the (\Sigma local maxima)-first idea correctly, the usage up to the first polling cycle would always be discounted from any duration. No, because if you have: Time | Value 0| 10 1| 30 2| 50 3| 80 4| 100 If your delta-pollster is down at 1 and 2, you restart at 3, therefore at 4 you'll send 20 as usage (100 minus 80). So you miss the delta between 10 (time 0) and 80 (time 3) (therefore 70 for free!). If you send right away 80 at time 3 when restarting, the API will be able to guess that between 0 and 3 the value went from 10 to 80. With delta approach, the API cannot guess that. Sure it can, you just need to move where the caching is done. Using a local cache to maintain the previous time a value was published you would know at time 3 that the last published value was 10, and so send 70. So the total will be correct. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer dan.dye...@gmail.com wrote: In some cases, the service controller is actually running inside a VM. It would not have access to the internals of the VM's. It maintains its metadata separately from the Nova infrastructure. It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or join it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug DD On 10/25/2012 2:25 AM, Nick Barcet wrote: Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each Unit of the service such as constant:uuid If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specified so that we can ignore the proper VM's. There has also been talk of hybrid billing models that charge for some part of the VM usage as well as other metrics. Once again we need a way to characterize the relationships so that processing can associate and filter correctly. Dan On 10/24/2012 3:35 PM, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate table and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable product/service In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Kind of the same here, if you don't want to really bill the vm, just don't meter them (or ignore the meters) and emit your own meter via your PaaS platform to bill your customer. Or is there a limitation I miss? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help :
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 1:21 PM, Zehnder Toni (zehndton) zehnd...@students.zhaw.ch wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: On Thu, Nov 01 2012, Julien Danjou wrote: On every physical compute node is the Ceilometer compute agent installed, right?! Yes. 1) Does the compute agent collect data of the physical machine as well or is it just collecting data of the virtual machines? Only virtual machines. 2) Could it be useful to enhance the Ceilometer agent to collect data from the physical servers? Why not. What do you have in mind exactly? My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? We are looking into adding monitoring features to ceilometer, but it is likely that your admin will want fresher (and different) data than we are collecting right now (the agent only polls every 10 minutes at this point). If you have a strong need for this, it would be great if you could work with us to help get it implemented more quickly. Doug Cheers, Toni ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Thu, Nov 1, 2012 at 1:31 PM, Eoghan Glynn egl...@redhat.com wrote: if you have: Time | Value 0 | 10 1 | 30 2 | 50 3 | 80 4 | 100 If your delta-pollster is down at 1 and 2, you restart at 3, therefore at 4 you'll send 20 as usage (100 minus 80). So you miss the delta between 10 (time 0) and 80 (time 3) (therefore 70 for free!). If you send right away 80 at time 3 when restarting, the API will be able to guess that between 0 and 3 the value went from 10 to 80. With delta approach, the API cannot guess that. Sure it can, you just need to move where the caching is done. Using a local cache to maintain the previous time a value was published you would know at time 3 that the last published value was 10, and so send 70. So the total will be correct. Good point, previously IIUC there was an implicit assumption that any prev time caching would be done in-memory, hence lost across process restarts. But as you point out, these data could be persisted locally by the compute agent. What would be the best way to achieve this? A small sqlite DB per-agent, or even simpler just a pickled dict? The latter would avoid the complexity of DB versioning and migration. I discussed this issue at the summit with James Penick of Yahoo, and he showed me some code in their agent that is using a sqllite db. We will want to build a nice API so pollsters can use the cache without having to worry about how it is implemented, which would let us deal with any versioning issues in a central spot. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 2:13 PM, Julien Danjou jul...@danjou.info wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. Good points. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 10:23 AM, Eoghan Glynn egl...@redhat.com wrote: Hi Yawei Wu, The root of the confusion is the fact the cpu meter is reporting the cumlative cpu_time stat from libvirt. This libvirt counter is reset when the associated qemu process is restarted (an artifact of how cpuacct works). So when you stop/start or suspend/resume, a fresh qemu process is sparked up, then the cumulative time is reset. Thanks for bringing this up, as it has implications as to how we meter CPU time and utilization[1]. We may need to start metering the delta between CPU times on subsequent polling cycles, instead of using a cumulative meter (dealing with the edge case where the instance has been restarted within a polling period). Good idea. We need to capture this issue to make sure we get it onto the roadmap for this cycle. Is there a bug or blueprint for it yet? Doug Cheers, Eoghan [1] https://review.openstack.org/14921 I am still testing ceilometer now. I am confused about the meter volume in the mongodb. Let's talk about cpu usage. After I create and boot a vm named vm_1, meter data record about cpu usage will be inserted into db in cycle(default 10 minutes). For example,the 'counter_volume' of the first record is '5206000',and the second one is '12389000'. 1) '12389000' nanoseconds means '123.89' seconds or two minutes,it seem like to be 1238.9 seconds actually, is there something wrong ? 2) If I never reboot or suspend vm_1, will the 'counter_volume' of cpu usage record increase all the time ? Just like '8 minutes' - '18 minutes' - '28 minutes' ? 3) If I reboot or suspend vm_1, I find that the 'counter_volume' of cpu usage record will count from zero. Just like '8 minutes' - '18 minutes' - '28 minutes' [- '0 minutes'] -'5 minutes' - '15 minutes'. Does it mean that 'counter_volume' just represents how long has vm_1 been booted up ? 4) This one is about Web API. I find that GET /v1/resources/(resource)/meters/(meter)/volume/sum just return the sum value of all the cpu 'counter_volume', like '8 minutes' + '18 minutes'. Is it reduplicate ? 5) If I want to know how long has vm_1's cpu been used yesterday, how can I do ? It seems like that I have too many questions.. Thank you very much ! --- Yawei Wu Dalian Hi-Think Computer Technology,Corp. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 11:25 AM, Eoghan Glynn egl...@redhat.com wrote: I don't think (max - min) would suffice to give an accurate measure of the actual CPU time used, as the counter may have reset multiple times in the course of the requested duration. It is, because /max in the API should be aware of the fact a reset can occur and computes accordingly. We started to discuss this a bit in: https://bugs.launchpad.net/ceilometer/+bug/1061817 A-ha, OK, so not so much (max - min) as: (\Sigma local maxima) - first Sounds computationally expensive to produce on the fly, but maybe the local maxima can be efficiently recorded as the data is being ingested. Is that better than just reporting the data in a more easily digested format in the first place? Julien, I don't understand your comment about losing data if your system is not launched to compute delta. Can you clarify what you mean there? I do understand that the agent would need to store state about the counter locally in order to track the delta value, but I think we could provide a convenient way for pollsters to do that without complicating them excessively. Doug Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up
I can't meet later than that tonight. Doug On Mon, Oct 29, 2012 at 10:21 AM, Sandy Walsh sandy.wa...@rackspace.comwrote: Ugh, I just realized I have a conflict. Can we push it an hour later? (sorry!) -S -- *From:* openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh= rackspace@lists.launchpad.net] on behalf of Annie Cheng [ ann...@yahoo-inc.com] *Sent:* Monday, October 29, 2012 11:18 AM *To:* openstack@lists.launchpad.net *Subject:* Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up Apologize .. Correction Time: Monday (10/29/2012) 2200 UTC – 2300 UTC (*3pm PDT *if you are in California) Location: IRC #openstack-meeting From: Annie Cheng ann...@yahoo-inc.com Date: Mon, 29 Oct 2012 07:14:08 -0700 To: openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up Reminder: Time: Monday (10/29/2012) 2200 UTC – 2300 UTC (2pm PDT if you are in California) Location: IRC #openstack-meeting Thanks! Annie From: Annie Cheng ann...@yahoo-inc.com Date: Thu, 25 Oct 2012 14:17:09 -0700 To: openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: [Openstack] Instrumentation Monitoring Next Step - quick meet up Hi all, Couple of us chat in the summit design sessions and and after summit on #openstack irc regarding topic of Monitoring. We think it's best to do a quick meeting to get everyone on the same page, split works, and get at least a prototype going in Grizzly. Time: Monday (10/29/2012) 2200 UTC – 2300 UTC Location: IRC #openstack-meeting I checked http://wiki.openstack.org/Meetings, this tme slot seems to be empty Top level agenda would be 1. Get everyone on the same page on high level direction 2. Discuss different design/implementation possibility 3. Split up works Before the meeting, if you want to read up, here are some links I know. Please jump in with others I missed: Blueprint: https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring Etherpad: https://etherpad.openstack.org/grizzly-common-instrumentation Different code samples: https://github.com/asalkeld/statgen Looking forward, some of those conversation probably will fold into the regular Metering meeting. Just like to do a one off for now so we can go deeper on monitoring specific topics. Thanks! Annie ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Oct 25, 2012, at 6:05 PM, Angus Salkeld asalk...@redhat.com wrote: On 25/10/12 17:04 -0400, Doug Hellmann wrote: On Thu, Oct 25, 2012 at 10:22 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Doug Hellmann wrote: That would be one way, but adding dimensions to the meters also makes sense because it reduces the need to collect the data more than once. In case of group, the other problem is how to emit instance counter with group metadata (assuming this group implementation is not part of Nova but Heat). Good point. I was assuming the values would be available as metadata of the underlying resource, but that may not always be the case. Yea, we need a consistent way of doing this. That should work on different resource types. We could use the tags or a similar mechanism. Tags would be available as part of an objects normal metadata, right? Doug -A For instance, if flavor was a dimension of the instance meter I wouldn't need the separate meter instance:flavor. These sorts of use cases were part of the original motivation for collecting all of the metadata about a resource, but what we have now isn't structured enough to let the API user query into it. IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. How, then, do we define the dimensions for a given meter in a more structured way? Some built-in values (like flavor) can be pulled automatically based on the resource type, but what about settings controlled by the deployer and end-user (for purposes other than billing)? Do we have to define dimensions explicitely, or isn't what's needed just ways to filter and/or group events by metadata fields? Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Doug -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info g ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Oct 26, 2012, at 4:29 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Doug Hellmann wrote: IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. Oh, sure it is. But adding feature and making things more efficient are different things. :) Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Ah, thanks to remind me how ORM are bad and that we now have to fight against it. :) I wish we could use JSON native type from PostgreSQL directly and be efficient! You could write a different storage driver. ;) Doug -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
On Oct 25, 2012, at 9:44 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: As for statgen, I think that¹s just a temp repo, it'd be nice to have the end result of this be a library that provides somewhat generic metrics and plugins and such so that stacktech could use the outputs of it, ceilometer could the outputs and other systems could use the outputs (where an output goes would be configurable so that each system can configure its outputs as the operator desires, ie I want my MONITOR metrics to go to MQ in ceilomter and stacktech consumable formats, or to files or to...). I think when this gets going we should have some repo/project name that goes on stackforge so that even while this is being developed it can go through the normal review process and such (and not in someones special github repo). But we have to start somewhere, something we can discuss as to what a good solution is @ the meeting. At the summit, as part of the discussions around expanding the scope of ceilometer to cover measurement for monitoring, we discussed developing the library as part of ceilometer for now and either moving it to Oslo for release as a library or just releasing the library as a separate package from the ceilometer project directly. Doug -Josh On 10/25/12 5:47 PM, Jeffrey Budzinski jeffr...@yahoo-inc.com wrote: Yes, I think support for metrics objects that can be leveraged both by monkey patches and decorators was what we'd been thinking along the lines of. The metrics would be controlled via config both in what scopes are active (e.g. on|off for a package, module, etc.) and also the outlet for the metrics (file, datagram, rpc). Support for instrumentation levels would also be nice so that metric flow could be controlled (i.e. instrumentation point is annotated as METRIC, MONITOR, PROFILE and then level to actually emit can be set in conjunction with a metric outlet/sink). With this approach, folks could control both the volume of metrics and also the outlet for the metrics. Ceilometer would also be an outlet that could be leveraged via RPC flow for metrics -- though I'd expect some would want to send via datagram to statsd or file for offline log aggregation. I'll post a diagram tomorrow for review and comment. Oh btw, I updated the spec with most of what was in the etherpad. We can update the spec to reflect whatever we decide is the preferred approach. -jeff On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: On 25/10/12 11:13 +, Sandy Walsh wrote: grizzly-common-instrumentation seems to be the best choice ... hopefully the other groups will use this etherpad too. We need a proper blueprint to nail down the approach. IRC is great, but doesn't retain history for other groups. I think we need to get a plan for translating the etherpad into something concise and nailed down. Agree. statgen should really just be a new notifier in Tach (or Scrutinize) ... vs copy-pasting the code into yet-another repo. Hopefully that's the plan? Tach should remain a generic tool and not pegged to OpenStack. Well that was just an ideas play pen not serious code. I might be coming at this from a slightly different angle... I was looking at a library that can be used to generate trace, monitoring and metering data (kind of like log levels for logging). Currently both Tach and Scrutinize don't have enough fields (of course that could be changed). Also I think we should be able to insert instrumentation into the code as well as have the function level performance metrics monkey patched. Then we could have a config that directed metric data to different notifiers like how you do it in Scrutinize perhaps. Also enforcing data rate limits and possible data aggregation could be neat configurable features. Anyway more at the meeting... -Angus -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Angus Salkeld [asalk...@redhat.com] Sent: Thursday, October 25, 2012 1:00 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems every time I turn around there's a new metric/instrumentation effort, which is good I guess :) Here is some fun I have been having with a bit of tach+ceilometer code.
Re: [Openstack] Ceilometer API Glossary
On Thu, Oct 25, 2012 at 5:39 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, 吴亚伟 wrote: If it is just like what Eoghan said that it is unused right now as there is only a single source currently, what does the ? represent? If I want to establish a customer billing system,do I need it ? It represents the fact we had no idea to use it when we write the code the first time. We discussed this yesterday, now we have a good plan. See https://bugs.launchpad.net/ceilometer/+bug/1070857 I do mean the cinder volumes , but I don't know how to configure it ,could you tell me more detailed steps on how to do this ? You need to configure the notifier to use rabbitmq (this is commented out in the default configuration file IIRC), to set the audit interval to something low like daily or hourlay (this should also be in the default configuration file) and to run this program into a cronjob. Same to cinder , detailed steps will be highly appreciated . Like cinder, you need to configure the notification backend to use rabbitmq. We have special instructions for configuring glance to use rabbit for notifications in http://ceilometer.readthedocs.org/en/latest/install.html#configuring-devstackbut I don't see anything about cinder there. Do we need to add another step to set the notification_driver for cinder? Doug If I modify the configuration file of service like nova or cinder , how to restart the related service in devstack ? Just press C-c in the screen window, up arrow and return. -- Julien Danjou /* Free Software hacker freelance http://julien.danjou.info */ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
That would be one way, but adding dimensions to the meters also makes sense because it reduces the need to collect the data more than once. For instance, if flavor was a dimension of the instance meter I wouldn't need the separate meter instance:flavor. These sorts of use cases were part of the original motivation for collecting all of the metadata about a resource, but what we have now isn't structured enough to let the API user query into it. How, then, do we define the dimensions for a given meter in a more structured way? Some built-in values (like flavor) can be pulled automatically based on the resource type, but what about settings controlled by the deployer and end-user (for purposes other than billing)? Doug On Thu, Oct 25, 2012 at 7:11 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Angus Salkeld wrote: So we need normal stuff like cpu/mem usage but aggregated over the instances in the group. So this is not easy to do externally. Interesting use case. I think for such a thing, a way would be to have a component listening to meters (e.g. on the bus directly or via PubSubHubbub-like) to re-emit consolidated meters. -- Julien Danjou /* Free Software hacker freelance http://julien.danjou.info */ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Oct 25, 2012 at 10:22 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Doug Hellmann wrote: That would be one way, but adding dimensions to the meters also makes sense because it reduces the need to collect the data more than once. In case of group, the other problem is how to emit instance counter with group metadata (assuming this group implementation is not part of Nova but Heat). Good point. I was assuming the values would be available as metadata of the underlying resource, but that may not always be the case. For instance, if flavor was a dimension of the instance meter I wouldn't need the separate meter instance:flavor. These sorts of use cases were part of the original motivation for collecting all of the metadata about a resource, but what we have now isn't structured enough to let the API user query into it. IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. How, then, do we define the dimensions for a given meter in a more structured way? Some built-in values (like flavor) can be pulled automatically based on the resource type, but what about settings controlled by the deployer and end-user (for purposes other than billing)? Do we have to define dimensions explicitely, or isn't what's needed just ways to filter and/or group events by metadata fields? Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Doug -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info g ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Ceilometer API Glossary
On Tue, Oct 23, 2012 at 5:34 AM, Eoghan Glynn egl...@redhat.com wrote: I am testing ceilometer in my devstack virtual machine. Although I can see the meter data model in the mongodb, I am confused about some glossary when I test its Web API. I am not very clear about the resource in the GET /v1/resources,either source in the GET /v1/sources/(source)/meters/(meter). Can someone explain these two concept? Thanks a lot. Hi Yawei, Good question! My understanding is that the resources collection refers to the set of UUIDs identifying the openstack entities (e.g. instance, volume, image ... etc.) being metered, whereas the source refers to the origin of the metering data and is effectively unused right now as there is only a single source currently (though we could in the future for example distinguish between metric and metering data in this way). That's exactly right. I'll open a ticket to add a glossary to the documentation. Doug Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Versioning for notification messages
On Oct 10, 2012, at 6:02 AM, Day, Phil philip@hp.com wrote: Whilst a version number would allow a consumer to detect that something has changed, it doesn’t really help in providing any kind of backward compatibility. Consider the following scenario: There are a bunch of systems external to Nova developed to consume notification messages, and someone introduces a change to the notification system that changes the message content. They do the right thing in updating the version number, but all of those external systems now need to change as well. The new version number lets them fail explicitly, but it could still have a significant impact on a production system. Where I’d like to get to make the notification system a formal external interface, that has the same degree of stability, version control, and rigor around changes that the inbound API has. I would guess that Ceilometer will have some requirement around this ? Yes, definitely. We will be resilient in the face of additions to the notification payloads, but our event processing plugins may break if fields are removed (mostly the envelope fields). Our primary requirement, though, is that notifications include all of the metadata for the object in question so we can track that accurately. We are still working with some of the projects to make that true in all cases. Doug Phil From: Diego Parrilla Santamaría [mailto:diego.parrilla.santama...@gmail.com] Sent: 10 October 2012 09:18 To: Day, Phil Cc: David Ripton; openstack@lists.launchpad.net Subject: Re: [Openstack] Versioning for notification messages If we want to have a notification system that could handle messages with different payloads and different versions, we have two options: 1) detect the version of the payload in the notification message 2) add a version number in the notification message Option 1 sounds to me like something hard to maintain. Option 2 seems to be correct way to do it in the long term. +1 for a version number in the notification message Cheers Diego -- Diego Parrilla CEO www.stackops.com | diego.parri...@stackops.com | +34 649 94 43 29 | skype:diegoparrilla On Wed, Oct 10, 2012 at 9:27 AM, Day, Phil philip@hp.com wrote: Hi All, I guess I may have mis-stated the problem a tad in talking about version numbering. The notification system is an outbound interface, and my interest is in being able to write consumers with some guarantee that they won't be broken as the notification message format evolves. Having a version number gives the client a way to know that it may now be broken, but that's not really the same as having an interface with some degree of guaranteed compatibility, Phil -Original Message- From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto:openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of David Ripton Sent: 09 October 2012 20:59 To: openstack@lists.launchpad.net Subject: Re: [Openstack] Versioning for notification messages On 10/09/2012 01:07 PM, Day, Phil wrote: What do people think about adding a version number to the notification systems, so that consumers of notification messages are protected to some extent from changes in the message contents ? For example, would it be enough to add a version number to the messages - or should we have the version number as part of the topic itself (so that the notification system can provide both a 1.0 and 1.1 feed), etc ? Putting a version number in the messages is easy, and should work fine. Of course it only really helps if someone writes clients that can deal with multiple versions, or at least give helpful error messages when they get an unexpected version. I think using separate topics for each version would be inefficient and error-prone. Inefficient because you'd have to send out multiples of each message, some of which would probably never be read. Obviously, if you're sending out N copies of each message then you expect only 1/N the queue performance. Worse, if you're sending out N copies of each message but only 1 of them is being consumed, your queue server is using a lot more memory than it needs to, to hold onto old messages that nobody needs. (If you properly configure a high-water mark or timeout, then the old messages will eventually be thrown away. If you don't, then your queue server will eventually consume way too much memory and start swapping, your cloud will break, and someone will get paged at 2 a.m.) Error-prone because someone would end up reusing the notification queue code for less idempotent/safe uses of queues, like internal API calls. And then client A would pick up the message from topic_v1, and client B would pick up the same message from topic_v2, and they'd both perform the same API operation, resulting in wasted resources in the
Re: [Openstack] Versioning for notification messages
On Tue, Oct 9, 2012 at 4:12 PM, Eric Windisch e...@cloudscaling.com wrote: On Tuesday, October 9, 2012 at 15:58 PM, David Ripton wrote: On 10/09/2012 01:07 PM, Day, Phil wrote: What do people think about adding a version number to the notification systems, so that consumers of notification messages are protected to some extent from changes in the message contents ? Right now, there is no appropriate or acceptable way to consume notifications. Plainly, while notifications exist, they shouldn't be considered stable or even usable until this exists. Message formats and versioning should be a consideration in the effort to create a reusable consumption pattern. This will be part of what is covered during the Using the message bus for messages session Tuesday afternoon at the summit. http://summit.openstack.org/cfp/details/117 Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Discussion / proposal: deleted column marker
On Wed, Oct 3, 2012 at 4:46 AM, Haynes, Dave (Cloud Services) dave.hay...@hp.com wrote: +1. Let's do it. If we need to add some extra tests to protect against regressions, then so be it. I will help. I also think better use could be made of the notifications system. A properly defined topic namespace would go a long way to assist that. The ceilometer project is making *extensive* use of the existing notifications to collect metering data about resources (I expect us to be consuming notifications from all of the other OpenStack services by the time we're done). So, if you're going to change the way notifications work, please bring us into that discussion. Here are a few proposed summit sessions related to notifications, messaging in general, and ceilometer integration: http://summit.openstack.org/cfp/details/110 http://summit.openstack.org/cfp/details/113 http://summit.openstack.org/cfp/details/117 http://summit.openstack.org/cfp/details/128 Both Nick Barcet and I will be at the summit. -Original Message- From: openstack-bounces+dave.haynes=hp@lists.launchpad.net [mailto: openstack-bounces+dave.haynes=hp@lists.launchpad.net] On Behalf Of Pitucha, Stanislaw Izaak Sent: 02 October 2012 17:43 To: openstack@lists.launchpad.net Subject: [Openstack] Discussion / proposal: deleted column marker Hi all, I'd like to open a discussion on a topic that's been bugging me for a number of reasons - soft deletes (by that I mean marking rows with deleted=1 in the db) and related - actions audit. Some research and speculations first... To be honest I could not find any reason why the feature is there in the first place. Here's the commit that introduced the 'deleted' columns: https://github.com/openstack/nova/commit/ae6905b9f1ef97206ee3c8722cec3b26fc0 64f38 - unfortunately the description says only Refactored orm to support atomic actions. So the guessing part starts here. These are the possible uses for soft-deletion of the database records that I could come up with: 1. safety net (recover data what was deleted by accident) 2. audit / log (preserve the information about past data) 3. some kind of micro-optimisation where update is more useful than deletion - be it speed or ease of handling foreign constraints (or not handling them straight away more likely) 4. ... no... that's all But I think there's a number of issues with that approach. First - what are the issues with the possible uses above. Then - issues that I can see otherwise. Point by point: 1. Soft-deletion probably makes some restoration possible, but I doubt there's much that could be done without full analysis of the situation. Mainly because the database is only about metainformation - the actual data users care about either goes away (ephemeral disks, memory, ...) or not (volumes, networks, ...) and is not recoverable. Since resources like ips and volumes can be just reused in other instances, not all recovery is possible anyway. Most hardcore fixes could be done by reinserting the original/reconstructed data just as easily as verifying what's safe to undelete. Both actions require looking at existing data and locking out information so it doesn't get reused while we're messing with the previous state. 2. Soft-deleted records are not great as a source of old information. This is connected to the previous point - some resources are just reused / rewritten instead of created and deleted. For example there's no record of what happens with old floating ips - the information gets overwritten when the IP is reassigned to the new instance, so the useful bits are gone. 3. This is the only thing I could come up with related to the commit message itself and the support atomic actions part. Maybe it was sometimes easier to mark something as deleted rather than managing and properly ordering deletes of a number of related entries. So with that out of the way, here's a number of issues related to soft-deletes that I run into myself: 4. Indexing all this data on a busy system is getting a bit silly. Unless you do your own cleanup of old entries, you will end up in a situation where looking up instances on a host actually looks through thousands of deleted rows even if only around 20 or so can be live and interesting. I know it's not a huge deal, but still an unnecessary cpu cycle burning. 5. Some things are just not possible to do in a safe and portable way at the moment. For example adding a new network and fixed IPs (there's a bug for that https://bugs.launchpad.net/nova/+bug/755138). I tried to fix this situation, but actually discovered that this is not possible to do using only sessions and with the 'deleted' column in place. There are ways to do it in a specific database (you can lock the whole table in mysql for example), but it's not portable then. The best you can do easily is limit the issue and hope that two inserts in different sessions won't happen at the
Re: [Openstack] Discussion / proposal: deleted column marker
On Wed, Oct 3, 2012 at 9:08 AM, Day, Phil philip@hp.com wrote: I *think* deleted flavours used to be needed as there could still be instances running against them and the flavour definition was used by the quota calculations. Not sure if this is still the case, or if the data now comes straight from the instances table.Some aspects of a flavour (e.g. rxtx_factor) could be useful to a scheduler, and that data currently isn't saved into the instance. I guess the usage audit type functionality (i.e. tell me about all instances that have run sometime in this audit period) may be another case where deleted instances are required at the moment. That data is being gathered by ceilometer now, so maybe we can do away with it in the nova database during Grizzly. Doug -Original Message- From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto: openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of Pitucha, Stanislaw Izaak Sent: 03 October 2012 13:09 To: openstack@lists.launchpad.net Subject: Re: [Openstack] Discussion / proposal: deleted column marker Hi Johannes, I know the names collide here, but since this technique is known as soft-deletes... We need more namespaces :) Thanks for the idea of grepping for read_deleted. Fortunately I think the situation isn't as bad as it would seem. Let me group the places which change read_deleted in the code (many results from grep are comments). Reading only deleted entries, or all: - xenserver (instance) - cleanup tool - I don't do xen, so I'm not sure how useful is it. Anyone? - tests - can be ignored - if there is no functionality, tests can be killed - sqlalchemy api (instance) - fixed ip can reference a deleted instance (tricky situation; from the commit message: It adds a test to verify that the code works with a duplicate deleted floating_ip - this seems very wrong...) - sqlalchemy api (iscsi) - getting deleted iscsi targets which are still referenced by volume - sqlalchemy api (various) - instance migration, s3image, bandwidth, storage manager, flavors (only available from nova-manage) - compute manager (instance) - reaping deleted instances - I can't see why the same logic wouldn't apply if the rows are actually missing (needs analysis, maybe there's a reason) - compute instance_types (flavour) - apparently flavours are usually read even if they're deleted - network manager (instance) - making sure that ips/networks can be removed even if the instance is already deleted So here's what I can see: pretty much all the usage is about deleting instances or making sure parts connected to instances go away if the instance is deleted earlier. It doesn't seem right, but could be progressively fixed. It looks like another state of the instance, which could be integrated into the other state fields. Nothing else uses the deleted column explicitly (unless I missed something - possible). Ips, networks, keys, anything that actually goes away permanently (and doesn't involve a big chain of cleanup events) is never read back once it's marked as deleted. So maybe a better approach is not to remove the deleted column completely, but to start stripping it from places where it's not needed (fixed, floating ips, networks, ssh keys being good candidates). This could be done by creating a new layer over NovaBase and removing the deleted marker from NovaBase itself. Or maybe even by creating a SoftDeleteMixin and applying it to all current models, then removing it where unnecessary? That would keep the current behaviour where it's currently needed, but at the same time it would provide a known migration path to get rid of it. We could start stripping the new layer then table by table and adding unique constraints where they make sense, before trying to tackle the really tricky parts (for instances table, maybe the marker actually makes sense? maybe not? - it's definitely not going to be an easy decision/fix) Regards, Stanisław Pitucha Cloud Services Hewlett Packard -Original Message- From: openstack-bounces+stanislaw.pitucha=hp@lists.launchpad.net [mailto:openstack-bounces+stanislaw.pitucha=hp@lists.launchpad.net] On Behalf Of Johannes Erdfelt Sent: Tuesday, October 02, 2012 6:12 PM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Discussion / proposal: deleted column marker On Tue, Oct 02, 2012, Pitucha, Stanislaw Izaak stanislaw.pitu...@hp.com wrote: Does anyone know why soft-delete is still in place? Are there any reasons it can't / shouldn't be removed at this time? If it's possible to remove it, would you miss it? I'm certainly not a fan of the database soft-delete for many of the same reasons you've described, but there are some places where removing it would require code changes. Off the top of my head would be pretty much anywhere a context sets read_deleted to 'yes' or 'only', which is a surprising number
Re: [Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)
On Tue, Sep 11, 2012 at 5:22 PM, Nick Barcet nick.bar...@canonical.comwrote: PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC The metering project team will hold its next meeting at alternate time on *Wednesday* at 9PM UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=21min=0sec=0 . Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * gmb to a plan on the ml with fixed dates * dhellmann make sure flask is listed as a dependency of ceilometer In case I can't make it to the meeting, I did check that Flask is listed in the tools/pip-requires file. It is pegged to version 0.9. * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On Fri, Sep 7, 2012 at 6:39 AM, Nick Barcet nick.bar...@canonical.comwrote: On 09/05/2012 11:51 AM, Nick Barcet wrote: Thanks for asking, I was just about to come up with this. So based on the poll, it seems that the 3-4pm UTC time slot received the most favors with 9 yes, 1 if need be and 1 no. So I guess we'll have to do without Angus, unless we want to do alternating meetings to satisfy both sides of the planet. In the later case we could do one week at 3pm UTC, and the other at 9PM. What would the others think about this? In any case, the next meeting will be tomorrow at 3pm UTC. I'll be sending a formal invite later today. Based on yesterday's meeting, we decided to try alternating time. Since 9PM UTC is taken on thursdays, I am proposing to move it on Wednesdays. If Nobody objects, this means that starting next week we'll have our meetings on: Wednesday at 9PM UTC for odd weeks (September 12, 26) Thursday at 3PM UTC for even weeks (September 20 and October 4) Does this work for everyone? That time on Wednesday does not work for me. An hour earlier or later would work, if you want to try to stay close to the same time. I am available during the Thursday time slot. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On Wed, Sep 5, 2012 at 5:51 AM, Nick Barcet nick.bar...@canonical.comwrote: On 09/05/2012 08:55 AM, SPN wrote: On 2012-08-31 21:34, Nick Barcet wrote: On 08/30/2012 09:49 AM, Nick Barcet wrote: I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on Google. Will reply to this thread when it opens. We'll have to go with what fits most. The poll is now open at [1], please submit your preference(s) there. I will close the poll on tuesday. http://doodle.com/srxx5244qg9pz2pm Cheers, Nick Is there a concenses on the timings? Thanks for asking, I was just about to come up with this. So based on the poll, it seems that the 3-4pm UTC time slot received the most favors with 9 yes, 1 if need be and 1 no. So I guess we'll have to do without Angus, unless we want to do alternating meetings to satisfy both sides of the planet. In the later case we could do one week at 3pm UTC, and the other at 9PM. What would the others think about this? That might be a little confusing, but we could give it a try. I would like to have as much input as possible. Doug In any case, the next meeting will be tomorrow at 3pm UTC. I'll be sending a formal invite later today. Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] experimenting with ceilometer via devstack
John Tran 's patch just landed in devstack to enable ceilometer support. Thanks, John! To turn on ceilometer, add this line to your localrc before running devstack: enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector The default configuration results in a MongoDB database called ceilometer containing all of the metering data. You can access it directly using the mongo shell via the command mongo ceilometer. The ceilometer API server does not have a wrapper script, but you can start the dev server by running python -m ceilometer.api The server will listen for requests on port 9000 by default. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Project Technical Leader election result
Congratulations, Nick! On Fri, Aug 3, 2012 at 4:22 AM, Julien Danjou jul...@danjou.info wrote: Hi, The PTL election for Ceilometer¹ is over. The winner is Nick Barcet. Congratulations ! The results are available here: http://www.opavote.org/vote/491028 ¹ http://wiki.openstack.org/EfficientMetering/PTLElectionProcess Cheers, -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keyring support in openstack
On Sun, Jul 29, 2012 at 1:37 AM, Bhuvaneswaran A bhu...@apache.org wrote: Team, As per patch https://review.openstack.org/#/c/9497/ we are adding keyring support for openstack client. If password is not specified in command line or environment variable, the user is prompted to enter password. During this time, the password is stored in keyring. During next time, the password is read from keyring, instead of prompt. It is true, if password is not specified in command line or environment variable. This behavior is documented in this wiki page: http://wiki.openstack.org/KeyringSupport If you have any comments, please let us know. You've already answered several of my questions on the ticket, but I still have some usability concerns. How does the keyring system support a single person logging in using multiple user accounts? For example, if I have an admin account and a regular user, how do I switch between them based on the operations I need to perform? Is there a way to disable the behavior of having a password saved to a keyring for a particular user, without uninstalling the python-keyring package (and therefore disabling keyring support for all users)? The wiki mentions the password being saved using keyring.backend.UncryptedFileKeyring. Does that mean the password is saved in cleartext? Is the file protected in some way besides filesystem permissions? The mention of one backend implies that there are others. Should we give users a way to choose the backend, in case they have a preference? How does the use of the keyring affect scripting using the command line tool? Can a script access the keyring, or does it need to use the other options? In one review comment you mention a few desktop apps that know how to manipulate the keyring to manage its contents. What about remote access via ssh, where a desktop environment is not available? Does the keyring library include tools for manipulating the file, or do we need to build our own? If so, what tools would be needed? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] python 3 advocacy (was Re: Hiding complexity of paste config files from operators)
On Mon, Jul 30, 2012 at 5:12 AM, Thierry Carrez thie...@openstack.orgwrote: Assuming that the *-paste.ini files always need to be there, is there some way we could avoid requiring admins to edit these files, and instead make it more like editing the .conf files? For example, could the paste.ini files be generated from the corresponding .conf file as needed? I would not assume that *-paste.ini files always need to be there... Paste is a pain point if we are to support Python 3 one day, so it's also on the black list of the (still inexistant) OpenStack Python3 advocacy group. We exist, we're just not organized, yet. :-) Who else is interested in contributing to Python 3 support in a future version of OpenStack? Should we start planning for some discussions at the Grizzly summit? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] API docs - spec and dev guide split proposal
+1 to separating specifications from documentation for API users On Mon, Jul 30, 2012 at 1:09 PM, Anne Gentle a...@openstack.org wrote: Hi again all, your friendly doc coordinator here. I'd like to get a big push towards documenting reality instead of relying only on the specs stored in the compute-api repository. As an example, recently a Rackspace writer inserted min_count and max_count to the compute-api repository but the change was reversed after a request by Brian Waldon and Jorge Williams to leave the spec as-is. I agree that the spec has value but I believe we need to push towards reality and creating developer guides. So what I'd like to propose here is that we govern these repos as specs and change the titles of those documents to API specification: compute-api image-api identity-api object-api netconn-api At the same time, we'd start new developer guides in the openstack-manuals repository that document reality. We can track the work needed through the openstack-manuals bug and blueprint system. I went up to Brian Aker after his keynote at OSCon seeking contacts at HP who are interested in doing this type of work, and I have started some one-on-one meetings, but I'd like to find more interested collaborators. Anyone at Rightscale or Enstratus interested? Also are there other API implementers who are experts in how the APIs really work? Please join in finding the right solution here. Does this proposal sound like a workable solution? Any tweaks or other suggestions? Thanks, Anne ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keyring support in openstack
On Mon, Jul 30, 2012 at 4:50 PM, Bhuvaneswaran A bhu...@apache.org wrote: On Mon, Jul 30, 2012 at 6:31 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: You've already answered several of my questions on the ticket, but I still have some usability concerns. How does the keyring system support a single person logging in using multiple user accounts? For example, if I have an admin account and a regular user, how do I switch between them based on the operations I need to perform? The password is stored in keyring, for a given user. It also support multiple users. The password is stored against the user specified in command line, --os-username or environment variable OS_USERNAME. The sample content of the keyring file ~/.openstack-keyring.cfg is as follows: [openstack] bhuvan = dG4wN2FjxA== test = xYwN2FjxA== OK, that's good to know. Is there a way to disable the behavior of having a password saved to a keyring for a particular user, without uninstalling the python-keyring package (and therefore disabling keyring support for all users)? The simplest alternative is to specify password using other mechanism, in command line or environment variable. It's not possible to prevent using keyring, if password is not specified in any of these 2 mechanisms. The purpose of this patch is, to prevent password prompt. We're going to need to include a way in the openstack cli to disable the use of the keyring. There will be times when users won't want passwords saved to a keyring, or where the password that is in the keyring is wrong or shouldn't be used for some reason. It seems like an environment variable and a command line switch would cover all of the ways to turn the keyring off, don't you think? The wiki mentions the password being saved using keyring.backend.UncryptedFileKeyring. Does that mean the password is saved in cleartext? Is the file protected in some way besides filesystem permissions? As mentioned in wiki page, the password is stored in base64 format. That doesn't seem any more secure than an environment variable set from a user's login script. What benefit does keyring give us with this configuration? The mention of one backend implies that there are others. Should we give users a way to choose the backend, in case they have a preference? python-keyring also support several other backends: 1.CryptedFileKeyring 2. GnomeKeyring 3. KDEKWallet 4. OSXKeychain 5. Win32CryptoKeyring 6. ... and more. The behaviour of these backends vary for each desktop. For instance, GnomeKeyring may prompt for keyring password, once per login session. CryptedFileKeyring may prompt for keyring password, every time. It's as good as not using keyring. On the other hand, different users will be running in different configurations. Maybe they *do* have a desktop environment, and want to use one of those real keyring managers, instead of the simple INI file described above. Does the keyring library have some way to detect which backends are available at runtime? Or does the application (or user) have to specify one explicitly? How does the use of the keyring affect scripting using the command line tool? Can a script access the keyring, or does it need to use the other options? Yes. The script could be managed with any python script, using the same methods exposed in keyring python module. -- get_password() -- to get the password for given user. -- set_password() -- to set the password in keyring. I was not clear. I meant could a shell script running the new cli access the keyring. It sounds like that is not an issue, based on what you say below. In one review comment you mention a few desktop apps that know how to manipulate the keyring to manage its contents. What about remote access via ssh, where a desktop environment is not available? Does the keyring library include tools for manipulating the file, or do we need to build our own? If so, what tools would be needed? This was applicable for older patch, wherein we rely on desktop/environment specific backend. With older patch, if GNOME desktop is used, GnomeKeyring backend is used; if no desktop is used, CryptedFileKeyring backend is used. With new patch, irrespective of whether desktop is enabled, UncryptedFileKeyring backend is used. With this patch, the keyring behaviour is uniform across all systems in which we deploy openstack. That resolves my concern, but does not seem to give us any useful features. We could achieve the same effect using just the environment variable. It seems like we want to use the best keyring method available, if we're going to use one at all. In summary, the primary goal of this patch is to reuse the password entered in the prompt once, and prevent the user from entering the password again. Ultimately, the password is not exposed in environment or command line (ps). It also facilitate
[Openstack] [ceilometer] Metering team meeting agenda for Thursday 26 July, 2012 @ 16:00 UTC
The metering project team holds regular meetings via IRC in #openstack-meeting, Thursdays at 1600 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome to participate. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions * Discuss priority of maintaining Essex support and find contributor to work on it if we are going to do it * Open discussion If you have any additional topics you would like to cover, please update the agenda in the wiki. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] requesting feedback on priorities for metering data collection
When the ceilometer project started after the Folsom summit, we compiled a list of metrics to be collected [1]. We have reached a stage in the project where we are ready to start implementing more of these meters, and would like some input from the rest of the community about priorities. We have implemented the instance counter, disk I/O, and CPU usage meters, and have begun working on collecting some of the network statistics. If you are interested in using ceilometer for tracking usage in your cloud deployment, please let us know which other metrics you will find especially useful. Thanks, Doug 1 - http://wiki.openstack.org/EfficientMetering#Meters ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] requesting feedback on priorities for metering data collection
On Wed, Jul 25, 2012 at 10:45 AM, Aguiar, Glaucimar (Brazil RD-ECL) glaucimar.agu...@hp.com wrote: Doug, ** ** Thanks for sending this out. I am interested in this project and as I could not have it working yet with my Essex release of OpenStack Some of the pieces work with Essex, but the actual daemons don't, yet. We are planning to discuss this at our meeting this Thursday ( http://wiki.openstack.org/Meetings/MeteringAgenda). I would like to better understand the metrics collected you mention below. I have being reading the documentation and code extensively and would like to count on your help to better understand some aspects of the ceilometer. ** ** Some questions below, sorry if they sound so easy for you but if you could help me with these, this would be of a great help for me. These are good questions, thanks for bringing them up! When you say CPU usage meters, does it mean ceilometer is already capable of reporting the cpu consumption over an hour? We are currently polling libvirt periodically to get the total CPU time for an instance. The frequency of polling is adjustable, but I think the default is 10 minutes. The value returned by libvirt represents the total time for the instance, but a client could pull the data and perform the calculation over a period of time. The API we have defined allows for querying within a time range, for example. Does this take into account the number of CPUs of the instance or this should be derived from the instance data? The number of CPUs in an instance is part of the metadata associated with the counter, but does not enter into the calculation of CPU-time as far as I know. We ask libvirt for that info, and just use what we're given. ** ** Thank you very much for any help in increasing my understanding of the behavior of this component. I can also contact you in IRC if you prefer. I hope my answers help, Doug ** ** Regards, Glaucimar Aguiar ** ** ** ** *From:* openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net[mailto: openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net] *On Behalf Of *Doug Hellmann *Sent:* quarta-feira, 25 de julho de 2012 11:24 *To:* openstack-operat...@lists.openstack.org; openstack@lists.launchpad.net *Subject:* [Openstack] [ceilometer] requesting feedback on priorities for metering data collection ** ** When the ceilometer project started after the Folsom summit, we compiled a list of metrics to be collected [1]. We have reached a stage in the project where we are ready to start implementing more of these meters, and would like some input from the rest of the community about priorities. We have implemented the instance counter, disk I/O, and CPU usage meters, and have begun working on collecting some of the network statistics. If you are interested in using ceilometer for tracking usage in your cloud deployment, please let us know which other metrics you will find especially useful.*** * ** ** Thanks, Doug ** ** 1 - http://wiki.openstack.org/EfficientMetering#Meters ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fw:Re: Questions about ceilometer
--
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Mon, Jul 16, 2012 10:31 PM
*To: * 张家龙zhan...@awcloud.com; * *
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
An earlier message in this thread points to a bug (
https://bugs.launchpad.net/ceilometer/+bug/1024563) which has a review
patch attached.
On Mon, Jul 16, 2012 at 10:03 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
Thanks for reply.
While ,where to find the patch posted John ? If it`s
prossable,please point it out,Thanks.
And it`s my pleasure to be the first one to receive the message
about you fix this problems.
Good luck !
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Mon, Jul 16, 2012 08:55 PM
*To: * 张家龙zhan...@awcloud.com; * *
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Sat, Jul 14, 2012 at 10:02 PM, 张家龙 zhan...@awcloud.com wrote:
Hi,all.
Sorry for late reply.
Until now, I not test folsom. So, I`am not sure how it work in
folsom .
The follow is my qpid config file:
http://pastebin.com/sBXm6k7z
And Doug writed set driver to
nova.openstack.common.notifier.rabbit_notifier,
while , I cannot found this class or modules in exsse.
The notifier classes moved in Folsom, so that's the setting you would
need if you were working with Folsom.
I'm traveling this week, so I won't be able to set up a test environment
with Essex or qpid until next week some time.
Based on rereading the configuration files you posted, I do suspect
that this is a problem with the code, rather than your configuration. You
might want to try the patch John posted above. I don't think that's the
right long-term solution, but if it gets you past this situation we can
find a better solution later.
Doug
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Sat, Jul 14, 2012 00:17 AM
*To: * 张家龙zhan...@awcloud.com; * *
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 11:36 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
Qpid should work, but I haven't been testing with that so you might
have found a bug.
There are (at least) two differences between your setup and what I
normally use: Essex and qpid. Have you tried running against a folsom
install? That would at least tell us if the qpid configuration is correct
or if the problem is related to Essex.
Doug
In addition,my nova.conf,mongodb.conf and
ceilometer-collector.conf are here:
http://pastebin.com/sW5d8eRv
http://pastebin.com/D5GMkLsb
http://pastebin.com/u5vH22Lh
Were there some errors in my config files?
Waiting your reply. Thanks.
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Fri, Jul 13, 2012 11:28 PM
*To: * 张家龙zhan...@awcloud.com; Julien Danjou
jul...@danjou.info; * *
*Cc: * openstackopenstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention
to it.
According to the docs of ceilometer,I deploied it in openstack
exsse environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO
Re: [Openstack] best practices for merging common into specific projects
On Wed, Jul 18, 2012 at 7:00 PM, Thierry Carrez thie...@openstack.orgwrote: Mark McLoughlin wrote: Making our multiple projects converge onto consolidated and well-accepted APIs is a bit painful work, but it is a prerequisite to turning openstack-common into a proper library (or set of libraries). I'd say the whole thing suffers from not having a proper team/leader/coordinator dedicated to it: relying on existing, overstretched PTLs to lead that effort might not be the fastest path. While I was on vacation, I read in the weekly newsletter: It developed into a request for leadership for openstack-common and was like WTF do you call the work that e.g. I, Jason, Russell and Doug have been doing? But I see your point is a little different - you feel there should be an elected/appointed PTL without a PPB vote or whatever to represent the project. I guess that could help clarify things since it's what folks are used to with other projects. Right. So far we said that openstack-common was driven by all the PTLs, but that didn't prove particularly fast and efficient. Having a clear face associated with it, someone specific taking the lead on this project, will, I think, help a bit in getting to the next step. Sorry if this rekindles old arguments, but could someone summarize the reasons for an openstack-common PTL without voting rights? I would have defaulted to giving them a vote *especially* because the code in common is, well, common to all of the projects. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
On Mon, Jul 23, 2012 at 12:00 PM, Thierry Carrez thie...@openstack.orgwrote: Doug Hellmann wrote: On Wed, Jul 18, 2012 at 7:00 PM, Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org wrote: Mark McLoughlin wrote: Making our multiple projects converge onto consolidated and well-accepted APIs is a bit painful work, but it is a prerequisite to turning openstack-common into a proper library (or set of libraries). I'd say the whole thing suffers from not having a proper team/leader/coordinator dedicated to it: relying on existing, overstretched PTLs to lead that effort might not be the fastest path. While I was on vacation, I read in the weekly newsletter: It developed into a request for leadership for openstack-common and was like WTF do you call the work that e.g. I, Jason, Russell and Doug have been doing? But I see your point is a little different - you feel there should be an elected/appointed PTL without a PPB vote or whatever to represent the project. I guess that could help clarify things since it's what folks are used to with other projects. Right. So far we said that openstack-common was driven by all the PTLs, but that didn't prove particularly fast and efficient. Having a clear face associated with it, someone specific taking the lead on this project, will, I think, help a bit in getting to the next step. Sorry if this rekindles old arguments, but could someone summarize the reasons for an openstack-common PTL without voting rights? I would have defaulted to giving them a vote *especially* because the code in common is, well, common to all of the projects. So far, the PPB considered openstack-common to be driven by all PTLs, so it didn't have a specific PTL. As far as future governance is concerned (technical committee of the Foundation), openstack-common would technically be considered a supporting library (rather than a core project) -- those can have leads, but those do not get granted an automatic TC seat. OK, I can see the distinction there. I think the project needs an official leader, even if we don't call them a PTL in the sense meant for other projects. And I would expect anyone willing to take on the PTL role for common to be qualified to run for one of the open positions on the new TC, if they wanted to participate there. [ Avoiding the need to distinguish between worthy and unworthy projects leads was one of the many reasons why I preferred the TC to be completely directly-elected. ] That does make sense. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Questions about ceilometer
On Sat, Jul 14, 2012 at 10:02 PM, 张家龙 zhan...@awcloud.com wrote:
Hi,all.
Sorry for late reply.
Until now, I not test folsom. So, I`am not sure how it work in folsom .
The follow is my qpid config file:
http://pastebin.com/sBXm6k7z
And Doug writed set driver to
nova.openstack.common.notifier.rabbit_notifier,
while , I cannot found this class or modules in exsse.
The notifier classes moved in Folsom, so that's the setting you would need
if you were working with Folsom.
I'm traveling this week, so I won't be able to set up a test environment
with Essex or qpid until next week some time.
Based on rereading the configuration files you posted, I do suspect that
this is a problem with the code, rather than your configuration. You might
want to try the patch John posted above. I don't think that's the right
long-term solution, but if it gets you past this situation we can find a
better solution later.
Doug
**
--
Best Regards
ZhangJialong
**
** **
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Sat, Jul 14, 2012 00:17 AM
*To: * 张家龙zhan...@awcloud.com; **
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; **
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 11:36 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
Qpid should work, but I haven't been testing with that so you might have
found a bug.
There are (at least) two differences between your setup and what I
normally use: Essex and qpid. Have you tried running against a folsom
install? That would at least tell us if the qpid configuration is correct
or if the problem is related to Essex.
Doug
In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
are here:
http://pastebin.com/sW5d8eRv
http://pastebin.com/D5GMkLsb
http://pastebin.com/u5vH22Lh
Were there some errors in my config files?
Waiting your reply. Thanks.
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Fri, Jul 13, 2012 11:28 PM
*To: * 张家龙zhan...@awcloud.com; Julien Danjoujul...@danjou.info;
* *
*Cc: * openstackopenstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to
it.
According to the docs of ceilometer,I deploied it in openstack
exsse environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.create.end events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.exists events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.delete.start events
Traceback (most recent call last):
File /usr/lib/python2.6/site-packages/eventlet/hubs/poll.py,
line 97, in wait
readers.get(fileno, noop).cb(fileno)
File /usr/lib/python2.6/site-packages/eventlet/greenthread.py,
line 192, in main
result = function(*args, **kwargs)
File /usr/lib/python2.6/site-packages/nova/service.py, line
101, in run_server
server.start()
File /usr/lib/python2.6/site-packages/nova/service.py, line
162, in start
self.manager.init_host()
File
/usr
Re: [Openstack] Questions about ceilometer
An earlier message in this thread points to a bug (
https://bugs.launchpad.net/ceilometer/+bug/1024563) which has a review
patch attached.
On Mon, Jul 16, 2012 at 10:03 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
Thanks for reply.
While ,where to find the patch posted John ? If it`s prossable,please
point it out,Thanks.
And it`s my pleasure to be the first one to receive the message about
you fix this problems.
Good luck !
**
--
Best Regards
ZhangJialong
**
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Mon, Jul 16, 2012 08:55 PM
*To: * 张家龙zhan...@awcloud.com; **
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; **
*Subject: * Re: [Openstack] Questions about ceilometer
On Sat, Jul 14, 2012 at 10:02 PM, 张家龙 zhan...@awcloud.com wrote:
Hi,all.
Sorry for late reply.
Until now, I not test folsom. So, I`am not sure how it work in folsom
.
The follow is my qpid config file:
http://pastebin.com/sBXm6k7z
And Doug writed set driver to
nova.openstack.common.notifier.rabbit_notifier,
while , I cannot found this class or modules in exsse.
The notifier classes moved in Folsom, so that's the setting you would
need if you were working with Folsom.
I'm traveling this week, so I won't be able to set up a test environment
with Essex or qpid until next week some time.
Based on rereading the configuration files you posted, I do suspect that
this is a problem with the code, rather than your configuration. You might
want to try the patch John posted above. I don't think that's the right
long-term solution, but if it gets you past this situation we can find a
better solution later.
Doug
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Sat, Jul 14, 2012 00:17 AM
*To: * 张家龙zhan...@awcloud.com; * *
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 11:36 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
Qpid should work, but I haven't been testing with that so you might
have found a bug.
There are (at least) two differences between your setup and what I
normally use: Essex and qpid. Have you tried running against a folsom
install? That would at least tell us if the qpid configuration is correct
or if the problem is related to Essex.
Doug
In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
are here:
http://pastebin.com/sW5d8eRv
http://pastebin.com/D5GMkLsb
http://pastebin.com/u5vH22Lh
Were there some errors in my config files?
Waiting your reply. Thanks.
* *
--
Best Regards
ZhangJialong
* *
* * * *
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Fri, Jul 13, 2012 11:28 PM
*To: * 张家龙zhan...@awcloud.com; Julien Danjoujul...@danjou.info;
* *
*Cc: * openstackopenstack@lists.launchpad.net; * *
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to
it.
According to the docs of ceilometer,I deploied it in openstack
exsse environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing
Re: [Openstack] Questions about ceilometer
I have been doing all of my development with Folsom HEAD because DreamHost
plans to deploy ceilometer with our Folsom-based cloud. I know Julien has
done some work on Essex compatibility, but I don't know where that stands
or if he is still working on it.
Doug
On Mon, Jul 16, 2012 at 10:12 AM, Aguiar, Glaucimar (Brazil RD-ECL)
glaucimar.agu...@hp.com wrote:
Doug,
** **
I will start ceilometer installation now. Would you recommend installing
Folsom then instead of essex? (I already have essex installed so, this is
the reason for asking).
** **
Thanks,
Glaucimar Aguiar
** **
** **
*From:* openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net[mailto:
openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net] *On Behalf
Of *Doug Hellmann
*Sent:* segunda-feira, 16 de julho de 2012 09:55
*To:* 张家龙
*Cc:* openstack
*Subject:* Re: [Openstack] Questions about ceilometer
** **
** **
On Sat, Jul 14, 2012 at 10:02 PM, 张家龙 zhan...@awcloud.com wrote:
Hi,all.
Sorry for late reply.
Until now, I not test folsom. So, I`am not sure how it work in folsom .
The follow is my qpid config file:
*http://pastebin.com/sBXm6k7z*
And Doug writed set driver to
nova.openstack.common.notifier.rabbit_notifier,
while , I cannot found this class or modules in exsse.
** **
The notifier classes moved in Folsom, so that's the setting you would need
if you were working with Folsom.
I'm traveling this week, so I won't be able to set up a test environment
with Essex or qpid until next week some time.
** **
Based on rereading the configuration files you posted, I do suspect that
this is a problem with the code, rather than your configuration. You might
want to try the patch John posted above. I don't think that's the right
long-term solution, but if it gets you past this situation we can find a
better solution later.
** **
Doug
** **
--
Best Regards
ZhangJialong
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Sat, Jul 14, 2012 00:17 AM
*To: * 张家龙zhan...@awcloud.com;
*Cc: * Julien Danjoujul...@danjou.info; openstack
openstack@lists.launchpad.net;
*Subject: * Re: [Openstack] Questions about ceilometer
** **
On Fri, Jul 13, 2012 at 11:36 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
** **
Qpid should work, but I haven't been testing with that so you might have
found a bug.
** **
There are (at least) two differences between your setup and what I
normally use: Essex and qpid. Have you tried running against a folsom
install? That would at least tell us if the qpid configuration is correct
or if the problem is related to Essex.
** **
Doug
In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
are here:
*http://pastebin.com/sW5d8eRv *
*http://pastebin.com/D5GMkLsb *
*http://pastebin.com/u5vH22Lh *
Were there some errors in my config files?
Waiting your reply. Thanks.
--
Best Regards
ZhangJialong
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Fri, Jul 13, 2012 11:28 PM
*To: * 张家龙zhan...@awcloud.com; Julien Danjoujul...@danjou.info; **
**
*Cc: * openstackopenstack@lists.launchpad.net;
*Subject: * Re: [Openstack] Questions about ceilometer
** **
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
** **
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to it.
According to the docs of ceilometer,I deploied it in openstack exsse
environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
*http://pastebin.com/sW5d8eRv *
Here is the /etc/ceilometer-collector.conf file is here:
*http://pastebin.com/u5vH22Lh *
And the /etc/mongodb.conf is here:
*http://pastebin.com/D5GMkLsb *
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb
Re: [Openstack] Questions about ceilometer
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to it.
According to the docs of ceilometer,I deploied it in openstack exsse
environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.create.end events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.exists events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.delete.start events
Traceback (most recent call last):
File /usr/lib/python2.6/site-packages/eventlet/hubs/poll.py, line
97, in wait
readers.get(fileno, noop).cb(fileno)
File /usr/lib/python2.6/site-packages/eventlet/greenthread.py,
line 192, in main
result = function(*args, **kwargs)
File /usr/lib/python2.6/site-packages/nova/service.py, line 101,
in run_server
server.start()
File /usr/lib/python2.6/site-packages/nova/service.py, line 162,
in start
self.manager.init_host()
File
/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py,
line 69, in init_host
topic='%s.info' % flags.FLAGS.notification_topics[0],
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 867,
in __getattr__
return self._substitute(self._get(name))
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1070,
in _get
info = self._get_opt_info(name, group)
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1161,
in _get_opt_info
raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: notification_topics
Removing descriptor: 12
If anyone can help me ? Waiting your reply. Thanks !
It looks like we have a problem with the way we are loading nova's settings
under Essex. I know some of the flags and cfg code was moved around and
changed between Essex and Folsom as part of it moved into openstack-common.
Julien, you're more familiar with Essex than I am, do you have any ideas
about this problem?
Doug
**
--
Best Regards
ZhangJialong
**
** **
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann doug.hellm...@dreamhost.com
wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to it.
According to the docs of ceilometer,I deploied it in openstack exsse
environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.create.end events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.exists events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.delete.start events
Traceback (most recent call last):
File /usr/lib/python2.6/site-packages/eventlet/hubs/poll.py, line
97, in wait
readers.get(fileno, noop).cb(fileno)
File /usr/lib/python2.6/site-packages/eventlet/greenthread.py,
line 192, in main
result = function(*args, **kwargs)
File /usr/lib/python2.6/site-packages/nova/service.py, line 101,
in run_server
server.start()
File /usr/lib/python2.6/site-packages/nova/service.py, line 162,
in start
self.manager.init_host()
File
/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py,
line 69, in init_host
topic='%s.info' % flags.FLAGS.notification_topics[0],
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 867,
in __getattr__
return self._substitute(self._get(name))
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1070,
in _get
info = self._get_opt_info(name, group)
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1161,
in _get_opt_info
raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: notification_topics
Removing descriptor: 12
If anyone can help me ? Waiting your reply. Thanks !
It looks like we have a problem with the way we are loading nova's
settings under Essex. I know some of the flags and cfg code was moved
around and changed between Essex and Folsom as part of it moved into
openstack-common. Julien, you're more familiar with Essex than I am, do you
have any ideas about this problem?
I think I left out a configuration step in the instructions at
http://ceilometer.readthedocs.org/en/latest/install.html.
Did you configure your system to use the Rabbit notifier?
Doug
Doug
**
--
Best Regards
ZhangJialong
**
** **
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 11:36 AM, 张家龙 zhan...@awcloud.com wrote:
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
Qpid should work, but I haven't been testing with that so you might have
found a bug.
There are (at least) two differences between your setup and what I normally
use: Essex and qpid. Have you tried running against a folsom install? That
would at least tell us if the qpid configuration is correct or if the
problem is related to Essex.
Doug
In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
are here:
http://pastebin.com/sW5d8eRv
http://pastebin.com/D5GMkLsb
http://pastebin.com/u5vH22Lh
Were there some errors in my config files?
Waiting your reply. Thanks.
**
--
Best Regards
ZhangJialong
**
-- Original --
*From: * Doug Hellmanndoug.hellm...@dreamhost.com;
*Date: * Fri, Jul 13, 2012 11:28 PM
*To: * 张家龙zhan...@awcloud.com; Julien Danjoujul...@danjou.info; **
*Cc: * openstackopenstack@lists.launchpad.net; **
*Subject: * Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to
it.
According to the docs of ceilometer,I deploied it in openstack exsse
environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
storage engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.create.end events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.exists events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.delete.start events
Traceback (most recent call last):
File /usr/lib/python2.6/site-packages/eventlet/hubs/poll.py,
line 97, in wait
readers.get(fileno, noop).cb(fileno)
File /usr/lib/python2.6/site-packages/eventlet/greenthread.py,
line 192, in main
result = function(*args, **kwargs)
File /usr/lib/python2.6/site-packages/nova/service.py, line 101,
in run_server
server.start()
File /usr/lib/python2.6/site-packages/nova/service.py, line 162,
in start
self.manager.init_host()
File
/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py,
line 69, in init_host
topic='%s.info' % flags.FLAGS.notification_topics[0],
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 867,
in __getattr__
return self._substitute(self._get(name))
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1070,
in _get
info = self._get_opt_info(name, group)
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1161,
in _get_opt_info
raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: notification_topics
Removing descriptor: 12
If anyone can help me ? Waiting your reply. Thanks !
It looks like we have a problem with the way we are loading nova's
settings under Essex. I know some of the flags and cfg code was moved
around and changed between Essex and Folsom as part of it moved into
openstack-common. Julien, you're more familiar with Essex than I am, do you
have any ideas about this problem?
I think I left out a configuration step in the instructions at
http://ceilometer.readthedocs.org/en/latest/install.html.
Did you configure your system to use the Rabbit notifier?
Doug
Doug
* *
--
Best Regards
ZhangJialong
Re: [Openstack] Questions about ceilometer
I *just* ran into a problem today triggered by some code moving around in
nova and common. I had to set my driver to
nova.openstack.common.notifier.rabbit_notifier (in both files).
I didn't see any errors from that until I tried to launch an instance, so
it doesn't seem to prevent any of the nova services from starting up.
Doug
On Fri, Jul 13, 2012 at 1:17 PM, TRAN, JOHN jt7...@att.com wrote:
I'm having the same problem. I'm running against nova trunk and I have
rabbit notifier enabled.
/etc/nova/nova.conf:notification_driver=nova.notifier.rabbit_notifier
/home/stack/ceilometer-collector.conf:notification_driver=nova.notifier.rabbit_notifier
From:
openstack-bounces+jhtran=att@lists.launchpad.net[openstack-bounces+jhtran=
att@lists.launchpad.net] on behalf of 张家龙 [zhan...@awcloud.com]
Sent: Friday, July 13, 2012 8:36 AM
To: Doug Hellmann; Julien Danjou
Cc: openstack
Subject: Re: [Openstack] Questions about ceilometer
Dear Doug,
I`m use Qpid instead of Rabbit .
Did it cause the error ?
In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
are here:
http://pastebin.com/sW5d8eRv
http://pastebin.com/D5GMkLsb
http://pastebin.com/u5vH22Lh
Were there some errors in my config files?
Waiting your reply. Thanks.
--
Best Regards
ZhangJialong
-- Original --
From: Doug Hellmanndoug.hellm...@dreamhost.com;
Date: Fri, Jul 13, 2012 11:28 PM
To: 张家龙zhan...@awcloud.com; Julien Danjoujul...@danjou.info;
Cc: openstackopenstack@lists.launchpad.net;
Subject: Re: [Openstack] Questions about ceilometer
On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann
doug.hellm...@dreamhost.commailto:doug.hellm...@dreamhost.com wrote:
On Thu, Jul 12, 2012 at 9:42 PM, 张家龙 zhan...@awcloud.commailto:
zhan...@awcloud.com wrote:
Dear all,
As the project named ceilometer appeared,I paid close attention to it.
According to the docs of ceilometer,I deploied it in openstack exsse
environment.
While,I cannot start the ceilometer collector and agent.
The follows are my operations.
1.Install openstack nova ,mongodb and ceilometer.
2.configurate nova ,mongodb and ceilometer
The /etc/nova/nova.conf file is here:
http://pastebin.com/sW5d8eRv
Here is the /etc/ceilometer-collector.conf file is here:
http://pastebin.com/u5vH22Lh
And the /etc/mongodb.conf is here:
http://pastebin.com/D5GMkLsb
3.Start openstack nova ,mongodb
4.Then I start the ceilometer-collector
/usr/bin/ceilometer-collector start
While,some errors occurred:
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
engine EntryPoint.parse('mongodb =
ceilometer.storage.impl_mongodb:MongoDBStorage')
2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
connecting to MongoDB on localhost:27017
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
attempting to load notification handler for
ceilometer.collector.compute:instance
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.create.end events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.exists events
2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
subscribing instance handler to compute.instance.delete.start events
Traceback (most recent call last):
File /usr/lib/python2.6/site-packages/eventlet/hubs/poll.py, line
97, in wait
readers.get(fileno, noop).cb(fileno)
File /usr/lib/python2.6/site-packages/eventlet/greenthread.py,
line 192, in main
result = function(*args, **kwargs)
File /usr/lib/python2.6/site-packages/nova/service.py, line 101,
in run_server
server.start()
File /usr/lib/python2.6/site-packages/nova/service.py, line 162,
in start
self.manager.init_host()
File
/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py,
line 69, in init_host
topic='%s.infohttp://s.info' %
flags.FLAGS.notification_topics[0],
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 867,
in __getattr__
return self._substitute(self._get(name))
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1070,
in _get
info = self._get_opt_info(name, group)
File
/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py, line 1161,
in _get_opt_info
raise NoSuchOptError(opt_name, group)
NoSuchOptError: no such option: notification_topics
Removing descriptor: 12
If anyone can help me ? Waiting your reply
Re: [Openstack] [metering] Ceilometer PTL Election Process - Call for candidate
I am running for PTL for the ceilometer project. I have posted some information about myself and my thoughts for the project to the wiki under http://wiki.openstack.org/EfficientMetering/PTLElectionProcess/DougHellmann Doug On Wed, Jul 11, 2012 at 8:24 AM, Nick Barcet nick.bar...@canonical.comwrote: As voted on the July 5th meeting the Ceilometer team members will soon vote on electing it's project team lead (PTL). Even though Ceilometer is not yet an official OpenStack project, we are applying for it, so we should be following the standard process as much as possible. The details of the process we will be following is described at [1]. This email is a formal call for candidates, which is step one of the process. Candidates for the PTL role must have contributed either to the source or to the wiki pages for Ceilometer prior to the call for candidates. The list that we assembled can be found on the above mentioned wiki page, but in the event we have missed someone, please shout. Candidates should, before July 24th: 1/ declare themselves on this mailing list 2/ add their name on the wiki page [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PEP8 checks
On Mon, Jul 9, 2012 at 9:16 PM, Dan Prince dpri...@redhat.com wrote: - Original Message - From: Dave Walker davewal...@ubuntu.com To: Monty Taylor mord...@inaugust.com Cc: openstack@lists.launchpad.net, John Garbutt john.garb...@citrix.com Sent: Monday, July 9, 2012 6:01:19 PM Subject: Re: [Openstack] PEP8 checks On Mon, Jul 02, 2012 at 08:28:04AM -0400, Monty Taylor wrote: On 07/02/2012 06:46 AM, John Garbutt wrote: Hi, I noticed I can now run the pep8 tests like this (taken from Jenkins job): tox -v -epep8 ... pep8: commands succeeded congratulations :) But the old way to run tests seems to fail: ./run-tests.sh -p ... File /home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py, line 1220, in check_logical for result in self.run_check(check, argument_names): TypeError: 'NoneType' object is not iterable Is this expected? Did I just miss an email about this change? I cannot reproduce this on my system. Can you run bash -x run_tests.sh -p and pastebin the output? Also, tools/with_venv.sh pep8 --version just to be sure. Hi, The issue is that as of a recent change to upstream pep8 [1], the additional pep8 rules in tools/hacking.py need to be changed from returns to yields.. :( This brings up a good point. Why are we following the latest pep8 release so closely in Nova? The latest release is hardly a month old and we are already using it? We aren't necessarily doing the same for all the other OpenStack projects... (nor am I suggesting that we do). So why Nova? I'm not convinced the latest pep8 features really provide us enough benefit that we need to bump our pep8 baseline every month or two. in fact they may be hurting us in terms of churn, extra work, back-portability of upstream patches, etc. Ultimately is tracking the latest pep8 really worth it? +1 Some of the features are requiring useless whitespace changes to working code just to get through the gating tests. In ceilometer we've pegged our pep8 tests to 1.1. Doug [1] https://github.com/jcrocholl/pep8/commit/b9f72b16011aac981ce9e3a47fd0ffb1d3d2e085 Kind Regards, Dave Walker dave.wal...@canonical.com Engineering Manager, Ubuntu Server Infrastructure ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer application for incubation
On Fri, Jul 6, 2012 at 3:20 AM, Nick Barcet nick.bar...@canonical.comwrote: On 07/05/2012 09:22 PM, Jonathan Bryce wrote: Thanks, Nick. I've added it to the agenda for next Tuesday's meeting at 20:00 UTC/3:00 PM CDT. If you can join the meeting to field any questions, that would be helpful. Jonathan Great! I'll be there, hopefully joined by other team members. I will try to make it, too. Doug Thanks, Nick On Jul 5, 2012, at 12:52 PM, Nick Barcet wrote: Dear members of the Project Policy Board, After 3 month of work on the Ceilometer project, and great progress being made, our last meeting IRC meeting [1] validated that we should be submitting this project for incubation. Following the OpenStack project rules, we have completed the incubation application form which you will find at [2]. We would be happy to answer any question you may have about our application and hope that you will give a favorable answer to our request. [1] http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html [2] http://wiki.openstack.org/Governance/Proposed/Ceilometer On behalf of the Ceilometer project team, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] ceilometer dev docs on readthedocs.org
On Tue, Jul 3, 2012 at 3:38 PM, Loic Dachary l...@enovance.com wrote: On 07/03/2012 07:46 PM, Doug Hellmann wrote: I've set up the ceilometer development documentation build on RTD at http://ceilometer.readthedocs.org/en/latest/index.html Hi, I've updated https://launchpad.net/ceilometer to list this link. Thanks, Loic! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] resource metadata changes and billing
On Wed, Jul 4, 2012 at 1:52 PM, Nick Barcet nick.bar...@canonical.comwrote: On 06/29/2012 03:04 PM, Doug Hellmann wrote: [..] My conclusion from all of this (over-)thinking is that the ceilometer API should assume the simple case and ignore the metadata changes when computing the sum or maximum value for a counter over a range of time. More complex processing will be left up to the caller, who can ask for raw metering data in manageable chunks and process them outside of the API. I could be persuaded to do something more complicated if the problems described above can be solved in a relatively simple way, but even then I think we should push that to the v2 API. [..] Sorry for my late reply on this, but... So, if I summarize what you are saying, the problem is that for a given Instance ID, a given meter may have to be interpreted as if the Instance ID was changing over time. Example: t1: Instance A - Has 1 CPU - 64G ram - runs in zone 1 t2: Instance A - Has 2 CPU - 64G ram - runs in zone 1 t3: Instance A - Has 2 CPU - 128G ram - runs in zone 1 t4: Instance A - Has 2 CPU - 128G ram - runs in zone 3 t5: Instance A is stopped From a billing point of view, what is important here is that even though the Instance ID remains the same, we have in fact 4 different segments of time which could lead to 4 different pricing being applied to the same instance: t1-t2: price 1 t2-t3: price 2 t3-t4: price 3 t4-t5: price 4 So we need to be able to inform the rating engine that these events have occurred so that it does not uniformly apply a billing price to from a sum of a given meter volume. But in fact this information is indeed captured and accessible to rating engines via their respective meters. Yes, that is exactly it. Thank you for clarifying what I was trying to say. :-) What is interesting here is that, in my mind, the sum and duration function of the API, when I proposed it, were only meant to be able to: * In a simple amazon type billing model where instances cannot change zone, add CPU or add ram, * In a Private cloud scenario where you only need simple usage stats to inform your users, * in a horizon plugin to give a quick summary of use, and would never be used by any serious rating engines that would in each and every case require to have access to the raw list of events so that it can recreate the full time line of the events. This is where we need to draw the line between metering and rating. I didn't realize that was your intent. I therefore propose that we leave the API as is, knowing the side effects of such high level sum and duration calculations. If we agree on this, I take the action to document the limitation of the summary functions of the API. +1, and thank you for offering to document it. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Stackforge migrating
On Wed, Jul 4, 2012 at 9:00 AM, Andrew Hutchings and...@linuxjedi.co.ukwrote: Hi guys, On 04/07/12 13:20, Andrew Hutchings wrote: There was an initial plan to migrate Stackforge Gerrit and Jenkins to OpenStack Gerrit/Jenkins. There are many reasons for this such as: As far as I can tell the hard work of the migration is now complete. For anyone using Stackforge branches please pull the latest master and rebase any branches you were working on with this. It will update .gitreview to point to the correct server. From then on all reviews should be on review.openstack.org and tests on jenkins.openstack.org. If there are any problems please get in touch with the CI team (note it is 4th July holidays so probably just me today) Thanks, Andrew, it looks like the ceilometer parts are working just fine! Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
On Tue, Jul 3, 2012 at 2:59 PM, Gabriel Hurley gabriel.hur...@nebula.comwrote: The notion that copying code is any protection against APIs that may change is a red herring. It's the exact same effect as pegging a version of a dependency (whether it's a commit hash or a real version number), except now you have code duplication. An unstable upgrade path is an unstable upgrade path, and copying the code into the project doesn't alleviate the pain for the project if the upstream library decides to change its APIs. It does if the API being changed is the one that was copied because it means the project doesn't have to respond to the change immediately. For example, if we add something to the rpc library that ceilometer needs, nova doesn't have to stop what they're doing to handle any potential changes. If openstack-common was installed as a separate library, there wouldn't be a (good) way to have both versions installed so ceilometer and nova couldn't run on the same host (a requirement for the ceilometer agent). So, I'm +1 on turning common into *several* libraries with their own versioning scheme, when the components are ready. I don't see a need to rush that as part of Folsom, though. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] new locations for some things
On Jul 2, 2012, at 7:02 PM, Monty Taylor mord...@inaugust.com wrote: Secondly, in addition to the normal per-commit tarballs, we're now publishing tarballs of the form $project-$branch.tar.gz which will get overwritten with each commit - that way, if you need to track trunk from a pip-requires file, (such as ceilometer, which needs to track nova trunk) you can simply plop in something like http://tarballs.openstack.org/nova/nova-master.tar.gz - and it'll work for both pip installs AND easy_install/distutils based installs. Yay! Yay indeed!! Thank you from the entire ceilometer team! Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] best practices for merging common into specific projects
On Jul 3, 2012, at 5:31 AM, Thierry Carrez thie...@openstack.org wrote: Gabriel Hurley wrote: On a more fundamental level, did I miss some tremendous reason why we have this merge from common pattern instead of making OpenStack Common a standard python dependency just like anything else? Especially with the work Monty has recently done on versioning and packaging the client libs from Jenkins, I can't see a reason to keep following this update common and merge to everything else pattern at all... This discussion should probably wait for markmc to come back, since he set up most of this framework in the first place. He would certainly produce a more compelling rationale than I can :) IIRC the idea was to have openstack-common APIs in incubation until some of them are stable enough that we can apply backward compatibility for them at the level expected from any other decent Python library. When we reach this point, those stable modules would be out of incubation and released in a real openstack-common library. Unstable APIs would stay in incubation and still use the merge model. My understanding is that we are not yet at this point, especially as we tweak/enrich openstack-common modules to make them acceptable by all the projects... Ideally when we reach that point the libraries will be released as individual components instead of a monolithic shared library, too. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Single global dependency list
On Jul 2, 2012, at 6:40 PM, Monty Taylor mord...@inaugust.com wrote: Hey all! One of the tasks from the last ODS was to implement a single global dependency list. Turns out the more you think about it, the more important it is... because of the way we use devstack as part of the gate, we actually _currently_ have a de facto global dependency list, it's just not declared anywhere. (oops) Anyway - the original thought was to put the depends in openstack-common. We'd use update.py to copy the depends in to the project, so that projects could align on their own timeframe. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. The mechanics of that all work and are ready - but then bcwaldon pointed out that it didn't make a ton of sense for them to go in openstack-common, since that has its own lifecycle and is a place for common code to go - not just a catch all place. To that end, I took the code we had written for the update logic and put it, along with the depends lists, into its own repo. I think we're ready to start actually moving forward with it - but we've run up against the hardest problem we every have: naming openstack-depends already got vetoed on IRC because it makes people think of adult diapers. I'm proposing openstack-requires, since the files we're talking about are actually python requirements files. Any objections? +0 on the name As an alternative, how about combining the requirements file with the other packaging related stuff from openstack-common and calling the result openstack-packaging? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PyPI uploads for client libs live
On Jul 3, 2012, at 5:57 AM, Thierry Carrez thie...@openstack.org wrote: Monty Taylor wrote: At the moment, the only people with permission to upload tags is the openstack-release team. However, since we're letting client libs manage their own versions, I kinda think we should give PTLs the right on their own project - so Vish would get tag access to python-novaclient, Brian to python-glanceclient, etc. Ideally it would be a two-side approval process (PTL + openstack-release), because openstack-release shouldn't be able to tag without PTL approval, and openstack-release should still be kept in the loop before a tag is pushed by PTLs (there are multiple reasons why a few hours delay before tagging would be a good idea, and the openstack-release people actually keep track of those). That said, we don't have that approval mechanism available yet (same issue with the core projects release tagging) so in the mean time we should probably let the small set of individuals with an understanding of the issues (PTLs + openstack-release) have the power to do it. Within that group, we can have a soft two-side approval process (based on IRC pings) to check everything is OK before triggering a release. Could we use gerrit's 2-step approval like we do for other projects and combine it with a fancy tag detector like was just added for DocImpact? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PyPI uploads for client libs live
On Tue, Jul 3, 2012 at 9:54 AM, Monty Taylor mord...@inaugust.com wrote: On 07/03/2012 08:47 AM, Doug Hellmann wrote: On Jul 3, 2012, at 5:57 AM, Thierry Carrez thie...@openstack.org wrote: Monty Taylor wrote: At the moment, the only people with permission to upload tags is the openstack-release team. However, since we're letting client libs manage their own versions, I kinda think we should give PTLs the right on their own project - so Vish would get tag access to python-novaclient, Brian to python-glanceclient, etc. Ideally it would be a two-side approval process (PTL + openstack-release), because openstack-release shouldn't be able to tag without PTL approval, and openstack-release should still be kept in the loop before a tag is pushed by PTLs (there are multiple reasons why a few hours delay before tagging would be a good idea, and the openstack-release people actually keep track of those). That said, we don't have that approval mechanism available yet (same issue with the core projects release tagging) so in the mean time we should probably let the small set of individuals with an understanding of the issues (PTLs + openstack-release) have the power to do it. Within that group, we can have a soft two-side approval process (based on IRC pings) to check everything is OK before triggering a release. Could we use gerrit's 2-step approval like we do for other projects and combine it with a fancy tag detector like was just added for DocImpact? I have an outstanding bug/feature request for gerrit to allow for the review of and approval or disapproval of tags. Ah, I actually meant the magical text feature you mention below, but approving tags sounds like a good thing to have. That being said - if we wanted to go the route you're talking about in the mean time, instead of actually using the git tag route we could have an additional commit with a magical text in the commit message which, on merging, would cause a tag to be created... We've got guys on the team who hack gerrit now though - lemme get some feedback on how much it would take to actually get proper tag reviewing. Sounds good. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] resource metadata changes and billing
On Mon, Jul 2, 2012 at 4:58 PM, Julien Danjou jul...@danjou.info wrote: On Fri, Jun 29 2012, Doug Hellmann wrote: Hi Doug, Sorry for the late reply. I don't think I've made the problem clear. I'm not talking about wanting to calculate the different usage for CPU, RAM, etc. The different counters are calculated separately, so we can keep the amounts for CPU and RAM completely separate, and the API allows the outside user to ask for the amounts for each counter for a resource (or globally for a user/project). The problem is in deciding how the metadata associated with a meter event might cause the provider to change the rate they want to charge for that usage. It's not metadata of a counter that cause the provider to change the rate. It's a meter of a resource that can do that. No, there are cases where the metadata will affect the rate. For instance, it costs a different amount to have an instance in each of Amazon's availability zones (data centers). The counter would still say that the instance has been running for a certain amount of time, but the *rate* for charging for that time would depend on where it is. A representative from HP requested the same thing in ceilometer, and we may use it at DreamHost, too, eventually. That only solves part of the problem, though. As a provider I may want to charge different flat rates for the amount of RAM being used. For example, 1 unit for 1024 MB of RAM but 2 units for 4096 MB. That means when the size of the VM changes, we need to produce multiple totals (the length of time that the VM had 1024 MB RAM and then the length of time it had 4096 MB RAM). Yeah, like I said, for the meter 'RAM' of resource 'instance' you can't request a total amount used, because the type of this meter (I don't know how to call it, it's the kind I named if-i-change-you-need-to-split-the-resource-in-several-stuff in my latest email) don't have this semantic. I might also want to change the rate I bill when a VM is migrated between hosts or availability zones (I think we said migration caused a new instance to be created, but bear with me). The availability zone for an instance is clearly metadata and not something we can track via a counter. Again, that's also a meter that has the same type of 'RAM' for a resource 'instance'. That's an interesting idea, and it might solve the problem. At this point in the Folsom schedule though, I would much rather implement a pared down API that handles the simple cases but makes the caller do a bit more data manipulation for complex cases, in favor of focusing on counting more things than we do now. Is that a reasonable approach? Problem is that we might break the API a bit with this. This would not be the first time an OpenStack API is broken, but if we can avoid it, it'd be better. I am not sure you can really bill anything if you're not able to handle a simple thing such a VM resize. So currently, it seems that the API is not designed correctly to handle such a case, and since it's not yet implemented, maybe it's still time to fix it? We probably have time to fix it before the release. On the other hand, it seems much more important to use work on writing data collectors (new pollsters, adding notifications to other projects, etc.). I don't think we can do both things. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] resource metadata changes and billing
tl;dr: Ceilometer should ignore resource metadata when computing sums or maximum values for counters through the API. One of the things we discussed early during the design meetings was the need to track metadata along with resources so providers could use the metadata to determine the rate to charge for using the resource (for example, flavor or availability group of an instance). While working on the mongodb driver, I've been thinking about how that requirement changes what the API we defined needs to do. At first I thought we would need to try to return multiple values from the sum and max calls so the values could be associated with the resource metadata and a given time range. I've decided that implementing that inside the ceilometer API will be very complex, and unlikely to produce the correct result. I want to work through my reasoning here in case someone else can find a fault in it or propose a solution I wasn't able to find. First, the scenario: A user boots an instance, lets it run for some time (period 1), then changes the metadata in some way that does not result in the instance being recreated but does result in something the provider would decide introduces a different charge structure. For example, the amount of RAM allocated to the instance might be increased. After running with the new settings for a period of time (period 2), the user changes them back to their original value and the instance continues to run (period 3). The specific change to the metadata doesn't matter (RAM is just an example), except that the metadata change should not require an instance to be recreated because when that happens the user actually gets a new instance (at least I believe they do based on feedback during an early meeting, please correct me if I'm wrong). Getting a new instance simplifies things immensely, since the new instance is a completely new resource and so we can ignore those cases for the rest of this discussion. Another important point in the way the scenario is constructed is that the metadata values go from value A to B and then back to their original value A. That means any signature we calculate for the metadata will be the same during periods 1 and 3. Now we would like for v1/USERS/USER_ID/RESOURCES/instance_id/cpu/VOLUME to return the amount of CPU time used by the instance. However, if that time is billed at a different rate depending on the size of the server then the RAM change will cause a difference in billing rates. We therefore need to return a sequence of 3-tuples containing the total for the counter, the resource metadata, and the time range during which the metadata was in effect. There are two problems implementing this in a generic way in the API server. First, it turns out to be surprisingly difficult to write an efficient query to compute that time range in the case described because it is not easy to recognize the ranges for period 1 and period 3 as being interrupted by period 2 (finding min or max for a value is easy, but finding the endpoint of period 1 is not because the signature for the metadata is the same for period 1 and 3). Second, while calculating ranges is difficult in itself, what is even more difficult is recognizing *important* changes in the metadata that actually imply a change in the billing rate. The logic for that is up to the deployer and their billing rules. There are ways to compute the ranges by using multiple queries [1], and we could create some sort of way for the query to specify which fields in the metadata for a given type of resource are important. Both calculations would be expensive to apply in the API server, though, and I think they can be solved more efficiently on the client-side. If the client grabs all (or a large portion) of the data and processes it sequentially, it is easy to test the metadata fields to find changes and treat that condition as the boundary between the time ranges. My conclusion from all of this (over-)thinking is that the ceilometer API should assume the simple case and ignore the metadata changes when computing the sum or maximum value for a counter over a range of time. More complex processing will be left up to the caller, who can ask for raw metering data in manageable chunks and process them outside of the API. I could be persuaded to do something more complicated if the problems described above can be solved in a relatively simple way, but even then I think we should push that to the v2 API. Thoughts? There-and-back-again-ly, Doug [1] Find the min time for a (resource, metadata) pair; find min time for any different (resource, metadata) pair greater than the first time; find max for original pair with timestamp less than second min; use the max as starting point to determine the next range; loop. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help :
Re: [Openstack] [metering] resource metadata changes and billing
On Fri, Jun 29, 2012 at 1:19 PM, Julien Danjou jul...@danjou.info wrote: On Fri, Jun 29 2012, Doug Hellmann wrote: We do have counters for RAM and CPU separate from instance. But the rate at which the provider bills for those things may vary based on metadata. My example may be bad because it uses 2 values we're measuring, one of which also shows up in the metadata for another. As a different example, take the instance display name. The display name is under the control of the user and is extremely unlikely to reflect a change in the billing rate. However, changing the display name changes the metadata for the instance. A naive implementation of the processing loop would pick that up and generate multiple documents even though there is no need to do so. Yep, but the display name is not a counter. Memory is a counter. An instance is made of several counter. We need to split metered objects based on their absolute counter changing (memory, number of core…), not based on random metadata, i.e. a resource have several meters. I don't think I've made the problem clear. I'm not talking about wanting to calculate the different usage for CPU, RAM, etc. The different counters are calculated separately, so we can keep the amounts for CPU and RAM completely separate, and the API allows the outside user to ask for the amounts for each counter for a resource (or globally for a user/project). The problem is in deciding how the metadata associated with a meter event might cause the provider to change the rate they want to charge for that usage. So what was considered as metadata (like memory) so far should changed to become a meter of an resource (like an instance) and have for this one a special type (not sure about the type name to use). That only solves part of the problem, though. As a provider I may want to charge different flat rates for the amount of RAM being used. For example, 1 unit for 1024 MB of RAM but 2 units for 4096 MB. That means when the size of the VM changes, we need to produce multiple totals (the length of time that the VM had 1024 MB RAM and then the length of time it had 4096 MB RAM). I might also want to change the rate I bill when a VM is migrated between hosts or availability zones (I think we said migration caused a new instance to be created, but bear with me). The availability zone for an instance is clearly metadata and not something we can track via a counter. We may need to refine our model to be a bit more hierarhical like: resource -- counter #1 of type 'relative' | `- counter #2 of type 'absolute' ` counter #3 of type 'if-i-change-you-need-to-split-the-resource-in-several-stuff' That's an interesting idea, and it might solve the problem. At this point in the Folsom schedule though, I would much rather implement a pared down API that handles the simple cases but makes the caller do a bit more data manipulation for complex cases, in favor of focusing on counting more things than we do now. Is that a reasonable approach? etc… -- Julien ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] proposed changes to pollster plugin API
The Common Pollster class changes [1] from Kobagana Kumar reminded me of something I thought of when I was trying to figure out how to solve the instance-delete-notification problem in bug #1005944 [2]. The current implementation of the pollster plugins assumes that they could be polling anything at all (compute, network, volume, etc.) and leaves it up to the plugin to determine what resources to scan. That means each plugin queries the database for the list of instances, connects to libvirt, then asks for the data it needs. Kobagana is correct that we should only have one copy of that code, but the placement in the patch is wrong, for a reason that is only really clear if you think about both problems. In order to poll about an instance before it is deleted, we will need to add a new way to call the pollsters. That calling path will need to let the caller pass in the instance being deleted to avoid polling *all* instances on the compute node (primarily because if there are a lot of instances, polling all of them may take longer than we want to wait before finishing the delete operation). I propose that we move the code that connects to libvirt and gets the list of instances into the class that calls the pollsters (AgentManager) so we can support both calling patterns. That will make the AgentManager the ComputeAgentManager (since network pollsters won't necessarily need to talk to libvirt) and we will need other managers for calling pollsters that do not use compute resources. I don't see that as a problem, since those pollsters probably won't run on the compute nodes anyway so we will want them to be loaded in another agent process. What do the rest of you think about this change? Doug [1] https://review.stackforge.org/#/c/259/ [2] https://bugs.launchpad.net/ceilometer/+bug/1005944 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] proposed changes to pollster plugin API
On Thu, Jun 28, 2012 at 7:52 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Jun 28 2012, Doug Hellmann wrote: I propose that we move the code that connects to libvirt and gets the list of instances into the class that calls the pollsters (AgentManager) so we can support both calling patterns. That will make the AgentManager the ComputeAgentManager (since network pollsters won't necessarily need to talk to libvirt) and we will need other managers for calling pollsters that do not use compute resources. I don't see that as a problem, since those pollsters probably won't run on the compute nodes anyway so we will want them to be loaded in another agent process. What do the rest of you think about this change? That recalls me my first idea of having one nova.service.Service (but only one agent daemon) by OS component. Except this time we would put them in only one daemon. So that sounds fine to me. I'm not sure about using one daemon. It could work in some cases, but may not scale. If we rename bin/ceilometer-agent to bin/ceilometer-compute-agent and we add a bin/ceilometer-central-agent then that central agent could poll a bunch of services. If we have a lot of data to work with, though, we might want a bin/ceilometer-volume-agent and bin/ceilometer-network-agent so those daemons can poll the separate services more quickly. We could probably do that by making bin/ceilometer-central-agent take a command line argument to control which plugin(s) it loads. How do you propose we configure the enabled manager then? Using the same kind of system that we have for plugins? I'm not sure we need that many different managers. If we only need a couple, we could just have separate wrapper scripts like we do for the collector and agent now. That'd basically makes us have a 2 levels plugin system in the end, which doesn't bother me anyway since for now everything is automagically configured and running. Right. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Adding docs gating jobs?
On Thu, Jun 28, 2012 at 12:19 PM, Chmouel Boudjnah chmo...@chmouel.comwrote: Just to make sure this gating test will just run python setup.py build_sphinx, right? (if it was a gating spellcheck I'll be in big trouble :)) http://www.doughellmann.com/docs/sphinxcontrib.spelling/ /shameless-plug On Tue, Jun 26, 2012 at 4:02 PM, Monty Taylor mord...@inaugust.com wrote: Hey guys! We have all of the projects properly and consistently building and uploading sphinx docs from in tree. This is pretty exciting, because it means one more resource we can expect to work. So related to that, we were talking about putting in a gating job for each project to prevent changes from breaking the docs. I don't really expect these jobs to fail builds very often, as the jobs themselves are pretty stable - but obviously it's the kind of thing people might have an opinion on. Thoughts? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] proposed changes to pollster plugin API
On Thu, Jun 28, 2012 at 1:28 PM, Julien Danjou jul...@danjou.info wrote: On Thu, Jun 28 2012, Doug Hellmann wrote: I'm not sure we need that many different managers. If we only need a couple, we could just have separate wrapper scripts like we do for the collector and agent now. That makes me think about how nova-api works. We can mimic that. I'll have a look at how it works. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] horizon install from devstack fails to find glanceclient/versioninfo
On Tue, Jun 26, 2012 at 8:14 PM, Dean Troyer dtro...@gmail.com wrote: On Tue, Jun 26, 2012 at 6:37 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: OK, I got past that part but now I'm seeing an error about a missing volume group stack-volumes. Is that related to enabling cinder? The only settings I have in my localrc file are passwords. Is there something else I need to do to ensure cinder is on? I use this to enable cinder: ENABLED_SERVICES=$(echo $ENABLED_SERVICES | sed 's/n-vol/c-api,c-sch,c-vol/') But the volume group should be created either way. The cinder patch did change its name from nova-volumes to stack-volumes though. I'm not doing anything that makes me care which volume storage I'm using, so I don't necessarily need to enable cinder. For whatever reason, with the defaults, the volume group isn't being created (or has the wrong name). I'm using a fresh vagrant VM with all of my sandboxes updated to HEAD this morning. I'll run it again and get a list of the volume groups that do exist and see if I can figure out what's going wrong. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] horizon install from devstack fails to find glanceclient/versioninfo
On Wed, Jun 27, 2012 at 8:23 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Tue, Jun 26, 2012 at 8:14 PM, Dean Troyer dtro...@gmail.com wrote: On Tue, Jun 26, 2012 at 6:37 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: OK, I got past that part but now I'm seeing an error about a missing volume group stack-volumes. Is that related to enabling cinder? The only settings I have in my localrc file are passwords. Is there something else I need to do to ensure cinder is on? I use this to enable cinder: ENABLED_SERVICES=$(echo $ENABLED_SERVICES | sed 's/n-vol/c-api,c-sch,c-vol/') But the volume group should be created either way. The cinder patch did change its name from nova-volumes to stack-volumes though. I'm not doing anything that makes me care which volume storage I'm using, so I don't necessarily need to enable cinder. For whatever reason, with the defaults, the volume group isn't being created (or has the wrong name). I'm using a fresh vagrant VM with all of my sandboxes updated to HEAD this morning. I'll run it again and get a list of the volume groups that do exist and see if I can figure out what's going wrong. It looks like a problem creating the volume group on the loopback device. I get: Device /dev/loop0 not found (or ignored by filtering). Unable to add physical volume '/dev/loop0' to volume group 'stack-volumes'. This is on Ubuntu Precise. It was working Monday in the same image. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] integrating Quantum and Ceilometer
Hello, As part of the ceilometer project¹, we're working on retrieving usage data from various OpenStack components. We would like to integrate with Quantum for information about network resource utilization that a deployer might want to bill their tenants for. Ceilometer has a plugin-based architecture, which makes it easy to add new measurement types and data collectors. Our approach with other measurements is to collect everything and let the ceilometer user decide what to bill for and what to ignore (they can turn off measurements for things they do not care about). Depending on the project and the type of data, we can either use the notification events generated by allocating/deallocating a resource, or we can poll for metrics being collected elsewhere. We will probably want to use both approaches for integrating with Quantum (e.g., use events for things like IP allocation and polling for I/O). Do you have any advice on how to integrate Ceilometer and Quantum? For example, does Quantum emit notifications and does it collect (or provide an API to query) I/O statistics? Thanks, Doug ¹ http://launchpad.net/ceilometer ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] horizon install from devstack fails to find glanceclient/versioninfo
On Wed, Jun 27, 2012 at 9:44 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Wed, Jun 27, 2012 at 8:23 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Tue, Jun 26, 2012 at 8:14 PM, Dean Troyer dtro...@gmail.com wrote: On Tue, Jun 26, 2012 at 6:37 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: OK, I got past that part but now I'm seeing an error about a missing volume group stack-volumes. Is that related to enabling cinder? The only settings I have in my localrc file are passwords. Is there something else I need to do to ensure cinder is on? I use this to enable cinder: ENABLED_SERVICES=$(echo $ENABLED_SERVICES | sed 's/n-vol/c-api,c-sch,c-vol/') But the volume group should be created either way. The cinder patch did change its name from nova-volumes to stack-volumes though. I'm not doing anything that makes me care which volume storage I'm using, so I don't necessarily need to enable cinder. For whatever reason, with the defaults, the volume group isn't being created (or has the wrong name). I'm using a fresh vagrant VM with all of my sandboxes updated to HEAD this morning. I'll run it again and get a list of the volume groups that do exist and see if I can figure out what's going wrong. It looks like a problem creating the volume group on the loopback device. I get: Device /dev/loop0 not found (or ignored by filtering). Unable to add physical volume '/dev/loop0' to volume group 'stack-volumes'. This is on Ubuntu Precise. It was working Monday in the same image. Deleting the existing backing file and letting devstack re-create it fixed the problem. I've never had to do that before, so I have no idea what might have caused it to be necessary this time. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] integrating Quantum and Ceilometer
On Wed, Jun 27, 2012 at 5:48 PM, Dan Wendlandt d...@nicira.com wrote: Hi Doug, Thanks for sending this out. Pinging the ceilometer team about Quantum support has been on my todo list for a while now. Troy (CC'd) did some thinking about notifications for quantum, but we haven't implemented this yet. Here's the existing blueprint: https://blueprints.launchpad.net/quantum/+spec/quantum-notifications . My assumption is that we would model what Nova is doing for notifications, but to be honest, I'm not really familiar with that code. Anyone interested in taking this blueprint on? Using the nova-based notifications library would make it very easy to tie in with the work we have already done to catch compute notifications. The libraries for notifications were just moved into openstack.common, which will make them easy for you to pick up and reuse. We do have an extension to support port-statistics to indicate the amount of traffic sent to/from a VM that could be used by ceilometer, though that would likely be more of a poll model, at least with the current implementation. That makes sense, and is similar to what we're doing for things like CPU and disk I/O by polling libvirt. One thing that comes up frequently is the desire to only charge for traffic outside of a cloud. Do the ports know how to tell the difference between internal and external traffic? Dan On Wed, Jun 27, 2012 at 3:16 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: Hello, As part of the ceilometer project¹, we're working on retrieving usage data from various OpenStack components. We would like to integrate with Quantum for information about network resource utilization that a deployer might want to bill their tenants for. Ceilometer has a plugin-based architecture, which makes it easy to add new measurement types and data collectors. Our approach with other measurements is to collect everything and let the ceilometer user decide what to bill for and what to ignore (they can turn off measurements for things they do not care about). Depending on the project and the type of data, we can either use the notification events generated by allocating/deallocating a resource, or we can poll for metrics being collected elsewhere. We will probably want to use both approaches for integrating with Quantum (e.g., use events for things like IP allocation and polling for I/O). Do you have any advice on how to integrate Ceilometer and Quantum? For example, does Quantum emit notifications and does it collect (or provide an API to query) I/O statistics? Thanks, Doug ¹ http://launchpad.net/ceilometer ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- ~~~ Dan Wendlandt Nicira, Inc: www.nicira.com twitter: danwendlandt ~~~ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] horizon install from devstack fails to find glanceclient/versioninfo
I'm having trouble running devstack today. I'm getting the following error
during the horizon installation. Does anyone have any idea what would be
causing that?
Doug
Downloading/unpacking python-glanceclient (from -r
horizon.egg-info/requires.txt (line 7))
Running setup.py egg_info for package python-glanceclient
Traceback (most recent call last):
File string, line 14, in module
File /opt/stack/horizon/build/python-glanceclient/setup.py, line
22, in module
version=setup.get_post_version('glanceclient'),
File glanceclient/openstack/common/setup.py, line 329, in
get_post_version
return open(os.path.join(projectname, 'versioninfo'),
'r').read().strip()
IOError: [Errno 2] No such file or directory: 'glanceclient/versioninfo'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File string, line 14, in module
File /opt/stack/horizon/build/python-glanceclient/setup.py, line 22, in
module
version=setup.get_post_version('glanceclient'),
File glanceclient/openstack/common/setup.py, line 329, in
get_post_version
return open(os.path.join(projectname, 'versioninfo'),
'r').read().strip()
IOError: [Errno 2] No such file or directory: 'glanceclient/versioninfo'
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
