Re: [Openstack] Cinder Storage Server Statistics

[Doug Hellmann]

When I said, "we", I meant "the ceilometer team". If the auditing app isn't
finding any volumes, it's not going to notify us.

If you just want to know how much data is being used by cinder, there may
be a way to get that from their admin API, but I'm not sure.

On Mon, Jul 15, 2013 at 7:08 PM, Ray Sun  wrote:

> D
> ​oug,
> Thanks. I tried it in grizzly, here's the return:
> sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit
> Starting volume usage audit
> Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00
> Found 0 volumes
> Volume usage audit completed​
>
> ​Actually, I want to get some data like this:
> Total Cinder Storage on Physical Machine: 100G
> Used Cinder Storage on Physical Machine: 10G​
>
> Is there any way to get this?
>
>
> Best Regards
> -- Ray
>
>
> On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>> We rely on a similar audit program to get the "exists" notifications
>> about cinder volumes. Look for "cinder-volume-usage-audit".
>>
>> Doug
>>
>>
>> On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun  wrote:
>>
>>> Yes, it should be, but seems not at least in grizzly. Any update of
>>> Ceilometer?
>>>
>>> Best Regards
>>> -- Ray
>>>
>>>
>>> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang wrote:
>>>
>>>> I think Statistics should be find in Ceilometer. Ceilometer may provide
>>>> with
>>>> enough information you need.
>>>>
>>>> Best regards,
>>>> Haomai Wang, UnitedStack Inc.
>>>>
>>>> 在 2013-7-14，上午8:09，Ray Sun  写道：
>>>>
>>>> In nova, we have a period task to report the usage of the physical
>>>> server, including CPU, Memory and Local Disk, but I don't think I can find
>>>> the same strategy in cinder service. Is there any way to do this or is
>>>> there any blueprint for this?
>>>>
>>>> Thanks.
>>>>
>>>> Best Regards
>>>> -- Ray
>>>>  ___
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>>
>>>
>>> ___
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to : openstack@lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Cinder Storage Server Statistics

[Doug Hellmann]

We rely on a similar audit program to get the "exists" notifications about
cinder volumes. Look for "cinder-volume-usage-audit".

Doug


On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun  wrote:

> Yes, it should be, but seems not at least in grizzly. Any update of
> Ceilometer?
>
> Best Regards
> -- Ray
>
>
> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang wrote:
>
>> I think Statistics should be find in Ceilometer. Ceilometer may provide
>> with
>> enough information you need.
>>
>> Best regards,
>> Haomai Wang, UnitedStack Inc.
>>
>> 在 2013-7-14，上午8:09，Ray Sun  写道：
>>
>> In nova, we have a period task to report the usage of the physical
>> server, including CPU, Memory and Local Disk, but I don't think I can find
>> the same strategy in cinder service. Is there any way to do this or is
>> there any blueprint for this?
>>
>> Thanks.
>>
>> Best Regards
>> -- Ray
>>  ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Ceilometer][Healthnmon]

[Doug Hellmann]

The work to merge ceilometer and healthnmon hasn't made much progress, yet.

The ceilometer team is working on collecting information about CPU and RAM
utilization for triggering alarms for Heat. That work is schedule for the
havana release, and is moving along nicely. We could use some help with
code reviews, but I think the overall design is worked out.

Doug


On Mon, Jul 1, 2013 at 5:51 AM, Emanuel Marzini  wrote:

> Hi,
> I want to retrive Vm information from Openstack. I am interested of CPU &
> RAM utilization.
> I known that someone use collectd, libvirt ecc.. or product like
> Ceilometer or Healthnmon.
> I am also interested to receive alarm information from the cloud provider
> if the CPU or RAM value exceeds a threshold.
> I read that from Havana, Ceilometer and Healthnmon will be merged, it's
> true? Might be a good solution to my problems?
>
> Thanks in advance
> Emanuel
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] [Metering] Options to obtain Notifications when ceilometer generates a meter entry in the DB

[Doug Hellmann]

On Fri, Jun 21, 2013 at 9:37 PM, Deepak Bangalore Hariyanna <
dbang...@usc.edu> wrote:

> Hi All,
>
> I am currently working on an charging application to work with
> ceilometer(Grizzly release).
> Have been working on openstack for just couple weeks and hence have a
> couple of questions:
> 1). I read in the documentation that the only way to access the data
> generated by collector is through the API(i.e either use the cURL option or
> the ceilometer pythonclient to query the required information). I am able
> to do this and fetch the required data.
> However, for an application to access these, is there a better way? Like,
> whenever the ceilometer generates an event entry in the db, does it provide
> any notification on any queues, or is there a way we can configure this to
> happen?
>
> If there is no such option, I plan to periodically poll the ceilometer
> through API for the usage data. (which I sincerely feel is a bad idea).
>
> 2). From the Havana roadmap, I see an objective "Publish meters to other
> systems"
> Does this mean, other systems can configure to bind themselves on an AMQP
> queue and ceilometer would publish notifications on this exchange?
> If so, this would be a feature of Havana?
> Because I was confused with the publisher feature of Grizzzly, as below
> [image: Inline image 1]
>
> Would be great if someone can help me on this.
>

The architecture of ceilometer supports plugging in different publishers.
We have one that reproduces the grizzly/folsom feature of sending data on
the message bus so the collector can receive it. We also have one that
sends UDP messages that are consumed by the collector (useful for
monitoring, but not for metering). The API is designed to allow you to
write your own if you have a custom consumer, and we would welcome other
reusable publishers in the core code base.

Doug


>
> Thanks a lot,
> Deepak
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer-api Auth Error

[Doug Hellmann]

This is also a good point. I think you're getting an unscoped token, which
won't be useful for authenticating to ceilometer.

Doug


On Thu, Jun 6, 2013 at 12:33 PM, Guangyu Suo wrote:

> Hello, claudio
>
> I don't think you get the real token, because you did't specify the
> tenantName or tenantID in curl command, even if you used admin and
> admin_pass. Please read this document
> http://docs.openstack.org/api/quick-start/content/ for details.
>
>
> 2013/6/6 claudio marques 
>
>> Hi Doug
>>
>> I send authentications from the admin user. In order for me to explain
>> you better the issue i will paste here the phases that i am trying to do:
>>
>> I use curl and send the admin user/password and ask for a token:
>>
>> curl -d '{"auth":{"passwordCredentials":{"username": "admin", "password":
>> "admin_pass"}}}' -H "Content-type: application/json"
>> http://localhost:35357/v2.0/tokens
>>
>> It returns this
>>
>> {"access": {"token": {"issued_at": "2013-06-06T14:11:26.005501",
>> "expires": "2013-06-07T14:11:26Z", "id":
>> "MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE="},
>> "serviceCatalog": [], "user": {"username": "admin", "roles_links": [],
>> "id": "0b14e1644ffa4336967087458cd85c56", "roles": [], "name": "admin"},
>> "metadata": {"is_admin": 0, "roles": []}}}
>>
>> Then, i use curl again with the token that i just received with the
>> following command
>>
>>
>> curl -k -D -H "X-Auth-Token:
>> MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE="
>> -X 'GET' -v http://localhost:8777/v2/meters
>>
>> The return value is this
>>
>> ** getaddrinfo(3) failed for X-Auth-Token:80*
>> ** Couldn't resolve host 'X-Auth-Token'*
>> ** Closing connection 0*
>> *curl: (6) Couldn't resolve host 'X-Auth-Token'*
>> ** About to connect() to localhost port 8777 (#1)*
>> **   Trying 127.0.0.1...*
>> ** Connected to localhost (127.0.0.1) port 8777 (#1)*
>> *> GET /v2/meters HTTP/1.1*
>> *> User-Agent: curl/7.29.0*
>> *> Host: localhost:8777*
>> *> Accept: */**
>> *>*
>> ** HTTP 1.0, assume close after body*
>> *< HTTP/1.0 401 Unauthorized*
>> *< Date: Thu, 06 Jun 2013 14:14:06 GMT*
>> *< Server: WSGIServer/0.1 Python/2.7.4*
>> *< WWW-Authenticate: Keystone uri='http://127.0.0.1:35357'*
>> *< Content-Length: 381*
>> *< Content-Type: text/html; charset=UTF-8*
>> *<*
>> **
>> * *
>> *  401 Unauthorized*
>> * *
>> * *
>> *  401 Unauthorized*
>> *  This server could not verify that you are authorized to access the
>> document you requested. Either you supplied the wrong credentials (e.g.,
>> bad password), or your browser does not understand how to supply the
>> credentials required.*
>> *Authentication required*
>> *
>> *
>> *
>> *
>> * *
>> ** Closing connection 1*
>>
>> No data is returned in any case.
>>
>> I manually installed 3 node openStack and ceilometer , so i am not using
>> devStack.
>>
>> Even when i try to send manually credentials of the admin user:
>>
>> *ceilometer --os-username admin --os-password password --os-tenant-name
>> admin --os-auth-url http://localhost:5000/v2.0 resource-list*
>>
>> the result is this:
>>
>> *No handlers could be found for logger "ceilometerclient.common.http"*
>> *Invalid OpenStack Identity credentials.*
>>
>> Is there any possibility that keystone is not validating all the Tokens?
>>
>>
>> Claudio Marques
>>
>>
>> --
>> Date: Thu, 6 Jun 2013 09:42:38 -0400
>> From: doug.hellm...@dreamhost.com
>> To: clau...@onesour

Re: [Openstack] Ceilometer-api Auth Error

[Doug Hellmann]

On Thu, Jun 6, 2013 at 10:43 AM, claudio marques wrote:

> Hi Doug
>
> I send authentications from the admin user. In order for me to explain you
> better the issue i will paste here the phases that i am trying to do:
>
> I use curl and send the admin user/password and ask for a token:
>
> curl -d '{"auth":{"passwordCredentials":{"username": "admin", "password":
> "admin_pass"}}}' -H "Content-type: application/json"
> http://localhost:35357/v2.0/tokens
>
> It returns this
>
> {"access": {"token": {"issued_at": "2013-06-06T14:11:26.005501",
> "expires": "2013-06-07T14:11:26Z", "id":
> "MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE="},
> "serviceCatalog": [], "user": {"username": "admin", "roles_links": [],
> "id": "0b14e1644ffa4336967087458cd85c56", "roles": [], "name": "admin"},
> "metadata": {"is_admin": 0, "roles": []}}}
>
> Then, i use curl again with the token that i just received with the
> following command
>
>
> curl -k -D -H "X-Auth-Token:
> MIICbgYJKoZIhvcNAQcCoIICXzCCAlsCAQExCTAHBgUrDgMCGjCCAUcGCSqGSIb3DQEHAaCCATgEggE0eyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0wNi0wNlQxNDoxMToyNi4wMDU1MDEiLCAiZXhwaXJlcyI6ICIyMDEzLTA2LTA3VDE0OjExOjI2WiIsICJpZCI6ICJwbGFjZWhvbGRlciJ9LCAic2VydmljZUNhdGFsb2ciOiBbXSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICIwYjE0ZTE2NDRmZmE0MzM2OTY3MDg3NDU4Y2Q4NWM1NiIsICJyb2xlcyI6IFtdLCAibmFtZSI6ICJhZG1pbiJ9LCAibWV0YWRhdGEiOiB7ImlzX2FkbWluIjogMCwgInJvbGVzIjogW119fX0xgf8wgfwCAQEwXDBXMQswCQYDVQQGEwJVUzEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEYMBYGA1UEAxMPd3d3LmV4YW1wbGUuY29tAgEBMAcGBSsOAwIaMA0GCSqGSIb3DQEBAQUABIGALO7HS8ddSpYzEmRK9eLHOkFQPifzbNSHrf9I62keB+BDmBHAD47Lhz+jg-SkRJXKlyWVL0YG3Mhd3R9srSRC15rMGNhC0wSt0ohcppjzIr-OT8x6UabTYdU0We-54+4dEbyIgMH6fIuWKLq3DKvk+Qb-57JGknBemnFSrZHZjNE="
> -X 'GET' -v http://localhost:8777/v2/meters
>
> The return value is this
>
> ** getaddrinfo(3) failed for X-Auth-Token:80*
> ** Couldn't resolve host 'X-Auth-Token'*
> ** Closing connection 0*
> *curl: (6) Couldn't resolve host 'X-Auth-Token'*
>

It looks like curl does not like the way you are passing some of the
parameters. It is interpreting the header name as a host name.

Doug


> ** About to connect() to localhost port 8777 (#1)*
> **   Trying 127.0.0.1...*
> ** Connected to localhost (127.0.0.1) port 8777 (#1)*
> *> GET /v2/meters HTTP/1.1*
> *> User-Agent: curl/7.29.0*
> *> Host: localhost:8777*
> *> Accept: */**
> *>*
> ** HTTP 1.0, assume close after body*
> *< HTTP/1.0 401 Unauthorized*
> *< Date: Thu, 06 Jun 2013 14:14:06 GMT*
> *< Server: WSGIServer/0.1 Python/2.7.4*
> *< WWW-Authenticate: Keystone uri='http://127.0.0.1:35357'*
> *< Content-Length: 381*
> *< Content-Type: text/html; charset=UTF-8*
> *<*
> **
> * *
> *  401 Unauthorized*
> * *
> * *
> *  401 Unauthorized*
> *  This server could not verify that you are authorized to access the
> document you requested. Either you supplied the wrong credentials (e.g.,
> bad password), or your browser does not understand how to supply the
> credentials required.*
> *Authentication required*
> *
> *
> *
> *
> * *
> ** Closing connection 1*
>
> No data is returned in any case.
>
> I manually installed 3 node openStack and ceilometer , so i am not using
> devStack.
>
> Even when i try to send manually credentials of the admin user:
>
> *ceilometer --os-username admin --os-password password --os-tenant-name
> admin --os-auth-url http://localhost:5000/v2.0 resource-list*
>
> the result is this:
>
> *No handlers could be found for logger "ceilometerclient.common.http"*
> *Invalid OpenStack Identity credentials.*
>
> Is there any possibility that keystone is not validating all the Tokens?
>
>
> Claudio Marques
>
>
> --
> Date: Thu, 6 Jun 2013 09:42:38 -0400
> From: doug.hellm...@dreamhost.com
> To: clau...@onesource.pt
> CC: openstack@lists.launchpad.net
> Subject: Re: [Openstack] Ceilometer-api Auth Error
>
>
>
>
>
> On Thu, Jun 6, 2013 at 7:22 AM, Claudio Marques wrote:
>
> Hi Stackers
>
>
> Hi have a problem with ceilometer-api. I want access it via curl or http
> and every time i try to do it i simple get the same errors.
>
> This server could not verify that you are authorized t

Re: [Openstack] Ceilometer-api Auth Error

[Doug Hellmann]

On Thu, Jun 6, 2013 at 7:22 AM, Claudio Marques wrote:

> Hi Stackers
>
>
> Hi have a problem with ceilometer-api. I want access it via curl or http
> and every time i try to do it i simple get the same errors.
>
> This server could not verify that you are authorized to access the
> document you requested. Either you supplied the wrong credentials (e.g.,
> bad password), or your browser does not understand how to supply the
> credentials required.
>
> My ceilometer.conf file is like this:
>
> [DEFAULT]
> os_username=admin
> os_password=admin_pass
> os_tenant_name=admin
> os_auth_url=http://10.0.1.167:5000/v2.0/
> signing_dirname = /tmp/keystone-signing-ceilometer
> metering_api_port=8777
> auth_strategy=keystone
> nova_control_exchange=nova
> hypervisor_inspector=libvirt
> libvirt_type=qemu
> glance_control_exchange=glance
> quantum_control_exchange=quantum
> debug=true
> verbose=true
>
> log_dir=/var/log/ceilometer
> rpc_backend=ceilometer.openstack.common.rpc.impl_kombu
> rabbit_host=localhost
> rabbit_port=5672
> rabbit_userid=guest
> rabbit_password=guest
> rabbit_retry_backoff=2
> rabbit_max_retries=0
> rabbit_use_ssl=False
>
> database_connection=mongodb://10.0.1.25:27017/ceilometer
> sql_connection_debug=0
> cinder_control_exchange=cinder
> enable_v1_api=true
>
> [keystone_authtoken]
>
> auth_host = localhost
> auth_port = 5000
> admin_user = admin
> admin_password = admin_pass
> admin_tenant_name = admin
> auth_uri = http://10.0.1.167:5000/v2.0/
>
> What auth chould i pass in order to get metrics form ceilometer?
>

The ceilometer API uses keystone authentication, just like the other
OpenStack services. If you pass credentials for a regular user, you can see
data about the tenant/project you're authenticating with. If you pass
credentials for an admin user, you can see all data.

Doug


>
> Thank's for any reply
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] support for older versions of ceilometer

[Doug Hellmann]

On Fri, May 31, 2013 at 4:11 AM, Julien Danjou  wrote:

> On Thu, May 30 2013, Tim Bell wrote:
>
> > I hope that ceilometer can also include this within the Havana timeframe
> as
> > it becomes a key component of production, large scale
> > clouds.
>
> I think we all agree in principle. We'd be happy to enhance Ceilometer
> and fix potential bugs in this direction, but we'll require some help
> testing the upgrade path *before* Havana is released.
>

Right, and upgrades are a different issue than running production systems
long-term on mixed versions.

Most of the problems between grizzly or trunk ceilometer and folsom have
had to do with proof-of-concept deployments where people were trying to
install a "modern" ceilometer on the same server as another component that
was older. Because the services require incompatible libraries and were
running on the same host using the same site-packages directories, there
are conflicts and things broke (either ceilometer, or the other service).
Deploying into virtualenvs or using separate hosts will reduce (if not
eliminate) these problems. The protocol ceilometer uses to talk over the
message bus is compatible between versions, AFAIK, so should not have
issues with rolling upgrades.

Doug


>
> --
> Julien Danjou
> # Free Software hacker # freelance consultant
> # http://julien.danjou.info
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] support for older versions of ceilometer

[Doug Hellmann]

>From what I understand, it is unusual to support mixing components from
different releases like that. Am I wrong?

Doug


On Thu, May 30, 2013 at 3:29 PM, Tim Bell  wrote:

> ** **
>
> Doug,
>
> ** **
>
> Can you advise on what the plan/policy will be for Havana ?
>
> ** **
>
> **-  **Will I be able to run Havana core components such as Nova
> with Grizzly ceilometer ?
>
> **-  **Will I be able to run Havana ceilometer with Grizzly core
> components (with reduced functionality compared to the Havana core
> components) ?
>
> ** **
>
> Tim
>
> ** **
>
> *From:* Openstack [mailto:openstack-bounces+tim.bell=
> cern...@lists.launchpad.net] *On Behalf Of *Doug Hellmann
> *Sent:* 30 May 2013 18:59
> *To:* openstack@lists.launchpad.net
> *Subject:* [Openstack] [ceilometer] support for older versions of
> ceilometer
>
> ** **
>
> The ceilometer team has had a few requests for help with older versions of
> ceilometer or running the grizzly version of ceilometer with older versions
> of other OpenStack components lately. We appreciate the level of interest
> in the project but, as much as we would like to, unfortunately we are not
> always able to help everyone with these "mixed" configurations. During the
> early phases of our development we tried to maintain compatibility with
> folsom, even well into the grizzly development cycle. We are no longer
> doing that, however, and so to make clear what support we can offer, the
> ceilometer team has put together the statement below. It has been added to
> our documentation, and we are posting here to make sure everyone has a
> chance to see it.
>
> ** **
>
> Regards,
>
> Doug
>
> ** **
>
> ** **
>
> The ceilometer team has limited capacity to provide support for older
> versions of the project. Because the project graduated from incubation
> around the time of the grizzly release, that is the first version for which
> we will provide regular ongoing support following the standard deprecation
> cycle for OpenStack [1]. The grizzly version of ceilometer  cannot be
> installed on the same server with earlier versions of OpenStack because of
> conflicting package requirements, but is API compatible with the folsom
> release if installed separately.
>
> ** **
>
> [1] https://wiki.openstack.org/wiki/StableBranch
>
> ** **
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [ceilometer] support for older versions of ceilometer

[Doug Hellmann]

The ceilometer team has had a few requests for help with older versions of
ceilometer or running the grizzly version of ceilometer with older versions
of other OpenStack components lately. We appreciate the level of interest
in the project but, as much as we would like to, unfortunately we are not
always able to help everyone with these "mixed" configurations. During the
early phases of our development we tried to maintain compatibility with
folsom, even well into the grizzly development cycle. We are no longer
doing that, however, and so to make clear what support we can offer, the
ceilometer team has put together the statement below. It has been added to
our documentation, and we are posting here to make sure everyone has a
chance to see it.

Regards,
Doug


The ceilometer team has limited capacity to provide support for older
versions of the project. Because the project graduated from incubation
around the time of the grizzly release, that is the first version for which
we will provide regular ongoing support following the standard deprecation
cycle for OpenStack [1]. The grizzly version of ceilometer  cannot be
installed on the same server with earlier versions of OpenStack because of
conflicting package requirements, but is API compatible with the folsom
release if installed separately.

[1] https://wiki.openstack.org/wiki/StableBranch
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Ceilometer][Ceilometer-API] Ceilometer-API Error 401 Unauthorized

[Doug Hellmann]

Right now we only have a web API, which you can use via the client library,
curl, or the command line tool. There are a few people working on
integrating ceilometer data with horizon, but I don't know the status of
that project.

Doug


On Tue, May 28, 2013 at 3:49 AM, Ildiko Vancsa wrote:

> Hi All,
>
> I'm new to OpenStack and Ceilometer as well, so I have a few questions. :)
>
> Does Ceilometer supports a Web UI or it is available via command line and
> curl only? I installed the environment with DevStack as I wanted to check
> how it works and looks like and it sets ceilometer in the apache2 serveice
> as an enabled site, but I could not find any information about how to set
> it up correctly.
>
> Thanks and Best Regards,
> Ildiko
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New code name for networks

[Doug Hellmann]

On Mon, May 13, 2013 at 11:30 AM, Sean Dague  wrote:

> On 05/13/2013 11:03 AM, Doug Hellmann wrote:
>
>>
>>
>> +1
>>
>> Use a namespace package "openstack" then each project has a unique
>> package under that for their meaningfully named package (compute,
>> networking, etc.).
>>
>
> Sounds great and all, except when you try to do something like quantum, or
> even cinder, where you break out function into another project.
>
> from openstack import network <- wait, is that what used to be nova
> network, or quantum, or some abstraction?
>

Fair point, and those specific examples of names may not be good. :-)

On the other hand, there's no reason each team needs to be limited to
deploying a single package. If nova network was "openstack.network" to
begin with, then perhaps quantum could have just been an evolution of that.


>
> from openstack.compute import network as compute_network ?
>
> Code names are actually incredibly useful in developing this stuff,
> because it lets us think about an implementation separate from a concept,
> and work with non native english speakers a lot easier where concept vs.
> implementation.
>
> Honestly, there is already incredible confusion when you talk with people
> about "compute". Where you have to be really paying attention to nuance to
> figure out if people meant "Nova as a whole", "the Nova compute daemon", "A
> nova compute node, which might also have nova-network on it". The number of
> times we had to explain no-db-compute blueprint because of that speaks to
> the fact that generic names do not make anything easier, they generate more
> confusion quite often.
>
> Code names are useful because it gives us a whole other namespace of words
> to work with to be very specific about what we mean, that can't be confused
> for the generic concepts of networking or computing. Yes, it's inside
> baseball, but when you are dealing with code as complicated as OpenStack,
> not having that inside baseball really slows things down.
>

This seems like a stronger argument than "it's hard to rename things"
(which I think is at least a large part of the argument against moving away
from code names during/after incubation).

FWIW, I'm in favor of not branding every little thing we do "openstack,"
especially when components can be reused by other projects (I would love to
see us acquire a few downstream users of some of the Oslo libraries, for
example). But for the big core stuff, it feels weird to have separate
commands, separate package namespaces, etc.


>
> Just look at the regular confusion new people have about the 2 uses of the
> term migrations in Nova, one for the database schema, and one for moving
> guests around. :)


I'm not sure we're going to eliminate confusion entirely. I'm not even sure
changing anything will reduce it. But it does seem like we can minimize
legal issues by avoiding using code names in the code, which is where this
thread started.

Doug


>
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] New code name for networks

[Doug Hellmann]

On Sat, May 11, 2013 at 5:48 PM, Anne Gentle  wrote:

>
>
>
> On Sat, May 11, 2013 at 3:12 PM, Monty Taylor wrote:
>
>>
>>
>> On 05/11/2013 04:07 PM, Asher Newcomer wrote:
>> > Or even better, just continue to call it openstack networking. The code
>> > names only serve to confuse the uninitiated. They needlessly steepen the
>> > learning curve and slow uptake.
>>
>> The problem with OpenStack Networking (or getting rid of codenames) is
>> seen with pre-incubation->incubation->integrated cycle.
>>
>> A project cannot call itself "OpenStack Foo" until it actually _is_
>> openstack foo. So any new project by necessity has to start off as
>> something else.
>>
>> But - if we then require them to drop that name and become openstack foo
>> when they become incubated or integrated, then we've got what's become a
>> stable project with a decent amount of contributors renaming itself.
>>
>> Every. Time.
>>
>> The code names aren't just cute. I kind of wish they were, because it
>> would make several things much easier if we could just ditch them and do
>> a pure openstack. code namespace. But the reality is that it gets really
>> really tricky to deal with a bunch of things if they go away.
>>
>
> I told Monty and the TC this at the Summit (sorry I couldn't attend the
> session about code names). I find this trickiness a weak argument in the
> face of the invented names that are getting to be as bad as U.S.
> pharmaceuticals. Plus it forces us to put a "lookup" table in the docs
> forever. [1] Let's find a process for naming that meets a few reqs:
> - describes the service
> - goes through a legal review
> - enables new eyes to understanding it by reading the English word that
> the service represents (that can be translated into a meaningful word in
> other languages)
>
> If we have to preface with Stackforge to indicate pre-incubation, that's
> fine. Use Incubating or some such to indicate incubation. Meaningful words
> have meaning.
>

+1

Use a namespace package "openstack" then each project has a unique package
under that for their meaningfully named package (compute, networking, etc.).

We only have to rename projects if they start out with "bad" names to begin
with. Projects that want to be integrated should be developing on
stackforge and using the openstack namespace package.


>
> I acknowledge we still have to indicate what commands, daemon names, and
> so on are related to a particular service. That issue is also fixable with
> some resources and effort and pays off in easier adoption and understanding.
>

The unified command line project will resolve the command issue because all
commands will be "openstack $noun $verb" (end users shouldn't have to know
which OpenStack component owns a resource in order to operate on it via the
command line). Daemon startup scripts could use a similar framework, or
just a naming convention ("openstack-compute-api", "openstack-network-api",
etc.).

Doug


>
> Anne
>
> 1.
> http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/terminology-codenames.html
>
>
>
>>
>> > On May 11, 2013 3:59 PM, "Davanum Srinivas" > > > wrote:
>> >
>> > Lattice
>> >
>> > -- dims
>> >
>> > On Sat, May 11, 2013 at 3:52 PM, Mark Turner > > > wrote:
>> > > Tubes
>> > >
>> > > ;-)
>> > >
>> > >
>> > > On Sat, May 11, 2013 at 12:51 PM, Jason Smith
>> > mailto:jason.sm...@rackspace.com>>
>> > > wrote:
>> > >>
>> > >> Hello,
>> > >> I understand why we had to give up Quantum code name but rather
>> > than just
>> > >> refer to it as networking let's come up with a new code name!
>> > >>
>> > >> Thoughts?
>> > >>
>> > >> Thanks,
>> > >> -js
>> > >> ___
>> > >> Mailing list: https://launchpad.net/~openstack
>> > >> Post to : openstack@lists.launchpad.net
>> > 
>> > >> Unsubscribe : https://launchpad.net/~openstack
>> > >> More help : https://help.launchpad.net/ListHelp
>> > >
>> > >
>> > >
>> > > ___
>> > > Mailing list: https://launchpad.net/~openstack
>> > > Post to : openstack@lists.launchpad.net
>> > 
>> > > Unsubscribe : https://launchpad.net/~openstack
>> > > More help   : https://help.launchpad.net/ListHelp
>> > >
>> >
>> >
>> >
>> > --
>> > Davanum Srinivas :: http://davanum.wordpress.com
>> >
>> > ___
>> > Mailing list: https://launchpad.net/~openstack
>> > Post to : openstack@lists.launchpad.net
>> > 
>> > Unsubscribe : https://launchpad.net/~openstack
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>> >
>> >
>> > ___
>> > Mailing list: h

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

The database schema isn't part of the formal API, so if you're OK with code
breaking as we make changes to that schema then it would be fine to run
without the API. If you want to ensure your app continues to work across
those changes, it should be straightforward to set up the API server in a
virtualenv.

Doug


On Tue, May 7, 2013 at 7:13 AM, Riki Arslan wrote:

> Thank you for the through explanation. I have no problem running the
> Collector, Central and Compute Agents. So, I believe only the API Server is
> trying to use the old oslo-incubator version.
>
> I am still weighing the options.
>
> Just a quick question; since the only thing that does not work in my
> environment is the API Server, I believe -as long as I can query MongoDB
> directly-, I think I wouldn't need it anyway. Would you say this is correct?
>
>
> On Mon, May 6, 2013 at 6:08 PM, Doug Hellmann  > wrote:
>
>> It looks like you still have incompatible versions of things installed.
>>
>> The configuration library changed during grizzly. The old version and new
>> version cannot be used together in the same program because they both try
>> to modify different copies of a global variable. The exception you're
>> getting is, I think, due to the fact that the API service loads the
>> keystone middleware to handle authentication. You have a version of the
>> middleware that uses oslo.config, and a version of ceilometer that uses the
>> older oslo-incubator version of the configuration library.
>>
>> The ceilometer team is small, so we have limited capacity to support
>> "old" versions (especially pre-incubated versions). We do intend to support
>> grizzly, but can only offer moderate help with folsom. The g2 release
>> tarballs *should* be compatible at the communication layer with folsom
>> versions of the other components, but it looks like you can't install them
>> into the same Python installation as the other services.
>>
>> You can separate ceilometer code from the other services a couple of
>> different ways. The simplest would be to use a separate VM to run
>> ceilometer. That would let you follow all of the normal instructions, and
>> ensure that you don't have mismatched versions of libraries. The other way
>> is to install ceilometer into a virtualenv. That would take more care,
>> since you need to ensure that the virtualenv does not look at the globally
>> installed site-packages. I haven't tried doing this, so I can't provide
>> more detailed steps, and you will likely need to experiment a bit to get it
>> right.
>>
>> The one piece of ceilometer that does *need* to be installed in the same
>> location as the other services is the plugin for the nova compute agent. We
>> spent a fair amount of time making sure there was a version of that plugin
>> compatible with folsom, so we believe it should work. However, if you are
>> just testing ceilometer, or not using it for billing instance-hours, you
>> could skip deploying that piece entirely.
>>
>> Doug
>>
>>
>>
>> On Mon, May 6, 2013 at 9:43 AM, Riki Arslan wrote:
>>
>>> I have also installed ceilometer-2013.1~g2~20130107.449.tar.gz from the
>>> tarballs list and still getting the same error:
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/local/bin/ceilometer-api", line 5, in 
>>> pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
>>>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in
>>> run_script
>>> self.require(requires)[0].run_script(script_name, ns)
>>>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235,
>>> in run_script
>>> execfile(script_filename, namespace, namespace)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api",
>>> line 37, in 
>>> cfg.CONF(sys.argv[1:])
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
>>> line 1024, in __call__
>>> self._cli_values, leftovers = self._parse_cli_opts(args)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
>>> line 1527, in _parse_cli_opts
>>> opt._add_to_cli(self._oparser, group)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
>>&g

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

It looks like you still have incompatible versions of things installed.

The configuration library changed during grizzly. The old version and new
version cannot be used together in the same program because they both try
to modify different copies of a global variable. The exception you're
getting is, I think, due to the fact that the API service loads the
keystone middleware to handle authentication. You have a version of the
middleware that uses oslo.config, and a version of ceilometer that uses the
older oslo-incubator version of the configuration library.

The ceilometer team is small, so we have limited capacity to support "old"
versions (especially pre-incubated versions). We do intend to support
grizzly, but can only offer moderate help with folsom. The g2 release
tarballs *should* be compatible at the communication layer with folsom
versions of the other components, but it looks like you can't install them
into the same Python installation as the other services.

You can separate ceilometer code from the other services a couple of
different ways. The simplest would be to use a separate VM to run
ceilometer. That would let you follow all of the normal instructions, and
ensure that you don't have mismatched versions of libraries. The other way
is to install ceilometer into a virtualenv. That would take more care,
since you need to ensure that the virtualenv does not look at the globally
installed site-packages. I haven't tried doing this, so I can't provide
more detailed steps, and you will likely need to experiment a bit to get it
right.

The one piece of ceilometer that does *need* to be installed in the same
location as the other services is the plugin for the nova compute agent. We
spent a fair amount of time making sure there was a version of that plugin
compatible with folsom, so we believe it should work. However, if you are
just testing ceilometer, or not using it for billing instance-hours, you
could skip deploying that piece entirely.

Doug



On Mon, May 6, 2013 at 9:43 AM, Riki Arslan wrote:

> I have also installed ceilometer-2013.1~g2~20130107.449.tar.gz from the
> tarballs list and still getting the same error:
>
> Traceback (most recent call last):
>   File "/usr/local/bin/ceilometer-api", line 5, in 
> pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in
> run_script
> self.require(requires)[0].run_script(script_name, ns)
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235, in
> run_script
> execfile(script_filename, namespace, namespace)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api",
> line 37, in 
> cfg.CONF(sys.argv[1:])
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1024, in __call__
> self._cli_values, leftovers = self._parse_cli_opts(args)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1527, in _parse_cli_opts
> opt._add_to_cli(self._oparser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 591, in _add_to_cli
> container = self._get_argparse_container(parser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 633, in _get_argparse_container
> return group._get_argparse_group(parser)
> AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
>
>
> On Mon, May 6, 2013 at 3:56 PM, Riki Arslan wrote:
>
>> Hi Doug,
>>
>> I actually got it from a link on your website:
>>
>>
>> http://doughellmann.com/2013/01/ceilometer-grizzly-2-milestone-available.html
>>
>> So, do you think this one is not good?
>>
>>
>> On Thu, May 2, 2013 at 7:33 PM, Doug Hellmann <
>> doug.hellm...@dreamhost.com> wrote:
>>
>>>
>>>
>>>
>>> On Mon, Apr 29, 2013 at 6:42 PM, Riki Arslan 
>>> wrote:
>>>
>>>> I thought it might help if mentioned little more:
>>>>
>>>> /etc/ceilometer.conf file has the following parameters added:
>>>>
>>>> os_username=ceilometer
>>>> os_password=$PASSWORD
>>>> os_tenant_name=service
>>>> os_auth_url=http://localhost:5000/v2.0/
>>>>
>>>> I checked CLI_OPTIONS in service.py and it looks allright:
>>>>
>>>> CLI_OPTIONS = [
>>>> 

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

On Mon, Apr 29, 2013 at 6:42 PM, Riki Arslan wrote:

> I thought it might help if mentioned little more:
>
> /etc/ceilometer.conf file has the following parameters added:
>
> os_username=ceilometer
> os_password=$PASSWORD
> os_tenant_name=service
> os_auth_url=http://localhost:5000/v2.0/
>
> I checked CLI_OPTIONS in service.py and it looks allright:
>
> CLI_OPTIONS = [
> cfg.StrOpt('os-username',
>default=os.environ.get('OS_USERNAME', 'ceilometer'),
>help='Username to use for openstack service access'),
> cfg.StrOpt('os-password',
>default=os.environ.get('OS_PASSWORD', 'admin'),
>help='Password to use for openstack service access'),
> cfg.StrOpt('os-tenant-id',
>default=os.environ.get('OS_TENANT_ID', ''),
>help='Tenant ID to use for openstack service access'),
> cfg.StrOpt('os-tenant-name',
>default=os.environ.get('OS_TENANT_NAME', 'admin'),
>help='Tenant name to use for openstack service access'),
> cfg.StrOpt('os-auth-url',
>default=os.environ.get('OS_AUTH_URL',
>   'http://localhost:5000/v2.0'),
>help='Auth URL to use for openstack service access'),
> ]
>
> But still, according to the error I am getting, it can not parse
> _parse_cli_opts:
>
> Traceback (most recent call last):
>   File "/usr/local/bin/ceilometer-api", line 5, in 
> pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in
> run_script
> self.require(requires)[0].run_script(script_name, ns)
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235, in
> run_script
> execfile(script_filename, namespace, namespace)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api",
> line 38, in 
> service.prepare_service()
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py",
> line 80, in prepare_service
> cfg.CONF(argv[1:], project='ceilometer')
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1024, in __call__
> self._cli_values, leftovers = self._parse_cli_opts(args)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1527, in _parse_cli_opts
> opt._add_to_cli(self._oparser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 591, in _add_to_cli
> container = self._get_argparse_container(parser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 633, in _get_argparse_container
> return group._get_argparse_group(parser)
> AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
>
> I am really puzzled as Collector, Computer Agent and Central Agent are
> working fine and Api Server is not.
>

I don't see a 2013.1~g2.tar.gz tarball listed under
http://tarballs.openstack.org/ceilometer/. Where did you get the source you
are working with?

You may have a bad snapshot, since it is trying to combine
ceilometer/openstack/common/cfg.py with oslo.config.

Doug


>
>
> On Tue, Apr 30, 2013 at 12:56 AM, Riki Arslan 
> wrote:
>
>> Hi Doug,
>>
>> I have followed the document. The only thing that is different from the
>> docs is that I did not copy the yaml file (it does not exist in tarball):
>>
>> cp etc/ceilometer/*.yaml /etc/ceilometer
>>
>> However, the tarball is the g2 version, which is the last version that
>> was supposed to work with Folsom.
>>
>> It seems like Collector, Computer Agent and Central Agent are working. I
>> only can't get the Api Server working.
>>
>>
>> On Fri, Apr 26, 2013 at 6:19 PM, Doug Hellmann <
>> doug.hellm...@dreamhost.com> wrote:
>>
>>> It sounds like you haven't completed the installation instructions. I
>>> don't know if the manual steps listed at
>>> http://docs.openstack.org/developer/ceilometer/install/manual.html work
>>> with the tarball, but they should be close.
>>>

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

It sounds like you haven't completed the installation instructions. I don't
know if the manual steps listed at
http://docs.openstack.org/developer/ceilometer/install/manual.html work
with the tarball, but they should be close.

Doug


On Fri, Apr 26, 2013 at 3:46 AM, Riki Arslan wrote:

> The command line I am using is: "sudo /usr/local/bin/ceilometer-api".
>
> However, the ceilometer.ini file is missing. The version of Ceilometer I
> am using is "ceilometer-2013.1~g2.tar.gz". And, I only have the following
> configuration files:
>
> /etc/ceilometer/ceilometer.conf
> /etc/ceilometer/policy.json
> /etc/ceilometer/sources.json
>
> Thanks.
>
>
> On Fri, Apr 26, 2013 at 1:10 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>>
>>
>>
>> On Thu, Apr 25, 2013 at 8:37 AM, Riki Arslan 
>> wrote:
>>
>>> I thought Ceilometer did not set a dependency on any DB drivers. I have
>>> installed the driver Mongo using "sudo pip install pymongo".
>>>
>>
>> Ceilometer does use a database. You have to install the right driver. If
>> you want Mongo, then it sounds like you've done the right thing. It's
>> possible mako is also being used somewhere else, I'm not sure.
>>
>>
>>>
>>> Regarding the current problem; the traceback is as follows:
>>>
>>> Traceback (most recent call last):
>>>   File "/usr/local/bin/ceilometer-api", line 5, in 
>>> pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
>>>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in
>>> run_script
>>> self.require(requires)[0].run_script(script_name, ns)
>>>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235,
>>> in run_script
>>> execfile(script_filename, namespace, namespace)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api",
>>> line 38, in 
>>> service.prepare_service()
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py",
>>> line 80, in prepare_service
>>> cfg.CONF(argv[1:], project='ceilometer')
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
>>> line 1024, in __call__
>>> self._cli_values, leftovers = self._parse_cli_opts(args)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
>>> line 1527, in _parse_cli_opts
>>> opt._add_to_cli(self._oparser, group)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
>>> line 591, in _add_to_cli
>>> container = self._get_argparse_container(parser, group)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
>>> line 633, in _get_argparse_container
>>> return group._get_argparse_group(parser)
>>> AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
>>>
>>
>> That is coming from oslo.config. Can you post the ceilometer.ini file and
>> command line you are using to start the service?
>>
>> Doug
>>
>>
>>>
>>> Thank for the help.
>>>
>>>
>>> On Thu, Apr 25, 2013 at 3:27 PM, Doug Hellmann <
>>> doug.hellm...@dreamhost.com> wrote:
>>>
>>>>
>>>>
>>>> On Thursday, April 25, 2013, Riki Arslan wrote:
>>>>
>>>>> I have encountered other problems too.
>>>>>
>>>>> First of all, when starting the Central Agent I have had Glance
>>>>> endpoint 404 not found errors. As, Julien pointed out (
>>>>> https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed
>>>>> the "v1" from the Glance URLs and it worked well.
>>>>>
>>>>> Secondly, when starting the API Server, I have received "ImportError:
>>>>> No module named mako.template" error. Thus, I have installed python-mako
>>>>> module (sudo apt-get install python-mako), and the error disappeared.
>>>>>
>>>>
>>>> Mako is a dependency do sqlalchemy, I think. Are you using the
&

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

On Thu, Apr 25, 2013 at 8:37 AM, Riki Arslan wrote:

> I thought Ceilometer did not set a dependency on any DB drivers. I have
> installed the driver Mongo using "sudo pip install pymongo".
>

Ceilometer does use a database. You have to install the right driver. If
you want Mongo, then it sounds like you've done the right thing. It's
possible mako is also being used somewhere else, I'm not sure.


>
> Regarding the current problem; the traceback is as follows:
>
> Traceback (most recent call last):
>   File "/usr/local/bin/ceilometer-api", line 5, in 
> pkg_resources.run_script('ceilometer==0.0.0', 'ceilometer-api')
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 499, in
> run_script
> self.require(requires)[0].run_script(script_name, ns)
>   File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1235, in
> run_script
> execfile(script_filename, namespace, namespace)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/EGG-INFO/scripts/ceilometer-api",
> line 38, in 
> service.prepare_service()
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/service.py",
> line 80, in prepare_service
> cfg.CONF(argv[1:], project='ceilometer')
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1024, in __call__
> self._cli_values, leftovers = self._parse_cli_opts(args)
>   File
> "/usr/local/lib/python2.7/dist-packages/ceilometer-0.0.0-py2.7.egg/ceilometer/openstack/common/cfg.py",
> line 1527, in _parse_cli_opts
> opt._add_to_cli(self._oparser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 591, in _add_to_cli
> container = self._get_argparse_container(parser, group)
>   File
> "/usr/local/lib/python2.7/dist-packages/oslo.config-1.1.0-py2.7.egg/oslo/config/cfg.py",
> line 633, in _get_argparse_container
> return group._get_argparse_group(parser)
> AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'
>

That is coming from oslo.config. Can you post the ceilometer.ini file and
command line you are using to start the service?

Doug


>
> Thank for the help.
>
>
> On Thu, Apr 25, 2013 at 3:27 PM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>>
>>
>> On Thursday, April 25, 2013, Riki Arslan wrote:
>>
>>> I have encountered other problems too.
>>>
>>> First of all, when starting the Central Agent I have had Glance endpoint
>>> 404 not found errors. As, Julien pointed out (
>>> https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed the
>>> "v1" from the Glance URLs and it worked well.
>>>
>>> Secondly, when starting the API Server, I have received "ImportError: No
>>> module named mako.template" error. Thus, I have installed python-mako
>>> module (sudo apt-get install python-mako), and the error disappeared.
>>>
>>
>> Mako is a dependency do sqlalchemy, I think. Are you using the sqlalchemy
>> storage driver for ceilometer?
>>
>>
>>>
>>> Now, I am receiving another error within the API Server. The error is as
>>> follows:
>>> "AttributeError: 'OptGroup' object has no attribute
>>> '_get_argparse_group'"
>>>
>>
>> That sounds like a problem with the config module. Was there a full
>> traceback? If not, try adding the --debug option when starting the service.
>>
>> Doug
>>
>>
>>>
>>> Do you think it has something to do with mod_wsgi (
>>> http://docs.openstack.org/developer/ceilometer/install/mod_wsgi.html)?
>>>
>>> I would appreciate your help on this.
>>>
>>> Thanks.
>>>
>>>
>>> On Thu, Apr 25, 2013 at 12:27 AM, Riki Arslan >> > wrote:
>>>
>>>> Hi Doug,
>>>>
>>>> Your email helped me. It was actually glanceclient version 0.5.1 that
>>>> was causing the conflict. After updating it, the conflict error 
>>>> disappeared.
>>>>
>>>> I hope this would help someone else too.
>>>>
>>>> Thanks again.
>>>>
>>>>
>>>> On Wed, Apr 24, 2013 at 11:49 PM, Doug Hellmann <
>>>> doug.hellm...@dreamhost.com> wrote:
>>>>
>>>>>
>>>>>
&g

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

On Thursday, April 25, 2013, Riki Arslan wrote:

> I have encountered other problems too.
>
> First of all, when starting the Central Agent I have had Glance endpoint
> 404 not found errors. As, Julien pointed out (
> https://bugs.launchpad.net/ceilometer/+bug/1083104), I have removed the
> "v1" from the Glance URLs and it worked well.
>
> Secondly, when starting the API Server, I have received "ImportError: No
> module named mako.template" error. Thus, I have installed python-mako
> module (sudo apt-get install python-mako), and the error disappeared.
>

Mako is a dependency do sqlalchemy, I think. Are you using the sqlalchemy
storage driver for ceilometer?


>
> Now, I am receiving another error within the API Server. The error is as
> follows:
> "AttributeError: 'OptGroup' object has no attribute '_get_argparse_group'"
>

That sounds like a problem with the config module. Was there a full
traceback? If not, try adding the --debug option when starting the service.

Doug


>
> Do you think it has something to do with mod_wsgi (
> http://docs.openstack.org/developer/ceilometer/install/mod_wsgi.html)?
>
> I would appreciate your help on this.
>
> Thanks.
>
>
> On Thu, Apr 25, 2013 at 12:27 AM, Riki Arslan 
>  'riki.ars...@cloudturk.net');>
> > wrote:
>
>> Hi Doug,
>>
>> Your email helped me. It was actually glanceclient version 0.5.1 that was
>> causing the conflict. After updating it, the conflict error disappeared.
>>
>> I hope this would help someone else too.
>>
>> Thanks again.
>>
>>
>> On Wed, Apr 24, 2013 at 11:49 PM, Doug Hellmann <
>> doug.hellm...@dreamhost.com > 'doug.hellm...@dreamhost.com');>> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan 
>>> >> 'riki.ars...@cloudturk.net');>
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> We are trying to install "ceilometer-2013.1~g2.tar.gz" which presumably
>>>> has Folsom compatibility.
>>>>
>>>> The requirment is "python-keystoneclient>=0.2,<0.3" and we have
>>>> the version 2.3.
>>>>
>>>> But, still, setup quits with the following message:
>>>>
>>>> "error: Installed distribution python-keystoneclient 0.2.3
>>>> conflicts with requirement python-keystoneclient>=0.1.2,<0.2"
>>>>
>>>> The funny thing is, although pip-requires states
>>>> "python-keystoneclient>=0.2,<0.3", the error message complains that it is
>>>> not "python-keystoneclient>=0.1.2,<0.2".
>>>>
>>>
>>> Something else you have installed already wants an older version of the
>>> keystone client, so the installation of ceilometer is not able to upgrade
>>> to the version we need.
>>>
>>> Doug
>>>
>>>
>>>>
>>>> Your help is greatly appreciated.
>>>>
>>>> Thank you in advance.
>>>>
>>>> ___
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@lists.launchpad.net >>> 'openstack@lists.launchpad.net');>
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer Install

[Doug Hellmann]

On Wed, Apr 24, 2013 at 9:17 AM, Riki Arslan wrote:

> Hi,
>
> We are trying to install "ceilometer-2013.1~g2.tar.gz" which presumably
> has Folsom compatibility.
>
> The requirment is "python-keystoneclient>=0.2,<0.3" and we have
> the version 2.3.
>
> But, still, setup quits with the following message:
>
> "error: Installed distribution python-keystoneclient 0.2.3 conflicts with
> requirement python-keystoneclient>=0.1.2,<0.2"
>
> The funny thing is, although pip-requires states
> "python-keystoneclient>=0.2,<0.3", the error message complains that it is
> not "python-keystoneclient>=0.1.2,<0.2".
>

Something else you have installed already wants an older version of the
keystone client, so the installation of ceilometer is not able to upgrade
to the version we need.

Doug


>
> Your help is greatly appreciated.
>
> Thank you in advance.
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] ceilometer-agent-central starting fail

[Doug Hellmann]

On Thu, Apr 11, 2013 at 11:34 PM, Liu Wenmao  wrote:

> Thanks, the ceilometer seems to lack some default options in configuration
> files and the official guidance. （
> http://docs.openstack.org/developer/ceilometer/configuration.html）
>

I have opened a bug to address the missing details in the configuration
documentation. https://bugs.launchpad.net/ceilometer/+bug/1168375

Doug


>
> So maybe it is not ready for users yet?
>
>
> On Wed, Apr 10, 2013 at 8:28 PM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>>
>>
>>
>> On Wed, Apr 10, 2013 at 6:10 AM, Liu Wenmao  wrote:
>>
>>> Actually this is not over.
>>>
>>> The main reason of service failure is that central/manager.py
>>> and service.py use different vairables:
>>>
>>> central/manager.py
>>>  70 def interval_task(self, task):
>>>  71 self.keystone = ksclient.Client(
>>>  72 username=cfg.CONF.*os_username*,
>>>  73 password=cfg.CONF.os_password,
>>>  74 tenant_id=cfg.CONF.os_tenant_id,
>>>  75 tenant_name=cfg.CONF.os_tenant_name,
>>>  76 auth_url=cfg.CONF.os_auth_url)
>>>
>>> 44 CLI_OPTIONS = [
>>>  45 cfg.StrOpt('*os-username*',
>>>  46default=os.environ.get('OS_USERNAME', 'ceilometer'),
>>>  47help='Username to use for openstack service access'),
>>>  48 cfg.StrOpt('os-password',
>>>  49default=os.environ.get('OS_PASSWORD', 'admin'),
>>>  50help='Password to use for openstack service access'),
>>>  51 cfg.StrOpt('os-tenant-id',
>>>  52default=os.environ.get('OS_TENANT_ID', ''),
>>>  53help='Tenant ID to use for openstack service access'),
>>>  54 cfg.StrOpt('os-tenant-name',
>>>  55default=os.environ.get('OS_TENANT_NAME', 'admin'),
>>>  56help='Tenant name to use for openstack service
>>> access'),
>>>  57 cfg.StrOpt('os_auth_url',
>>>  58default=os.environ.get('OS_AUTH_URL',
>>>  59   'http://localhost:5000/v2.0'),
>>>
>>> So after I change all "-" to "_" and modify all options in
>>> /etc/ceilometer/ceilometer.conf, the service starts OK.
>>>
>>
>> The thing that fixed it was changing "-" to "_" in your configuration
>> file. The options library allows option names to have "-" in them so they
>> look nice as command line switches, but the option name uses the "_".
>>
>> Doug
>>
>>
>>>
>>>
>>>
>>> On Wed, Apr 10, 2013 at 2:02 PM, Liu Wenmao  wrote:
>>>
>>>> I solve this problem by two steps:
>>>>
>>>> 1 modify /etc/init/ceilometer-agent-central.conf
>>>> exec start-stop-daemon --start --chuid ceilometer --exec
>>>> /usr/local/bin/ceilometer-agent-central --
>>>> --config-file=/etc/ceilometer/ceilometer.conf
>>>> 2 add some lines to /etc/ceilometer/ceilometer.conf:
>>>> os-username=ceilometer
>>>> os-password=nsfocus
>>>> os-tenant-name=service
>>>> os-auth-url=http://controller:5000/v2.0
>>>>
>>>>
>>>>
>>>> On Wed, Apr 10, 2013 at 1:36 PM, Liu Wenmao wrote:
>>>>
>>>>> Hi all:
>>>>>
>>>>> I have just install ceilometer grizzly github version, but fail to
>>>>> start ceilometer-agent-central service. I think it is due to that I didn't
>>>>> set up the keystone user/password like other projects. but I follow the
>>>>> instructions(
>>>>> http://docs.openstack.org/developer/ceilometer/install/manual.html#configuring-keystone-to-work-with-api)
>>>>> but it does not include the ceilometer configuration.
>>>>>
>>>>> # service ceilometer-agent-central start
>>>>> ceilometer-agent-central start/running, process 5679
>>>>>
>>>>> # cat /etc/init/ceilometer-agent-central.conf
>>>>> description "ceilometer-agent-compute"
>>>>> author "Chuck Short "
>>>>>
>>>>> start on runlevel [2345]
>>>&

Re: [Openstack] ceilometer-agent-central starting fail

[Doug Hellmann]

On Wed, Apr 10, 2013 at 6:10 AM, Liu Wenmao  wrote:

> Actually this is not over.
>
> The main reason of service failure is that central/manager.py
> and service.py use different vairables:
>
> central/manager.py
>  70 def interval_task(self, task):
>  71 self.keystone = ksclient.Client(
>  72 username=cfg.CONF.*os_username*,
>  73 password=cfg.CONF.os_password,
>  74 tenant_id=cfg.CONF.os_tenant_id,
>  75 tenant_name=cfg.CONF.os_tenant_name,
>  76 auth_url=cfg.CONF.os_auth_url)
>
> 44 CLI_OPTIONS = [
>  45 cfg.StrOpt('*os-username*',
>  46default=os.environ.get('OS_USERNAME', 'ceilometer'),
>  47help='Username to use for openstack service access'),
>  48 cfg.StrOpt('os-password',
>  49default=os.environ.get('OS_PASSWORD', 'admin'),
>  50help='Password to use for openstack service access'),
>  51 cfg.StrOpt('os-tenant-id',
>  52default=os.environ.get('OS_TENANT_ID', ''),
>  53help='Tenant ID to use for openstack service access'),
>  54 cfg.StrOpt('os-tenant-name',
>  55default=os.environ.get('OS_TENANT_NAME', 'admin'),
>  56help='Tenant name to use for openstack service access'),
>  57 cfg.StrOpt('os_auth_url',
>  58default=os.environ.get('OS_AUTH_URL',
>  59   'http://localhost:5000/v2.0'),
>
> So after I change all "-" to "_" and modify all options in
> /etc/ceilometer/ceilometer.conf, the service starts OK.
>

The thing that fixed it was changing "-" to "_" in your configuration file.
The options library allows option names to have "-" in them so they look
nice as command line switches, but the option name uses the "_".

Doug


>
>
>
> On Wed, Apr 10, 2013 at 2:02 PM, Liu Wenmao  wrote:
>
>> I solve this problem by two steps:
>>
>> 1 modify /etc/init/ceilometer-agent-central.conf
>> exec start-stop-daemon --start --chuid ceilometer --exec
>> /usr/local/bin/ceilometer-agent-central --
>> --config-file=/etc/ceilometer/ceilometer.conf
>> 2 add some lines to /etc/ceilometer/ceilometer.conf:
>> os-username=ceilometer
>> os-password=nsfocus
>> os-tenant-name=service
>> os-auth-url=http://controller:5000/v2.0
>>
>>
>>
>> On Wed, Apr 10, 2013 at 1:36 PM, Liu Wenmao  wrote:
>>
>>> Hi all:
>>>
>>> I have just install ceilometer grizzly github version, but fail to
>>> start ceilometer-agent-central service. I think it is due to that I didn't
>>> set up the keystone user/password like other projects. but I follow the
>>> instructions(
>>> http://docs.openstack.org/developer/ceilometer/install/manual.html#configuring-keystone-to-work-with-api)
>>> but it does not include the ceilometer configuration.
>>>
>>> # service ceilometer-agent-central start
>>> ceilometer-agent-central start/running, process 5679
>>>
>>> # cat /etc/init/ceilometer-agent-central.conf
>>> description "ceilometer-agent-compute"
>>> author "Chuck Short "
>>>
>>> start on runlevel [2345]
>>> stop on runlelvel [!2345]
>>>
>>> chdir /var/run
>>>
>>> pre-start script
>>> mkdir -p /var/run/ceilometer
>>> chown ceilometer:ceilometer /var/run/ceilometer
>>>
>>> mkdir -p /var/lock/ceilometer
>>> chown ceilometer:ceilometer /var/lock/ceilometer
>>> end script
>>>
>>> exec start-stop-daemon --start --chuid ceilometer --exec
>>> /usr/local/bin/ceilometer-agent-central
>>>
>>>
>>> /var/log/ceilometer/ceilometer-agent-central.log
>>> 2013-04-10 13:01:39ERROR [ceilometer.openstack.common.loopingcall]
>>> in looping call
>>> Traceback (most recent call last):
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/openstack/common/loopingcall.py",
>>> line 67, in _inner
>>> self.f(*self.args, **self.kw)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/ceilometer-2013.1-py2.7.egg/ceilometer/central/manager.py",
>>> line 76, in interval_task
>>> auth_url=cfg.CONF.os_auth_url)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py",
>>> line 134, in __init__
>>> self.authenticate()
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/client.py",
>>> line 205, in authenticate
>>> token)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py",
>>> line 174, in get_raw_token_from_identity_servicetoken=token)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/v2_0/client.py",
>>> line 202, in _base_authN
>>> resp, body = self.request(url, 'POST', body=params, headers=headers)
>>>   File
>>> "/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.1.g3a3e254-py2.7.egg/keystoneclient/client.py",
>>> line 366, in request
>>> raise 

Re: [Openstack] How to purge old meter data of Ceilometer

[Doug Hellmann]

I created a blueprint for this:
https://blueprints.launchpad.net/ceilometer/+spec/purge-data


On Thu, Apr 4, 2013 at 4:06 AM, Julien Danjou  wrote:

> On Thu, Apr 04 2013, Harri Pyy wrote:
>
> > With the default 1 minute interval, Ceilometer collects quite large
> amounts of meter data.
> >
> > Does Ceilometer provide a TTL configuration option for the meter data, or
> > some other functionality or API for purging old meter data?
>
> Not yet unfortunately.
>
> --
> Julien Danjou
> ;; Free Software hacker ; freelance consultant
> ;; http://julien.danjou.info
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer question

[Doug Hellmann]

You want the "statistics" command from the v2 API.

ceilometer --ceilometer-api-version 2 help statistics

The CLI needs a little work to make the options clear, but the API
documentation may help explain:
http://docs.openstack.org/developer/ceilometer/webapi/v2.html

Doug

On Tue, Mar 26, 2013 at 12:22 PM, Wyllys Ingersoll <
wyllys.ingers...@evault.com> wrote:

>
> Perhaps this is documented but I can't seem to find it…
>
> I want to use ceilometer to show usage from the swift objectstore.  What
> would the correct command be to extract that information? Currently, the
> various "-list" commands just spit out various meter types along with
> resource and project id values.
>
> Thanks,
>   Wyllys Ingersoll
>   EVault
>
>
> On Mar 26, 2013, at 11:47 AM, Julien Danjou  wrote:
>
> > Hi,
> >
> > We're pleased to announce that the RC1 version for the Grizzly release
> > of Ceilometer is available!
> >
> >   https://launchpad.net/ceilometer/grizzly/grizzly-rc1
> >
> > Unless release-critical issues are found that warrant a release
> > candidate respin, this version will be formally released as the
> > Ceilometer 2013.1 version on April 4. You are therefore strongly
> > encouraged to test and validate this tarball.
> >
> > If you find an issue that could be considered release-critical, please
> > file it at:
> >
> > https://bugs.launchpad.net/ceilometer/+filebug
> >
> > Note that the "master" branch for Ceilometer is now open for Havana
> > development, and feature freeze restriction no longer apply.
> >
> > --
> > Julien Danjou
> > -- Free Software hacker - freelance consultant
> > -- http://julien.danjou.info
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Regarding openstack metering

[Doug Hellmann]

Hi, Arumon,

You will find the ceilometer documentation at
http://docs.openstack.org/developer/ceilometer/

There are some basic installation and configuration instructions, as well
as an architectural overview. Please do not hesitate to ask questions, so
we can expand and improve what is documented so far based on your feedback.

Doug

On Fri, Mar 8, 2013 at 5:36 AM, Aru s  wrote:

> Hi,
>
> I am new to openstack and looking for the metering tools.
> I read about the ceilometer, but not found any proper documentation for
> this.
> Please help me understand regarding the available metering tools.
>
> Regards,
> Arumon
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] UK Trademark issue with "Python"

[Doug Hellmann]

The Python Software Foundation, maintainers of the IP behind the Python
language, are currently fighting a trademark claim against a company in the
UK that is trying to trademark the use of the term "Python" for all
software, services, and servers apparently as part of branding their cloud
service. If you work for a company with an office in an EU Community member
state, the PSF would appreciate your help documenting the community's prior
claim to the name "Python". For details about where to send supporting
evidence, please see the PSF blog post [1].

Thanks,
Doug

[1]
http://pyfound.blogspot.com/2013/02/python-trademark-at-risk-in-europe-we.html
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Name it Hood!

[Doug Hellmann]

Will we have hoodies instead of t-shirts for ODS attendees?

Doug

On Thu, Jan 24, 2013 at 2:50 PM, Monty Taylor  wrote:

> Hey all!
>
> Here's my pitch for Hood:
>
> a) It's the tallest mountain in Oregon, and honestly, it's a pretty
> kick-ass mountain in general
> b) Being in the pacific northwest, the mountain itself is quite
> regularly in the clouds. That's gotta count for something.
> c) It's actually a volcano.
> d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in
> Cuba. (We should have a design summit in cuba!!!)
> e) Harbor is super-problematic because of the US/UK clash in spelling.
> Half of us will spell it wrong no matter what.
> f) Hood is only 4 letters. Think about that when you think about typing
> hatfield a lot. Also, if we name it hatfield, we're going to have to
> have the M summit somewhere that has a town called McCoy.
> g) I'll buy you a beer at the summit if you vote for Hood.
>
> Monty
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Calling all user group and meetup organizers

[Doug Hellmann]

Someone on the call today offered meeting space in Alpharetta, GA but I
couldn't hear your name clearly at the time. Could you drop me a note off
list, please, so we can talk about those arrangements?

Thanks,
Doug


On Tue, Jan 8, 2013 at 7:54 PM, Sean Roberts  wrote:

> We are going to have a planning meeting on Tuesday, 15 Jan 2012 11:00am to
> 1:00pm PST. RSVP via  http://www.meetup.com/openstack/events/93593062/
>  Connect remotely via webex
> https://yahoomeetings.webex.com/yahoomeetings/j.php?ED=160663792&UID=492396097&RT=MiM0
> If this time doesn't work for you, get ahold of me directly via
> skype:seanroberts66, email, mobile, irc:sarob, twitter:sarob, or carrier
> pigeon.
>
> Stefano and Thierry will be joining us. I want to get input from people
> that run user groups and meetups.
>
> See you then!
>
> Sean Roberts
> Infrastructure Strategy
> sean...@yahoo-inc.com
> Direct (408) 349-5234  Mobile (925) 980-4729
>
> 701 First Avenue, Sunnyvale, CA, 94089-0703, US
> Phone (408) 349-3300  Fax (408) 349-3301
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Is python-openstackclient dead?

[Doug Hellmann]

On Sun, Dec 23, 2012 at 9:39 AM, Alessio Ababilov <
aababi...@griddynamics.com> wrote:

> Hi everyone!
>
> There is a nice project in OpenStack - python-openstackclient. Its
> purpose is to provide a convenient command line interface to all
> OpenStack services, including nova. glance, and keystone, according to
> http://wiki.openstack.org/UnifiedCLI.
>
> I noticed that the last commit that added functionality to this
> project was on August 20, 2012. In current state,
> python-openstackclient implements only operations with nova instances
> and several keystone commands, so, the most of required functionality
> is not ready.
>
> In the same time, python-novaclient and python-keystoneclient still
> extend their command line interfaces - the latest commits were
> accepted in December, 2012. So, instead of moving CLI to
> python-openstackclient, it is developed in separate python-*client
> projects.
>
> Is python-openstackclient considered to be dead?
>
> Please, do not kill this project! Now it is really convenient, and all
> it needs is just more attention from the community that will add
> lacked functionality! Lets us drop CLI support from miscellaneous
> python-PROJECTclients and add it to python-openstackclient!
>

It's not dead, but most of the contributors haven't had time to prioritize
working on it during this cycle. If you are interested in helping out, I'll
be happy to do code reviews for you.

Doug


>
> --
> Alessio Ababilov
> Software Engineer
> Grid Dynamics
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

[Doug Hellmann]

On Wed, Nov 7, 2012 at 10:21 PM, Sandy Walsh wrote:

> Hey!
>
> (sorry for the top-posting, crappy web client)
>
> There is a periodic task already in the compute manager that can handle
> this:
> https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3021
>
> There seems to be some recent (to me) changes in the manager now wrt the
> resource_tracker.py and stats.py files about how this information gets
> relayed. Now it seems it only goes to the db, but previously it was sent to
> a fanout queue that the schedulers could use.
>
> Regardless, this is done at a high enough level that it doesn't really
> care about the underlying virt layer, so long at the virt layer supports
> the get_available_resource() method.
>
>
> https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L152
>
> https://github.com/openstack/nova/blob/master/nova/virt/xenapi/driver.py#L392
>
> https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2209
>
> I'd add a hook in there do what we want with this data. Write it to the
> db, send it over the wire, whatever. If there is additional information
> required, it should go in this dictionary (or we should define a format for
> extensions to it).
>

It looks like that is collecting resource data, but not usage data. For
example, there's no disk I/O information, just disk space. Is that what you
mean by adding extra information to the dictionary?

Doug


>
> The --periodic_interval value is meant to be the "fastest tick" approach
> and the individual methods have to deal with how many multiples of the base
> tick it should use. So you can have different data reported at different
> intervals.
>
> Now, the question of polling vs. pushing shouldn't really matter if the
> sampling rate is predetermined. We can push when the sample is taken or we
> can read from some other store from an external process ... but the
> sampling should only be done in one place, once.
>
> Hope I answered your question? If not, just repeat it in another way and
> I'll try again :)
>
> -S
>
>
>
> 
> From: 
> openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh=
> rackspace@lists.launchpad.net] on behalf of Eoghan Glynn [
> egl...@redhat.com]
> Sent: Wednesday, November 07, 2012 4:32 PM
> To: OpenStack Development Mailing List
> Cc: openstack@lists.launchpad.net
> Subject: Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified
> Instrumentation, Metering, Monitoring ...
>
> > Here's a first pass at a proposal for unifying StackTach/Ceilometer
> > and other instrumentation/metering/monitoring efforts.
> >
> > It's v1, so bend, spindle, mutilate as needed ... but send feedback!
> >
> > http://wiki.openstack.org/UnifiedInstrumentationMetering
>
> Thanks for putting this together Sandy,
>
> We were debating on IRC (#heat) earlier the merits of moving the
> ceilometer "emission" logic into the services, e.g. directly into the
> nova-compute node. At first sight, this seemed to be what you were
> getting at with the suggestion:
>
>  "Remove the Compute service that Ceilometer uses and integrate the
>   existing fanout compute notifications into the data collected by the
>   workers. There's no need for yet-another-worker."
>
> While this could be feasible for measurements driven directly by
> notifications, I'm struggling with the idea of moving say the libvirt
> polling out of the ceilometer compute agent, as this seems to leak too
> many monitoring-related concerns directly into nova (cadence of polling,
> semantics of libvirt stats reported etc.).
>
> So I just wanted to clarify whether the type of low level unification
> you're proposing includes both push & pull (i.e. notification & polling)
> or whether you mainly had just former in mind when it comes to ceilometer.
>
> Cheers,
> Eoghan
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?

[Doug Hellmann]

On Thu, Nov 8, 2012 at 9:33 AM,  wrote:

> Hi Doug,
>
> ** **
>
> >+1
>
> >It will be good to see if the storage API design works for other
> backends like HBase.
>
> ** **
>
> The first glance I had on ceilometer/storage/base.py, the design of basic
> storage class does look ok to work with. 
>
> ** **
>
> The only challenge is that the Hbase schema is not as flexible as
> traditional RDBMS or MongoDB, so the efforts would be more on the Hbase
> implementation side. Eg. the design of row key, column family etc.
>

I hope that the API provides enough encapsulation to let you implement that
in a reasonable way.


> 
>
> ** **
>
> We can discuss more once I had a blueprint draft registered J
>

I'm looking forward to it!

Doug


> 
>
> ** **
>
> --
>
> Thanks,
>
> Shengjie
>
> ** **
>
> ** **
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?

[Doug Hellmann]

+1

It will be good to see if the storage API design works for other backends
like HBase.

Doug

On Thu, Nov 8, 2012 at 5:50 AM,  wrote:

> Thanks Julien, I will write up a blueprint soon.
>
> Shengjie
>
> -Original Message-
> From: Julien Danjou [mailto:jul...@danjou.info]
> Sent: 07 November 2012 18:12
> To: Min, Shengjie
> Cc: openstack@lists.launchpad.net
> Subject: Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer
> - blueprint?
>
> On Wed, Nov 07 2012, shengjie_...@dell.com wrote:
>
> Hi Shengjie,
>
> > I am looking for a way to propose a blueprint to the ceilometer team -
> > "Adding Hbase storage backend for ceilometer". Currently we have only
> > MongoDB and SQLalchemy as the possible ceilometer storage backend.
> > Given the storage interface is clearly designed and allows us to have
> > different backend, it would be nice to have more options which make
> > Ceilometer more adoptive to some providers' existing architectures. As
> > a cloud IAAS provider, Ceilometer might not be just considered as a
> > usage db, also it's being seen as a centralized place to store all the
> > usage data cross platforms. Additionally, all the historical usage
> > data stored can be used for historical usage analysis as well, this is
> > where MapReduce comes into play. With Hadoop Hbase, it allows usage
> > data to be stored in HDFS, also it gives us the ability to run large
> > scale massive MapRed analysis jobs in the future.
>
> This is indeed a good idea, and as you pointed out, Ceilometer collector
> has been designed so this is being possible.
>
> I think you can create a blueprint here:
>
>   https://blueprints.launchpad.net/ceilometer
>
> And write in what should be done.
>
> --
> Julien Danjou
> ;; Free Software hacker & freelance
> ;; http://julien.danjou.info
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Nov 6, 2012, at 6:45 AM, Tim Bell  wrote:

> 
> There is a use case for base metal hardware metering in the private cloud
> where the user allocated the machine does not have root access to kill the
> metering.

How can we detect that special case?

> 
> Being able to create a single metering infrastructure for the entire private
> cloud, virtual or bare-metal allocation, is a need technically, it is
> not clear how to guarantee it but it is worth exploring.

I agree, it would be good to have an answer. Ceilometer can already hold the 
data, even if the agent to collect it is a custom solution. 

Doug

> 
> Tim
> 
>> -Original Message-
>> From: openstack-bounces+tim.bell=cern...@lists.launchpad.net
>> [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf
>> Of Robert Collins
>> Sent: 06 November 2012 11:00
>> To: Graf Lucas (graflu0); Zehnder Toni (zehndton);
>> openstack@lists.launchpad.net; Doug Hellmann
>> Subject: Re: [Openstack] [ceilometer] Monitoring physical devices
>> 
>> On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou  wrote:
>>> On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote:
>>> 
>>>> I'm a little confused now... ;) Is the bare metal run by the platform
>>>> operator the physical machine? What do you mean with the bare metal
>>>> run to replace virtual instances for any project?
>>> 
>>> AFAIU, bare-metal provisionning is about using hardware (bare-metal)
>>> rather than virtual instances as a flavor in Nova.
>>> For such case, we won't be able to poll anything about the hardware.
>>> 
>>> For hardware ran by the operator, this will be doable.
>> 
>> We can still talk to the IPMI controller for the machine, which will get
> us lots
>> of info, without running agents in the host os.
>> 
>> -Rob
>> 
>> 
>> 
>> --
>> Robert Collins 
>> Distinguished Technologist
>> HP Cloud Services
>> 
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Nov 6, 2012, at 4:59 AM, Robert Collins  wrote:

> On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou  wrote:
>> On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote:
>> 
>>> I'm a little confused now... ;) Is the bare metal run by the platform
>>> operator the physical machine? What do you mean with the bare metal
>>> run to replace virtual instances for any project?
>> 
>> AFAIU, bare-metal provisionning is about using hardware (bare-metal)
>> rather than virtual instances as a flavor in Nova.
>> For such case, we won't be able to poll anything about the hardware.
>> 
>> For hardware ran by the operator, this will be doable.
> 
> We can still talk to the IPMI controller for the machine, which will
> get us lots of info, without running agents in the host os.

I think for billing it is enough to know that the server is allocated, since 
the hardware won't be shared between tenants (so it wouldn't make much sense to 
charge less if they only use 1/2 of the CPU capacity, for example). Using IPMI 
sounds like an interesting option for monitoring, though. We would need to add 
a pollster to a central agent, and it would need to know which instances are 
bare metal. I assume that info is available through a query somehow. 

Doug

> 
> -Rob
> 
> 
> 
> --
> Robert Collins 
> Distinguished Technologist
> HP Cloud Services

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Mon, Nov 5, 2012 at 7:37 AM, Julien Danjou  wrote:

> On Mon, Nov 05 2012, Doug Hellmann wrote:
>
> > If we make the current compute agent take an option telling it which
> > pollster namespace to use, then the same framework can load different
> > pollsters. However, there is a fundamental security issue with
> > communicating from an agent running inside a tenant's OS image using the
> > RPC stack. At DreamHost, and I suspect at other providers, that RPC
> network
> > is completely isolated from any tenant networks. We would not want a
> tenant
> > to be able to listen to the message bus, and definitely would not want it
> > to be able to write anything to the message bus.
>
> What makes you think an agent would run inside an instance? I mean, this
> is not what this is about, we're talking about hardware running OS.
>

When an image is deployed to bare metal, there is no container, right?

Doug


>
> --
> Julien Danjou
> # Free Software hacker & freelance
> # http://julien.danjou.info
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Mon, Nov 5, 2012 at 3:32 AM, Zehnder Toni (zehndton) <
zehnd...@students.zhaw.ch> wrote:

> > On Thu, Nov 01 2012, Julien Danjou wrote:
> >> On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote:
>
> >> My goal is to offer monitored data to the admin and customers. The
> >> admin is interested in the utilization of the physical components and
> >> the virtual machines and the customer is interested to know what his
> VMs do or can do.
> >> It would be nice to get the data from a single point. I thought I can
> >> enhance the Ceilometer compute agent to get this data out. Does this
> >> make sense or is it better to use another monitoring tool for the
> >> physical components?
>
> > I think the pollster implementation can be done. I wouldn't implement
> this in the compute agent, but probably in some hardware agent, because
> it's likely it would be used in different kinds of environment and not only
> on compute node, i.e. you may also want to meter hardware usage for you
> cinder or glance node anyway.
>
> I think also the best way to implement this is to integrate a new
> (hardware) agent. Then we have a clear delineation. I'm very interested in
> helping to develop this.
>

If we make the current compute agent take an option telling it which
pollster namespace to use, then the same framework can load different
pollsters. However, there is a fundamental security issue with
communicating from an agent running inside a tenant's OS image using the
RPC stack. At DreamHost, and I suspect at other providers, that RPC network
is completely isolated from any tenant networks. We would not want a tenant
to be able to listen to the message bus, and definitely would not want it
to be able to write anything to the message bus.

We will need an API endpoint to receive data from agents running inside a
tenant image, examine it, and republish it on the message bus for other
consumers. Heat has something similar, although I think they just store the
data immediately.

Doug


>
> Toni
>
> > About the 10 minutes polling interval Doug mentionned, this can be a
> problem indeed, but it's still solvable later and would be easy to solve if
> this in a different agent, since you could change the periodic interval for
> pollster runs to something like 1 or 5 minutes.
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Fri, Nov 2, 2012 at 3:07 AM, Patrick Petit <
patrick.michel.pe...@gmail.com> wrote:

> Folks,
> I'd like to add to this that physical server metering shouldn't be treated
> differently in Ceilometer now that bare metal provisioning framework enters
> into Grizzly. Physical servers will just become billable resources much
> like VMs. I am not speaking of physical server monitoring here. Just
> extending Ceilometer agent to also report usage data out of the physical
> box.
>

Thanks for posting this, Patrick. I understood "physical devices" as "host
server" not as "bare-metal server".

I suspect we could use the same agent framework, but almost all of the
pollsters would need to be different because they would be running inside
the guest OS rather than on a host VM, so the APIs they will use to collect
the same data will be different.

Doug


> Cheers
> Patrick
>
> Envoyé de mon iPad
>
> Le 1 nov. 2012 à 19:13, Julien Danjou  a écrit :
>
> > On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote:
> >
> >> My goal is to offer monitored data to the admin and customers. The
> admin is
> >> interested in the utilization of the physical components and the virtual
> >> machines and the customer is interested to know what his VMs do or can
> do.
> >> It would be nice to get the data from a single point. I thought I can
> >> enhance the Ceilometer compute agent to get this data out. Does this
> make
> >> sense or is it better to use another monitoring tool for the physical
> >> components?
> >
> > I think the pollster implementation can be done. I wouldn't implement
> > this in the compute agent, but probably in some hardware agent, because
> > it's likely it would be used in different kinds of environment and not
> > only on compute node, i.e. you may also want to meter hardware usage for
> > you cinder or glance node anyway.
> >
> > About the 10 minutes polling interval Doug mentionned, this can be a
> > problem indeed, but it's still solvable later and would be easy to solve
> > if this in a different agent, since you could change the periodic
> > interval for pollster runs to something like 1 or 5 minutes.
> >
> > --
> > Julien Danjou
> > // Free Software hacker & freelance
> > // http://julien.danjou.info
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

On Fri, Nov 2, 2012 at 4:42 PM, Dan Dyer  wrote:

>  Yes, I am assuming the service controller provides a different stream of
> data from the lower level VM events. So the question is how to represent
> and store this additional meta data in ceilometer. Note that there doesn't
> necessarily need to be a linkage/grouping between the resources since the
> association is what is actually contained in the metadata that is provided
> by the service controller.
>
> As a summary
> Nova provides its normal events for usage
> Service controller provides a mapping of nova instances to service type
> and actual end user
>

So the problem isn't necessarily that you want to measure something
different, but that the "ownership" in the existing data is not "correct"
from the perspective of the billing system.

We have a similar issue at DreamHost. Our existing user database has
account ids that need to be mapped to tenant ids from keystone. Rather than
putting that information in keystone, or ceilometer, we decided to store it
in our system and have the DreamHost billing system drive the ceilometer
API. Does it make sense to do something similar here?

If we definitely want ceilometer to hold the metadata, then I could also
see adding an API to let an outside system add metadata to a resource. That
would let the PaaS code, which knows about each VM, store extra data that
would be returned with the VM metadata when a caller visits
/resources/.

Would you expect to be able to query using the metadata? For example,
"provide the total instance hours for all instances with paas_tag=foo"?

Doug


>
>
> Dan
>
>
> On 11/1/2012 11:25 AM, Doug Hellmann wrote:
>
>
>
> On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer  wrote:
>
>>  In some cases, the service controller is actually running inside a VM.
>> It would not have access to the internals of the VM's. It maintains its
>> metadata separately from the Nova infrastructure.
>>
>
>  It doesn't need internal access to the VM, but something has to share
> the metadata with ceilometer (or "join" it to the data ceilometer has) at
> some point. If it would be too difficult to get the data into the events,
> then it could be done by the app that uses the ceilometer API to query for
> usage. For example, the app that loads data from ceilometer to your real
> billing system could be driven by data saved by the service controller in
> whatever database it uses.
>
>  Doug
>
>
>>
>>
>> DD
>>
>>
>> On 10/25/2012 2:25 AM, Nick Barcet wrote:
>>
>> Let's imagine that the service that launch instances can tag the
>> instance with:
>> a) a common service identifier (constant)
>> b) a uuid unique for each "Unit" of the service
>> such as :
>>
>> If that tag is passed onto the events which ceilometer stores in its
>> entirety as meta, I do not see what the difficulty would be for the
>> rating engine to be able to reconcile the information to handle your 2
>> use cases.  Am I missing something?
>>
>> Nick
>>
>> On 10/25/2012 12:03 AM, Dan Dyer wrote:
>>
>>  I don't think its just a matter of adding more meters or events for a
>> couple of reasons:
>> 1. In many cases the metadata I am referring to comes from a different
>> source than the base usage data. Nova is still emitting its normal
>> events, but we get the service/user mapping from a different source. I
>> would not characterize this data as usage metrics but more data about
>> the system relationships.
>> 2. in the multiple VM case, we need to have the relationships specified
>> so that we can ignore the proper VM's. There has also been talk of
>> hybrid billing models that charge for some part of the VM usage as well
>> as other metrics. Once again we need a way to characterize the
>> relationships so that processing can associate and filter correctly.
>>
>> Dan
>>
>> On 10/24/2012 3:35 PM, Julien Danjou wrote:
>>
>>  On Wed, Oct 24 2012, Dan Dyer wrote:
>>
>>
>>  Use Case 1
>> Service Owned Instances
>> There are a set of use cases where a service is acting on behalf of a
>> user,
>> the service is the owner of the VM but billing needs to be attributed
>> to the
>> end user of the system.This scenario drives two requirements:
>> 1. Pricing is similar to base VM's but with a premium. So the type of
>> service for a VM needs to be identifiable so that the appropriate
>> pricing
>> can be applied.
>> 2. The actual end user of the VM needs to be identified so usage can be
>> properly attributed
>>

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Thu, Nov 1, 2012 at 2:13 PM, Julien Danjou  wrote:

> On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote:
>
> > My goal is to offer monitored data to the admin and customers. The admin
> is
> > interested in the utilization of the physical components and the virtual
> > machines and the customer is interested to know what his VMs do or can
> do.
> > It would be nice to get the data from a single point. I thought I can
> > enhance the Ceilometer compute agent to get this data out. Does this make
> > sense or is it better to use another monitoring tool for the physical
> > components?
>
> I think the pollster implementation can be done. I wouldn't implement
> this in the compute agent, but probably in some hardware agent, because
> it's likely it would be used in different kinds of environment and not
> only on compute node, i.e. you may also want to meter hardware usage for
> you cinder or glance node anyway.
>
> About the 10 minutes polling interval Doug mentionned, this can be a
> problem indeed, but it's still solvable later and would be easy to solve
> if this in a different agent, since you could change the periodic
> interval for pollster runs to something like 1 or 5 minutes.
>

Good points.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] meter data volume

[Doug Hellmann]

On Thu, Nov 1, 2012 at 1:31 PM, Eoghan Glynn  wrote:

>
>
> > > if you have:
> > >
> > > Time | Value
> > > 0 | 10
> > > 1 | 30
> > > 2 | 50
> > > 3 | 80
> > > 4 | 100
> > >
> > > If your delta-pollster is down at 1 and 2, you restart at 3,
> > > therefore at 4 you'll send "20" as usage (100 minus 80).
> > > So you miss the delta between 10 (time 0) and 80 (time 3)
> > > (therefore 70 for free!).  If you send right away 80 at
> > > time 3 when restarting, the API will be able to guess that
> > > between 0 and 3 the value went from 10 to 80.  With delta
> > > approach, the API cannot guess that.
> >
> >
> > Sure it can, you just need to move where the caching is done. Using a
> > local cache to maintain the previous time a value was published you
> > would know at time 3 that the last published value was 10, and so
> > send 70. So the total will be correct.
>
> Good point, previously IIUC there was an implicit assumption that
> any prev time caching would be done in-memory, hence lost across
> process restarts.
>
> But as you point out, these data could be persisted locally by the
> compute agent.
>
> What would be the best way to achieve this? A small sqlite DB
> per-agent, or even simpler just a pickled dict? The latter would
> avoid the complexity of DB versioning and migration.
>

I discussed this issue at the summit with James Penick of Yahoo, and he
showed me some code in their agent that is using a sqllite db. We will want
to build a nice API so pollsters can use the cache without having to worry
about how it is implemented, which would let us deal with any versioning
issues in a central spot.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Monitoring physical devices

[Doug Hellmann]

On Thu, Nov 1, 2012 at 1:21 PM, Zehnder Toni (zehndton) <
zehnd...@students.zhaw.ch> wrote:

> > On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote:
>
> >> On Thu, Nov 01 2012, Julien Danjou wrote:
>
> >> On every physical compute node is the Ceilometer compute agent
> >> installed, right?!
>
> > Yes.
>
> >> 1) Does the compute agent collect data of the physical machine as well
> >> or is it just collecting data of the virtual machines?
>
> > Only virtual machines.
>
> >> 2) Could it be useful to enhance the Ceilometer agent to collect data
> from the physical servers?
>
> > Why not. What do you have in mind exactly?
>
> My goal is to offer monitored data to the admin and customers. The admin
> is interested in the utilization of the physical components and the virtual
> machines and the customer is interested to know what his VMs do or can do.
> It would be nice to get the data from a single point. I thought I can
> enhance the Ceilometer compute agent to get this data out. Does this make
> sense or is it better to use another monitoring tool for the physical
> components?
>

We are looking into adding monitoring features to ceilometer, but it is
likely that your admin will want fresher (and different) data than we are
collecting right now (the agent only polls every 10 minutes at this point).

If you have a strong need for this, it would be great if you could work
with us to help get it implemented more quickly.

Doug


>
> Cheers,
>
> Toni
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer  wrote:

>  In some cases, the service controller is actually running inside a VM.
> It would not have access to the internals of the VM's. It maintains its
> metadata separately from the Nova infrastructure.
>

It doesn't need internal access to the VM, but something has to share the
metadata with ceilometer (or "join" it to the data ceilometer has) at some
point. If it would be too difficult to get the data into the events, then
it could be done by the app that uses the ceilometer API to query for
usage. For example, the app that loads data from ceilometer to your real
billing system could be driven by data saved by the service controller in
whatever database it uses.

Doug


>
>
> DD
>
>
> On 10/25/2012 2:25 AM, Nick Barcet wrote:
>
> Let's imagine that the service that launch instances can tag the
> instance with:
> a) a common service identifier (constant)
> b) a uuid unique for each "Unit" of the service
> such as :
>
> If that tag is passed onto the events which ceilometer stores in its
> entirety as meta, I do not see what the difficulty would be for the
> rating engine to be able to reconcile the information to handle your 2
> use cases.  Am I missing something?
>
> Nick
>
> On 10/25/2012 12:03 AM, Dan Dyer wrote:
>
>  I don't think its just a matter of adding more meters or events for a
> couple of reasons:
> 1. In many cases the metadata I am referring to comes from a different
> source than the base usage data. Nova is still emitting its normal
> events, but we get the service/user mapping from a different source. I
> would not characterize this data as usage metrics but more data about
> the system relationships.
> 2. in the multiple VM case, we need to have the relationships specified
> so that we can ignore the proper VM's. There has also been talk of
> hybrid billing models that charge for some part of the VM usage as well
> as other metrics. Once again we need a way to characterize the
> relationships so that processing can associate and filter correctly.
>
> Dan
>
> On 10/24/2012 3:35 PM, Julien Danjou wrote:
>
>  On Wed, Oct 24 2012, Dan Dyer wrote:
>
>
>  Use Case 1
> Service Owned Instances
> There are a set of use cases where a service is acting on behalf of a
> user,
> the service is the owner of the VM but billing needs to be attributed
> to the
> end user of the system.This scenario drives two requirements:
> 1. Pricing is similar to base VM's but with a premium. So the type of
> service for a VM needs to be identifiable so that the appropriate
> pricing
> can be applied.
> 2. The actual end user of the VM needs to be identified so usage can be
> properly attributed
>
>  I think that for this, you just need to add more meters on top of the
> existing one with your own user and project id information.
>
>
>  As an example, in some of our PAAS use cases, there is a service
> controller
> running on top of the base VM that maintains the control and and
> manages the
> customer experience. The idea is to expose the service and not have the
> customer have to (or even be able to) manipulate the virtual machine
> directly. So in this case, from a Nova perspective, the PAAS service
> owns
> the VM and it's tenantID is what is reported back in events. The way we
> resolve this is to query the service controller for meta data about that
> instances they own. This is stored off in a separate "table" and used to
> determine the real user at aggregation time.
>
>  This is probably where you should emit the meters you need.
>
>
>  Use Case 2
> Multple Instances combine to make a billable "product/service"
> In this use case, a service might consist of several VM's, but the
> actual
> number does not directly drive the billing.  An example of this might
> be a
> redundant service that has a primary and two backup VM's that make up a
> deployment. The customer is charged for the service, not the fact
> that there
> are 3 VM's running. Once again, we need meta data that is able to
> describe
> this relationship so that when the billing records are processed, this
> relationship can be identified and billed properly.
>
>  Kind of the same here, if you don't want to really bill the vm, just
> don't meter them (or ignore the meters) and emit your own meter via your
> PaaS platform to bill your customer.
>
> Or is there a limitation I miss?
>
>
>  ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : opensta

Re: [Openstack] [ceilometer] meter data volume

[Doug Hellmann]

On Wed, Oct 31, 2012 at 1:39 PM, Julien Danjou  wrote:

> On Wed, Oct 31 2012, Eoghan Glynn wrote:
>
> > Would we have also have some 'misses' with the cumulative approach
> > when the ceilometer agent was down?
>
> No, unless the counter resets several times while your agent is down.
> But delta has the same issue.
>
> > If I understood the (\Sigma local maxima)-first idea correctly,
> > the usage up to the first polling cycle would always be
> > discounted from any duration.
>
> No, because if you have:
>
> Time | Value
> 0| 10
> 1| 30
> 2| 50
> 3| 80
> 4| 100
>
> If your delta-pollster is down at 1 and 2, you restart at 3, therefore
> at 4 you'll send "20" as usage (100 minus 80). So you miss the delta
>
between 10 (time 0) and 80 (time 3) (therefore 70 for free!).
> If you send right away 80 at time 3 when restarting, the API will be
> able to guess that between 0 and 3 the value went from 10 to 80.
> With delta approach, the API cannot guess that.
>

Sure it can, you just need to move where the caching is done. Using a local
cache to maintain the previous time a value was published you would know at
time 3 that the last published value was 10, and so send 70. So the total
will be correct.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] meter data volume

[Doug Hellmann]

On Wed, Oct 31, 2012 at 11:25 AM, Eoghan Glynn  wrote:

>
>
> > > I don't think (max - min) would suffice to give an accurate
> > > measure of the actual CPU time used, as the counter may have
> > > reset multiple times in the course of the requested duration.
> >
> > It is, because /max in the API should be aware of the fact a
> > reset can occur and computes accordingly. We started to discuss
> > this a bit in:
> >
> >   https://bugs.launchpad.net/ceilometer/+bug/1061817
>
> A-ha, OK, so not so much (max - min) as:
>
>(\Sigma local maxima) - first
>
> Sounds computationally expensive to produce on the fly, but maybe
> the local maxima can be efficiently recorded as the data is being
> ingested.
>

Is that better than just reporting the data in a more easily digested
format in the first place?

Julien, I don't understand your comment about losing data "if your system
is not launched to compute delta". Can you clarify what you mean there? I
do understand that the agent would need to store state about the counter
locally in order to track the delta value, but I think we could provide a
convenient way for pollsters to do that without complicating them
excessively.

Doug



>
> Cheers,
> Eoghan
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] meter data volume

[Doug Hellmann]

On Wed, Oct 31, 2012 at 10:23 AM, Eoghan Glynn  wrote:

>
>
> Hi Yawei Wu,
>
> The root of the confusion is the fact the cpu meter is reporting
> the cumlative cpu_time stat from libvirt. This libvirt counter is
> reset when the associated qemu process is restarted (an artifact
> of how cpuacct works).
>
> So when you stop/start or suspend/resume, a fresh qemu process
> is sparked up, then the cumulative time is reset.
>
> Thanks for bringing this up, as it has implications as to how
> we meter CPU time and utilization[1].
>
> We may need to start metering the delta between CPU times on
> subsequent polling cycles, instead of using a cumulative meter
> (dealing with the edge case where the instance has been restarted
> within a polling period).
>

Good idea. We need to capture this issue to make sure we get it onto the
roadmap for this cycle. Is there a bug or blueprint for it yet?

Doug


>
> Cheers,
> Eoghan
>
> [1] https://review.openstack.org/14921
>
>
> > I am still testing ceilometer now. I am confused about the meter
> > volume
> > in the mongodb. Let's talk about cpu usage.
> >
> > After I create and boot a vm named vm_1, meter data record about cpu
> > usage will be inserted into db in cycle(default 10 minutes). For
> > example,the 'counter_volume' of the first record is '5206000',and
> > the second one is '12389000'.
> >
> > 1) '12389000' nanoseconds means '123.89' seconds or two
> > minutes,it
> > seem like to be 1238.9 seconds actually, is there something wrong ?
> >
> > 2) If I never reboot or suspend vm_1, will the 'counter_volume' of
> > cpu
> > usage record increase all the time ? Just like '8 minutes' -> '18
> > minutes' -> '28 minutes' ?
> >
> > 3) If I reboot or suspend vm_1, I find that the 'counter_volume' of
> > cpu
> > usage record will count from zero. Just like '8 minutes' -> '18
> > minutes'
> > -> '28 minutes' [-> '0 minutes'] ->'5 minutes' -> '15 minutes'. Does
> > it
> > mean that 'counter_volume' just represents how long has vm_1 been
> > booted
> > up ?
> >
> > 4) This one is about Web API. I find that "GET
> > /v1/resources/(resource)/meters/(meter)/volume/sum" just return the
> > sum
> > value of all the cpu 'counter_volume', like '8 minutes' + '18
> > minutes'.
> > Is it reduplicate ?
> >
> > 5) If I want to know how long has vm_1's cpu been used yesterday, how
> > can I do ?
> >
> > It seems like that I have too many questions..
> >
> > Thank you very much !
> >
> >
> > ---
> > Yawei Wu
> > Dalian Hi-Think Computer Technology,Corp.
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up

[Doug Hellmann]

I can't meet later than that tonight.

Doug

On Mon, Oct 29, 2012 at 10:21 AM, Sandy Walsh wrote:

>  Ugh, I just realized I have a conflict. Can we push it an hour later?
> (sorry!)
>
> -S
>
>  --
> *From:* 
> openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net[openstack-bounces+sandy.walsh=
> rackspace@lists.launchpad.net] on behalf of Annie Cheng [
> ann...@yahoo-inc.com]
> *Sent:* Monday, October 29, 2012 11:18 AM
> *To:* openstack@lists.launchpad.net
>
> *Subject:* Re: [Openstack] Instrumentation Monitoring Next Step - quick
> meet up
>
>   Apologize .. Correction
>
>  Time: Monday (10/29/2012) 2200 UTC – 2300 UTC (*3pm PDT *if you are in
> California)
>  Location: IRC #openstack-meeting
>
>
>   From: Annie Cheng 
> Date: Mon, 29 Oct 2012 07:14:08 -0700
> To: "openstack@lists.launchpad.net" 
> Subject: Re: [Openstack] Instrumentation Monitoring Next Step - quick
> meet up
>
>   Reminder:
> Time: Monday (10/29/2012) 2200 UTC – 2300 UTC (2pm PDT if you are in
> California)
>  Location: IRC #openstack-meeting
>
>  Thanks!
>
>  Annie
>
>   From: Annie Cheng 
> Date: Thu, 25 Oct 2012 14:17:09 -0700
> To: "openstack@lists.launchpad.net" 
> Subject: [Openstack] Instrumentation Monitoring Next Step - quick meet up
>
>   Hi all,
>
>  Couple of us chat in the summit design sessions and and after summit on
> #openstack irc regarding topic of Monitoring.  We think it's best to do a
> quick meeting to get everyone on the same page, split works, and get at
> least a prototype going in Grizzly.
>
>  Time: Monday (10/29/2012) 2200 UTC – 2300 UTC
> Location: IRC #openstack-meeting
> I checked http://wiki.openstack.org/Meetings, this tme slot seems to be
> empty
>
>  Top level agenda would be
>
>1. Get everyone on the same page on high level direction
>2. Discuss different design/implementation possibility
>3. Split up works
>
> Before the meeting, if you want to read up, here are some links I know.
>  Please jump in with others I missed:
> Blueprint:
>
> https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring
> Etherpad:
> https://etherpad.openstack.org/grizzly-common-instrumentation
> Different code samples:
> https://github.com/asalkeld/statgen
>
>  Looking forward, some of those conversation probably will fold into the
> regular Metering meeting.  Just like to do a one off for now so we can go
> deeper on monitoring specific topics.
>
>  Thanks!
>
>  Annie
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup

[Doug Hellmann]

On Oct 25, 2012, at 9:44 PM, Joshua Harlow  wrote:

> As for statgen, I think that¹s just a temp repo, it'd be nice to have the
> end result of this be a library that provides somewhat generic metrics and
> plugins and such so that stacktech could use the outputs of it, ceilometer
> could the outputs and other systems could use the outputs (where an output
> goes would be configurable so that each system can configure its outputs
> as the operator desires, ie I want my MONITOR metrics to go to MQ in
> ceilomter and stacktech consumable formats, or to files or to...).
> 
> I think when this gets going we should have some repo/project name that
> goes on stackforge so that even while this is being developed it can go
> through the normal review process and such (and not in someones special
> github repo). But we have to start somewhere, something we can discuss as
> to what a good solution is @ the meeting.

At the summit, as part of the discussions around expanding the scope of 
ceilometer to cover measurement for monitoring, we discussed developing the 
library as part of ceilometer for now and either moving it to Oslo for release 
as a library or just releasing the library as a separate package from the 
ceilometer project directly.

Doug

> 
> -Josh 
> 
> On 10/25/12 5:47 PM, "Jeffrey Budzinski"  wrote:
> 
>> 
>> Yes, I think support for metrics objects that can be leveraged both by
>> monkey patches and decorators was what we'd been thinking along the lines
>> of. The metrics would be controlled via config both in what scopes are
>> active (e.g. on|off for a package, module, etc.) and also the outlet for
>> the metrics (file, datagram, rpc). Support for instrumentation levels
>> would also be nice so that metric flow could be controlled (i.e.
>> instrumentation point is annotated as METRIC, MONITOR, PROFILE and then
>> level to actually emit can be set in conjunction with a metric
>> outlet/sink). With this approach, folks could control both the volume of
>> metrics and also the outlet for the metrics. Ceilometer would also be an
>> outlet that could be leveraged via RPC flow for metrics -- though I'd
>> expect some would want to send via datagram to statsd or file for offline
>> log aggregation.
>> 
>> I'll post a diagram tomorrow for review and comment.
>> 
>> Oh btw, I updated the spec with most of what was in the etherpad. We can
>> update the spec to reflect whatever we decide is the preferred approach.
>> 
>> -jeff
>> 
>> On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote:
>> 
>>> On 25/10/12 11:13 +, Sandy Walsh wrote:
 grizzly-common-instrumentation seems to be the best choice ...
 hopefully the other groups will use this etherpad too.
 
 We need a proper blueprint to nail down the approach. IRC is great,
 but doesn't retain history for other groups. I think we need to get a
 plan for translating the etherpad into something concise and nailed
 down.
>>> 
>>> Agree.
>>> 
 
 statgen should really just be a new notifier in Tach (or Scrutinize)
 ... vs copy-pasting the code into yet-another repo.  Hopefully that's
 the plan? Tach should remain a generic tool and not pegged to OpenStack.
>>> 
>>> Well that was just an "ideas play pen" not serious code.
>>> 
>>> I might be coming at this from a slightly different angle...
>>> I was looking at a library that can be used to generate trace,
>>> monitoring
>>> and metering data (kind of like log levels for logging). Currently both
>>> Tach and Scrutinize don't have enough fields (of course that could be
>>> changed).
>>> 
>>> Also I think we should be able to insert instrumentation into the code
>>> as well
>>> as have the function level performance metrics monkey patched.
>>> 
>>> Then we could have a config that directed metric data to different
>>> notifiers
>>> like how you do it in Scrutinize perhaps. Also enforcing data rate
>>> limits
>>> and possible data aggregation could be neat configurable features.
>>> 
>>> Anyway more at the meeting...
>>> 
>>> -Angus
>>> 
 
 -S
 
 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on
 behalf of Angus Salkeld [asalk...@redhat.com]
 Sent: Thursday, October 25, 2012 1:00 AM
 To: openstack@lists.launchpad.net
 Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize,
 CloudWatch integration ... Summit followup
 
 On 24/10/12 23:35 +, Sandy Walsh wrote:
> Hey y'all,
> 
> Great to chat during the summit last week, but it's been a crazy few
> days of catch-up since then.
> 
> The main takeaway for me was the urgent need to get some common
> libraries under these efforts.
 
 Yip.
 
> 
> So, to that end ...
> 
> 1. To those that asked, I'm going to get my slides / video
> presentation made available via the list. Stay tuned.
>>

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

On Oct 26, 2012, at 4:29 AM, Julien Danjou  wrote:

> On Thu, Oct 25 2012, Doug Hellmann wrote:
> 
>>> IIUC, what's need here is a GROUP BY operator in the API.
>>> 
>>> Correct me if I'm wrong, but this is still doable via the API if you
>>> request /users//meters/instance and treats the events in the
>>> client, no?
>> 
>> It is possible, but very very inefficient.
> 
> Oh, sure it is. But adding feature and making things more efficient are
> different things. :)
> 
>> Querying against arbitrary metadata fields is easy in the MongoDB driver,
>> but not in the SQLAlchemy driver. Adding explicit handling for dimensions
>> would let us implement it in SQL and improve performance with indexes in
>> Mongo.
> 
> Ah, thanks to remind me how ORM are bad and that we now have to fight
> against it. :)
> 
> I wish we could use JSON native type from PostgreSQL directly and be
> efficient!

You could write a different storage driver. ;)

Doug

> 
> -- 
> Julien Danjou
> # Free Software hacker & freelance
> # http://julien.danjou.info

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

On Oct 25, 2012, at 6:05 PM, Angus Salkeld  wrote:

> On 25/10/12 17:04 -0400, Doug Hellmann wrote:
>> On Thu, Oct 25, 2012 at 10:22 AM, Julien Danjou  wrote:
>> 
>>> On Thu, Oct 25 2012, Doug Hellmann wrote:
>>> 
>>> > That would be one way, but adding "dimensions" to the meters also makes
>>> > sense because it reduces the need to collect the data more than once.
>>> 
>>> In case of group, the other problem is how to emit instance counter with
>>> group metadata (assuming this group implementation is not part of Nova
>>> but Heat).
>>> 
>> 
>> Good point. I was assuming the values would be available as metadata of the
>> underlying resource, but that may not always be the case.
>> 
> 
> Yea, we need a consistent way of doing this. That should work on different
> resource types. We could use the tags or a similar mechanism.

Tags would be available as part of an objects normal metadata, right?

Doug

> 
> -A
>> 
>>> 
>>> > For instance, if "flavor" was a dimension of the "instance" meter I
>>> > wouldn't need the separate meter "instance:". These sorts of
>>> > use cases were part of the original motivation for collecting all of
>>> > the metadata about a resource, but what we have now isn't structured
>>> > enough to let the API user query into it.
>>> 
>>> IIUC, what's need here is a GROUP BY operator in the API.
>>> 
>>> Correct me if I'm wrong, but this is still doable via the API if you
>>> request /users//meters/instance and treats the events in the
>>> client, no?
>>> 
>> 
>> It is possible, but very very inefficient.
>> 
>> 
>>> 
>>> > How, then, do we define the dimensions for a given meter in a more
>>> > structured way? Some built-in values (like flavor) can be pulled
>>> > automatically based on the resource type, but what about settings
>>> > controlled by the deployer and end-user (for purposes other than
>>> billing)?
>>> 
>>> Do we have to define dimensions explicitely, or isn't what's needed just
>>> ways to filter and/or group events by metadata fields?
>>> 
>> 
>> Querying against arbitrary metadata fields is easy in the MongoDB driver,
>> but not in the SQLAlchemy driver. Adding explicit handling for dimensions
>> would let us implement it in SQL and improve performance with indexes in
>> Mongo.
>> 
>> Doug
>> 
>> 
>>> 
>>> --
>>> Julien Danjou
>>> // Free Software hacker & freelance
>>> // http://julien.danjou.info
>>> g
>>> 
> 
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

On Thu, Oct 25, 2012 at 10:22 AM, Julien Danjou  wrote:

> On Thu, Oct 25 2012, Doug Hellmann wrote:
>
> > That would be one way, but adding "dimensions" to the meters also makes
> > sense because it reduces the need to collect the data more than once.
>
> In case of group, the other problem is how to emit instance counter with
> group metadata (assuming this group implementation is not part of Nova
> but Heat).
>

Good point. I was assuming the values would be available as metadata of the
underlying resource, but that may not always be the case.


>
> > For instance, if "flavor" was a dimension of the "instance" meter I
> > wouldn't need the separate meter "instance:". These sorts of
> > use cases were part of the original motivation for collecting all of
> > the metadata about a resource, but what we have now isn't structured
> > enough to let the API user query into it.
>
> IIUC, what's need here is a GROUP BY operator in the API.
>
> Correct me if I'm wrong, but this is still doable via the API if you
> request /users//meters/instance and treats the events in the
> client, no?
>

It is possible, but very very inefficient.


>
> > How, then, do we define the dimensions for a given meter in a more
> > structured way? Some built-in values (like flavor) can be pulled
> > automatically based on the resource type, but what about settings
> > controlled by the deployer and end-user (for purposes other than
> billing)?
>
> Do we have to define dimensions explicitely, or isn't what's needed just
> ways to filter and/or group events by metadata fields?
>

Querying against arbitrary metadata fields is easy in the MongoDB driver,
but not in the SQLAlchemy driver. Adding explicit handling for dimensions
would let us implement it in SQL and improve performance with indexes in
Mongo.

Doug


>
> --
> Julien Danjou
> // Free Software hacker & freelance
> // http://julien.danjou.info
> g
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Potential New Use Cases

[Doug Hellmann]

That would be one way, but adding "dimensions" to the meters also makes
sense because it reduces the need to collect the data more than once. For
instance, if "flavor" was a dimension of the "instance" meter I wouldn't
need the separate meter "instance:". These sorts of use cases were
part of the original motivation for collecting all of the metadata about a
resource, but what we have now isn't structured enough to let the API user
query into it.

How, then, do we define the dimensions for a given meter in a more
structured way? Some built-in values (like flavor) can be pulled
automatically based on the resource type, but what about settings
controlled by the deployer and end-user (for purposes other than billing)?

Doug

On Thu, Oct 25, 2012 at 7:11 AM, Julien Danjou  wrote:

> On Thu, Oct 25 2012, Angus Salkeld wrote:
>
> > So we need normal stuff like cpu/mem usage but aggregated over the
> instances
> > in the group. So this is not easy to do externally.
>
> Interesting use case. I think for such a thing, a way would be to have a
> component listening to meters (e.g. on the bus directly or via
> PubSubHubbub-like) to re-emit consolidated meters.
>
> --
> Julien Danjou
> /* Free Software hacker & freelance
>http://julien.danjou.info */
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer API Glossary

[Doug Hellmann]

On Thu, Oct 25, 2012 at 5:39 AM, Julien Danjou  wrote:

> On Thu, Oct 25 2012, 吴亚伟 wrote:
>
> > If it is just like what Eoghan said that  "it is unused right now as
> there
> > is only a single source currently", what does the "?" represent?
> > If I want to establish a customer billing system,do I need it ?
>
> It represents the fact we had no idea to use it when we write the code
> the first time. We discussed this yesterday, now we have a good plan.
>
> See https://bugs.launchpad.net/ceilometer/+bug/1070857
>
> > I do mean the cinder volumes , but I don't know how to configure it
> ,could
> > you tell me more detailed steps on how to do this ?
>
> You need to configure the notifier to use rabbitmq (this is commented
> out in the default configuration file IIRC), to set the audit interval
> to something low like daily or hourlay (this should also be in the
> default configuration file) and to run this program into a cronjob.
>
> > Same to cinder , detailed steps will be highly appreciated .
>
> Like cinder, you need to configure the notification backend to use
> rabbitmq.
>

We have special instructions for configuring glance to use rabbit for
notifications in
http://ceilometer.readthedocs.org/en/latest/install.html#configuring-devstackbut
I don't see anything about cinder there. Do we need to add another
step
to set the notification_driver for cinder?

Doug


>
> > If I modify the configuration file of service like nova or  cinder , how
> to
> > restart the related service in devstack ?
>
> Just press C-c in the screen window, up arrow and return.
>
> --
> Julien Danjou
> /* Free Software hacker & freelance
>http://julien.danjou.info */
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [OpenStack] Ceilometer API Glossary

[Doug Hellmann]

On Tue, Oct 23, 2012 at 5:34 AM, Eoghan Glynn  wrote:

>
>
> > I am testing ceilometer in my devstack virtual machine. Although I
> > can see the meter data model in the mongodb, I am confused about
> > some glossary when I test its Web API. I am not very clear about the
> > " resource" in the " GET /v1/resources",either "source" in the "GET
> > /v1/sources/(source)/meters/(meter)".
> > Can someone explain these two concept? Thanks a lot.
>
> Hi Yawei,
>
> Good question!
>
> My understanding is that the resources collection refers to the set
> of UUIDs identifying the openstack entities (e.g. instance, volume,
> image ... etc.) being metered, whereas the source refers to the
> origin of the metering data and is effectively unused right now
> as there is only a single source currently (though we could in the
> future for example distinguish between metric and metering data
> in this way).
>

That's exactly right. I'll open a ticket to add a glossary to the
documentation.

Doug


>
> Cheers,
> Eoghan
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Versioning for notification messages

[Doug Hellmann]

On Oct 10, 2012, at 6:02 AM, "Day, Phil"  wrote:

> Whilst a version number would allow a consumer to detect that something has 
> changed, it doesn’t really help in providing any kind of backward 
> compatibility.
> Consider the following scenario:  There are a bunch of systems external to 
> Nova developed to consume notification messages, and someone introduces a 
> change to the notification system that changes the message content.   They do 
> the right thing in updating the version number, but all of those external 
> systems now need to change as well.   The new version number lets them fail 
> explicitly, but it could still have a significant impact on a production 
> system.
>  
> Where I’d like to get to make the notification system a formal external 
> interface, that has the same degree of stability, version control, and rigor 
> around changes that the inbound API has.   
>  
> I would guess that Ceilometer will have some requirement around this ?

Yes, definitely. We will be resilient in the face of additions to the 
notification payloads, but our event processing plugins may break if fields are 
removed (mostly the envelope fields). 

Our primary requirement, though, is that notifications include all of the 
metadata for the object in question so we can track that accurately. We are 
still working with some of the projects to make that true in all cases. 

Doug

>  
> Phil
>  
> From: Diego Parrilla Santamaría [mailto:diego.parrilla.santama...@gmail.com] 
> Sent: 10 October 2012 09:18
> To: Day, Phil
> Cc: David Ripton; openstack@lists.launchpad.net
> Subject: Re: [Openstack] Versioning for notification messages
>  
> If we want to have a notification system that could handle messages with 
> different payloads and different versions, we have two options:
> 
> 1) detect the version of the payload in the notification message
> 2) add a version number in the notification message
> 
> Option 1 sounds to me like something hard to maintain. Option 2 seems to be 
> correct way to do it in the long term.
> 
> +1 for a version number in the notification message
> 
> Cheers
> Diego
>  -- 
> Diego Parrilla
> CEO
> www.stackops.com |  diego.parri...@stackops.com | +34 649 94 43 29 | 
> skype:diegoparrilla
> 
> 
>  
> 
> 
> 
> On Wed, Oct 10, 2012 at 9:27 AM, Day, Phil  wrote:
> Hi All,
> 
> I guess I may have mis-stated the problem a tad in talking about version 
> numbering.  The notification system is an outbound interface, and my interest 
> is in being able to write consumers with some guarantee that they won't be 
> broken as the notification message format evolves.
> 
> Having a version number gives the client a way to know that it may now be 
> broken, but that's not really the same as having an interface with some 
> degree of guaranteed compatibility,
> 
> Phil
> 
> -Original Message-
> From: openstack-bounces+philip.day=hp@lists.launchpad.net 
> [mailto:openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of 
> David Ripton
> Sent: 09 October 2012 20:59
> To: openstack@lists.launchpad.net
> Subject: Re: [Openstack] Versioning for notification messages
> 
> On 10/09/2012 01:07 PM, Day, Phil wrote:
> 
> > What do people think about adding a version number to the notification
> > systems, so that consumers of notification messages are protected to
> > some extent from changes in the message contents ?
> >
> > For example, would it be enough to add a version number to the
> > messages - or should we have the version number as part of the topic
> > itself (so that the notification system can provide both a 1.0 and 1.1 
> > feed), etc ?
> 
> Putting a version number in the messages is easy, and should work fine.
>   Of course it only really helps if someone writes clients that can deal with 
> multiple versions, or at least give helpful error messages when they get an 
> unexpected version.
> 
> I think using separate topics for each version would be inefficient and 
> error-prone.
> 
> Inefficient because you'd have to send out multiples of each message, some of 
> which would probably never be read.  Obviously, if you're sending out N 
> copies of each message then you expect only 1/N the queue performance.  
> Worse, if you're sending out N copies of each message but only 1 of them is 
> being consumed, your queue server is using a lot more memory than it needs 
> to, to hold onto old messages that nobody needs.
> (If you properly configure a high-water mark or timeout, then the old 
> messages will eventually be thrown away.  If you don't, then your queue 
> server will eventually consume way too much memory and start swapping, your 
> cloud will break, and someone will get paged at 2 a.m.)
> 
> Error-prone because someone would end up reusing the notification queue code 
> for less idempotent/safe uses of queues, like internal API calls.
> And then client A would pick up the message from topic_v1, and client B would 
> pick up the same message from topic_v2, and the

Re: [Openstack] Versioning for notification messages

[Doug Hellmann]

On Tue, Oct 9, 2012 at 4:12 PM, Eric Windisch  wrote:

>
>
>
> On Tuesday, October 9, 2012 at 15:58 PM, David Ripton wrote:
>
> > On 10/09/2012 01:07 PM, Day, Phil wrote:
> >
> > > What do people think about adding a version number to the notification
> > > systems, so that consumers of notification messages are protected to
> > > some extent from changes in the message contents ?
> >
>
> Right now, there is no appropriate or acceptable way to consume
> notifications. Plainly, while notifications exist, they shouldn't be
> considered stable or even usable until this exists.
>
> Message formats and versioning should be a consideration in the effort to
> create a reusable consumption pattern.
>

This will be part of what is covered during the "Using the message bus for
messages" session Tuesday afternoon at the summit.

http://summit.openstack.org/cfp/details/117

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Discussion / proposal: deleted column marker

[Doug Hellmann]

On Wed, Oct 3, 2012 at 9:08 AM, Day, Phil  wrote:

> I *think* deleted flavours used to be needed as there could still be
> instances running against them and the flavour definition was used by the
> quota calculations.  Not sure if this is still the case, or if the data now
> comes straight from the instances table.Some aspects of a flavour (e.g.
> rxtx_factor) could be useful to a scheduler, and that data currently isn't
> saved into the instance.
>
> I guess the usage audit type functionality (i.e. tell me about all
> instances that have run sometime in this audit period) may be another case
> where deleted instances are required at the moment.
>

That data is being gathered by ceilometer now, so maybe we can do away with
it in the nova database during Grizzly.

Doug


>
>
>
> -Original Message-
> From: openstack-bounces+philip.day=hp@lists.launchpad.net [mailto:
> openstack-bounces+philip.day=hp@lists.launchpad.net] On Behalf Of
> Pitucha, Stanislaw Izaak
> Sent: 03 October 2012 13:09
> To: openstack@lists.launchpad.net
> Subject: Re: [Openstack] Discussion / proposal: deleted column marker
>
> Hi Johannes,
> I know the names collide here, but since this technique is known as
> soft-deletes... We need more namespaces :)
>
> Thanks for the idea of grepping for read_deleted. Fortunately I think the
> situation isn't as bad as it would seem. Let me group the places which
> change read_deleted in the code (many results from grep are comments).
> Reading only deleted entries, or all:
>
> - xenserver (instance) - cleanup tool - I don't do xen, so I'm not sure
> how useful is it. Anyone?
> - tests - can be ignored - if there is no functionality, tests can be
> killed
> - sqlalchemy api (instance) - fixed ip can reference a deleted instance
> (tricky situation; from the commit message: "It adds a test  to verify that
> the code works with a duplicate deleted floating_ip" - this seems very
> wrong...)
> - sqlalchemy api (iscsi) - getting deleted iscsi targets which are still
> referenced by volume
> - sqlalchemy api (various) - instance migration, s3image, bandwidth,
> storage manager, flavors (only available from nova-manage)
> - compute manager (instance) - reaping deleted instances - I can't see why
> the same logic wouldn't apply if the rows are actually missing (needs
> analysis, maybe there's a reason)
> - compute instance_types (flavour) - apparently flavours are usually read
> even if they're deleted
> - network manager (instance) - making sure that ips/networks can be
> removed even if the instance is already deleted
>
> So here's what I can see: pretty much all the usage is about deleting
> instances or making sure parts connected to instances go away if the
> instance is deleted earlier. It doesn't seem right, but could be
> progressively fixed. It looks like another "state" of the instance, which
> could be integrated into the other state fields.
>
> Nothing else uses the deleted column explicitly (unless I missed something
> - possible). Ips, networks, keys, anything that actually goes away
> permanently (and doesn't involve a big chain of cleanup events) is never
> read back once it's marked as deleted.
> So maybe a better approach is not to remove the deleted column completely,
> but to start stripping it from places where it's not needed (fixed,
> floating ips, networks, ssh keys being good candidates). This could be done
> by creating a new layer over NovaBase and removing the "deleted" marker
> from NovaBase itself. Or maybe even by creating a SoftDeleteMixin and
> applying it to all current models, then removing it where unnecessary? That
> would keep the current behaviour where it's currently needed, but at the
> same time it would provide a known migration path to get rid of it.
> We could start stripping the new layer then table by table and adding
> unique constraints where they make sense, before trying to tackle the
> really tricky parts (for instances table, maybe the marker actually makes
> sense? maybe not? - it's definitely not going to be an easy decision/fix)
>
> Regards,
> Stanisław Pitucha
> Cloud Services
> Hewlett Packard
>
>
> -Original Message-
> From: openstack-bounces+stanislaw.pitucha=hp@lists.launchpad.net
> [mailto:openstack-bounces+stanislaw.pitucha=hp@lists.launchpad.net] On
> Behalf Of Johannes Erdfelt
> Sent: Tuesday, October 02, 2012 6:12 PM
> To: openstack@lists.launchpad.net
> Subject: Re: [Openstack] Discussion / proposal: deleted column marker
>
> On Tue, Oct 02, 2012, Pitucha, Stanislaw Izaak 
> wrote:
> > Does anyone know why soft-delete is still in place?
> > Are there any reasons it can't / shouldn't be removed at this time?
> > If it's possible to remove it, would you miss it?
>
> I'm certainly not a fan of the database soft-delete for many of the same
> reasons you've described, but there are some places where removing it would
> require code changes.
>
> Off the top of my head would be pretty much anywhere a context se

Re: [Openstack] Discussion / proposal: deleted column marker

[Doug Hellmann]

On Wed, Oct 3, 2012 at 4:46 AM, Haynes, Dave (Cloud Services) <
dave.hay...@hp.com> wrote:

> +1. Let's do it.
>
> If we need to add some extra tests to protect against regressions, then so
> be it. I will help.
> I also think better use could be made of the notifications system. A
> properly defined topic namespace would go a long way to assist that.
>

The ceilometer project is making *extensive* use of the existing
notifications to collect metering data about resources (I expect us to be
consuming notifications from all of the other OpenStack services by the
time we're done). So, if you're going to change the way notifications work,
please bring us into that discussion.

Here are a few proposed summit sessions related to notifications, messaging
in general, and ceilometer integration:

http://summit.openstack.org/cfp/details/110
http://summit.openstack.org/cfp/details/113
http://summit.openstack.org/cfp/details/117
http://summit.openstack.org/cfp/details/128

Both Nick Barcet and I will be at the summit.


>
> -Original Message-
> From: openstack-bounces+dave.haynes=hp@lists.launchpad.net [mailto:
> openstack-bounces+dave.haynes=hp@lists.launchpad.net] On Behalf Of
> Pitucha, Stanislaw Izaak
> Sent: 02 October 2012 17:43
> To: openstack@lists.launchpad.net
> Subject: [Openstack] Discussion / proposal: deleted column marker
>
> Hi all,
> I'd like to open a discussion on a topic that's been bugging me for a
> number of reasons - soft deletes (by that I mean marking rows with
> deleted=1 in the
> db) and related - actions audit. Some research and speculations first...
> To be honest I could not find any reason why the feature is there in the
> first place. Here's the commit that introduced the 'deleted' columns:
>
> https://github.com/openstack/nova/commit/ae6905b9f1ef97206ee3c8722cec3b26fc0
> 64f38 - unfortunately the description says only "Refactored orm to
> support atomic actions". So the guessing part starts here. These are the
> possible uses for soft-deletion of the database records that I could come
> up with:
>
> 1. safety net (recover data what was deleted by accident) 2. audit / log
> (preserve the information about past data) 3. some kind of
> micro-optimisation where update is more useful than deletion
> - be it speed or ease of handling foreign constraints (or not handling
> them straight away more likely) 4. ... no... that's all
>
> But I think there's a number of issues with that approach. First - what
> are the issues with the possible uses above. Then - issues that I can see
> otherwise. Point by point:
>
> 1. Soft-deletion probably makes some restoration possible, but I doubt
> there's much that could be done without full analysis of the situation.
> Mainly because the database is only about metainformation - the actual
> data users care about either goes away (ephemeral disks, memory, ...) or
> not (volumes, networks, ...) and is not recoverable. Since resources like
> ips and volumes can be just reused in other instances, not all recovery is
> possible anyway. Most hardcore fixes could be done by reinserting the
> original/reconstructed data just as easily as verifying what's safe to
> undelete. Both actions require looking at existing data and locking out
> information so it doesn't get reused while we're messing with the previous
> state.
>
> 2. Soft-deleted records are not great as a source of old information. This
> is connected to the previous point - some resources are just reused /
> rewritten instead of created and deleted. For example there's no record of
> what happens with old floating ips - the information gets overwritten when
> the IP is reassigned to the new instance, so the useful bits are gone.
>
> 3. This is the only thing I could come up with related to the commit
> message itself and the "support atomic actions" part. Maybe it was
> sometimes easier to mark something as deleted rather than managing and
> properly ordering deletes of a number of related entries.
>
> So with that out of the way, here's a number of issues related to
> soft-deletes that I run into myself:
>
> 4. Indexing all this data on a busy system is getting a bit silly. Unless
> you do your own cleanup of old entries, you will end up in a situation
> where looking up instances on a host actually looks through thousands of
> "deleted"
> rows even if only around 20 or so can be live and interesting. I know it's
> not a huge deal, but still an unnecessary cpu cycle burning.
>
> 5. Some things are just not possible to do in a safe and portable way at
> the moment. For example adding a new network and fixed IPs (there's a bug
> for that https://bugs.launchpad.net/nova/+bug/755138). I tried to fix
> this situation, but actually discovered that this is not possible to do
> using only sessions and with the 'deleted' column in place. There are ways
> to do it in a specific database (you can lock the whole table in mysql for
> example), but it's not portable then. The best you can do easily i

Re: [Openstack] [ceilometer] Release plan for 0.1

[Doug Hellmann]

On Wed, Sep 12, 2012 at 12:47 PM, Graham Binns  wrote:

> Hi all,
>
> Based on the discussion in the last Ceilometer meeting[1], here's my
> proposal for dates for the Ceilometer 0.1 release calendar:
>
>  - Feature freeze for 0.1 QA: 2012-09-28
>  - Release: 2012-10-12
>
> Arguments for / against welcome.
>

Those dates seem reasonable to me.


>
>  [1]
> http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-06-15.00.log.html
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)

[Doug Hellmann]

On Tue, Sep 11, 2012 at 5:22 PM, Nick Barcet wrote:

>  PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC 
>
> The metering project team will hold its next meeting at alternate time
> on *Wednesday* at 9PM UTC
>  >.
>
> Everyone is welcome.
>
> Agenda:
> http://wiki.openstack.org/Meetings/MeteringAgenda
>
> * Review last week's actions
>   * nijaba to see with ttx how our sessions can be shown on official
> program
>   * nijaba to update meeting page to note new alternating meeting time
>   * gmb to a plan on the ml with fixed dates
>   * dhellmann make sure flask is listed as a dependency of ceilometer
>

In case I can't make it to the meeting, I did check that Flask is listed in
the tools/pip-requires file. It is pegged to version 0.9.


> * Open discussion
>
> If you are not able to attend or have additional topic you would like to
> cover, please update the agenda on the wiki.
>
> Cheers,
> --
> Nick Barcet 
> aka: nijaba, nicolas
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Weekly irc meetings time change?

[Doug Hellmann]

On Fri, Sep 7, 2012 at 6:39 AM, Nick Barcet wrote:

> On 09/05/2012 11:51 AM, Nick Barcet wrote:
> > Thanks for asking, I was just about to come up with this.  So based on
> > the poll, it seems that the 3-4pm UTC time slot received the most favors
> > with 9 yes, 1 "if need be" and 1 no.  So I guess we'll have to do
> > without Angus, unless we want to do alternating meetings to satisfy both
> > sides of the planet. In the later case we could do one week at 3pm UTC,
> > and the other at 9PM.  What would the others think about this?
> >
> > In any case, the next meeting will be tomorrow at 3pm UTC.  I'll be
> > sending a formal invite later today.
>
> Based on yesterday's meeting, we decided to try alternating time. Since
> 9PM UTC is taken on thursdays, I am proposing to move it on Wednesdays.
>  If Nobody objects, this means that starting next week we'll have our
> meetings on:
>
> Wednesday at 9PM UTC for odd weeks (September 12, 26)
> Thursday at 3PM UTC for even weeks (September 20 and October 4)
>
> Does this work for everyone?
>

That time on Wednesday does not work for me. An hour earlier or later would
work, if you want to try to stay close to the same time.

I am available during the Thursday time slot.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Weekly irc meetings time change?

[Doug Hellmann]

On Wed, Sep 5, 2012 at 5:51 AM, Nick Barcet wrote:

> On 09/05/2012 08:55 AM, SPN wrote:
> > On 2012-08-31 21:34, Nick Barcet wrote:
> >> On 08/30/2012 09:49 AM, Nick Barcet wrote:
> >>> I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on
> >>> Google.  Will reply to this thread when it opens.  We'll have to go
> with
> >>> what fits most.
> >>
> >> The poll is now open at [1], please submit your preference(s) there.  I
> >> will close the poll on tuesday.
> >>
> >> http://doodle.com/srxx5244qg9pz2pm
> >>
> >> Cheers,
> >> Nick
> >
> >
> > Is there a concenses on the timings?
>
> Thanks for asking, I was just about to come up with this.  So based on
> the poll, it seems that the 3-4pm UTC time slot received the most favors
> with 9 yes, 1 "if need be" and 1 no.  So I guess we'll have to do
> without Angus, unless we want to do alternating meetings to satisfy both
> sides of the planet. In the later case we could do one week at 3pm UTC,
> and the other at 9PM.  What would the others think about this?
>

That might be a little confusing, but we could give it a try. I would like
to have as much input as possible.

Doug


>
> In any case, the next meeting will be tomorrow at 3pm UTC.  I'll be
> sending a formal invite later today.
>
> Nick
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] experimenting with ceilometer via devstack

[Doug Hellmann]

John Tran 's patch just landed in devstack to enable ceilometer support.
Thanks, John!

To turn on ceilometer, add this line to your localrc before running
devstack:

enable_service ceilometer-acompute,ceilometer-acentral,
ceilometer-collector

The default configuration results in a MongoDB database called "ceilometer"
containing all of the metering data. You can access it directly using the
mongo shell via the command "mongo ceilometer".

The ceilometer API server does not have a wrapper script, but you can start
the dev server by running

python -m ceilometer.api

The server will listen for requests on port 9000 by default.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [Ceilometer] Project Technical Leader election result

[Doug Hellmann]

Congratulations, Nick!

On Fri, Aug 3, 2012 at 4:22 AM, Julien Danjou  wrote:

> Hi,
>
> The PTL election for Ceilometer¹ is over.
>
> The winner is Nick Barcet.
>
> Congratulations !
>
>
> The results are available here:
>
> http://www.opavote.org/vote/491028
>
> ¹  http://wiki.openstack.org/EfficientMetering/PTLElectionProcess
>
> Cheers,
> --
> Julien Danjou
> // Free Software hacker & freelance
> // http://julien.danjou.info
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Keyring support in openstack

[Doug Hellmann]

On Mon, Jul 30, 2012 at 5:30 PM, Adam Young  wrote:

> On 07/30/2012 05:17 PM, Kevin L. Mitchell wrote:
>
>> On Mon, 2012-07-30 at 13:50 -0700, Bhuvaneswaran A wrote:
>>
>>> The wiki mentions the password being saved using
 keyring.backend.**UncryptedFileKeyring. Does that mean the password is

>>> saved
>>>
 in cleartext? Is the file protected in some way besides filesystem
 permissions?

>>> As mentioned in wiki page, the password is stored in base64 format.
>>>
>> Which means it's stored in cleartext.  That is Not Good(tm) :)
>>
> Can Keyring be used to store a token instead?  That would A)  be better
> than password and B)  avoid a Keystone hit.


Don't tokens expire?

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Keyring support in openstack

[Doug Hellmann]

On Mon, Jul 30, 2012 at 4:51 PM, Bhuvaneswaran A  wrote:

> On Mon, Jul 30, 2012 at 7:46 AM, David Kranz 
> wrote:
> > I share Doug's concerns but would state some more strongly. IMO, it is
> > simply unacceptable to modify user-visible behavior based on whether some
> > package that happens to be used in an implementation is installed or not.
> > This package is installed on Ubuntu by default and may be used by other
> > applications that have nothing to do with OpenStack at all.
>
> Yes, as python-keyring is installed in almost all systems, the
> behaviour is unchanged.
>
> > If we really want to go down this road there should be an environment
> > variable that can be set to turn off this behavior for applications that
> do
> > not want it.
>
> David, good point. I'll revise the patch to not use keyring, if
> environment variable USE_KEYRING=0. If environment variable is not set
> or if it is USE_KEYRING=1, then keyring is used to store password.
>

How about OS_USE_KEYRING so it is clearer that the variable is related to
openstack?


>
> Doug, agree?
>
> --
> Regards,
> Bhuvaneswaran A
> www.livecipher.com
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Keyring support in openstack

[Doug Hellmann]

On Mon, Jul 30, 2012 at 4:50 PM, Bhuvaneswaran A  wrote:

> On Mon, Jul 30, 2012 at 6:31 AM, Doug Hellmann
>  wrote:
>
> > You've already answered several of my questions on the ticket, but I
> still
> > have some "usability" concerns.
> >
> > How does the keyring system support a single person logging in using
> > multiple user accounts? For example, if I have an admin account and a
> > "regular" user, how do I switch between them based on the operations I
> need
> > to perform?
>
> The password is stored in keyring, for a given user. It also support
> multiple users. The password is stored against the user specified in
> command line, --os-username or environment variable OS_USERNAME.
>
> The sample content of the keyring file ~/.openstack-keyring.cfg is as
> follows:
> [openstack]
> bhuvan = dG4wN2FjxA==
> test = xYwN2FjxA==
>

OK, that's good to know.


>
> > Is there a way to disable the behavior of having a password saved to a
> > keyring for a particular user, without uninstalling the python-keyring
> > package (and therefore disabling keyring support for all users)?
>
> The simplest alternative is to specify password using other mechanism,
> in command line or environment variable. It's not possible to prevent
> using keyring, if password is not specified in any of these 2
> mechanisms. The purpose of this patch is, to prevent password prompt.
>

We're going to need to include a way in the openstack cli to disable the
use of the keyring. There will be times when users won't want passwords
saved to a keyring, or where the password that is in the keyring is wrong
or shouldn't be used for some reason. It seems like an environment variable
and a command line switch would cover all of the ways to turn the keyring
off, don't you think?


>
> > The wiki mentions the password being saved using
> > keyring.backend.UncryptedFileKeyring. Does that mean the password is
> saved
> > in cleartext? Is the file protected in some way besides filesystem
> > permissions?
>
> As mentioned in wiki page, the password is stored in base64 format.
>

That doesn't seem any more secure than an environment variable set from a
user's login script. What benefit does keyring give us with this
configuration?


>
> > The mention of one backend implies that there are others. Should we give
> > users a way to choose the backend, in case they have a preference?
>
> python-keyring also support several other backends:
>   1.CryptedFileKeyring
>   2. GnomeKeyring
>   3. KDEKWallet
>   4. OSXKeychain
>   5. Win32CryptoKeyring
>   6. ... and more.
>
> The behaviour of these backends vary for each desktop. For instance,
> GnomeKeyring may prompt for keyring password, once per login session.
> CryptedFileKeyring may prompt for keyring password, every time. It's
> as good as not using keyring.
>

On the other hand, different users will be running in different
configurations. Maybe they *do* have a desktop environment, and want to use
one of those "real" keyring managers, instead of the simple INI file
described above. Does the keyring library have some way to detect which
backends are available at runtime? Or does the application (or user) have
to specify one explicitly?


>
> > How does the use of the keyring affect scripting using the command line
> > tool? Can a script access the keyring, or does it need to use the other
> > options?
>
> Yes. The script could be managed with any python script, using the
> same methods exposed in "keyring" python module.
>   -- get_password() -- to get the password for given user.
>   -- set_password() -- to set the password in keyring.
>

I was not clear. I meant could a shell script running the new cli access
the keyring. It sounds like that is not an issue, based on what you say
below.


>
> > In one review comment you mention a few desktop apps that know how to
> > manipulate the keyring to manage its contents. What about remote access
> via
> > ssh, where a desktop environment is not available? Does the keyring
> library
> > include tools for manipulating the file, or do we need to build our own?
> If
> > so, what tools would be needed?
>
> This was applicable for older patch, wherein we rely on
> desktop/environment specific backend. With older patch, if GNOME
> desktop is used, GnomeKeyring backend is used; if no desktop is used,
> CryptedFileKeyring backend is used. With new patch, irrespective of
> whether desktop is enabled, UncryptedFileKeyring backend is used. With
> this patch, the keyring behaviour is uniform across all systems in
> which we deploy openstack.
>

Re: [Openstack] API docs - spec and dev guide split proposal

[Doug Hellmann]

+1 to separating specifications from documentation for API users

On Mon, Jul 30, 2012 at 1:09 PM, Anne Gentle  wrote:

> Hi again all, your friendly doc coordinator here.
>
> I'd like to get a big push towards documenting reality instead of
> relying only on the specs stored in the compute-api repository. As an
> example, recently a Rackspace writer inserted min_count and max_count
> to the compute-api repository but the change was reversed after a
> request by Brian Waldon and Jorge Williams to leave the spec as-is. I
> agree that the spec has value but I believe we need to push towards
> reality and creating developer guides.
>
> So what I'd like to propose here is that we govern these repos as
> specs and change the titles of those documents to API specification:
>
> compute-api
> image-api
> identity-api
> object-api
> netconn-api
>
> At the same time, we'd start new "developer guides" in the
> openstack-manuals repository that document reality. We can track the
> work needed through the openstack-manuals bug and blueprint system.
>
> I went up to Brian Aker after his keynote at OSCon seeking contacts at
> HP who are interested in doing this type of work, and I have started
> some one-on-one meetings, but I'd like to find more interested
> collaborators. Anyone at Rightscale or Enstratus interested? Also are
> there other API implementers who are experts in how the APIs really
> work? Please join in finding the right solution here.
>
> Does this proposal sound like a workable solution? Any tweaks or other
> suggestions?
>
> Thanks,
> Anne
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] python 3 advocacy (was Re: Hiding complexity of paste config files from operators)

[Doug Hellmann]

On Mon, Jul 30, 2012 at 5:12 AM, Thierry Carrez wrote:

>
> > Assuming that the *-paste.ini files always need to be there, is there
> some way we could avoid requiring admins to edit these files, and instead
> make it more like editing the .conf files? For example, could the paste.ini
> files be generated from the corresponding .conf file as needed?
>
> I would not assume that *-paste.ini files always need to be there...
> Paste is a pain point if we are to support Python 3 one day, so it's
> also on the black list of the (still inexistant) OpenStack Python3
> advocacy group.
>

We exist, we're just not organized, yet. :-)

Who else is interested in contributing to Python 3 support in a future
version of OpenStack? Should we start planning for some discussions at the
Grizzly summit?

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Keyring support in openstack

[Doug Hellmann]

On Sun, Jul 29, 2012 at 1:37 AM, Bhuvaneswaran A  wrote:

> Team,
>
> As per patch https://review.openstack.org/#/c/9497/ we are adding
> keyring support for "openstack" client.  If password is not specified
> in command line or environment variable, the user is prompted to enter
> password. During this time, the password is stored in keyring. During
> next time, the password is read from keyring, instead of prompt. It is
> true, if password is not specified in command line or environment
> variable.
>
> This behavior is documented in this wiki page:
>   http://wiki.openstack.org/KeyringSupport
>
> If you have any comments, please let us know.
>

You've already answered several of my questions on the ticket, but I still
have some "usability" concerns.

How does the keyring system support a single person logging in using
multiple user accounts? For example, if I have an admin account and a
"regular" user, how do I switch between them based on the operations I need
to perform?

Is there a way to disable the behavior of having a password saved to a
keyring for a particular user, without uninstalling the python-keyring
package (and therefore disabling keyring support for all users)?

The wiki mentions the password being saved
using keyring.backend.UncryptedFileKeyring. Does that mean the password is
saved in cleartext? Is the file protected in some way besides filesystem
permissions?

The mention of one backend implies that there are others. Should we give
users a way to choose the backend, in case they have a preference?

How does the use of the keyring affect scripting using the command line
tool? Can a script access the keyring, or does it need to use the other
options?

In one review comment you mention a few desktop apps that know how to
manipulate the keyring to manage its contents. What about remote access via
ssh, where a desktop environment is not available? Does the keyring library
include tools for manipulating the file, or do we need to build our own? If
so, what tools would be needed?

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] Metering team meeting agenda for Thursday 26 July, 2012 @ 16:00 UTC

[Doug Hellmann]

On Thu, Jul 26, 2012 at 5:54 AM, Doug Hellmann
wrote:

> The metering project team holds regular meetings via IRC in
> #openstack-meeting, Thursdays at 1600 UTC <
> http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.
>
> Everyone is welcome to participate.
>
> Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda
>
> * Review last week's actions
> * Discuss priority of maintaining Essex support and find contributor to
> work on it if we are going to do it
> * Open discussion
>
> If you have any additional topics you would like to cover, please update
> the agenda in the wiki.
>
> Doug
>
>
==
#openstack-meeting Meeting
==


Meeting started by dhellmann at 16:01:14 UTC.  The full logs are
available at
http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-26-16.01.log.html
.



Meeting summary
---

* LINK: https://lists.launchpad.net/openstack/msg15134.html  (dhellmann,
  16:01:19)
* dhellmann to get some feedback now about the sorts of meters users
  want from the mailing list  (dhellmann, 16:03:13)
  * My message to list about meters:
https://lists.launchpad.net/openstack/msg15104.html  (dhellmann,
16:03:19)
  * I've had no feedback on the meters yet  (dhellmann, 16:03:25)
  * I don't have feedback at the specific meters yet but I do believe it
would be helpful to see a clear definition and colletion of the
meters and their meaning  (glauaguiar, 16:05:27)
  * LINK: http://wiki.openstack.org/EfficientMetering#Meters
(dhellmann, 16:06:08)
  * ACTION: dhellmann to open a ticket to add documentation about the
meters to the rst docs based on the wiki  (dhellmann, 16:09:40)

* jd to setup opa voting system to start on 26th and end on Aug 3rd
  (dhellmann, 16:14:45)

* dhellmann to open a bug and work on devstack integration  (dhellmann,
  16:18:00)
  * LINK: https://bugs.launchpad.net/ceilometer/+bug/1023972
(dhellmann, 16:18:08)
  * Have not started dev work, yet. Focusing on the API server first.
(dhellmann, 16:18:14)
  * ACTION: dhellmann to open a bug and work on devstack integration
(dhellmann, 16:19:31)

* dhellmann create a diagram of ceilometer architecture  (dhellmann,
  16:20:24)
  * No progress  (dhellmann, 16:20:29)
  * ACTION: dhellmann create a diagram of ceilometer architecture
(dhellmann, 16:20:45)
  * jd did one  (jd___, 16:21:34)
  * LINK: http://xlcloud.org/bin/view/Main/  (jd___, 16:22:25)

* dhellmann write a walk-through of setting up ceilometer and collecting
  data  (dhellmann, 16:25:54)
  * No progress  (dhellmann, 16:25:58)
  * ACTION: dhellmann open a ticket to write a walk-through of setting
up ceilometer and collecting data  (dhellmann, 16:26:36)

* lzyeval create a blueprint to add counter for nova-volume disk usage
  retrieval  (dhellmann, 16:29:48)
  * LINK:
https://blueprints.launchpad.net/nova/+spec/volume-usage-metering
(dhellmann, 16:32:36)

* dricco work and report about nova-volume metering notification
  https://blueprints.launchpad.net/nova/+spec/volume-usage-metering
  (dhellmann, 16:34:11)

* Discuss priority of maintaining Essex support and find contributor to
  work on it if we are going to do it  (dhellmann, 16:34:43)
  * ACTION: jtran investigate and report on the amount of work needed to
support metering essex  (dhellmann, 16:45:20)

* PTL election  (dhellmann, 16:45:33)

* Open discussion  (dhellmann, 16:47:44)



Meeting ended at 16:50:44 UTC.



Action items, by person
---

* dhellmann
  * dhellmann to open a ticket to add documentation about the meters to
the rst docs based on the wiki
  * dhellmann to open a bug and work on devstack integration
  * dhellmann create a diagram of ceilometer architecture
  * dhellmann open a ticket to write a walk-through of setting up
ceilometer and collecting data
* jtran
  * jtran investigate and report on the amount of work needed to support
metering essex



People present (lines said)
---

* dhellmann (124)
* jd___ (49)
* dricco (11)
* jtran (6)
* glauaguiar (5)
* Vincent_Hou (3)
* uvirtbot (2)
* openstack (2)



Generated by `MeetBot`_ 0.1.4
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [ceilometer] Metering team meeting agenda for Thursday 26 July, 2012 @ 16:00 UTC

[Doug Hellmann]

The metering project team holds regular meetings via IRC in
#openstack-meeting, Thursdays at 1600 UTC <
http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>.

Everyone is welcome to participate.

Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda

* Review last week's actions
* Discuss priority of maintaining Essex support and find contributor to
work on it if we are going to do it
* Open discussion

If you have any additional topics you would like to cover, please update
the agenda in the wiki.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [ceilometer] requesting feedback on priorities for metering data collection

[Doug Hellmann]

On Wed, Jul 25, 2012 at 10:45 AM, Aguiar, Glaucimar (Brazil R&D-ECL) <
glaucimar.agu...@hp.com> wrote:

>  Doug,
>
> ** **
>
> Thanks for sending this out. I am interested in this project and as I
> could not have it working yet with my Essex release of OpenStack
>

Some of the pieces work with Essex, but the actual daemons don't, yet. We
are planning to discuss this at our meeting this Thursday (
http://wiki.openstack.org/Meetings/MeteringAgenda).


> I would like to better understand the metrics collected you mention below.
> 
>
> I have being reading the documentation and code extensively and would like
> to count on your help to better understand some aspects of the ceilometer.
> 
>
> ** **
>
> Some questions below, sorry if they sound so easy for you but if you could
> help me with these, this would be of a great help for me.
>

These are good questions, thanks for bringing them up!


> 
>
> 
>
> When you say CPU usage meters, does it mean ceilometer is already capable
> of reporting the cpu consumption over an hour?
>

We are currently polling libvirt periodically to get the total CPU time for
an instance. The frequency of polling is adjustable, but I think the
default is 10 minutes. The value returned by libvirt represents the total
time for the instance, but a client could pull the data and perform the
calculation over a period of time. The API we have defined allows for
querying within a time range, for example.


> Does this take into account the number of CPUs of the instance or this
> should be derived from the instance data?
>

The number of CPUs in an instance is part of the metadata associated with
the counter, but does not enter into the calculation of CPU-time as far as
I know. We ask libvirt for that info, and just use what we're given.


> 
>
> ** **
>
> Thank you very much for any help in increasing my understanding of the
> behavior of this component. I can also contact you in IRC if you prefer.
>

I hope my answers help,
Doug


> 
>
> ** **
>
> Regards,
>
> Glaucimar Aguiar
>
> ** **
>
> ** **
>
> *From:* openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net[mailto:
> openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net] *On Behalf
> Of *Doug Hellmann
> *Sent:* quarta-feira, 25 de julho de 2012 11:24
> *To:* openstack-operat...@lists.openstack.org;
> openstack@lists.launchpad.net
> *Subject:* [Openstack] [ceilometer] requesting feedback on priorities for
> metering data collection
>
> ** **
>
> When the ceilometer project started after the Folsom summit, we compiled a
> list of metrics to be collected [1]. We have reached a stage in the project
> where we are ready to start implementing more of these meters, and would
> like some input from the rest of the community about priorities. We have
> implemented the instance counter, disk I/O, and CPU usage meters, and have
> begun working on collecting some of the network statistics. If you are
> interested in using ceilometer for tracking usage in your cloud deployment,
> please let us know which other metrics you will find especially useful.***
> *
>
> ** **
>
> Thanks,
>
> Doug
>
> ** **
>
> 1 - http://wiki.openstack.org/EfficientMetering#Meters
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [ceilometer] requesting feedback on priorities for metering data collection

[Doug Hellmann]

When the ceilometer project started after the Folsom summit, we compiled a
list of metrics to be collected [1]. We have reached a stage in the project
where we are ready to start implementing more of these meters, and would
like some input from the rest of the community about priorities. We have
implemented the instance counter, disk I/O, and CPU usage meters, and have
begun working on collecting some of the network statistics. If you are
interested in using ceilometer for tracking usage in your cloud deployment,
please let us know which other metrics you will find especially useful.

Thanks,
Doug

1 - http://wiki.openstack.org/EfficientMetering#Meters
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] best practices for merging common into specific projects

[Doug Hellmann]

On Mon, Jul 23, 2012 at 12:00 PM, Thierry Carrez wrote:

> Doug Hellmann wrote:
> >
> > On Wed, Jul 18, 2012 at 7:00 PM, Thierry Carrez  > <mailto:thie...@openstack.org>> wrote:
> >
> > Mark McLoughlin wrote:
> > >> Making our multiple projects converge onto consolidated and
> > >> well-accepted APIs is a bit painful work, but it is a
> prerequisite to
> > >> turning openstack-common into a proper library (or set of
> libraries).
> > >>
> > >> I'd say the whole thing suffers from not having a proper
> > >> team/leader/coordinator dedicated to it: relying on existing,
> > >> overstretched PTLs to lead that effort might not be the fastest
> path.
> > >
> > > While I was on vacation, I read in the weekly newsletter:
> > >
> > >   "It developed into a request for leadership for openstack-common"
> > >
> > > and was like "WTF do you call the work that e.g. I, Jason, Russell
> and
> > > Doug have been doing?"
> > >
> > > But I see your point is a little different - you feel there should
> > be an
> > > elected/appointed "PTL without a PPB vote" or whatever to
> > represent the
> > > project. I guess that could help clarify things since it's what
> folks
> > > are used to with other projects.
> >
> > Right. So far we said that openstack-common was driven by "all the
> > PTLs", but that didn't prove particularly fast and efficient. Having
> a
> > clear face associated with it, someone specific taking the "lead" on
> > this project, will, I think, help a bit in getting to the next step.
> >
> >
> > Sorry if this rekindles old arguments, but could someone summarize the
> > reasons for an openstack-common "PTL" without voting rights? I would
> > have defaulted to giving them a vote *especially* because the code in
> > common is, well, common to all of the projects.
>
> So far, the PPB considered openstack-common to be driven by "all PTLs",
> so it didn't have a specific PTL.
>
> As far as future governance is concerned (technical committee of the
> Foundation), openstack-common would technically be considered a
> supporting library (rather than a core project) -- those can have leads,
> but those do not get granted an automatic TC seat.
>

OK, I can see the distinction there. I think the project needs an official
leader, even if we don't call them a PTL in the sense meant for other
projects. And I would expect anyone willing to take on the PTL role for
common to be qualified to run for one of the open positions on the new TC,
if they wanted to participate there.


>
> [ Avoiding the need to distinguish between "worthy" and "unworthy"
> projects leads was one of the many reasons why I preferred the TC to be
> completely directly-elected. ]


That does make sense.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] best practices for merging common into specific projects

[Doug Hellmann]

On Wed, Jul 18, 2012 at 7:00 PM, Thierry Carrez wrote:

> Mark McLoughlin wrote:
> >> Making our multiple projects converge onto consolidated and
> >> well-accepted APIs is a bit painful work, but it is a prerequisite to
> >> turning openstack-common into a proper library (or set of libraries).
> >>
> >> I'd say the whole thing suffers from not having a proper
> >> team/leader/coordinator dedicated to it: relying on existing,
> >> overstretched PTLs to lead that effort might not be the fastest path.
> >
> > While I was on vacation, I read in the weekly newsletter:
> >
> >   "It developed into a request for leadership for openstack-common"
> >
> > and was like "WTF do you call the work that e.g. I, Jason, Russell and
> > Doug have been doing?"
> >
> > But I see your point is a little different - you feel there should be an
> > elected/appointed "PTL without a PPB vote" or whatever to represent the
> > project. I guess that could help clarify things since it's what folks
> > are used to with other projects.
>
> Right. So far we said that openstack-common was driven by "all the
> PTLs", but that didn't prove particularly fast and efficient. Having a
> clear face associated with it, someone specific taking the "lead" on
> this project, will, I think, help a bit in getting to the next step.


Sorry if this rekindles old arguments, but could someone summarize the
reasons for an openstack-common "PTL" without voting rights? I would have
defaulted to giving them a vote *especially* because the code in common is,
well, common to all of the projects.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Fw:Re: Questions about ceilometer

[Doug Hellmann]

 callback=self.compute_handler.notify)
>
>   # Set ourselves up as a separate worker for the metering data,
>
> On Mon, Jul 16, 2012 at 7:41 PM, 张家龙  wrote:
>
>> Hi,Doug,
>>It`s a bad news that the patch (
>> https://bugs.launchpad.net/ceilometer/+bug/1024563) has been removed .
>> This page showed "page not found".
>>   Anyway,Thanks for your help.
>>
>>  * *
>> --
>>  Best Regards
>>
>> ZhangJialong
>>  * *
>>  * * * *
>>
>>
>>
>>  -- Original --
>>  *From: * "Doug Hellmann";
>>  *Date: * Mon, Jul 16, 2012 10:31 PM
>>  *To: * "张家龙"; * *
>>  *Cc: * "Julien Danjou"; "openstack"<
>> openstack@lists.launchpad.net>; * *
>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>
>> An earlier message in this thread points to a bug (
>> https://bugs.launchpad.net/ceilometer/+bug/1024563) which has a review
>> patch attached.
>>
>>  On Mon, Jul 16, 2012 at 10:03 AM, 张家龙  wrote:
>>
>>> Dear Doug,
>>> Thanks for reply.
>>> While ,where to find the patch posted John ? If it`s
>>> prossable,please point it out,Thanks.
>>> And it`s my pleasure to be the first one to receive the message
>>> about you fix this problems.
>>>
>>> Good luck !
>>>  * *
>>> --
>>>  Best Regards
>>>
>>> ZhangJialong
>>>  * *
>>>  * * * *
>>>
>>>
>>>
>>>  -- Original --
>>>  *From: * "Doug Hellmann";
>>>  *Date: * Mon, Jul 16, 2012 08:55 PM
>>>   *To: * "张家龙"; * *
>>>  *Cc: * "Julien Danjou"; "openstack"<
>>> openstack@lists.launchpad.net>; * *
>>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>>
>>>
>>>
>>> On Sat, Jul 14, 2012 at 10:02 PM, 张家龙  wrote:
>>>
>>>> Hi,all.
>>>> Sorry for late reply.
>>>> Until now, I not test folsom. So, I`am not sure how it work in
>>>> folsom .
>>>> The follow is my qpid config file:
>>>> http://pastebin.com/sBXm6k7z
>>>> And Doug writed set driver to
>>>> "nova.openstack.common.notifier.rabbit_notifier",
>>>> while , I cannot found this class or modules in exsse.
>>>>
>>>
>>>  The notifier classes moved in Folsom, so that's the setting you would
>>> need if you were working with Folsom.
>>>
>>> I'm traveling this week, so I won't be able to set up a test environment
>>> with Essex or qpid until next week some time.
>>>
>>>  Based on rereading the configuration files you posted, I do suspect
>>> that this is a problem with the code, rather than your configuration. You
>>> might want to try the patch John posted above. I don't think that's the
>>> right long-term solution, but if it gets you past this situation we can
>>> find a better solution later.
>>>
>>>  Doug
>>>
>>>
>>>>
>>>>  * *
>>>> --
>>>>  Best Regards
>>>>
>>>> ZhangJialong
>>>>  * *
>>>>  * * * *
>>>>
>>>>
>>>>
>>>>  -- Original --
>>>>  *From: * "Doug Hellmann";
>>>>  *Date: * Sat, Jul 14, 2012 00:17 AM
>>>>  *To: * "张家龙"; * *
>>>>  *Cc: * "Julien Danjou"; "openstack"<
>>>> openstack@lists.launchpad.net>; * *
>>>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>>>
>>>>
>>>>
>>>>  On Fri, Jul 13, 2012 at 11:36 AM, 张家龙  wrote:
>>>>
>>>>> Dear Doug,
>>>>> I`m use Qpid instead of Rabbit .
>>>>> Did it cause the error ?
>>>>>
>>>>
>>>>   Qpid should work, but I haven't been testing with that so you might
>>>> have found a bug.
>>>>
>>>>  There are (at least) two differences between your setup and what I
>>>> normally use: Essex and qpid. Have you tried running against a folsom
>>>> install? That would at least tell us if the qpid configuration is correct
>>>> or if the problem is related to Essex.
>>>>

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

I have been doing all of my development with Folsom HEAD because DreamHost
plans to deploy ceilometer with our Folsom-based cloud. I know Julien has
done some work on Essex compatibility, but I don't know where that stands
or if he is still working on it.

Doug

On Mon, Jul 16, 2012 at 10:12 AM, Aguiar, Glaucimar (Brazil R&D-ECL) <
glaucimar.agu...@hp.com> wrote:

>  Doug,
>
> ** **
>
> I will start ceilometer installation now. Would you recommend installing
> Folsom then instead of essex? (I already have essex installed so, this is
> the reason for asking).
>
> ** **
>
> Thanks,
>
> Glaucimar Aguiar
>
> ** **
>
> ** **
>
> *From:* openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net[mailto:
> openstack-bounces+glaucimar.aguiar=hp@lists.launchpad.net] *On Behalf
> Of *Doug Hellmann
> *Sent:* segunda-feira, 16 de julho de 2012 09:55
> *To:* 张家龙
> *Cc:* openstack
>
> *Subject:* Re: [Openstack] Questions about ceilometer
>
> ** **
>
> ** **
>
> On Sat, Jul 14, 2012 at 10:02 PM, 张家龙  wrote:
>
> Hi,all.
> Sorry for late reply.
> Until now, I not test folsom. So, I`am not sure how it work in folsom .
> The follow is my qpid config file:
> *http://pastebin.com/sBXm6k7z*
> And Doug writed set driver to
> "nova.openstack.common.notifier.rabbit_notifier",
> while , I cannot found this class or modules in exsse.
>
> ** **
>
> The notifier classes moved in Folsom, so that's the setting you would need
> if you were working with Folsom.
>
>
> I'm traveling this week, so I won't be able to set up a test environment
> with Essex or qpid until next week some time.
>
> ** **
>
> Based on rereading the configuration files you posted, I do suspect that
> this is a problem with the code, rather than your configuration. You might
> want to try the patch John posted above. I don't think that's the right
> long-term solution, but if it gets you past this situation we can find a
> better solution later.****
>
> ** **
>
> Doug
>
>  
>
> ** **
>
> --
>
> Best Regards
>
>  
>
> ZhangJialong
>
>  
>
>  
>
>  
>
> -- Original --
>
> *From: * "Doug Hellmann";
>
> *Date: * Sat, Jul 14, 2012 00:17 AM
>
> *To: * "张家龙"; 
>
> *Cc: * "Julien Danjou"; "openstack"<
> openstack@lists.launchpad.net>; 
>
> *Subject: * Re: [Openstack] Questions about ceilometer
>
>  
>
> ** **
>
> On Fri, Jul 13, 2012 at 11:36 AM, 张家龙  wrote:
>
> Dear Doug,
> I`m use Qpid instead of Rabbit .
> Did it cause the error ?
>
>  ** **
>
> Qpid should work, but I haven't been testing with that so you might have
> found a bug.
>
> ** **
>
> There are (at least) two differences between your setup and what I
> normally use: Essex and qpid. Have you tried running against a folsom
> install? That would at least tell us if the qpid configuration is correct
> or if the problem is related to Essex.
>
> ** **
>
> Doug
>
>  
>
> In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
> are here:
>
>
> *http://pastebin.com/sW5d8eRv *
> *http://pastebin.com/D5GMkLsb *
> *http://pastebin.com/u5vH22Lh *
> Were there some errors in my config files?
>
> Waiting your reply. Thanks.
>
> --
>
> Best Regards
>
>  
>
> ZhangJialong
>
>  
>
>  
>
>  
>
> -- Original --
>
> *From: * "Doug Hellmann";
>
> *Date: * Fri, Jul 13, 2012 11:28 PM
>
> *To: * "张家龙"; "Julien Danjou"; **
> **
>
> *Cc: * "openstack"; 
>
> *Subject: * Re: [Openstack] Questions about ceilometer
>
>  
>
> ** **
>
> On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
> ** **
>
> On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:
>
>   Dear all,
>
> As the project named ceilometer appeared,I paid close attention to it.
> According to the docs of ceilometer,I deploied it in openstack exsse
> environment.
> While,I cannot start the ceilometer collector and agent.
>
> The follows are my operations.
>
> 1.Install openstack nova ,mongodb and ceilometer.
> 2.configurate nova ,mongodb and ceilometer
>
> The

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

An earlier message in this thread points to a bug (
https://bugs.launchpad.net/ceilometer/+bug/1024563) which has a review
patch attached.

On Mon, Jul 16, 2012 at 10:03 AM, 张家龙  wrote:

> Dear Doug,
> Thanks for reply.
> While ,where to find the patch posted John ? If it`s prossable,please
> point it out,Thanks.
> And it`s my pleasure to be the first one to receive the message about
> you fix this problems.
>
> Good luck !
> **
> --
> Best Regards
>
> ZhangJialong
> **
> 
>
>
>
> ------ Original --
> *From: * "Doug Hellmann";
> *Date: * Mon, Jul 16, 2012 08:55 PM
> *To: * "张家龙"; **
> *Cc: * "Julien Danjou"; "openstack"<
> openstack@lists.launchpad.net>; **
> *Subject: * Re: [Openstack] Questions about ceilometer
>
>
>
> On Sat, Jul 14, 2012 at 10:02 PM, 张家龙  wrote:
>
>> Hi,all.
>> Sorry for late reply.
>> Until now, I not test folsom. So, I`am not sure how it work in folsom
>> .
>> The follow is my qpid config file:
>> http://pastebin.com/sBXm6k7z
>> And Doug writed set driver to
>> "nova.openstack.common.notifier.rabbit_notifier",
>> while , I cannot found this class or modules in exsse.
>>
>
>  The notifier classes moved in Folsom, so that's the setting you would
> need if you were working with Folsom.
>
> I'm traveling this week, so I won't be able to set up a test environment
> with Essex or qpid until next week some time.
>
>  Based on rereading the configuration files you posted, I do suspect that
> this is a problem with the code, rather than your configuration. You might
> want to try the patch John posted above. I don't think that's the right
> long-term solution, but if it gets you past this situation we can find a
> better solution later.
>
>  Doug
>
>
>>
>>  * *
>> --
>>  Best Regards
>>
>> ZhangJialong
>>  * *
>>  * * * *
>>
>>
>>
>>  -- Original --
>>  *From: * "Doug Hellmann";
>>  *Date: * Sat, Jul 14, 2012 00:17 AM
>>  *To: * "张家龙"; * *
>>  *Cc: * "Julien Danjou"; "openstack"<
>> openstack@lists.launchpad.net>; * *
>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>
>>
>>
>>  On Fri, Jul 13, 2012 at 11:36 AM, 张家龙  wrote:
>>
>>> Dear Doug,
>>> I`m use Qpid instead of Rabbit .
>>> Did it cause the error ?
>>>
>>
>>   Qpid should work, but I haven't been testing with that so you might
>> have found a bug.
>>
>>  There are (at least) two differences between your setup and what I
>> normally use: Essex and qpid. Have you tried running against a folsom
>> install? That would at least tell us if the qpid configuration is correct
>> or if the problem is related to Essex.
>>
>>  Doug
>>
>>
>>> In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
>>> are here:
>>>
>>> http://pastebin.com/sW5d8eRv
>>> http://pastebin.com/D5GMkLsb
>>> http://pastebin.com/u5vH22Lh
>>> Were there some errors in my config files?
>>>
>>> Waiting your reply. Thanks.
>>>  * *
>>> --
>>>  Best Regards
>>>
>>> ZhangJialong
>>>  * *
>>>  * * * *
>>>
>>>
>>>
>>>   -- Original --
>>>  *From: * "Doug Hellmann";
>>>  *Date: * Fri, Jul 13, 2012 11:28 PM
>>>   *To: * "张家龙"; "Julien Danjou";
>>> * *
>>>  *Cc: * "openstack"; * *
>>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>>
>>>
>>>
>>> On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann <
>>> doug.hellm...@dreamhost.com> wrote:
>>>
>>>>
>>>>
>>>>   On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:
>>>>
>>>>>   Dear all,
>>>>>
>>>>> As the project named ceilometer appeared,I paid close attention to
>>>>> it.
>>>>> According to the docs of ceilometer,I deploied it in openstack
>>>>> exsse environment.
>>>>> While,I cannot start the ceilometer collector and agent.
>>>>>
>>>>> The follows are my operations.
>>>>>

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

On Sat, Jul 14, 2012 at 10:02 PM, 张家龙  wrote:

> Hi,all.
> Sorry for late reply.
> Until now, I not test folsom. So, I`am not sure how it work in folsom .
> The follow is my qpid config file:
> http://pastebin.com/sBXm6k7z
> And Doug writed set driver to
> "nova.openstack.common.notifier.rabbit_notifier",
> while , I cannot found this class or modules in exsse.
>

The notifier classes moved in Folsom, so that's the setting you would need
if you were working with Folsom.

I'm traveling this week, so I won't be able to set up a test environment
with Essex or qpid until next week some time.

Based on rereading the configuration files you posted, I do suspect that
this is a problem with the code, rather than your configuration. You might
want to try the patch John posted above. I don't think that's the right
long-term solution, but if it gets you past this situation we can find a
better solution later.

Doug


>
> **
> --
>  Best Regards
>
> ZhangJialong
>  **
>  ** **
>
>
>
>  -- Original --
>  *From: * "Doug Hellmann";
>  *Date: * Sat, Jul 14, 2012 00:17 AM
>  *To: * "张家龙"; **
>  *Cc: * "Julien Danjou"; "openstack"<
> openstack@lists.launchpad.net>; **
>  *Subject: * Re: [Openstack] Questions about ceilometer
>
>
>
> On Fri, Jul 13, 2012 at 11:36 AM, 张家龙  wrote:
>
>> Dear Doug,
>> I`m use Qpid instead of Rabbit .
>> Did it cause the error ?
>>
>
>  Qpid should work, but I haven't been testing with that so you might have
> found a bug.
>
>  There are (at least) two differences between your setup and what I
> normally use: Essex and qpid. Have you tried running against a folsom
> install? That would at least tell us if the qpid configuration is correct
> or if the problem is related to Essex.
>
>  Doug
>
>
>> In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
>> are here:
>>
>> http://pastebin.com/sW5d8eRv
>> http://pastebin.com/D5GMkLsb
>> http://pastebin.com/u5vH22Lh
>> Were there some errors in my config files?
>>
>> Waiting your reply. Thanks.
>>  * *
>> --
>>  Best Regards
>>
>> ZhangJialong
>>  * *
>>  * * * *
>>
>>
>>
>>  -- Original --
>>  *From: * "Doug Hellmann";
>>  *Date: * Fri, Jul 13, 2012 11:28 PM
>>  *To: * "张家龙"; "Julien Danjou";
>> * *
>>  *Cc: * "openstack"; * *
>>  *Subject: * Re: [Openstack] Questions about ceilometer
>>
>>
>>
>> On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann <
>> doug.hellm...@dreamhost.com> wrote:
>>
>>>
>>>
>>>  On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:
>>>
>>>>  Dear all,
>>>>
>>>> As the project named ceilometer appeared,I paid close attention to
>>>> it.
>>>> According to the docs of ceilometer,I deploied it in openstack
>>>> exsse environment.
>>>> While,I cannot start the ceilometer collector and agent.
>>>>
>>>> The follows are my operations.
>>>>
>>>> 1.Install openstack nova ,mongodb and ceilometer.
>>>> 2.configurate nova ,mongodb and ceilometer
>>>>
>>>> The /etc/nova/nova.conf file is here:
>>>> http://pastebin.com/sW5d8eRv
>>>>
>>>> Here is the /etc/ceilometer-collector.conf file is here:
>>>> http://pastebin.com/u5vH22Lh
>>>>
>>>> And the /etc/mongodb.conf is here:
>>>> http://pastebin.com/D5GMkLsb
>>>>
>>>> 3.Start openstack nova ,mongodb
>>>>
>>>> 4.Then I start the ceilometer-collector
>>>> /usr/bin/ceilometer-collector start
>>>> While,some errors occurred:
>>>>
>>>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>>>> storage engine EntryPoint.parse('mongodb =
>>>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>>>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>>>> storage engine EntryPoint.parse('mongodb =
>>>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>>>> 2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
>>>> connecting to MongoDB on localhost:27017
>>>> 2012-07-11 05:25:35 INFO ceilo

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

I *just* ran into a problem today triggered by some code moving around in
nova and common. I had to set my driver to
"nova.openstack.common.notifier.rabbit_notifier" (in both files).

I didn't see any errors from that until I tried to launch an instance, so
it doesn't seem to prevent any of the nova services from starting up.

Doug

On Fri, Jul 13, 2012 at 1:17 PM, TRAN, JOHN  wrote:

> I'm having the same problem.  I'm running against nova trunk and I have
> rabbit notifier enabled.
>
>
>
> /etc/nova/nova.conf:notification_driver=nova.notifier.rabbit_notifier
>
> /home/stack/ceilometer-collector.conf:notification_driver=nova.notifier.rabbit_notifier
>
> 
> From: 
> openstack-bounces+jhtran=att@lists.launchpad.net[openstack-bounces+jhtran=
> att@lists.launchpad.net] on behalf of 张家龙 [zhan...@awcloud.com]
> Sent: Friday, July 13, 2012 8:36 AM
> To: Doug Hellmann; Julien Danjou
> Cc: openstack
> Subject: Re: [Openstack] Questions about ceilometer
>
> Dear Doug,
> I`m use Qpid instead of Rabbit .
> Did it cause the error ?
> In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
> are here:
> http://pastebin.com/sW5d8eRv
> http://pastebin.com/D5GMkLsb
> http://pastebin.com/u5vH22Lh
> Were there some errors in my config files?
>
> Waiting your reply. Thanks.
> --
> Best Regards
>
> ZhangJialong
>
>
>
> -- Original --
> From:  "Doug Hellmann";
> Date:  Fri, Jul 13, 2012 11:28 PM
> To:  "张家龙"; "Julien Danjou";
> Cc:  "openstack";
> Subject:  Re: [Openstack] Questions about ceilometer
>
>
>
> On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com<mailto:doug.hellm...@dreamhost.com>> wrote:
>
>
> On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  zhan...@awcloud.com>> wrote:
> Dear all,
>
> As the project named ceilometer appeared,I paid close attention to it.
> According to the docs of ceilometer,I deploied it in openstack exsse
> environment.
> While,I cannot start the ceilometer collector and agent.
>
> The follows are my operations.
>
> 1.Install openstack nova ,mongodb and ceilometer.
> 2.configurate nova ,mongodb and ceilometer
>
> The /etc/nova/nova.conf file is here:
> http://pastebin.com/sW5d8eRv
>
> Here is the /etc/ceilometer-collector.conf file is here:
> http://pastebin.com/u5vH22Lh
>
> And the /etc/mongodb.conf is here:
> http://pastebin.com/D5GMkLsb
>
> 3.Start openstack nova ,mongodb
>
> 4.Then I start the ceilometer-collector
> /usr/bin/ceilometer-collector start
> While,some errors occurred:
>
> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
> engine EntryPoint.parse('mongodb =
> ceilometer.storage.impl_mongodb:MongoDBStorage')
> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
> engine EntryPoint.parse('mongodb =
> ceilometer.storage.impl_mongodb:MongoDBStorage')
> 2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
> connecting to MongoDB on localhost:27017
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> attempting to load notification handler for
> ceilometer.collector.compute:instance
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.create.end events
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.exists events
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.delete.start events
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/site-packages/eventlet/hubs/poll.py", line
> 97, in wait
> readers.get(fileno, noop).cb(fileno)
>   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py",
> line 192, in main
> result = function(*args, **kwargs)
>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 101,
> in run_server
> server.start()
>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 162,
> in start
> self.manager.init_host()
>   File
> "/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py",
> line 69, in init_host
> topic='%s.info<http://s.info>' %
> flags.FLAGS.notification_topics[0],
>   File
> "/usr/lib/python2.6/site-packages/nova/

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

On Fri, Jul 13, 2012 at 11:36 AM, 张家龙  wrote:

> Dear Doug,
> I`m use Qpid instead of Rabbit .
> Did it cause the error ?
>

Qpid should work, but I haven't been testing with that so you might have
found a bug.

There are (at least) two differences between your setup and what I normally
use: Essex and qpid. Have you tried running against a folsom install? That
would at least tell us if the qpid configuration is correct or if the
problem is related to Essex.

Doug


> In addition,my nova.conf,mongodb.conf and ceilometer-collector.conf
> are here:
> http://pastebin.com/sW5d8eRv
> http://pastebin.com/D5GMkLsb
> http://pastebin.com/u5vH22Lh
> Were there some errors in my config files?
>
> Waiting your reply. Thanks.
> **
> --
> Best Regards
>
> ZhangJialong
> **
> 
>
>
>
> -- Original --
> *From: * "Doug Hellmann";
> *Date: * Fri, Jul 13, 2012 11:28 PM
> *To: * "张家龙"; "Julien Danjou"; **
> *Cc: * "openstack"; **
> *Subject: * Re: [Openstack] Questions about ceilometer
>
>
>
> On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>>
>>
>>  On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:
>>
>>>  Dear all,
>>>
>>> As the project named ceilometer appeared,I paid close attention to
>>> it.
>>> According to the docs of ceilometer,I deploied it in openstack exsse
>>> environment.
>>> While,I cannot start the ceilometer collector and agent.
>>>
>>> The follows are my operations.
>>>
>>> 1.Install openstack nova ,mongodb and ceilometer.
>>> 2.configurate nova ,mongodb and ceilometer
>>>
>>> The /etc/nova/nova.conf file is here:
>>> http://pastebin.com/sW5d8eRv
>>>
>>> Here is the /etc/ceilometer-collector.conf file is here:
>>> http://pastebin.com/u5vH22Lh
>>>
>>> And the /etc/mongodb.conf is here:
>>> http://pastebin.com/D5GMkLsb
>>>
>>> 3.Start openstack nova ,mongodb
>>>
>>> 4.Then I start the ceilometer-collector
>>> /usr/bin/ceilometer-collector start
>>> While,some errors occurred:
>>>
>>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>>> storage engine EntryPoint.parse('mongodb =
>>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>>> storage engine EntryPoint.parse('mongodb =
>>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>>> 2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
>>> connecting to MongoDB on localhost:27017
>>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>>> attempting to load notification handler for
>>> ceilometer.collector.compute:instance
>>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>>> subscribing instance handler to compute.instance.create.end events
>>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>>> subscribing instance handler to compute.instance.exists events
>>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>>> subscribing instance handler to compute.instance.delete.start events
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.6/site-packages/eventlet/hubs/poll.py",
>>> line 97, in wait
>>> readers.get(fileno, noop).cb(fileno)
>>>   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py",
>>> line 192, in main
>>> result = function(*args, **kwargs)
>>>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 101,
>>> in run_server
>>> server.start()
>>>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 162,
>>> in start
>>> self.manager.init_host()
>>>   File
>>> "/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py",
>>> line 69, in init_host
>>> topic='%s.info' % flags.FLAGS.notification_topics[0],
>>>   File
>>> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 867,
>>> in __getattr__
>>> return self._substitute(self._get(name))
>>>   File
>>> &quo

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

On Fri, Jul 13, 2012 at 10:04 AM, Doug Hellmann  wrote:

>
>
> On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:
>
>> Dear all,
>>
>> As the project named ceilometer appeared,I paid close attention to it.
>> According to the docs of ceilometer,I deploied it in openstack exsse
>> environment.
>> While,I cannot start the ceilometer collector and agent.
>>
>> The follows are my operations.
>>
>> 1.Install openstack nova ,mongodb and ceilometer.
>> 2.configurate nova ,mongodb and ceilometer
>>
>> The /etc/nova/nova.conf file is here:
>> http://pastebin.com/sW5d8eRv
>>
>> Here is the /etc/ceilometer-collector.conf file is here:
>> http://pastebin.com/u5vH22Lh
>>
>> And the /etc/mongodb.conf is here:
>> http://pastebin.com/D5GMkLsb
>>
>> 3.Start openstack nova ,mongodb
>>
>> 4.Then I start the ceilometer-collector
>> /usr/bin/ceilometer-collector start
>> While,some errors occurred:
>>
>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>> storage engine EntryPoint.parse('mongodb =
>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb
>> storage engine EntryPoint.parse('mongodb =
>> ceilometer.storage.impl_mongodb:MongoDBStorage')
>> 2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
>> connecting to MongoDB on localhost:27017
>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>> attempting to load notification handler for
>> ceilometer.collector.compute:instance
>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>> subscribing instance handler to compute.instance.create.end events
>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>> subscribing instance handler to compute.instance.exists events
>> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
>> subscribing instance handler to compute.instance.delete.start events
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.6/site-packages/eventlet/hubs/poll.py", line
>> 97, in wait
>> readers.get(fileno, noop).cb(fileno)
>>   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py",
>> line 192, in main
>> result = function(*args, **kwargs)
>>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 101,
>> in run_server
>> server.start()
>>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 162,
>> in start
>> self.manager.init_host()
>>   File
>> "/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py",
>> line 69, in init_host
>> topic='%s.info' % flags.FLAGS.notification_topics[0],
>>   File
>> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 867,
>> in __getattr__
>> return self._substitute(self._get(name))
>>   File
>> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 1070,
>> in _get
>> info = self._get_opt_info(name, group)
>>   File
>> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 1161,
>> in _get_opt_info
>> raise NoSuchOptError(opt_name, group)
>> NoSuchOptError: no such option: notification_topics
>> Removing descriptor: 12
>>
>
>> If anyone can help me ? Waiting your reply. Thanks !
>>
>
> It looks like we have a problem with the way we are loading nova's
> settings under Essex. I know some of the flags and cfg code was moved
> around and changed between Essex and Folsom as part of it moved into
> openstack-common. Julien, you're more familiar with Essex than I am, do you
> have any ideas about this problem?
>

I think I left out a configuration step in the instructions at
http://ceilometer.readthedocs.org/en/latest/install.html.

Did you configure your system to use the Rabbit notifier?

Doug


>
> Doug
>
>
>>
>>  **
>> --
>>  Best Regards
>>
>> ZhangJialong
>>  **
>>  ** **
>>
>>
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Questions about ceilometer

[Doug Hellmann]

On Thu, Jul 12, 2012 at 9:42 PM, 张家龙  wrote:

> Dear all,
>
> As the project named ceilometer appeared,I paid close attention to it.
> According to the docs of ceilometer,I deploied it in openstack exsse
> environment.
> While,I cannot start the ceilometer collector and agent.
>
> The follows are my operations.
>
> 1.Install openstack nova ,mongodb and ceilometer.
> 2.configurate nova ,mongodb and ceilometer
>
> The /etc/nova/nova.conf file is here:
> http://pastebin.com/sW5d8eRv
>
> Here is the /etc/ceilometer-collector.conf file is here:
> http://pastebin.com/u5vH22Lh
>
> And the /etc/mongodb.conf is here:
> http://pastebin.com/D5GMkLsb
>
> 3.Start openstack nova ,mongodb
>
> 4.Then I start the ceilometer-collector
> /usr/bin/ceilometer-collector start
> While,some errors occurred:
>
> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
> engine EntryPoint.parse('mongodb =
> ceilometer.storage.impl_mongodb:MongoDBStorage')
> 2012-07-11 05:25:35 INFO ceilometer.storage [-] Loaded mongodb storage
> engine EntryPoint.parse('mongodb =
> ceilometer.storage.impl_mongodb:MongoDBStorage')
> 2012-07-11 05:25:35 INFO ceilometer.storage.impl_mongodb [-]
> connecting to MongoDB on localhost:27017
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> attempting to load notification handler for
> ceilometer.collector.compute:instance
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.create.end events
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.exists events
> 2012-07-11 05:25:35 INFO ceilometer.collector.dispatcher [-]
> subscribing instance handler to compute.instance.delete.start events
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/site-packages/eventlet/hubs/poll.py", line
> 97, in wait
> readers.get(fileno, noop).cb(fileno)
>   File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py",
> line 192, in main
> result = function(*args, **kwargs)
>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 101,
> in run_server
> server.start()
>   File "/usr/lib/python2.6/site-packages/nova/service.py", line 162,
> in start
> self.manager.init_host()
>   File
> "/usr/lib/python2.6/site-packages/ceilometer-0-py2.6.egg/ceilometer/collector/manager.py",
> line 69, in init_host
> topic='%s.info' % flags.FLAGS.notification_topics[0],
>   File
> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 867,
> in __getattr__
> return self._substitute(self._get(name))
>   File
> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 1070,
> in _get
> info = self._get_opt_info(name, group)
>   File
> "/usr/lib/python2.6/site-packages/nova/openstack/common/cfg.py", line 1161,
> in _get_opt_info
> raise NoSuchOptError(opt_name, group)
> NoSuchOptError: no such option: notification_topics
> Removing descriptor: 12
>

> If anyone can help me ? Waiting your reply. Thanks !
>

It looks like we have a problem with the way we are loading nova's settings
under Essex. I know some of the flags and cfg code was moved around and
changed between Essex and Folsom as part of it moved into openstack-common.
Julien, you're more familiar with Essex than I am, do you have any ideas
about this problem?

Doug


>
>  **
> --
>  Best Regards
>
> ZhangJialong
>  **
>  ** **
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] Ceilometer PTL Election Process - Call for candidate

[Doug Hellmann]

I am running for PTL for the ceilometer project. I have posted some
information about myself and my thoughts for the project to the wiki under
http://wiki.openstack.org/EfficientMetering/PTLElectionProcess/DougHellmann

Doug

On Wed, Jul 11, 2012 at 8:24 AM, Nick Barcet wrote:

> As voted on the July 5th meeting the Ceilometer team members will soon
> vote on electing it's project team lead (PTL). Even though Ceilometer is
> not yet an official OpenStack project, we are applying for it, so we
> should be following the standard process as much as possible. The
> details of the process we will be following is described at [1].
>
> This email is a formal call for candidates, which is step one of the
> process.  Candidates for the PTL role must have contributed either to
> the source or to the wiki pages for Ceilometer prior to the call for
> candidates. The list that we assembled can be found on the above
> mentioned wiki page, but in the event we have missed someone, please shout.
>
> Candidates should, before July 24th:
> 1/ declare themselves on this mailing list
> 2/ add their name on the wiki page
>
> [1] http://wiki.openstack.org/EfficientMetering/PTLElectionProcess
>
> Cheers,
> --
> Nick Barcet 
> aka: nijaba, nicolas
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PEP8 checks

[Doug Hellmann]

On Mon, Jul 9, 2012 at 9:16 PM, Dan Prince  wrote:

>
>
> - Original Message -
> > From: "Dave Walker" 
> > To: "Monty Taylor" 
> > Cc: openstack@lists.launchpad.net, "John Garbutt" <
> john.garb...@citrix.com>
> > Sent: Monday, July 9, 2012 6:01:19 PM
> > Subject: Re: [Openstack] PEP8 checks
> >
> > On Mon, Jul 02, 2012 at 08:28:04AM -0400, Monty Taylor wrote:
> > >
> > >
> > > On 07/02/2012 06:46 AM, John Garbutt wrote:
> > > > Hi,
> > > >
> > > > I noticed I can now run the pep8 tests like this (taken from
> > > > Jenkins job):
> > > >   tox -v -epep8
> > > >   ...
> > > >   pep8: commands succeeded
> > > >   congratulations :)
> > > >
> > > > But the old way to run tests seems to fail:
> > > >   ./run-tests.sh -p
> > > >   ...
> > > >   File
> > > >
> "/home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py",
> > > >   line 1220, in check_logical
> > > >   for result in self.run_check(check, argument_names):
> > > >   TypeError: 'NoneType' object is not iterable
> > > >
> > > > Is this expected?
> > > > Did I just miss an email about this change?
> > >
> > > I cannot reproduce this on my system. Can you run
> > > "bash -x run_tests.sh -p" and pastebin the output? Also,
> > > tools/with_venv.sh pep8 --version just to be sure.
> > >
> >
> > Hi,
> >
> > The issue is that as of a recent change to upstream pep8 [1], the
> > additional pep8 rules in tools/hacking.py need to be changed from
> > returns to yields.. :(
>
> This brings up a good point. Why are we following the latest pep8 release
> so closely in Nova? The latest release is hardly a month old and we are
> already using it? We aren't necessarily doing the same for all the other
> OpenStack projects... (nor am I suggesting that we do). So why Nova?
>
> I'm not convinced the latest pep8 "features" really provide us enough
> benefit that we need to bump our pep8 baseline every month or two.  in fact
> they may be hurting us in terms of churn, extra work, back-portability of
> upstream patches, etc. Ultimately is tracking the latest pep8 really worth
> it?
>

+1

Some of the "features" are requiring useless whitespace changes to working
code just to get through the gating tests. In ceilometer we've pegged our
pep8 tests to 1.1.

Doug


>
>
> >
> > [1]
> >
> https://github.com/jcrocholl/pep8/commit/b9f72b16011aac981ce9e3a47fd0ffb1d3d2e085
> >
> > Kind Regards,
> >
> > Dave Walker 
> > Engineering Manager,
> > Ubuntu Server Infrastructure
> >
> > ___
> > Mailing list: https://launchpad.net/~openstack
> > Post to : openstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~openstack
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Ceilometer application for incubation

[Doug Hellmann]

On Fri, Jul 6, 2012 at 3:20 AM, Nick Barcet wrote:

> On 07/05/2012 09:22 PM, Jonathan Bryce wrote:
> > Thanks, Nick. I've added it to the agenda for next Tuesday's meeting
> > at 20:00 UTC/3:00 PM CDT. If you can join the meeting to field any
> > questions, that would be helpful.
> >
> > Jonathan
>
> Great! I'll be there, hopefully joined by other team members.
>

I will try to make it, too.

Doug


>
> Thanks,
> Nick
>
> > On Jul 5, 2012, at 12:52 PM, Nick Barcet wrote:
> >
> >> Dear members of the Project Policy Board,
> >>
> >> After 3 month of work on the Ceilometer project, and great
> >> progress being made, our last meeting IRC meeting [1] validated
> >> that we should be submitting this project for incubation.
> >> Following the OpenStack project rules, we have completed the
> >> incubation application form which you will find at [2].
> >>
> >> We would be happy to answer any question you may have about our
> >> application and hope that you will give a favorable answer to our
> >> request.
> >>
> >> [1]
> >>
> http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-07-05-16.00.html
> >>
> >>
> [2] http://wiki.openstack.org/Governance/Proposed/Ceilometer
> >>
> >> On behalf of the Ceilometer project team, -- Nick Barcet
> >>  aka: nijaba, nicolas
> >>
> >> ___ Mailing list:
> >> https://launchpad.net/~openstack Post to :
> >> openstack@lists.launchpad.net Unsubscribe :
> >> https://launchpad.net/~openstack More help   :
> >> https://help.launchpad.net/ListHelp
> >
> >
> > ___ Mailing list:
> > https://launchpad.net/~openstack Post to :
> > openstack@lists.launchpad.net Unsubscribe :
> > https://launchpad.net/~openstack More help   :
> > https://help.launchpad.net/ListHelp
> >
>
>
>
>
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] best practices for merging common into specific projects

[Doug Hellmann]

On Tue, Jul 3, 2012 at 2:59 PM, Gabriel Hurley wrote:

> The notion that copying code is any protection against APIs that may
> change is a red herring. It's the exact same effect as pegging a version of
> a dependency (whether it's a commit hash or a real version number), except
> now you have code duplication. An unstable upgrade path is an unstable
> upgrade path, and copying the code into the project doesn't alleviate the
> pain for the project if the upstream library decides to change its APIs.
>

It does if the API being changed is the one that was copied because it
means the project doesn't have to respond to the change immediately. For
example, if we add something to the rpc library that ceilometer needs, nova
doesn't have to stop what they're doing to handle any potential changes. If
openstack-common was installed as a separate library, there wouldn't be a
(good) way to have both versions installed so ceilometer and nova couldn't
run on the same host (a requirement for the ceilometer agent).

So, I'm +1 on turning common into *several* libraries with their own
versioning scheme, when the components are ready. I don't see a need to
rush that as part of Folsom, though.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Stackforge migrating

[Doug Hellmann]

On Wed, Jul 4, 2012 at 9:00 AM, Andrew Hutchings wrote:

> Hi guys,
>
> On 04/07/12 13:20, Andrew Hutchings wrote:
> > There was an initial plan to migrate Stackforge Gerrit and Jenkins to
> > OpenStack Gerrit/Jenkins.  There are many reasons for this such as:
>
> As far as I can tell the hard work of the migration is now complete.
>
> For anyone using Stackforge branches please pull the latest master and
> rebase any branches you were working on with this.  It will update
> .gitreview to point to the correct server.
>
> >From then on all reviews should be on review.openstack.org and tests on
> jenkins.openstack.org.
>
> If there are any problems please get in touch with the CI team (note it
> is 4th July holidays so probably just me today)
>

Thanks, Andrew, it looks like the ceilometer parts are working just fine!

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] resource metadata changes and billing

[Doug Hellmann]

On Wed, Jul 4, 2012 at 1:52 PM, Nick Barcet wrote:

> On 06/29/2012 03:04 PM, Doug Hellmann wrote:
> [..]
> > My conclusion from all of this (over-)thinking is that the ceilometer
> > API should assume the simple case and ignore the metadata changes when
> > computing the sum or maximum value for a counter over a range of time.
> > More complex processing will be left up to the caller, who can ask for
> > raw metering data in manageable chunks and process them outside of the
> > API. I could be persuaded to do something more complicated if the
> > problems described above can be solved in a relatively simple way, but
> > even then I think we should push that to the v2 API.
> [..]
>
> Sorry for my late reply on this, but...
>
> So, if I summarize what you are saying, the problem is that for a given
> Instance ID, a given meter may have to be interpreted as if the Instance
> ID was changing over time.
>
> Example:
> t1: Instance A - Has 1 CPU - 64G ram - runs in zone 1
> t2: Instance A - Has 2 CPU - 64G ram - runs in zone 1
> t3: Instance A - Has 2 CPU - 128G ram - runs in zone 1
> t4: Instance A - Has 2 CPU - 128G ram - runs in zone 3
> t5: Instance A is stopped
>
> From a billing point of view, what is important here is that even though
> the Instance ID remains the same, we have in fact 4 different segments
> of time which could lead to 4 different pricing being applied to the
> same instance:
>
> t1->t2: price 1
> t2->t3: price 2
> t3->t4: price 3
> t4->t5: price 4
>
> So we need to be able to inform the rating engine that these events have
> occurred so that it does not uniformly apply a billing price to from a
> sum of a given meter volume.  But in fact this information is indeed
> captured and accessible to rating engines via their respective meters.
>

Yes, that is exactly it. Thank you for clarifying what I was trying to say.
:-)


>
> What is interesting here is that, in my mind, the sum and duration
> function of the API, when I proposed it, were only meant to be able to:
>  * In a simple amazon type billing model where instances cannot change
> zone, add CPU or add ram,
>  * In a Private cloud scenario where you only need simple usage stats to
> inform your users,
>  * in a horizon plugin to give a quick summary of use,
> and would never be used by any serious rating engines that would in each
> and every case require to have access to the raw list of events so that
> it can recreate the full time line of the events.  This is where we need
> to draw the line between metering and rating.
>

I didn't realize that was your intent.


> I therefore propose that we leave the API as is, knowing the side
> effects of such high level sum and duration calculations. If we agree on
> this, I take the action to document the limitation of the summary
> functions of the API.
>

+1, and thank you for offering to document it.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] ceilometer dev docs on readthedocs.org

[Doug Hellmann]

On Tue, Jul 3, 2012 at 3:38 PM, Loic Dachary  wrote:

> On 07/03/2012 07:46 PM, Doug Hellmann wrote:
> > I've set up the ceilometer development documentation build on RTD at
> http://ceilometer.readthedocs.org/en/latest/index.html
> >
> Hi,
>
> I've updated https://launchpad.net/ceilometer to list this link.
>

Thanks, Loic!
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [metering] ceilometer dev docs on readthedocs.org

[Doug Hellmann]

I've set up the ceilometer development documentation build on RTD at
http://ceilometer.readthedocs.org/en/latest/index.html

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PyPI uploads for client libs live

[Doug Hellmann]

On Tue, Jul 3, 2012 at 9:54 AM, Monty Taylor  wrote:

>
>
> On 07/03/2012 08:47 AM, Doug Hellmann wrote:
> >
> > On Jul 3, 2012, at 5:57 AM, Thierry Carrez 
> > wrote:
> >
> >> Monty Taylor wrote:
> >>> At the moment, the only people with permission to upload tags is
> >>> the openstack-release team. However, since we're letting client
> >>> libs manage their own versions, I kinda think we should give PTLs
> >>> the right on their own project - so Vish would get tag access to
> >>> python-novaclient, Brian to python-glanceclient, etc.
> >>
> >> Ideally it would be a two-side approval process (PTL +
> >> openstack-release), because openstack-release shouldn't be able to
> >> tag without PTL approval, and openstack-release should still be
> >> kept in the loop before a tag is pushed by PTLs (there are multiple
> >> reasons why a few hours delay before tagging would be a good idea,
> >> and the openstack-release people actually keep track of those).
> >>
> >> That said, we don't have that approval mechanism available yet
> >> (same issue with the core projects release tagging) so in the mean
> >> time we should probably let the small set of individuals with an
> >> understanding of the issues (PTLs + openstack-release) have the
> >> power to do it. Within that group, we can have a soft two-side
> >> approval process (based on IRC pings) to check "everything is OK"
> >> before triggering a release.
> >
> > Could we use gerrit's 2-step approval like we do for other projects
> > and combine it with a fancy tag detector like was just added for
> > DocImpact?
>
> I have an outstanding bug/feature request for gerrit to allow for the
> review of and approval or disapproval of tags.
>

Ah, I actually meant the "magical text" feature you mention below, but
approving tags sounds like a good thing to have.


>
> That being said - if we wanted to go the route you're talking about in
> the mean time, instead of actually using the git tag route we could have
> an additional commit with a magical text in the commit message which, on
> merging, would cause a tag to be created... We've got guys on the team
> who hack gerrit now though - lemme get some feedback on how much it
> would take to actually get proper tag reviewing.
>

Sounds good.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] PyPI uploads for client libs live

[Doug Hellmann]

On Jul 3, 2012, at 5:57 AM, Thierry Carrez  wrote:

> Monty Taylor wrote:
>> At the moment, the only people with permission to upload tags is the
>> openstack-release team. However, since we're letting client libs manage
>> their own versions, I kinda think we should give PTLs the right on their
>> own project - so Vish would get tag access to python-novaclient, Brian
>> to python-glanceclient, etc.
> 
> Ideally it would be a two-side approval process (PTL +
> openstack-release), because openstack-release shouldn't be able to tag
> without PTL approval, and openstack-release should still be kept in the
> loop before a tag is pushed by PTLs (there are multiple reasons why a
> few hours delay before tagging would be a good idea, and the
> openstack-release people actually keep track of those).
> 
> That said, we don't have that approval mechanism available yet (same
> issue with the core projects release tagging) so in the mean time we
> should probably let the small set of individuals with an understanding
> of the issues (PTLs + openstack-release) have the power to do it. Within
> that group, we can have a soft two-side approval process (based on IRC
> pings) to check "everything is OK" before triggering a release.

Could we use gerrit's 2-step approval like we do for other projects and combine 
it with a fancy tag detector like was just added for DocImpact?

Doug


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Single global dependency list

[Doug Hellmann]

On Jul 2, 2012, at 6:40 PM, Monty Taylor  wrote:

> Hey all!
> 
> One of the tasks from the last ODS was to implement a single global
> dependency list. Turns out the more you think about it, the more
> important it is... because of the way we use devstack as part of the
> gate, we actually _currently_ have a de facto global dependency list,
> it's just not declared anywhere. (oops)
> 
> Anyway - the original thought was to put the depends in
> openstack-common. We'd use update.py to copy the depends in to the
> project, so that projects could align on their own timeframe.
> Additionally, we'd make the copy only copy in the versions from
> openstack-common for package that were already listed in the target
> project, so that we wouldn't add django to python-swiftclient, for instance.
> 
> The mechanics of that all work and are ready - but then bcwaldon pointed
> out that it didn't make a ton of sense for them to go in
> openstack-common, since that has its own lifecycle and is a place for
> common code to go - not just a catch all place.
> 
> To that end, I took the code we had written for the update logic and put
> it, along with the depends lists, into its own repo. I think we're ready
> to start actually moving forward with it - but we've run up against the
> hardest problem we every have:
> 
> naming
> 
> openstack-depends already got vetoed on IRC because it makes people
> think of adult diapers. I'm proposing openstack-requires, since the
> files we're talking about are actually python requirements files.
> 
> Any objections?

+0 on the name

As an alternative, how about combining the requirements file with the other 
packaging related stuff from openstack-common and calling the result 
openstack-packaging? 

Doug


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] best practices for merging common into specific projects

[Doug Hellmann]

On Jul 3, 2012, at 5:31 AM, Thierry Carrez  wrote:

> Gabriel Hurley wrote:
>> On a more fundamental level, did I miss some tremendous reason why we have 
>> this "merge from common" pattern instead of making OpenStack Common a 
>> standard python dependency just like anything else? Especially with the work 
>> Monty has recently done on versioning and packaging the client libs from 
>> Jenkins, I can't see a reason to keep following this "update common and 
>> merge to everything else" pattern at all...
> 
> This discussion should probably wait for markmc to come back, since he
> set up most of this framework in the first place. He would certainly
> produce a more compelling rationale than I can :)
> 
> IIRC the idea was to have openstack-common APIs "in incubation" until
> some of them are stable enough that we can apply backward compatibility
> for them at the level expected from any other decent Python library.
> When we reach this point, those stable modules would be "out of
> incubation" and released in a real openstack-common library. Unstable
> APIs would stay "in incubation" and still use the merge model.
> 
> My understanding is that we are not yet at this point, especially as we
> tweak/enrich openstack-common modules to make them acceptable by all the
> projects...
> 

Ideally when we reach that point the libraries will be released as individual 
components instead of a monolithic shared library, too. 

Doug


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] new locations for some things

[Doug Hellmann]

On Jul 2, 2012, at 7:02 PM, Monty Taylor  wrote:

> Secondly, in addition to the normal per-commit tarballs, we're now
> publishing tarballs of the form "$project-$branch.tar.gz" which will get
> overwritten with each commit - that way, if you need to track trunk from
> a pip-requires file, (such as ceilometer, which needs to track nova
> trunk) you can simply plop in something like
> http://tarballs.openstack.org/nova/nova-master.tar.gz  - and it'll work
> for both pip installs AND easy_install/distutils based installs. Yay!

Yay indeed!!

Thank you from the entire ceilometer team!

Doug

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] resource metadata changes and billing

[Doug Hellmann]

On Mon, Jul 2, 2012 at 5:54 PM, Julien Danjou  wrote:

> On Mon, Jul 02 2012, Doug Hellmann wrote:
>
> > No, there are cases where the metadata will affect the rate. For
> instance,
> > it costs a different amount to have an instance in each of Amazon's
> > availability zones (data centers). The counter would still say that the
> > instance has been running for a certain amount of time, but the *rate*
> for
> > charging for that time would depend on where it is. A representative from
> > HP requested the same thing in ceilometer, and we may use it at
> DreamHost,
> > too, eventually.
>
> I totally agree with you, Doug. I'm just saying that's it's not *only* a
> metadata. The zone must be some kind of a meter, even if it's not
> numeric. It should be a meter with a type that causes the resource (here
> the instance) to be billed differently (and therefore to generate
> multiple "objects" when returning resource usage metering).
>

OK, I think I understand what you're saying, but I don't see how splitting
the metadata out to a separate meter helps. In the specific example I gave,
the rate for one item depends on the time for another value. Splitting them
out and measuring them separately makes it even harder to aggregate them,
doesn't it?

Clearly the term meter is probably not the good one, maybe we should
> split this, but to me it must be extracted from metadata to become
> something more. Something we can rely on to take the decision that "this
> is something worst splitting the metered resource in different parts
> because the billing must change" (zone, RAM, flavor, volume size…).
>
> Speaking of volume size, if you take the example of a storage volume,
> you likely to have the same issue. You may not charge the same thing if
> your volume total size is 1 GB or 10 GB, and if it has been resize you
> want (not sure it's possible, but one day) to know when precisely.
> Whereas size used is likely to be just a generic absolute meter.
>

A better example would be charging different rates for SSD vs. traditional
storage.

> We probably have time to fix it before the release. On the other hand, it
> > seems much more important to use work on writing data collectors (new
> > pollsters, adding notifications to other projects, etc.). I don't think
> we
> > can do both things.
>
> Sure. Anyway, we both know that's a do-ocracy. :-)
>

Darn, I was hoping you'd say you'd do it. ;-)

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] resource metadata changes and billing

[Doug Hellmann]

On Mon, Jul 2, 2012 at 4:58 PM, Julien Danjou  wrote:

> On Fri, Jun 29 2012, Doug Hellmann wrote:
>
> Hi Doug,
>
> Sorry for the late reply.
>
> > I don't think I've made the problem clear.
> >
> > I'm not talking about wanting to calculate the different usage for CPU,
> > RAM, etc. The different counters are calculated separately, so we can
> keep
> > the amounts for CPU and RAM completely separate, and the API allows the
> > outside user to ask for the amounts for each counter for a resource (or
> > globally for a user/project). The problem is in deciding how the metadata
> > associated with a meter event might cause the provider to change the rate
> > they want to charge for that usage.
>
> It's not metadata of a counter that cause the provider to change the
> rate. It's a meter of a resource that can do that.
>

No, there are cases where the metadata will affect the rate. For instance,
it costs a different amount to have an instance in each of Amazon's
availability zones (data centers). The counter would still say that the
instance has been running for a certain amount of time, but the *rate* for
charging for that time would depend on where it is. A representative from
HP requested the same thing in ceilometer, and we may use it at DreamHost,
too, eventually.

> That only solves part of the problem, though. As a provider I may want to
> > charge different flat rates for the amount of RAM being used. For
> example,
> > 1 unit for 1024 MB of RAM but 2 units for 4096 MB. That means when the
> size
> > of the VM changes, we need to produce multiple totals (the length of time
> > that the VM had 1024 MB RAM and then the length of time it had 4096 MB
> RAM).
>
> Yeah, like I said, for the meter 'RAM' of resource 'instance' you can't
> request a total amount used, because the type of this meter (I don't
> know how to call it, it's the kind I named
> if-i-change-you-need-to-split-the-resource-in-several-stuff in my latest
> email) don't have this semantic.
>
> > I might also want to change the rate I bill when a VM is migrated between
> > hosts or availability zones (I think we said migration caused a new
> > instance to be created, but bear with me). The availability zone for an
> > instance is clearly metadata and not something we can track via a
> counter.
>
> Again, that's also a meter that has the same type of 'RAM' for a
> resource 'instance'.
>
> > That's an interesting idea, and it might solve the problem. At this point
> > in the Folsom schedule though, I would much rather implement a pared down
> > API that handles the simple cases but makes the caller do a bit more data
> > manipulation for complex cases, in favor of focusing on counting more
> > things than we do now. Is that a reasonable approach?
>
> Problem is that we might break the API a bit with this. This would not
> be the first time an OpenStack API is broken, but if we can avoid it,
> it'd be better.


> I am not sure you can really bill anything if you're not able to handle
> a simple thing such a VM resize. So currently, it seems that the API is
> not designed correctly to handle such a case, and since it's not yet
> implemented, maybe it's still time to fix it?
>

We probably have time to fix it before the release. On the other hand, it
seems much more important to use work on writing data collectors (new
pollsters, adding notifications to other projects, etc.). I don't think we
can do both things.

Doug
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [metering] resource metadata changes and billing

[Doug Hellmann]

On Fri, Jun 29, 2012 at 1:19 PM, Julien Danjou  wrote:

> On Fri, Jun 29 2012, Doug Hellmann wrote:
>
> > We do have counters for RAM and CPU separate from instance. But the rate
> at
> > which the provider bills for those things may vary based on metadata. My
> > example may be bad because it uses 2 values we're measuring, one of which
> > also shows up in the metadata for another. As a different example, take
> the
> > instance display name. The display name is under the control of the user
> > and is extremely unlikely to reflect a change in the billing rate.
> However,
> > changing the display name changes the metadata for the instance. A naive
> > implementation of the processing loop would pick that up and generate
> > multiple documents even though there is no need to do so.
>
> Yep, but the display name is not a counter. Memory is a counter. An
> instance is made of several counter. We need to split metered objects
> based on their "absolute" counter changing (memory, number of core…),
> not based on random metadata, i.e. a resource have several meters.
>

I don't think I've made the problem clear.

I'm not talking about wanting to calculate the different usage for CPU,
RAM, etc. The different counters are calculated separately, so we can keep
the amounts for CPU and RAM completely separate, and the API allows the
outside user to ask for the amounts for each counter for a resource (or
globally for a user/project). The problem is in deciding how the metadata
associated with a meter event might cause the provider to change the rate
they want to charge for that usage.


> So what was considered as metadata (like memory) so far should changed
> to become a meter of an resource (like an instance) and have for this
> one a special type (not sure about the type name to use).
>

That only solves part of the problem, though. As a provider I may want to
charge different flat rates for the amount of RAM being used. For example,
1 unit for 1024 MB of RAM but 2 units for 4096 MB. That means when the size
of the VM changes, we need to produce multiple totals (the length of time
that the VM had 1024 MB RAM and then the length of time it had 4096 MB RAM).

I might also want to change the rate I bill when a VM is migrated between
hosts or availability zones (I think we said migration caused a new
instance to be created, but bear with me). The availability zone for an
instance is clearly metadata and not something we can track via a counter.


>
> We may need to refine our model to be a bit more hierarhical like:
>
> resource --> counter #1 of type 'relative'
>  |  `-> counter #2 of type 'absolute'
>  `> counter #3 of type
> 'if-i-change-you-need-to-split-the-resource-in-several-stuff'
>

That's an interesting idea, and it might solve the problem. At this point
in the Folsom schedule though, I would much rather implement a pared down
API that handles the simple cases but makes the caller do a bit more data
manipulation for complex cases, in favor of focusing on counting more
things than we do now. Is that a reasonable approach?


>
> etc…
>
> --
>   Julien
>
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] [metering] resource metadata changes and billing

[Doug Hellmann]

tl;dr: Ceilometer should ignore resource metadata when computing sums or
maximum values for counters through the API.

One of the things we discussed early during the design meetings was the
need to track metadata along with resources so providers could use the
metadata to determine the rate to charge for using the resource (for
example, flavor or availability group of an instance). While working on the
mongodb driver, I've been thinking about how that requirement changes what
the API we defined needs to do. At first I thought we would need to try to
return multiple values from the sum and max calls so the values could be
associated with the resource metadata and a given time range. I've decided
that implementing that inside the ceilometer API will be very complex, and
unlikely to produce the correct result. I want to work through my reasoning
here in case someone else can find a fault in it or propose a solution I
wasn't able to find.

First, the scenario: A user boots an instance, lets it run for some time
(period 1), then changes the metadata in some way that does not result in
the instance being recreated but does result in something the provider
would decide introduces a different charge structure. For example, the
amount of RAM allocated to the instance might be increased. After running
with the new settings for a period of time (period 2), the user changes
them back to their original value and the instance continues to run (period
3).

The specific change to the metadata doesn't matter (RAM is just an
example), except that the metadata change should not require an instance to
be recreated because when that happens the user actually gets a new
instance (at least I believe they do based on feedback during an early
meeting, please correct me if I'm wrong). Getting a new instance simplifies
things immensely, since the new instance is a completely new resource and
so we can ignore those cases for the rest of this discussion.

Another important point in the way the scenario is constructed is that the
metadata values go from value A to B and then back to their original value
A. That means any signature we calculate for the metadata will be the same
during periods 1 and 3.

Now we would like for v1/USERS//RESOURCES//cpu/VOLUME
to return the amount of CPU time used by the instance. However, if that
time is billed at a different rate depending on the "size" of the server
then the RAM change will cause a difference in billing rates. We therefore
need to return a sequence of 3-tuples containing the total for the counter,
the resource metadata, and the time range during which the metadata was in
effect. There are two problems implementing this in a generic way in the
API server.

First, it turns out to be surprisingly difficult to write an efficient
query to compute that time range in the case described because it is not
easy to recognize the ranges for period 1 and period 3 as being interrupted
by period 2 (finding min or max for a value is easy, but finding the
endpoint of period 1 is not because the signature for the metadata is the
same for period 1 and 3).

Second, while calculating ranges is difficult in itself, what is even more
difficult is recognizing *important* changes in the metadata that actually
imply a change in the billing rate. The logic for that is up to the
deployer and their billing rules.

There are ways to compute the ranges by using multiple queries [1], and we
could create some sort of way for the query to specify which fields in the
metadata for a given type of resource are important. Both calculations
would be expensive to apply in the API server, though, and I think they can
be solved more efficiently on the client-side. If the client grabs all (or
a large portion) of the data and processes it sequentially, it is easy to
test the metadata fields to find changes and treat that condition as the
boundary between the time ranges.

My conclusion from all of this (over-)thinking is that the ceilometer API
should assume the simple case and ignore the metadata changes when
computing the sum or maximum value for a counter over a range of time. More
complex processing will be left up to the caller, who can ask for raw
metering data in manageable chunks and process them outside of the API. I
could be persuaded to do something more complicated if the problems
described above can be solved in a relatively simple way, but even then I
think we should push that to the v2 API.

Thoughts?

There-and-back-again-ly,
Doug


[1] Find the min time for a (resource, metadata) pair; find min time for
any different (resource, metadata) pair greater than the first time; find
max for original pair with timestamp less than second min; use the max as
starting point to determine the next range; loop.
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchp