Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 2:13 PM, Julien Danjou jul...@danjou.info wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. Good points. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
Thanks for putting that together Sandy. Very nice! From my perspective, there are two major things that are undesirable for us: 1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data sinks from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation. 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc. Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different sinks. On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks, Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
On 01/11/12 13:58 -0700, Jeffrey Budzinski wrote: Thanks for putting that together Sandy. Very nice! From my perspective, there are two major things that are undesirable for us: 1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data sinks from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation. 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc. Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different sinks. Hi I Agree with what you said above, but lets not get bogged down with monkey patching/modifing the code as I think this is more a question for each ptl. Lets just make it possible to do both. btw: Has anyone started work on this? Is there a repo setup yet? -A On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks, Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ OpenStack-dev mailing list openstack-...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Julien Danjou wrote: On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: My goal is to offer monitored data to the admin and customers. The admin is interested in the utilization of the physical components and the virtual machines and the customer is interested to know what his VMs do or can do. It would be nice to get the data from a single point. I thought I can enhance the Ceilometer compute agent to get this data out. Does this make sense or is it better to use another monitoring tool for the physical components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. I think also the best way to implement this is to integrate a new (hardware) agent. Then we have a clear delineation. I'm very interested in helping to develop this. Toni About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
There is a use case for base metal hardware metering in the private cloud where the user allocated the machine does not have root access to kill the metering. Being able to create a single metering infrastructure for the entire private cloud, virtual or bare-metal allocation, is a need technically, it is not clear how to guarantee it but it is worth exploring. Tim -Original Message- From: openstack-bounces+tim.bell=cern...@lists.launchpad.net [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf Of Robert Collins Sent: 06 November 2012 11:00 To: Graf Lucas (graflu0); Zehnder Toni (zehndton); openstack@lists.launchpad.net; Doug Hellmann Subject: Re: [Openstack] [ceilometer] Monitoring physical devices On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
From: Doug Hellmann [doug.hellm...@dreamhost.com] Sent: Thursday, November 08, 2012 1:54 PM To: Sandy Walsh Cc: Eoghan Glynn; OpenStack Development Mailing List; openstack@lists.launchpad.net Subject: Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ... On Wed, Nov 7, 2012 at 10:21 PM, Sandy Walsh sandy.wa...@rackspace.commailto:sandy.wa...@rackspace.com wrote: Hey! (sorry for the top-posting, crappy web client) There is a periodic task already in the compute manager that can handle this: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3021 There seems to be some recent (to me) changes in the manager now wrt the resource_tracker.py and stats.py files about how this information gets relayed. Now it seems it only goes to the db, but previously it was sent to a fanout queue that the schedulers could use. Regardless, this is done at a high enough level that it doesn't really care about the underlying virt layer, so long at the virt layer supports the get_available_resource() method. https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L152 https://github.com/openstack/nova/blob/master/nova/virt/xenapi/driver.py#L392 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2209 I'd add a hook in there do what we want with this data. Write it to the db, send it over the wire, whatever. If there is additional information required, it should go in this dictionary (or we should define a format for extensions to it). Yes It looks like that is collecting resource data, but not usage data. For example, there's no disk I/O information, just disk space. Is that what you mean by adding extra information to the dictionary? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] not getting cinder volume meters/resources from ceilometer
I have created the openstack setup with devstack. I have created the volumes and instances. When I am listing resources using http://10.102.153.37:8777/v2/resources then I am getting only instnaces not volumes. Similarly I am not getting any meters. I tried setting notification_driver=cinder.openstack.common.notifier.rabbit_notifier notification_driver=cinder.openstack.common.notifier.rpc_notifier in cinder.conf and restarting cinder-volume service and ceilometer-collector service. Still I am not getting any volume related meters. Neither they are mentioned in resources. In c-vol screen logs I can see below kind of logs when creating and deleting volumes. f4-8ba8-8cdac08c7b21', 'project_id': u'39587e1865d14ea991b4ddf82c1dc1ab', 'read_deleted': u'no', 'tenant': u'39587e1865d14ea991b4ddf82c1dc1ab'} 2013-05-31 16:54:24 INFO [cinder.volume.manager] volume volume-147adb04-be41-4949-9fbf-52c871593de4: deleting 2013-05-31 16:54:24DEBUG [cinder.openstack.common.rpc.amqp] Sending volume.delete.start on notifications.info 2013-05-31 16:54:24DEBUG [cinder.openstack.common.rpc.amqp] UNIQUE_ID is fd2593f309ca45579955afa7eb20d6ca. 2013-05-31 16:54:24 INFO [cinder.volume.manager] Clear capabilities Thanks, Anshul___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] ceilometer client
Hi Stackers
I've installed the requirements from requirements.txt with pip, and then
tried to install the ceilometer client, and I get this error:
root@control:~/python-ceilometerclient# python setup.py install
running install
Traceback (most recent call last):
File setup.py, line 21, in module
d2to1=True)
File /usr/lib/python2.7/distutils/core.py, line 152, in setup
dist.run_commands()
File /usr/lib/python2.7/distutils/dist.py, line 953, in run_commands
self.run_command(cmd)
File /usr/lib/python2.7/distutils/dist.py, line 972, in run_command
cmd_obj.run()
File /usr/local/lib/python2.7/dist-packages/pbr/packaging.py, line 310,
in run
_pip_install(links, self.distribution.install_requires, self.root)
File /usr/local/lib/python2.7/dist-packages/pbr/packaging.py, line 95,
in _pip_install
.join(_wrap_in_quotes(_missing_requires(requires,
File /usr/local/lib/python2.7/dist-packages/pbr/packaging.py, line 83,
in _missing_requires
pkg_resources.Requirement.parse(r))]
File build/bdist.linux-x86_64/egg/pkg_resources.py, line 2515, in parse
try:
ValueError: ('No requirements found', '# This file is managed by
openstack-depends')
Does anyone got something like this?
Thanks in advance
Claudio Marques
clau...@onesource.pt
http://www.onesource.pt/
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] ceilometer not getting nova notifications
I am not getting disk.root.size meter notifications from nova to ceilometer. I am getting all ther pollster meters but not notification meters. Do I need to add any cron job like cinder to get notifications? I am using devstack and below are the contents of my nova.conf. firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver compute_driver = libvirt.LibvirtDriver flat_interface = eth0 flat_network_bridge = br100 vlan_interface = eth0 public_interface = br100 network_manager = nova.network.manager.FlatDHCPManager glance_api_servers = 10.102.153.5:9292 rabbit_password = freebsd rabbit_host = localhost rpc_backend = nova.openstack.common.rpc.impl_kombu ec2_dmz_host = 10.102.153.5 vncserver_proxyclient_address = 127.0.0.1 vncserver_listen = 127.0.0.1 vnc_enabled = true xvpvncproxy_base_url = http://10.102.153.5:6081/console novncproxy_base_url = http://10.102.153.5:6080/vnc_auto.html notification_driver = nova.openstack.common.notifier.rpc_notifier,ceilometer.compute.nova_notifier instance_usage_audit_period = hour instance_usage_audit = True logging_context_format_string = %(asctime)s.%(msecs)03d %(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s] %(instance)s%(message)s use_syslog = True instances_path = /opt/stack/data/nova/instances lock_path = /opt/stack/data/nova state_path = /opt/stack/data/nova volume_api_class = nova.volume.cinder.API enabled_apis = ec2,osapi_compute,metadata instance_name_template = instance-%08x libvirt_cpu_mode = none libvirt_type = qemu sql_connection = mysql://root:freebsd@localhost/nova?charset=utf8 my_ip = 10.102.153.5 osapi_compute_extension = nova.api.openstack.compute.contrib.standard_extensions s3_port = s3_host = 10.102.153.5 default_floating_pool = public fixed_range = force_dhcp_release = True dhcpbridge_flagfile = /etc/nova/nova.conf Thanks, Anshul 1,1 Top ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Availability of metrics from SWIFT - Object Storage
Hi All, I am able to configure Openstack Essex using Ubuntu 12.04 Precise Pangolin for Swift Proxy, and 3 storage nodes. In dashboard while I create a Container it is creating however a error pops up Error Unable to create container. I can upload objects but cant delete objects or container. Any help would be appreciated. Regards, Raghavendra Lad From: Openstack [mailto:openstack-bounces+raghavendra.lad=accenture@lists.launchpad.net] On Behalf Of Narayanan, Krishnaprasad Sent: Wednesday, June 26, 2013 3:10 PM To: openstack@lists.launchpad.net Subject: [Openstack] Availability of metrics from SWIFT - Object Storage Hallo All, Based on the documentation from Ceilometer, I see the metrics from all the components except SWIFT. Can I get to know whether Ceilometer offers any metrics from the SWIFT component? Thanks Krishnaprasad This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. __ www.accenture.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: Documentation for openstack-java-sdk
I think Luis can answer that? -- Videresendt melding -- Fra: Jobin Raju George jobin...@gmail.com Dato: 11. juli 2013 14:38 Emne: [Openstack] Documentation for openstack-java-sdk Til: openstack lista openstack@lists.launchpad.net Kopi: I am trying to query ceilometer using openstack-java-sdkhttps://github.com/woorea/openstack-java-sdkfor meters of VM's running on KVM. I am able to get the CPU meters via curl on the command line but unfortunately I don't find good documentation for the SDK's for ceilometer. I have seen this example programhttps://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java but most of it is commented(probably because it is deprecated). Where can I find good documentation/examples or java programs/snippets? -- Thanks and regards, Jobin Raju George Third Year, Information Technology College of Engineering Pune Alternate e-mail: georgejr10...@coep.ac.in ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Problems with query fields in filters
Hi Julien
thanks for your reply, unfortunately volume instead of counter_volume is
not working either.
Alex
-Original Message-
From: Julien Danjou [mailto:jul...@danjou.info]
Sent: venerdì 12 luglio 2013 11:34
To: Alessandro Barabesi
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Ceilometer] Problems with query fields in filters
On Fri, Jul 12 2013, Alessandro Barabesi wrote:
I have the following problems with query fields in filters.
1) Filtering by metadata.size in v2/meters/image apparently is not working.
The following request returns also samples with metadata.size=0.
GET http://10.10.10.10:8777/v2/meters/image
{
q: [{
field: project_id,
op: eq,
value: 77b461539c8542909f67b29939ec87dd
},
{
field: timestamp,
op: ge,
value: 2013-07-11T13:36:00
},
{
field: timestamp,
op: lt,
value: 2013-07-11T13:39:00
},
{
field: metadata.size,
op: gt,
value: 0
}]
}
I am probably doing something wrong but I can't figure out what.
That seems correct from the top of my head. If that really doesn't work, feel
free to open a bug.
2) Filtering by counter_volume returns the folloving error message:
error_message={debuginfo: null, faultcode: Client,
faultstring: Unknown argument: \counter_volume\: unrecognized
query field}
Is this a bug or filtering by counter-volume is not allowed?
Try `volume' instead of `counter_volume'. But I am not sure it's currently
allowed -- in that case opening a bug can be good idea too.
--
Julien Danjou
// Free Software hacker / freelance consultant // http://julien.danjou.info
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Where can I consume messages from metering system?
Hi,
We had a plan to write some plugin which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.
My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:
1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification Alert less than 15% free ram on host}
2. HostA hits less than 15% of free ram and sends notification Alert less
than 15% free ram on HostA.
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.
This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.
Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.
I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data) and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:
- write plugin for agent to send proper alert when metering data hits
proper rules (just like in above scenario with 15% free ram)
- write service which will consume this alert (listen proper topic in
queue) and fetch data from ceilometer collector?? (or it's backend via API)
and find out which VM should be migrated and where (or delegate Where to
nova-scheduler)
Another idea. Are there any Host states in Openstack? For example
Maintenance state? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to maintenance.
2. There goes alert HostA maintenance.
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to active.
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
Everything should be event-driven (alert-notification, host-state-change,
administrator's action).
I'm also wondering what about this
ResourceMonitorAlertsandNotificationshttp://wiki.openstack.org/ResourceMonitorAlertsandNotifications,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).
First question: Does it make sense? Are such mechanisms implemented in
Openstack right now? Or is it worth to implement?
Second question (if first's answer isn't negative): How and when can I use
ceilometer to do what I've described? From meeting's
topicshttp://wiki.openstack.org/Meetings/MeteringAgendaI know that
yesterday you were deciding what to use as notification bus,
and in june you will be thinking about storage backend, how is it going on
right now? Is such scenario possible in ceilometer (I mean plugins in
agents)? Collecting data not only from guests but also from hosts.
*Cheers,
*
*
*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Glance usage data retrieval
Hi Julien and Stuart! Comments inline... On 06/19/2012 07:12 AM, stuart.mcla...@hp.com wrote: Brian, Jay, I'll give you a chance to reply to Julien first, but I have a follow on query... It doesn't seem like right now Glance produces enough records for full metering of operations. Eg if you want to charge a specific user for every successful image upload/image download this information doesn't 'fall out' of the current logs. For example the eventlet output such as: Jun 19 09:18:39 az1-gl-api-0001 DEBUG 3795 [eventlet.wsgi.server] 127.0.0.1 - - [19/Jun/2012 09:18:39] GET /v1/images/4921 HTTP/1.1 200 104858310 2.098967 doesn't tie the GET operation to a particular user. As Julien mentions there is an image_upload notification, but I don't see an equivalent image_download notification. Yeah, there is one :) It's called image.send: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L892 (Note I'm using old Diablo code at the moment, so I'm going on code inspection of the latest code -- apologies if I missed anything.) Is the creation of records of operations in a well-defined format which can be consumed by eg a Metering and Billing team something we'd like to have? (I'm ignoring Data at Rest metering for now.) Not sure about this one, but I believe the message format is the same as Nova: https://github.com/openstack/glance/blob/master/glance/notifier/__init__.py#L55 With the exception of the request ID. The payload is structured here for downloads: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L882 If so, would something along the following lines be worth considering? A wsgi filter, using a mechanism similar to Swift's posthooklogger, which has a routine which is called when operations are about to complete. It should have access to the context (http return status, user, operation etc) so would be able to raise a notification if appropriate. Using a standard notifier would mean output could be to a log file or, perhaps in time, the notifications could be consumed by ceilometer. LOL, that's actually how it currently works :) https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L917 All the best, -jay -Stuart On Tue, 19 Jun 2012, Julien Danjou wrote: Hello, As part of the ceilometer project¹, we're working on usage data retrieval from various OpenStack components. One of them is Glance. We're targeting Folsom for the first release, therefore it seems important for both projects to be able to work together, this is why we're bringing ceilometer to your attention and asking for advices. :) What we want is to retrieve the maximum amount of data, so we can meter things, to bill them in the end. For now and for Glance, this only includes the number of image uploaded (per user/tenant), but we'll need probaly more in a near future. At this point we plan to plug into the notification system, since it seems to be what Glance provides to accomplish this. And so far, the notifications provided seem enough to accomplish what we want to do. Do you have any advice regarding integration of Ceilometer and Glance together? Is this something a stable interface we can rely on, or is there a better way to do so? Thanks in advance, Regards, ¹ http://launchpad.net/ceilometer -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Documentation for openstack-java-sdk
I know nothing about ceilometer.
I think the best thing is to checkout the classes on github and make a lot
of tests. Probably to functioning of objects and methods are the same in
ceilometer.
CHeckout the methods and try to workout with it.
:)
Good luck!
On Thu, Jul 11, 2013 at 11:59 AM, Jobin Raju George jobin...@gmail.comwrote:
Thanks a log, Gui! This helps but would be more useful if you could point
me to some *ceilometer-specific *guides/examples.
On Thu, Jul 11, 2013 at 8:25 PM, Gui Maluf guimal...@gmail.com wrote:
Surely Luis can help you, I've used openstack-java-sdk in one of my
projects, and this is the example Luis gave to me
private static final File TEST_FILE = new File(pom.xml);
private static final String KEYSTONE_AUTH_URL =
https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0;;
private static final String KEYSTONE_USERNAME = ;
private static final String KEYSTONE_PASSWORD = ;
/**
* @param args
*/
public static void main(String[] args) throws Exception {
KeystoneClient keystone = new KeystoneClient(KEYSTONE_AUTH_URL);
//access with unscoped token
Access access = keystone.execute(Authenticate.withPasswordCredentials(
KEYSTONE_USERNAME, KEYSTONE_PASSWORD));
//use the token in the following requests
keystone.setToken(access.getToken().getId());
Tenants tenants = keystone.execute(new ListTenants());
//try to exchange token using the first tenant
if(tenants.getList().size() 0) {
access =
keystone.execute(Authenticate.withToken(access.getToken().getId()).withTenantId(tenants.getList().get(0).getId()));
SwiftClient swiftClient =
newSwiftClient(KeystoneUtils.findEndpointURL(access.getServiceCatalog(),
object-store, null, public), access.getToken().getId());
//swiftClient.execute(new DeleteContainer(navidad2));
swiftClient.execute(new CreateContainer(navidad2));
System.out.println(swiftClient.execute(new ListContainers()));
ObjectForUpload upload = new ObjectForUpload();
upload.setContainer(navidad2);
upload.setName(example2);
upload.setInputStream(new FileInputStream(TEST_FILE));
swiftClient.execute(new UploadObject(upload));
System.out.println(swiftClient.execute(new ListObjects(navidad2, new
HashMapString,
String() {{
put(path, );
}})).get(0).getContentType());
}
}
On Thu, Jul 11, 2013 at 11:31 AM, Endre Karlson
endre.karl...@gmail.comwrote:
I think Luis can answer that?
-- Videresendt melding --
Fra: Jobin Raju George jobin...@gmail.com
Dato: 11. juli 2013 14:38
Emne: [Openstack] Documentation for openstack-java-sdk
Til: openstack lista openstack@lists.launchpad.net
Kopi:
I am trying to query ceilometer using
openstack-java-sdkhttps://github.com/woorea/openstack-java-sdkfor meters
of VM's running on KVM. I am able to get the CPU meters via curl
on the command line but unfortunately I don't find good documentation for
the SDK's for ceilometer.
I have seen this example
programhttps://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java
but
most of it is commented(probably because it is deprecated).
Where can I find good documentation/examples or java programs/snippets?
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
*guilherme* \n
\t *maluf*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
--
*guilherme* \n
\t *maluf*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Documentation for openstack-java-sdk
Okay, thanks Gui! Lets see how things go, but still I'm awaiting a little
help with respect to the documentation and examples :/
On Thu, Jul 11, 2013 at 8:31 PM, Gui Maluf guimal...@gmail.com wrote:
I know nothing about ceilometer.
I think the best thing is to checkout the classes on github and make a lot
of tests. Probably to functioning of objects and methods are the same in
ceilometer.
CHeckout the methods and try to workout with it.
:)
Good luck!
On Thu, Jul 11, 2013 at 11:59 AM, Jobin Raju George jobin...@gmail.comwrote:
Thanks a log, Gui! This helps but would be more useful if you could point
me to some *ceilometer-specific *guides/examples.
On Thu, Jul 11, 2013 at 8:25 PM, Gui Maluf guimal...@gmail.com wrote:
Surely Luis can help you, I've used openstack-java-sdk in one of my
projects, and this is the example Luis gave to me
private static final File TEST_FILE = new File(pom.xml);
private static final String KEYSTONE_AUTH_URL =
https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0;;
private static final String KEYSTONE_USERNAME = ;
private static final String KEYSTONE_PASSWORD = ;
/**
* @param args
*/
public static void main(String[] args) throws Exception {
KeystoneClient keystone = new KeystoneClient(KEYSTONE_AUTH_URL);
//access with unscoped token
Access access = keystone.execute(Authenticate.withPasswordCredentials(
KEYSTONE_USERNAME, KEYSTONE_PASSWORD));
//use the token in the following requests
keystone.setToken(access.getToken().getId());
Tenants tenants = keystone.execute(new ListTenants());
//try to exchange token using the first tenant
if(tenants.getList().size() 0) {
access =
keystone.execute(Authenticate.withToken(access.getToken().getId()).withTenantId(tenants.getList().get(0).getId()));
SwiftClient swiftClient =
newSwiftClient(KeystoneUtils.findEndpointURL(access.getServiceCatalog(),
object-store, null, public), access.getToken().getId());
//swiftClient.execute(new DeleteContainer(navidad2));
swiftClient.execute(new CreateContainer(navidad2));
System.out.println(swiftClient.execute(new ListContainers()));
ObjectForUpload upload = new ObjectForUpload();
upload.setContainer(navidad2);
upload.setName(example2);
upload.setInputStream(new FileInputStream(TEST_FILE));
swiftClient.execute(new UploadObject(upload));
System.out.println(swiftClient.execute(new ListObjects(navidad2,
new HashMapString, String() {{
put(path, );
}})).get(0).getContentType());
}
}
On Thu, Jul 11, 2013 at 11:31 AM, Endre Karlson endre.karl...@gmail.com
wrote:
I think Luis can answer that?
-- Videresendt melding --
Fra: Jobin Raju George jobin...@gmail.com
Dato: 11. juli 2013 14:38
Emne: [Openstack] Documentation for openstack-java-sdk
Til: openstack lista openstack@lists.launchpad.net
Kopi:
I am trying to query ceilometer using
openstack-java-sdkhttps://github.com/woorea/openstack-java-sdkfor meters
of VM's running on KVM. I am able to get the CPU meters via curl
on the command line but unfortunately I don't find good documentation for
the SDK's for ceilometer.
I have seen this example
programhttps://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java
but
most of it is commented(probably because it is deprecated).
Where can I find good documentation/examples or java programs/snippets?
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
*guilherme* \n
\t *maluf*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
--
*guilherme* \n
\t *maluf*
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https
[Openstack-ubuntu-testing-notifications] Build Still Failing: precise_havana_ceilometer_trunk #2
Title: precise_havana_ceilometer_trunk General InformationBUILD FAILUREBuild URL:https://jenkins.qa.ubuntu.com/job/precise_havana_ceilometer_trunk/2/Project:precise_havana_ceilometer_trunkDate of build:Thu, 28 Mar 2013 10:56:15 -0400Build duration:1 min 10 secBuild cause:Started by user James PageBuilt on:pkg-builderHealth ReportWDescriptionScoreBuild stability: All recent builds failed.0ChangesNo ChangesConsole Output[...truncated 1178 lines...]hard linking tests/storage/test_impl_log.py -> ceilometer-2013.2/tests/storagehard linking tests/storage/test_impl_mongodb.py -> ceilometer-2013.2/tests/storagehard linking tests/storage/test_impl_sqlalchemy.py -> ceilometer-2013.2/tests/storagehard linking tests/storage/test_register_opts.py -> ceilometer-2013.2/tests/storagehard linking tests/volume/__init__.py -> ceilometer-2013.2/tests/volumehard linking tests/volume/test_notifications.py -> ceilometer-2013.2/tests/volumehard linking tools/enable_notifications.sh -> ceilometer-2013.2/toolshard linking tools/flakes.py -> ceilometer-2013.2/toolshard linking tools/hacking.py -> ceilometer-2013.2/toolshard linking tools/make_test_data.py -> ceilometer-2013.2/toolshard linking tools/make_test_data.sh -> ceilometer-2013.2/toolshard linking tools/notificationclient.py -> ceilometer-2013.2/toolshard linking tools/pip-requires -> ceilometer-2013.2/toolshard linking tools/release-bugs.py -> ceilometer-2013.2/toolshard linking tools/show_data.py -> ceilometer-2013.2/toolshard linking tools/test-requires -> ceilometer-2013.2/toolshard linking tools/test-requires-folsom -> ceilometer-2013.2/toolshard linking tools/conf/extract_opts.py -> ceilometer-2013.2/tools/confhard linking tools/conf/generate_sample.sh -> ceilometer-2013.2/tools/confcopying setup.cfg -> ceilometer-2013.2Writing ceilometer-2013.2/setup.cfgcreating distCreating tar archiveremoving 'ceilometer-2013.2' (and everything under it)ERROR:root:Error occurred during package creation/build: 'bzr_testing_repo'ERROR:root:'bzr_testing_repo'INFO:root:Complete command log:INFO:root:Destroying schroot.bzr branch lp:~openstack-ubuntu-testing/ceilometer/grizzly /tmp/tmpZlDlTe/ceilometermk-build-deps -i -r -t apt-get -y /tmp/tmpZlDlTe/ceilometer/debian/controlpython setup.py sdistTraceback (most recent call last): File "/var/lib/jenkins/tools/openstack-ubuntu-testing/bin/build-package", line 139, in raise eKeyError: 'bzr_testing_repo'Error in sys.excepthook:Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 68, in apport_excepthookbinary = os.path.realpath(os.path.join(os.getcwd(), sys.argv[0]))OSError: [Errno 2] No such file or directoryOriginal exception was:Traceback (most recent call last): File "/var/lib/jenkins/tools/openstack-ubuntu-testing/bin/build-package", line 139, in raise eKeyError: 'bzr_testing_repo'Build step 'Execute shell' marked build as failureEmail was triggered for: FailureSending email for trigger: Failure-- Mailing list: https://launchpad.net/~openstack-ubuntu-testing-notifications Post to : openstack-ubuntu-testing-notifications@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-ubuntu-testing-notifications More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
Yes, I think support for metrics objects that can be leveraged both by monkey patches and decorators was what we'd been thinking along the lines of. The metrics would be controlled via config both in what scopes are active (e.g. on|off for a package, module, etc.) and also the outlet for the metrics (file, datagram, rpc). Support for instrumentation levels would also be nice so that metric flow could be controlled (i.e. instrumentation point is annotated as METRIC, MONITOR, PROFILE and then level to actually emit can be set in conjunction with a metric outlet/sink). With this approach, folks could control both the volume of metrics and also the outlet for the metrics. Ceilometer would also be an outlet that could be leveraged via RPC flow for metrics -- though I'd expect some would want to send via datagram to statsd or file for offline log aggregation. I'll post a diagram tomorrow for review and comment. Oh btw, I updated the spec with most of what was in the etherpad. We can update the spec to reflect whatever we decide is the preferred approach. -jeff On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: On 25/10/12 11:13 +, Sandy Walsh wrote: grizzly-common-instrumentation seems to be the best choice ... hopefully the other groups will use this etherpad too. We need a proper blueprint to nail down the approach. IRC is great, but doesn't retain history for other groups. I think we need to get a plan for translating the etherpad into something concise and nailed down. Agree. statgen should really just be a new notifier in Tach (or Scrutinize) ... vs copy-pasting the code into yet-another repo. Hopefully that's the plan? Tach should remain a generic tool and not pegged to OpenStack. Well that was just an ideas play pen not serious code. I might be coming at this from a slightly different angle... I was looking at a library that can be used to generate trace, monitoring and metering data (kind of like log levels for logging). Currently both Tach and Scrutinize don't have enough fields (of course that could be changed). Also I think we should be able to insert instrumentation into the code as well as have the function level performance metrics monkey patched. Then we could have a config that directed metric data to different notifiers like how you do it in Scrutinize perhaps. Also enforcing data rate limits and possible data aggregation could be neat configurable features. Anyway more at the meeting... -Angus -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Angus Salkeld [asalk...@redhat.com] Sent: Thursday, October 25, 2012 1:00 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems every time I turn around there's a new metric/instrumentation effort, which is good I guess :) Here is some fun I have been having with a bit of tach+ceilometer code. https://github.com/asalkeld/statgen Is there a single location I can place my feedback? If not, should we create one? I've got lots of suggestions/ideas and would hate to have to duplicate the threads or leave other groups out. I'll add some links here that I am aware of: https://bugs.launchpad.net/ceilometer/+bug/1071061 https://etherpad.openstack.org/grizzly-common-instrumentation https://etherpad.openstack.org/grizzly-ceilometer-actions https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring 3. I'm wrapping up the packaging / cleanup of StackTach v2 with Stacky and hope to make a more formal announcement on this by the end of the week. Lots of great changes to make it easier to use/deploy based on the Summit feedback! Unifying the stacktach worker (consumer of events) into ceilometer should be a first step to integration (or agree upon a common YAGI-based consumer?) 4. If you're looking at Tach, you should also consider looking at Scrutinize (my replacement effort) https://github.com/SandyWalsh/scrutinize (needs packaging/docs and some notifier tweaks on the cprofiler to be called done for now) Looks great! I like the monkey patching for performance as you have done here, but we also need a nice clean way of manually inserting instrumentation too (that is what I have
Re: [Openstack] [ceilometer] Potential New Use Cases
Yes, I am assuming the service controller provides a different stream of data from the lower level VM events. So the question is how to represent and store this additional meta data in ceilometer. Note that there doesn't necessarily need to be a linkage/grouping between the resources since the association is what is actually contained in the metadata that is provided by the service controller. As a summary Nova provides its normal events for usage Service controller provides a mapping of nova instances to service type and actual end user Dan On 11/1/2012 11:25 AM, Doug Hellmann wrote: On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer dan.dye...@gmail.com mailto:dan.dye...@gmail.com wrote: In some cases, the service controller is actually running inside a VM. It would not have access to the internals of the VM's. It maintains its metadata separately from the Nova infrastructure. It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or join it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug DD On 10/25/2012 2:25 AM, Nick Barcet wrote: Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each Unit of the service such as constant:uuid If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specified so that we can ignore the proper VM's. There has also been talk of hybrid billing models that charge for some part of the VM usage as well as other metrics. Once again we need a way to characterize the relationships so that processing can associate and filter correctly. Dan On 10/24/2012 3:35 PM, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate table and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable product/service In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Kind of the same here, if you don't want to really bill the vm, just
[Openstack] OpenStack Community Weekly Newsletter (Jan 11 – 18)
Highlights of the week My first week at OpenStack http://vmartinezdelacruz.com/my-first-week-at-openstack/ Victoria Martínez de la Cruz http://vmartinezdelacruz.com/ is one of the three interns working on OpenStack under the Outreach Program for Women (OPW –Anne Gentle shared some details about the program before.) She will be working on Tenant Deletion Workflow https://blueprints.launchpad.net/horizon/+spec/tenant-deletion in the next months. This first blog post about OpenStack contains lots of good advice for any developer joining the community: a must read, for experienced developers, hiring managers and newcomers to this great community. Ceilometer Grizzly 2 Milestone Available http://blog.doughellmann.com/2013/01/ceilometer-grizzly-2-milestone-available.html The Ceilometer team is proud to announce the first synchronous milestone delivery with the OpenStack http://openstack.org/ project. Grizzly-2 https://launchpad.net/ceilometer/+milestone/grizzly-2 is also the last Folsom compatible version of Ceilometer https://launchpad.net/ceilometer as we are planning to introduce some breaking changes very soon in our trunk to enable a totally new set of features bringing Ceilometer beyond basic metering into monitoring and alerting. How many people does one need to build a multi-region cloud? http://freedomhui.com/2013/01/stacklabhow-many-people-it-needs-to-build-a-multi-regions-cloud/ Hui Cheng describes in details the StackLab project. Spearheaded by Sina, Intel, Gamewave, Xi’an Jiaotong University, South China University of Technology and others, it’s a non profit platform to try and test OpenStack. Sina’s OpenStack development team was responsible for the development, operations, and go online initially. More volunteers have joined the effort and are actively being recruited to get involved to this great career, which will accelerate OpenStack popularizing in China. An Image Transfers Service For OpenStack https://tropicaldevel.wordpress.com/2013/01/11/an-image-transfers-service-for-openstack/ John Bresnahan https://tropicaldevel.wordpress.com/ makes the case for a new transfer service component in OpenStack. IaaS clouds must transfer VM images from the repositories in which they reside to compute nodes where they are booted. In the current state of OpenStack images download via HTTP to a nova-compute client speaking to the Glance image service. In the future proposed by John a transfers service would provide more predictable quality of service with horizontal scalability. Check his idea. Ceilomenter is looking for volunteers to take on unassigned blueprints http://lists.openstack/ The team is about to start implementing the blueprints for the g3 milestone, there are few blueprints which still don’t have anyone assigned to them. If you are looking for something useful to code, head over to see the list of things that you could be working on. Remember that contributions during the Grizzly lifecycle will get you a free ticket to the OpenStack Summit. Tips and tricks * By Patrick McGarry http://ceph.com/author/scuttlemonkey/: Building a Public AMI with Ceph and OpenStack http://ceph.com/howto/building-a-public-ami-with-ceph-and-openstack/ * By Davide Guerry: Using JuJu on OpenStack-based UniCloud http://youtu.be/K5-iJ37q23k * By John Bresnaha https://tropicaldevel.wordpress.com/: How to configure OpenStack Glance And Nova Backed By Red Hat Storage https://tropicaldevel.wordpress.com/2013/01/16/openstack-glance-and-nova-backed-by-red-hat-storage/ * By Loïc Dachary http://dachary.org/: Installing OpenStack Folsom on Debian GNU/Linux wheezy http://dachary.org/?p=1791 * By Robert Collins http://rbtcollins.wordpress.com/: Multi-machine parallel testing of nova with testrepository http://rbtcollins.wordpress.com/2013/01/14/multi-machine-parallel-testing-of-nova-with-testrepository/ * By Kyle Mestery http://www.siliconloons.com/: Multi-node OpenStack Folsom devstack http://www.siliconloons.com/?p=395 Upcoming Events * OpenStack Users January 2013 meetup http://www.meetup.com/Indian-OpenStack-User-Group/events/93144352/ Jan 19, 2013 – Bangalore, India Details http://www.meetup.com/Indian-OpenStack-User-Group/events/93144352/ * Openstack Developers Meetup http://wiki.openstack.org/DeveloperMeetupRaanana Jan 20, 2013 – Raanana, Israel Details http://wiki.openstack.org/DeveloperMeetupRaanana * Chef for OpenStack Hack Day http://www.eventbrite.com/event/4395344594 Jan 22, 2013 – Boston, MA Details http://www.eventbrite.com/event/4395344594 * oVirt Workshop http://www.ovirt.org/NetApp_Workshop_January_2013 Jan 22 – 24, 2013 – Sunnyvale, CA Details http://www.ovirt.org/NetApp_Workshop_January_2013 * OpenStack Mini-Conf at Linux Conf Australia https://lca2013.linux.org.au/wiki/Miniconfs/OpenStackJan 29, 2013 – Canberra
[Openstack] [ceilometer] Release status 2012-10-05
Hi all,
Here's a better-late-than-never release status update for Ceilometer 0.9. We're
now a week away from release.
Release in numbers
--
Bugs in progress: 5
Fixes committed: 39
Triaged bugs: 15
Confirmed bugs: 21
New bugs: 1
These numbers were generated using a launchpadlib API script (attached). We
haven't set up a milestone for the 0.9 release; I suggest that we do and will
take care of it today if no-one objects.
The full list of bugs by status is attached for completeness.
Status of roadmap
-
According to http://wiki.openstack.org/EfficientMetering/RoadMap, there's one
bug still in progress that should be targeted for Folsom.0:
1021775: Assignee: jdanjou; Listen Quantum notifications
Julien, what's the situation with this? I know we're technically in feature
freeze but we've got a week left until release and if we can get this committed
and QA'd in time, that would be lovely.
QA status
-
I've no information about the QA status of any of the above bugs. Is this
recorded anywhere? If not, I'd suggest that we record it as a tag on the bug,
thus (liberally stolen from the Launchpad project):
- Not QA'd yet: qa-needstesting
- QA'd; all good: qa-good
- QA'd; bad: qa-bad (can be updated to
qa-needstesting or -good once problems are fixed)
Objections, comments?
Further questions
-
Are there any bugs not listed on the roadmap that are essential for the 0.9
release?
#! /usr/bin/env python
from launchpadlib.launchpad import Launchpad
from launchpadlib.uris import LPNET_SERVICE_ROOT
project = ceilometer
lp = Launchpad.login_anonymously('grabber', LPNET_SERVICE_ROOT)
project_bugtasks = lp.projects[project].searchTasks()
by_status = {}
for bug_task in project_bugtasks:
if bug_task.status not in by_status:
by_status[bug_task.status] = set([bug_task])
else:
by_status[bug_task.status].add(bug_task)
bug_line = {bug}: Assignee: {assignee}; {title}
for status, tasks in by_status.items():
print status
print - * len(status)
print
print Total bugs: %i % len(tasks)
for task in tasks:
bug = task.bug
bug_dict = {
'bug': bug.id,
'assignee': task.assignee or None,
'title': bug.title,
}
print bug_line.format(**bug_dict)
print \n
In Progress
---
Total bugs: 5
1012242: Assignee: https://api.launchpad.net/1.0/~jtran; remove database
access from agent pollsters
1024563: Assignee: https://api.launchpad.net/1.0/~jtran; Problems starting
ceilometer-collector due to missing notifications_topics flag
1046404: Assignee: https://api.launchpad.net/1.0/~jaypipes; Ceilometer would
welcome a chef cookbook
1039069: Assignee: https://api.launchpad.net/1.0/~jdanjou; remove duration
field
1021350: Assignee: https://api.launchpad.net/1.0/~jsimms; add self-
enable/disable check on plugin load
Fix Committed
-
Total bugs: 39
1024093: Assignee: None; remove dependency on nova.service
1055319: Assignee: https://api.launchpad.net/1.0/~spn; Flask package is
missed since tools/pip-requires is not used
1004200: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; make
collector listen on metering queue
1004127: Assignee: https://api.launchpad.net/1.0/~jdanjou; convert to
openstack-common cfg
1023969: Assignee: https://api.launchpad.net/1.0/~jdanjou; Enhance/implement
counter types
1053515: Assignee: https://api.launchpad.net/1.0/~jtran; pep8 not checking
tests subdirectory
1060918: Assignee: https://api.launchpad.net/1.0/~jdanjou; Misleading network
meter names(net_in_int, net_out_int)
1004130: Assignee: https://api.launchpad.net/1.0/~jdanjou; fix logging
configuration
1004198: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; make agent
publish counters to metering queue(s)
1022679: Assignee: https://api.launchpad.net/1.0/~jtran; add authentication
options to mongodb driver
1018443: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; set up doc
build
1004560: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; add listener
for instance exists notifications
1023054: Assignee: https://api.launchpad.net/1.0/~nijaba; Need to add a link
to the roadmap in the docs
1005944: Assignee: https://api.launchpad.net/1.0/~jdanjou; need nova to tell
us when it is deleting instances
1006989: Assignee: https://api.launchpad.net/1.0/~heut2008; add emitting host
field to meter messages
1059765: Assignee: https://api.launchpad.net/1.0/~jdanjou; define constants
for counter types
1060939: Assignee: https://api.launchpad.net/1.0/~jdanjou; Wrong meter name
for disk IO
1006120: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; add metadata
to instance counters
1006366: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; create
developer documentation
1021775: Assignee: https://api.launchpad.net/1.0/~jdanjou; Listen Quantum
Re: [Openstack] [metering] Do we need an API and storage?
On Fri, May 18, 2012 at 4:53 PM, Francis J. Lacoste francis.laco...@canonical.com wrote: On 12-05-18 05:27 AM, Thierry Carrez wrote: You can certainly architect it in a way so that storage and API are optional: expose metering messages on the bus, and provide an optionally-run aggregation component that exposes a REST API (and that would use the metering-consumer client library). That would give deployers the option to poll via REST or implement their own alternate aggregation using the metering-consumer client lib, depending on the system they need to integrate with. Having the aggregation component clearly separate and optional will serve as a great example of how it could be done (and what are the responsibilities of the aggregation component). I would still do a (minimal) client library to facilitate integration, but maybe that's just me :) Right, I like this approach very much. The main thing I'm worried about is that we are building a system that has no use in _itself_. It's all about enabling integration in third-party billing system, but we aren't building such an integration as part of this project. Well, several of us actually *are* building such integration systems at the same time that we are building ceilometer. That's where these requirements are coming from! :-) I don't expect to be releasing all of the code for that integration, but I will release what I can and I am happy to talk about the general requirements and constraints for the rest on the list to help with the design of ceilometer. We could easily implement something that lacks our focus. Maybe, that's an argument for building a simple billing app as part of OpenStack as a proof of concept that the metering system can be integrated. I would not object if you wanted to do that, but I have a high degree of confidence that we can produce something usable and useful without going that far. Sure, interested parties will try to integrate it once we have early versions of it, but that still increase the feedback cycle on our API/architecture hypotheses. I could shorten that feedback cycle if folks would do code reviews for the outstanding items at https://review.stackforge.org/#/q/status:open+project:stackforge/ceilometer,n,zso I could stand up a copy of what has already been implemented. ;-) In all seriousness, it seems reasonable for us to concentrate on the front-end pieces (collecting and storing) for this release, and build a good enough API service to retrieve data. Even if that means I end up having to retrieve all of the raw records and process them myself, I can still get my project done as a proof of concept and we can refine the API as we go along using the experience I (and others) gain this time around. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [heat] Heat 8-13-2012 Planning and Status Meeting
= #heat Meeting = Meeting started by asalkeld at 21:14:37 UTC. The full logs are available at heat/2012/heat.2012-08-13-21.14.log.html . The full logs are available at: http://heat-api.org/heat-irc-meetings/heat.2012-08-13-21.14.log.html All IRC meeting logs are available at: http://heat-api.org/irclogs.html Meeting summary --- * rollcall (sdake, 21:17:23) * zaneb, sdake, imain, asalkeld, shardy, funzo, jpeeler present (sdake, 21:18:30) * shadower present too (sdake, 21:19:07) * v6 additional features (sdake, 21:19:14) * shardy wrapped up openshift (sdake, 21:23:54) * ACTION: shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort (sdake, 21:25:38) * ACTION: zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code (sdake, 21:26:11) * additional features from returning from pto devs - openstack rest orchestraition API and improved CloudWatch implementation (sdake, 21:28:10) * ACTION: slower to investigate s3 integration support with swift (sdake, 21:29:04) * 9/4 release target for v6 (sdake, 21:33:50) * zaneb to work on rest api for v6, sdake start sorting out quantum integration plans in v6, then v7 shardy,sdake,zaneb work on quantum vpc (sdake, 21:45:33) * open items (sdake, 21:45:53) * ACTION: sdake to find out how tempest expects test cases (sdake, 21:52:35) * LINK: https://github.com/openstack/tempest/ (asalkeld, 21:53:04) * unanimous vote to add tempest integration as roadmap item for future versions (sdake, 21:55:35) * as user base increases, add troubleshooting tips to troubleshooting wiki based upon user feedback (sdake, 21:58:49) Meeting ended at 22:06:14 UTC. Action Items * shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort * zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code * slower to investigate s3 integration support with swift * sdake to find out how tempest expects test cases Action Items, by person --- * sdake * sdake to find out how tempest expects test cases * shardy * shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort * zaneb * zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code * **UNASSIGNED** * slower to investigate s3 integration support with swift People Present (lines said) --- * sdake (134) * asalkeld (32) * zaneb (26) * shardy (24) * Slower (12) * jpeeler (7) * mheat-bot (5) * shadower (5) * russellb (2) * funzo (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
Hi Eoghan, Thanks for your reply. As we can see from the document: - Three type of meters are defined in ceilometer: TypeDefinition Cumulative Increasing over time (instance hours) Gauge Discrete items (floating IPs, image uploads) and fluctuating values (disk I/O) Delta Changing over time (bandwidth) Cumulative type is apparent, while even with descriptions gauge and delta type confuse me. Could you explain them through examples or by sharing an use case? Thanks --- Yawei Wu Dalian Hi-Think Computer Technology,Corp. Hi Yawei Wu, The root of the confusion is the fact the cpu meter is reporting the cumlative cpu_time stat from libvirt. This libvirt counter is reset when the associated qemu process is restarted (an artifact of how cpuacct works). So when you stop/start or suspend/resume, a fresh qemu process is sparked up, then the cumulative time is reset. Thanks for bringing this up, as it has implications as to how we meter CPU time and utilization[1]. We may need to start metering the delta between CPU times on subsequent polling cycles, instead of using a cumulative meter (dealing with the edge case where the instance has been restarted within a polling period). Cheers, Eoghan [1] https://review.openstack.org/14921 I am still testing ceilometer now. I am confused about the meter volume in the mongodb. Let's talk about cpu usage. After I create and boot a vm named vm_1, meter data record about cpu usage will be inserted into db in cycle(default 10 minutes). For example,the 'counter_volume' of the first record is '5206000',and the second one is '12389000'. 1) '12389000' nanoseconds means '123.89' seconds or two minutes,it seem like to be 1238.9 seconds actually, is there something wrong ? 2) If I never reboot or suspend vm_1, will the 'counter_volume' of cpu usage record increase all the time ? Just like '8 minutes' - '18 minutes' - '28 minutes' ? 3) If I reboot or suspend vm_1, I find that the 'counter_volume' of cpu usage record will count from zero. Just like '8 minutes' - '18 minutes' - '28 minutes' [- '0 minutes'] -'5 minutes' - '15 minutes'. Does it mean that 'counter_volume' just represents how long has vm_1 been booted up ? 4) This one is about Web API. I find that GET /v1/resources/(resource)/meters/(meter)/volume/sum just return the sum value of all the cpu 'counter_volume', like '8 minutes' + '18 minutes'. Is it reduplicate ? 5) If I want to know how long has vm_1's cpu been used yesterday, how can I do ? It seems like that I have too many questions.. Thank you very much ! --- Yawei Wu Dalian Hi-Think Computer Technology,Corp. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Grizzly-3 development milestone available (Keystone, Glance, Nova, Horizon, Quantum, Cinder)
Cool! Grizzly G3 is on Raring Ringtail! I'm seeing lots of AWESOME new packages here! Like Ajax Console, SPICE proxy, Ceilometer, Nova Cells and Baremetal and go on... But, no Heat package available... Is Heat a requirement to run Grizzly G3? Or can I start testing it now with those packages? Will Grizzly be available for Ubuntu 12.04 with all those features (Ceilometer, Ajax Console, SPICE and etc)? Tks! Thiago On 22 February 2013 10:49, Chuck Short chuck.sh...@canonical.com wrote: Hi. On 13-02-22 06:16 AM, Martinx - ジェームズ wrote: Hi! What is the status of Openstack Grizzly-3 Ubuntu packages? They will be uploaded to today for raring, precise should be uploaded a couple of hours later Can we already set it up using apt-get / aptitude? With packaged Heat, Ceilometer and etc? Yes Which version is recommended to test Grizzly-3, Precise (via testing UCA), Raring? Raring is the most easiest and fastest Is Grizzly planed to be the default Openstack for Raring? Yes Thanks for the AWESOME work on Openstack! Regards, Thiago On 22 February 2013 05:47, Thierry Carrez thie...@openstack.org mailto: thie...@openstack.org** wrote: Hi everyone, The last milestone of the Grizzly development cycle, grizzly-3 is now available for testing. This milestone contains almost all of the features that will be shipped in the final 2013.1 (Grizzly) release on April 4, 2013. This was an extremely busy milestone, with 100 blueprints implemented and more than 450 bugfixes overall. You can find the full list of new features and fixed bugs, as well as tarball downloads, at: https://launchpad.net/**keystone/grizzly/grizzly-3https://launchpad.net/keystone/grizzly/grizzly-3 https://launchpad.net/glance/**grizzly/grizzly-3https://launchpad.net/glance/grizzly/grizzly-3 https://launchpad.net/nova/**grizzly/grizzly-3https://launchpad.net/nova/grizzly/grizzly-3 https://launchpad.net/horizon/**grizzly/grizzly-3https://launchpad.net/horizon/grizzly/grizzly-3 https://launchpad.net/quantum/**grizzly/grizzly-3https://launchpad.net/quantum/grizzly/grizzly-3 https://launchpad.net/cinder/**grizzly/grizzly-3https://launchpad.net/cinder/grizzly/grizzly-3 Those projects are now temporarily feature-frozen (apart from already-granted exceptions) as we switch to testing and bugfixing mode in preparation for our first release candidates. Please test, try the new features, report bugs and help fix them ! Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack https://launchpad.net/%**7Eopenstackhttps://launchpad.net/%7Eopenstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.**launchpad.netopenstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack https://launchpad.net/%**7Eopenstackhttps://launchpad.net/%7Eopenstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Grizzly-3 development milestone available (Keystone, Glance, Nova, Horizon, Quantum, Cinder)
Hi On 13-02-25 01:34 PM, Martinx - ジェームズ wrote: Cool! Grizzly G3 is on Raring Ringtail! I'm seeing lots of AWESOME new packages here! Like Ajax Console, SPICE proxy, Ceilometer, Nova Cells and Baremetal and go on... But, no Heat package available... Is Heat a requirement to run Grizzly G3? Or can I start testing it now with those packages? Heat is not a requirement to run Grizzly G3 Will Grizzly be available for Ubuntu 12.04 with all those features (Ceilometer, Ajax Console, SPICE and etc)?. Most likely Tks! Thiago On 22 February 2013 10:49, Chuck Short chuck.sh...@canonical.com mailto:chuck.sh...@canonical.com wrote: Hi. On 13-02-22 06:16 AM, Martinx - ジェームズ wrote: Hi! What is the status of Openstack Grizzly-3 Ubuntu packages? They will be uploaded to today for raring, precise should be uploaded a couple of hours later Can we already set it up using apt-get / aptitude? With packaged Heat, Ceilometer and etc? Yes Which version is recommended to test Grizzly-3, Precise (via testing UCA), Raring? Raring is the most easiest and fastest Is Grizzly planed to be the default Openstack for Raring? Yes Thanks for the AWESOME work on Openstack! Regards, Thiago On 22 February 2013 05:47, Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org mailto:thie...@openstack.org mailto:thie...@openstack.org wrote: Hi everyone, The last milestone of the Grizzly development cycle, grizzly-3 is now available for testing. This milestone contains almost all of the features that will be shipped in the final 2013.1 (Grizzly) release on April 4, 2013. This was an extremely busy milestone, with 100 blueprints implemented and more than 450 bugfixes overall. You can find the full list of new features and fixed bugs, as well as tarball downloads, at: https://launchpad.net/keystone/grizzly/grizzly-3 https://launchpad.net/glance/grizzly/grizzly-3 https://launchpad.net/nova/grizzly/grizzly-3 https://launchpad.net/horizon/grizzly/grizzly-3 https://launchpad.net/quantum/grizzly/grizzly-3 https://launchpad.net/cinder/grizzly/grizzly-3 Those projects are now temporarily feature-frozen (apart from already-granted exceptions) as we switch to testing and bugfixing mode in preparation for our first release candidates. Please test, try the new features, report bugs and help fix them ! Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack https://launchpad.net/%7Eopenstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack https://launchpad.net/%7Eopenstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Documentation for openstack-java-sdk
Thanks a log, Gui! This helps but would be more useful if you could point
me to some *ceilometer-specific *guides/examples.
On Thu, Jul 11, 2013 at 8:25 PM, Gui Maluf guimal...@gmail.com wrote:
Surely Luis can help you, I've used openstack-java-sdk in one of my
projects, and this is the example Luis gave to me
private static final File TEST_FILE = new File(pom.xml);
private static final String KEYSTONE_AUTH_URL =
https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0;;
private static final String KEYSTONE_USERNAME = ;
private static final String KEYSTONE_PASSWORD = ;
/**
* @param args
*/
public static void main(String[] args) throws Exception {
KeystoneClient keystone = new KeystoneClient(KEYSTONE_AUTH_URL);
//access with unscoped token
Access access = keystone.execute(Authenticate.withPasswordCredentials(
KEYSTONE_USERNAME, KEYSTONE_PASSWORD));
//use the token in the following requests
keystone.setToken(access.getToken().getId());
Tenants tenants = keystone.execute(new ListTenants());
//try to exchange token using the first tenant
if(tenants.getList().size() 0) {
access =
keystone.execute(Authenticate.withToken(access.getToken().getId()).withTenantId(tenants.getList().get(0).getId()));
SwiftClient swiftClient =
newSwiftClient(KeystoneUtils.findEndpointURL(access.getServiceCatalog(),
object-store, null, public), access.getToken().getId());
//swiftClient.execute(new DeleteContainer(navidad2));
swiftClient.execute(new CreateContainer(navidad2));
System.out.println(swiftClient.execute(new ListContainers()));
ObjectForUpload upload = new ObjectForUpload();
upload.setContainer(navidad2);
upload.setName(example2);
upload.setInputStream(new FileInputStream(TEST_FILE));
swiftClient.execute(new UploadObject(upload));
System.out.println(swiftClient.execute(new ListObjects(navidad2, new
HashMapString,
String() {{
put(path, );
}})).get(0).getContentType());
}
}
On Thu, Jul 11, 2013 at 11:31 AM, Endre Karlson
endre.karl...@gmail.comwrote:
I think Luis can answer that?
-- Videresendt melding --
Fra: Jobin Raju George jobin...@gmail.com
Dato: 11. juli 2013 14:38
Emne: [Openstack] Documentation for openstack-java-sdk
Til: openstack lista openstack@lists.launchpad.net
Kopi:
I am trying to query ceilometer using
openstack-java-sdkhttps://github.com/woorea/openstack-java-sdkfor meters
of VM's running on KVM. I am able to get the CPU meters via curl
on the command line but unfortunately I don't find good documentation for
the SDK's for ceilometer.
I have seen this example
programhttps://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java
but
most of it is commented(probably because it is deprecated).
Where can I find good documentation/examples or java programs/snippets?
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
*guilherme* \n
\t *maluf*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] First meeting for the Metering project
On 04/25/2012 09:10 AM, Nick Barcet wrote: Thanks to the 16 persons that took the time to give their preference on Doodle [1]. The poll is now closed and the meeting time that came out of it is: Thursdays at 4pm UTC on #openstack-metering I was not sure if we could use #openstack-meeting as we are not yet an official project, lkm if you have an opinion on this. The proposed agenda can be found at [2]. Anyone is welcome to attend. The meeting occurred as planned, and overran... Nevertheless a lot of good decisions were made, the obviously most important one grin being the project name: ceilometer. For a more complete meeting summary, go visit: http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary Next week we will have for objective to find an agreement on schema and counter definitions. If anyone does not do it before me I'll fire an introduction email on the subject soon (sometime this we), as we agreed to start the discussion via mailing list and use the irc meeting to finalize the choices. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: ceilometer (java implementation)
Forgot to copy the list -- Forwarded message -- From: Luis Gervaso l...@woorea.es Date: Wed, May 9, 2012 at 7:14 PM Subject: Re: [Openstack] ceilometer (java implementation) To: Doug Hellmann doug.hellm...@dreamhost.com I asked for these fields in my previous email, they are not clear enough. After analyze the problem, i ended that metrics can not be calculated per event. We need at least 2 events to calculate the duration (start/end) in some metrics. We can afford this on the counter processes, but it requires an update of the previous pesisted start event. So as we can calculate this info everywhere, and i don't like updates on the counters logic i'm calculating the duration as part of agreggation/mediation process (this will be only a view of the raw data). This way we maintain a very simple logic on the counters: * counters listen for specific events * the interpretation of raw data can be outside On Wed, May 9, 2012 at 7:01 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Wed, May 9, 2012 at 12:46 PM, Luis Gervaso l...@woorea.es wrote: That's fantastic! What I mind is that the counters only should gather event info. Maybe multiple counters listening, collecting info about the system. But only one persist the event. That makes sense. It seems like we will need to split the event-based processing from the polling. The agent that runs on the compute node should poll libvirt (and any other services on that node) and generate counter messages using a metering exchange. A separate set of daemons running in a more central location will watch messages on the metering exchange and save them to the database. Those same processes can watch for notification messages and convert them directly to metering data to be written to the database. That saves the extra traffic from notification messages resulting in several extra metering messages. Other process agreggates the data and creates different views/reports/billing, this should be outside of openstack (since it depends on the business rules, not depends on the infrastructure) If I understand Nick's proposal for the API, we will have some minimal aggregation there but you are right that any other interpretation will probably need to be handled by a custom app. The agreggator, also should offer an api for polling or can be configured throught drivers to push the the 3rd systems Now, in my implementation i'm putting all the stuff on a directory but it can be as well: - AMQP - SQL / NoSQL (the persistent layer shold be interchangeable) So as response to your duration question, this think is out scope for the counter tasks. I think you misunderstood my question. The duration to which I referred was the field in the counter schema (counter_duration). Only the exists notification for instances includes enough information to calculate the duration. The create notification should result in a 0 duration, so that can be ignored. The delete notification should record the data since the end of the last cycle, but does not today (at least it does not seem to). On Wed, May 9, 2012 at 5:30 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Tue, May 8, 2012 at 7:19 PM, Luis Gervaso l...@woorea.es wrote: Hi, I have uploaded a toy version of ceilometer (java implementation). It does implement the first two counters (instance : rabbitmq listener and cpu : polling from libvirt) i need more clarification on the meaining: counter_volume counter_duration counter_datetaime I hope this helps to figure out how to agreggate these data. http://github.com/woorea/ceilometer-java Nice! I have also been experimenting. I have some Python code in https://github.com/dhellmann/metering-prototype that listens for notifications related to instances (create, delete, exists) and converts them to counter output (see spy.py). There is also a pair of scripts for recording an event stream and playing it back (useful for testing, see recorder.py and player.py). I don't have any libvirt polling, yet, though. In the course of looking at the notifications being generated for different scenarios, I discovered that the instance delete messages do not have any duration information right now. I don't know if that is a bug, or if the idea is to figure out the durations from looking at the most recent exists event. What do other people think? Doug -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es
Re: [Openstack] [metering] high-level design proposal
On May 22, 2012, at 5:15 AM, Tom tom.gal...@hp.com wrote: On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Hi Doug That looks nice t but I'm wondering why you've chosen to poll over an event driven approach? (libvirt supports events as far as I can tell). Using events only gets us some of the data we want. It isn't enough know that the vim was launched, we also want to know about the resources it uses over time. If we can get that without polling, then we should investigate that approach. Doug Thanks Tom ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
After experimenting with some of the implementation today, I modified the way the notification plugins list the event_types they want to see. http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1?action=diffrev2=10rev1=9 On Mon, May 21, 2012 at 5:52 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
A version of the code demonstrating using plugins in this way is up for review at https://review.stackforge.org/#/c/45/ On Tue, May 22, 2012 at 5:59 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: After experimenting with some of the implementation today, I modified the way the notification plugins list the event_types they want to see. http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1?action=diffrev2=10rev1=9 On Mon, May 21, 2012 at 5:52 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Agent configuration mechanism
On Tue, Jun 05 2012, Nick Barcet wrote: The main idea is that all agents of a given type should be sending similarly formatted information in order for the information to be usable, hence the need to ensure that configuration info is centrally stored and retrieved. This would rule out, in my mind, the idea that we could use the global flags object, as distribution of the configuration file is left to the cloud implementor and does not lend for easy and synchronized updates of agent config. IMHO this is solving a problem that already exists for all other OpenStack components. A problem that no other OpenStack components tried to resolved, AFAIK. So I don't see why we should try to resolve configuration deployment in ceilometer when it's a much larger issue in the project. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Agent configuration mechanism
On Tue, Jun 05 2012, Nick Barcet wrote: The main idea is that all agents of a given type should be sending similarly formatted information in order for the information to be usable, hence the need to ensure that configuration info is centrally stored and retrieved. This would rule out, in my mind, the idea that we could use the global flags object, as distribution of the configuration file is left to the cloud implementor and does not lend for easy and synchronized updates of agent config. IMHO this is solving a problem that already exists for all other OpenStack components. A problem that no other OpenStack components tried to resolved, AFAIK. So I don't see why we should try to resolve configuration deployment in ceilometer when it's a much larger issue in the project. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] notification metadata
See https://review.stackforge.org/163 for the code. On Tue, Jun 12, 2012 at 11:05 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: I am working on bug 1006120, adding metadata to the instance-related metering events. It's easy to add location data like availability zone to the pollsters because the agent has an object from the database with all of the information about the instance (we will have to change that, but presumably we will be able to get the data via RPC, too). The notification plugins however, only have the data in the incoming message, and those messages don't include all of the data about the instance. My current plan is to have the collector code that processes notifications look up the instance in the database to get the rest of the data. Does anyone have any thoughts on this plan? https://bugs.launchpad.net/ceilometer/+bug/1006120 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] Meeting agenda for Thursday at 16:00 UTC (June 14th, 2012)
Hi, The metering project team holds a weekly meeting in #openstack-meeting, Thursdays at 1600 UTC. Everyone is welcome. http://wiki.openstack.org/Meetings/MeteringAgenda Topics for this week: * Review last week's actions * dhellmann: submit plugin branch for review and merging * dhellmann rename plugin to engine for storage backend ex-plugin-now-engine system * dhellmann: start mapping API queries to database engine methods * nijaba to propose a google spreadsheet calculator to estimate volume of metering message (including nova, swift, cinder, quantum) * Discuss and hopefully agree on meter configuration mechanism [1] * Discuss proposing ceilometer as an incubated project * Prepare documentation and framework for plugin contributors * Establish communication with swift/quantum/cinder on best points of interaction for our agents * Status of the essex compatibility effort that jd___ is leading [1]http://www.mail-archive.com/openstack@lists.launchpad.net/msg12859.html Cheers, -- Nick Barcet nick.bar...@canonical.com aka nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] mapping query API to DB engine API
I have updated the proposed DB engine API to include query methods [1] we will need based on the REST API [2]. I also updated the REST API page in the wiki with references to which method implements each query. I'm not entirely happy with the results because the new DB API methods are all duplicated depending on whether user or project is passed. Exactly one of the two values is always required, so it also didn't feel right to have both be optional and force the engine to validate the arguments. Does anyone else have any thoughts on how to address that? Doug [1] https://github.com/dhellmann/ceilometer/commit/18130bcafabf23871831e9b4913752ebb7b9f3ef [2] http://wiki.openstack.org/EfficientMetering/APIProposalv1 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [infra] Stackforge issues: tests mails
Hi Julien, On 25/06/12 10:15, Julien Danjou wrote: We've a couple of problem on Stackforge currently with at least the ceilometer project. First, I don't receive any email from Gerrit anymore since several days. :-( As we have pointed out several times the Stackforge Gerrit mail is unreliable. This is because the Stackforge setup is hosted in HP Cloud and they have (quite rightly) anti-spam measures such as port 25 rate limiting. We have a plan to resolve this soon but it is not an easy fix. In the mean time the #stackforge-dev IRC channel can be used for Gerrit bot notifications. Secondly, I've added a bunch of tests for Essex on Jenkins days ago, but it seems they are either not complete or not deployed. The changes are: https://github.com/openstack/openstack-ci-puppet/commit/15e8d50de49dccb5958ec241638e7609d722d697 Could someone check this both issues? I can see the jobs on Jenkins just fine and puppet confirms there was no problem with deployment. The Zuul service, however, was not running. This has been fixed. Kind Regards -- Andrew Hutchings - LinuxJedi - http://www.linuxjedi.co.uk/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Stackforge migrating
On Wed, Jul 4, 2012 at 9:00 AM, Andrew Hutchings and...@linuxjedi.co.ukwrote: Hi guys, On 04/07/12 13:20, Andrew Hutchings wrote: There was an initial plan to migrate Stackforge Gerrit and Jenkins to OpenStack Gerrit/Jenkins. There are many reasons for this such as: As far as I can tell the hard work of the migration is now complete. For anyone using Stackforge branches please pull the latest master and rebase any branches you were working on with this. It will update .gitreview to point to the correct server. From then on all reviews should be on review.openstack.org and tests on jenkins.openstack.org. If there are any problems please get in touch with the CI team (note it is 4th July holidays so probably just me today) Thanks, Andrew, it looks like the ceilometer parts are working just fine! Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack] [horizon] Adding quota information to horizon.usage.base ?
Go for it. I assume this is related to https://bugs.launchpad.net/horizon/+bug/1018560 ? If you need to add stuff feel free. Were you thinking the end result would be to display those bars on the overview screen for the project? - Gabriel From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Matt Joyce Sent: Tuesday, August 14, 2012 12:24 PM To: openstack@lists.launchpad.net Subject: [Openstack] [openstack] [horizon] Adding quota information to horizon.usage.base ? Is anyone opposed to me adding quota information to usage.base. ? I'd like to add the graph bars from _launch_details_help.html to provide a quick heads up on where you are in terms of current allocation of resources against quotas. Maybe later we can provide some graphing of deltas in resource use against quota. In that, I don't know if we even track past quotas anywhere. So I am going to avoid jumping into that any time soon. Maybe that's a job for ceilometer. -Matt ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Weekly irc meetings day and time change?
Hello, As we are observing that the current meeting time is not very convenient for a few contributors, we would like to propose to change the date and time to something that would suit a larger number. I propose that we do this in a two phase process: 1/ Each project member that care should send a list of days and times that would be convenient for them as a reply to this message. Please try to avoid a day an time already used by another project as listed on [1] 2/ We then open a poll with a compiled list of proposals and get members vote on it. If you are not a contributor to the project, we are happy to receive your suggestions too, but since this meeting is meant to coordinate between contributors, the result of this poll will be heavily skewed toward active members preferences. [1] http://wiki.openstack.org/Meetings/ Thanks, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Billing in Openstack
Hi, You should read Ceilometer Project [1] and try it into DevStack directly in editing localrc before to run devstack and add : enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector Enjoy ! [1] http://wiki.openstack.org/EfficientMetering Regards, -Emilien On Thu, Aug 30, 2012 at 12:21 PM, Rajesh Avula avula.rajes...@gmail.comwrote: Greetings...!! I would like to generate billing of used resources (CPU, RAM, Disk) in openstack. Below are the details of my setup (Please ask me for more details if require regarding setup) Mainly I have used 2 system to make my setup, below are the details... 1. *On First Laptop *(Dell - Latitude, with 4 GB RAM, Intel Core i5 vpro Processor, 500 GB Hard disk) I have installed a *virtual machine with CentOS (6.2)* on *Xenserver (6.0.2)* and on that *virtual machine I installed openstack* by following below links and some other links from google https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova http://wiki.openstack.org/XenServer 2. *On Second Laptop* (same as above configuration) I have installed *openfiler and made NFS and iSCSI targets* for glance, swift, and for instances. 3. For network I am using BELKIN wireless adapter 5 port L2-switch) I am able to launch Instances from AMI's, instances are getting IP addresses, able to create volumes and attaching to the instances. All I need is, Is there any possibility in openstack where I can define the values / prices for the resources so that users will get the bills accordingly ? Below are the packages installed in my setup for dashboard python-django-horizon-2012.1.1-1.el6.noarch openstack-dashboard-2012.1.1-1.el6.noarch Is there any other packages should be installed to get billing option on dashboard? Any specific configuration required ? Please let me know how to get billing information of the used / using resources. -- Thanks Regards, Rajesh Avula 09831138115 07259024701. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Emilien Macchi *System Engineer* *www.stackops.com | *emilien.mac...@stackops.com** *|* skype:emilien.macchi* * http://www.stackops.com * * ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately. This message is intended exclusively for the person to whom it is addressed and contains privileged and confidential information protected from disclosure by law. If you are not the addressee indicated in this message, you should immediately delete it and any attachments and notify the sender by reply e-mail. In such case, you are hereby notified that any dissemination, distribution, copying or use of this message or any attachments, for any purpose, is strictly prohibited by law. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] StackSherpa JS Github Repository
Hi whether can share more in detail. how to installed and test? On Mon, Sep 3, 2012 at 4:35 AM, Luis Gervaso l...@woorea.es wrote: Hi folks, I just upload the first commit to stacksherpa-js (buggy ui web flows) https://github.com/woorea/stacksherpa-js.git This is will be an alternative OpenStack Management UI to integrate identity (keystone), compute (nova, +glance, +melange), storage (swift) and billing (ceilometer) The architecture proposed is Angular JS + Vertx.io (Java, Groovy and Python) Hope to see your forks! Cheers! -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@ luis.gerv...@gmail.comwoorea.es ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Shake Chen ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 09/05/2012 01:11 PM, surya_prabha...@dell.com wrote: On 09/05/12 15:49, Nick Barcet wrote: PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Isn't it 15.00 UTC? http://www.timeanddate.com/worldclock/fixedtime.html?hour=15min=0sec=0 Absolutely, my bad for thinking I had updated the URL while I obviously had not. Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Release status 2012-10-05
On Fri, Oct 05 2012, Graham Binns wrote: These numbers were generated using a launchpadlib API script (attached). We haven't set up a milestone for the 0.9 release; I suggest that we do and will take care of it today if no-one objects. Agreed, thanks! Status of roadmap - According to http://wiki.openstack.org/EfficientMetering/RoadMap, there's one bug still in progress that should be targeted for Folsom.0: 1021775: Assignee: jdanjou; Listen Quantum notifications Julien, what's the situation with this? I know we're technically in feature freeze but we've got a week left until release and if we can get this committed and QA'd in time, that would be lovely. This has been merged by the time you wrote the mail I think. :-) -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info pgpiYUgPutN1f.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Ceilometer API Glossary
On Wed, Oct 24 2012, 吴亚伟 wrote: I still have questions about the data in the mongodb. First: All the data about source in db is ?,for example: is it correct? Yes, that's what we used for now in the source code, so this is correct. second,when I create a instance,there will be meter data created in the mongodb as resource and meter,but for volume ,there is no any data about volume in the db,so why ?Is there something wrong with my devstack or there is configuration file should be modified? If you're talking about cinder volues, you need to configure and run cinder-volume-usage-audit regularly to get messages. third,I can only see cpu,disk,and network info for instance in the meter data by now ,but there is no memory info,so why? Make sure you enabled the notifications via RabbitMQ in nova. -- Julien Danjou ;; Free Software hacker freelance ;; http://julien.danjou.info pgp9T7Xj8PXY6.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Oct 25 2012, Doug Hellmann wrote: IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. Oh, sure it is. But adding feature and making things more efficient are different things. :) Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Ah, thanks to remind me how ORM are bad and that we now have to fight against it. :) I wish we could use JSON native type from PostgreSQL directly and be efficient! -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info pgplhhIAVMIOb.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Oct 26, 2012, at 4:29 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Doug Hellmann wrote: IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. Oh, sure it is. But adding feature and making things more efficient are different things. :) Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Ah, thanks to remind me how ORM are bad and that we now have to fight against it. :) I wish we could use JSON native type from PostgreSQL directly and be efficient! You could write a different storage driver. ;) Doug -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
Not at all. It means the CPU time consumed is reset to 0, but that's not an issue in itself, the API should be capable to deal with that if you ask for the total usage. Would that total usage be much more apparent if we started metering the delta between CPU times on subsequent polling periods as a gauge measure? (As opposed to treating it as a cumulative measure) /v1/resources/(resource)/meters/(meter)/volume/sum just return the sum value of all the cpu 'counter_volume', like '8 minutes' + '18 minutes'. Is it reduplicate ? Don't understand what you mean, but the CPU counter is a cumulative one, and asking for its sum is a non-sense. You want to ask for (max - min) to get the used value, something which is not in the API yet. I don't think (max - min) would suffice to give an accurate measure of the actual CPU time used, as the counter may have reset multiple times in the course of the requested duration. Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Debugging OpenStack - Video on StackTach v2 and Stacky
Hey! As promised at the summit the latest changes to StackTach are up on github. This is a major change from the original StackTach I introduced earlier this year (and left to wither). Also, I'm including Stacky, which is a new command line tool for StackTach. Here's a video that explains what it's all about, how to install and use it. http://youtu.be/pZgwDHZ3wm0 The repos: https://github.com/rackspace/stacktach https://github.com/rackspace/stacky What we need: + packaging + docs Where we're going: + worker and db improvement ... everything breaks at scale and stacktach is no exception. + Key Performance Indicators (kpi's). Useful for SLA's and other shirt stuff. There is some support in there now (via stacky), but it's busted and inefficient. I'm going to be getting this working again immediately. + Integration with ceilometer. We're duplicating effort here, and are planning to fix that. Please: install, experiment, report bugs, submit pull requests. Look forward to your feedback! -S ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Mon, Nov 05 2012, Doug Hellmann wrote: If we make the current compute agent take an option telling it which pollster namespace to use, then the same framework can load different pollsters. However, there is a fundamental security issue with communicating from an agent running inside a tenant's OS image using the RPC stack. At DreamHost, and I suspect at other providers, that RPC network is completely isolated from any tenant networks. We would not want a tenant to be able to listen to the message bus, and definitely would not want it to be able to write anything to the message bus. What makes you think an agent would run inside an instance? I mean, this is not what this is about, we're talking about hardware running OS. -- Julien Danjou # Free Software hacker freelance # http://julien.danjou.info pgp7Pk0gZAZEV.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou jul...@danjou.info wrote: On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: I'm a little confused now... ;) Is the bare metal run by the platform operator the physical machine? What do you mean with the bare metal run to replace virtual instances for any project? AFAIU, bare-metal provisionning is about using hardware (bare-metal) rather than virtual instances as a flavor in Nova. For such case, we won't be able to poll anything about the hardware. For hardware ran by the operator, this will be doable. We can still talk to the IPMI controller for the machine, which will get us lots of info, without running agents in the host os. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Cloud Services ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [StackTach][Metering] Nova summary report for the PHB in your life ...
Hey! We just added a feature to StackTach for generating 24-hr summary reports. The output can be seen here: https://gist.github.com/SandyWalsh/4946226 It will identify requests failures, with timing information, by major actions (create, resize, rescue, delete, snapshot, etc) by image type. (note: aux are all the minor actions, like list, show, etc) https://github.com/rackspace/stacktach (.../reports/pretty.py) We're in the process of porting all this stuff over to ceilometer, but that's a longer term effort. We've also got a more detailed report with causes of errors, cell breakdown, distros, etc. It's early and kind of thrown together, but handy all the same. Cheers! -Sandy PS PHB: http://en.wikipedia.org/wiki/Pointy-haired_Boss ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [DevStack] Does a Swift/Keystone only install require AMQP?
It doesn't - the AMQP is needed for the Nova/Glance/Cinder/Ceilometer integration and that internal RPC mechanism that they use. -joe On Feb 19, 2013, at 4:53 PM, Everett Toews everett.to...@rackspace.com wrote: Hi All, When I was doing a Swift/Keystone only install with DevStack I used the following in my localrc disable_all_services enable_service key swift mysql Then stack.sh paused with the error message ERROR: at least one rpc backend must be enabled, set one of 'rabbit', 'qpid', 'zeromq' via ENABLED_SERVICES. So I blindly added rabbit to enable_service and the error went away. Then I did the PR https://review.openstack.org/#/c/22333/ to change the DevStack README.md file. But then I got the question, is rabbit really necessary? Does anyone know if a Swift/Keystone only install requires AMQP? Thanks, Everett ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Grizzly-3 development milestone available (Keystone, Glance, Nova, Horizon, Quantum, Cinder)
Now that we are at feature freeze, is there a description of incompatible configuration or api changes that happened since Folsom? That is, a description of how deploying grizzly differs from deploying folsom. -David On 2/22/2013 7:21 AM, Thierry Carrez wrote: Martinx - ジェームズ wrote: What is the status of Openstack Grizzly-3 Ubuntu packages? Can we already set it up using apt-get / aptitude? With packaged Heat, Ceilometer and etc? Which version is recommended to test Grizzly-3, Precise (via testing UCA), Raring? Is Grizzly planed to be the default Openstack for Raring? I suspect it will take a few days for grizzly-3 to appear in Ubuntu, as the tarballs were cut a few hours ago. As far as I know, Grizzly is indeed the planned default OpenStack for Raring. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] StackTach and Stacy github repos have moved ...
Thanks for the heads up, Sandy. There was some talk in the last release cycle about possibly merging some or all of the StackTach functionality with Ceilometer. Is that still on the horizon or has that idea been scuttled? Best, -jay On 03/25/2013 09:42 AM, Sandy Walsh wrote: Hi, how are you? Me? Right as rain, thanks for asking. Due to a recent reorg of the Rackspace github repo, StackTach and Stacky are now moved under the rackerlabs organization. The new coordinates are: https://github.com/rackerlabs/stacktach https://github.com/rackerlabs/stacky Please update your .git/config accordingly. Cheers -S PS A bunch of other projects have moved to this new location, couldn't hurt to have a peek and see if it affects you. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Monitoring with Nagios
Hi Stackers, I am Krishnaprasad from University of Mainz, Germany and as a part of a project, we have developed APIs that gets monitoring information from Nagios for the virtual machines running in OpenStack cloud. The component aggregates basic VM information from OpenStack, available resource from Libvirt and run time resource usage details from Nagios. The aggregated information is published via REST APIs and the APIs are programmed in Java. I would like to have a feedback from the community whether our monitoring using Nagios can be used or integrated with Ceilometer / Healthnmon. In this regard, I can share information regarding what we have developed so far and what we intend to do further. Thanks Krishnaprasad ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] How to query healthnmon?
Hey stackers! I am trying to use healthnmon to get meters regarding CPU/Memory utilization. But I can't find any documentation as to how to query healthnmon to get these data. I would appreciate if someone would point me to a link for the same or give an example how I could query healthnmon. I have referred to this link but it isn't very helpful: - Utilization data https://wiki.openstack.org/wiki/Utilizationdata I have a controller node and a compute node on two different hosts(both on Ubuntu 12.04). I couldn't get ceilometer to fetch these data for me and I read healthnmon is meant for this purpose. Please correct me if I am wrong. -- Thanks and regards, Jobin Raju George Third Year, Information Technology College of Engineering Pune Alternate e-mail: georgejr10...@coep.ac.in ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Ceilometer] Problems with query fields in filters
Hi everybody,
I have the following problems with query fields in filters.
1) Filtering by metadata.size in v2/meters/image apparently is not working.
The following request returns also samples with metadata.size=0.
GET http://10.10.10.10:8777/v2/meters/image
{
q: [{
field: project_id,
op: eq,
value: 77b461539c8542909f67b29939ec87dd
},
{
field: timestamp,
op: ge,
value: 2013-07-11T13:36:00
},
{
field: timestamp,
op: lt,
value: 2013-07-11T13:39:00
},
{
field: metadata.size,
op: gt,
value: 0
}]
}
I am probably doing something wrong but I can't figure out what.
2) Filtering by counter_volume returns the folloving error message:
error_message={debuginfo: null, faultcode: Client, faultstring:
Unknown argument: \counter_volume\: unrecognized query field}
Is this a bug or filtering by counter-volume is not allowed?
Many thanks
Alex
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On 05/04/2012 03:11 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 6:03 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com wrote: On 05/04/2012 02:25 PM, Doug Hellmann wrote: On Fri, May 4, 2012 at 12:29 PM, Nick Barcet nick.bar...@canonical.com mailto:nick.bar...@canonical.com [...] In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional source field in the event record. This should be a short identifier. That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? Correct. We'll just need a sane default for Keystone. You're focusing on source as the origin for the identity information, but it seems like a layer sitting on top of OpenStack may well use Keystone for identity but generate different billable events. Should source be the origin of the metric data being sent to ceilometer? As I see it, source should remain the same as long as you can correlate project_id and user_id from the same source. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? True. It might be over-engineering to try to keep the info in the same place as well, but the impact on the system would be negligible. I'd be happy either ways :) It will be easier to add it later if we really need it than to take it out if we decide we don't. Agreed. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] schema and counter definitions
On 05/09/2012 11:11 PM, Doug Hellmann wrote: On Wed, May 9, 2012 at 3:07 PM, Tomasz Paszkowski ss7...@gmail.com mailto:ss7...@gmail.com wrote: On Wed, May 9, 2012 at 8:02 PM, Doug Hellmann doug.hellm...@dreamhost.com mailto:doug.hellm...@dreamhost.com wrote: Nice! For production code I think we are going to want to separate collection from storage, aren't we? We don't want each compute node to require access to the database server (that's an issue with nova that they are trying to fix during the folsom release, IIRC). Yes. Part of the code responsible for amqp support is not functional yet :( OK, that's what I thought. We all seem to be reinventing different parts of the services that we will eventually need, which is good for education but may be wasting a bit of energy. Is it premature to start talking a little more about architecture so we can start splitting up the implementation work and focusing that energy differently? There is a lot of work we can do independently of the remaining decisions outlined in http://wiki.openstack.org/Meetings/MeteringAgenda. Hi, It looks like the architecture of metering is indeed always implemented in similar ways. I had discussions with a company yesterday about their own metering implementation (which will be used in production soon) and it also has an architecture matching what has been proposed so far in ceilometer. I added a few points to the architecture chapter in the wiki: http://wiki.openstack.org/EfficientMetering#Architecture including a note summarizing the conclusions of the discussion regarding need for an independent ceilometer agent in addition to the existing meters provided by the OpenStack components. What do you think ? -- Tomasz Paszkowski SS7, Asterisk, SAN, Datacenter, Cloud Computing +48500166299 tel:%2B48500166299 ___ Mailing list: https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack https://launchpad.net/%7Eopenstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ? l...@enovance.com ? +33 1 49 70 99 82 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] API Extensibility (was: External API definition)
On Thu, May 10, 2012 at 9:22 AM, Loic Dachary l...@enovance.com wrote:
Another item that we need to discuss is extensibility of this API.
Hi,
Here is a proposal, which we could discuss further during the meeting.
GET extension=param1=fooparam2=bar
The API looks up /usr/share/ceilometer/extensions/.py and loads it.
The module defines a query function that takes the following arguments:
Andrew Bogott is doing some work with a standardized plugin mechanism for
Nova which will eventually be put in the common lib for all of the
projects. We should look at his work and use it, rather than inventing
something else. I think it will eventually use setuptools entrypoints,
which eliminates the need to worry about search paths.
Why would the extension be a query parameter, rather than a URL component?
That is, why wouldn't the extension just add new endpoints that could be
queried directly using their own API? Maybe I don't understand the types of
extensions you are thinking of.
* QUERY_STRING (i.e. extension=param1=fooparam2=bar )
* a handler to the storage
* a pointer to the configuration (assuming there is a /etc/ceilometer.ini
file, for instance)
The query function would return the result. For instance { 'in': 20001,
'out': 489324 } if asked for aggregated network usage.
Multiple extensions directories could be specified and searched, allowing
a mixture of extensions provided in ceilometer and custom extensions to
address specific needs or to mature an new extension.
The primary benefit of defining extensions in this way is to avoid complex
conventions for aggregations or other advanced operations. If the API was
to impose a syntax or conventions to say sum this field and this one and
display the result ordered in this way and grouped by this field and this
one, it would be redundant with the query language of the underlying data.
For instance, if using mongodb, it would be difficult to expose all the
features provided by http://www.mongodb.org/display/DOCS/Aggregation or
http://www.mongodb.org/display/DOCS/MapReduce
Cheers
--
Loïc Dachary Chief Research Officer
// eNovance labs http://labs.enovance.com
// ✉ l...@enovance.com ☎ +33 1 49 70 99 82
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 05/09/12 12:19 +0200, Nick Barcet wrote: PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. It's at 1am now (I might make it periodically). Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list - nijaba to open a bug for cookbook and assign to jaypipes - nijaba to add architecture image to project /doc rather than link from google - nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later * Discussion on session proposal for the summit http://wiki.openstack.org/EfficientMetering/GrizzlySummit * Discussion on alternating meeting time to allow presence from down under Well thanks for trying. I guess I am the only one down this end. The main things I'd like to achieve is to add alarms and monitoring to ceilometer. To achieve this we would need: - auth on api so normal users can access it. - ablility to post custom data via a rest api - ablility to increase the polling frequency on selected resources (this could it's self be metered and charged for) - add alarms and alarm history - add a notification mechanism - could just be rpc / http hook I am happy to wait for summit and discuss it there. I see you have added a wiki entry for it (let my know if you need me to do anything more to make this happen): http://wiki.openstack.org/EfficientMetering/GrizzlySummit/BeyondMetering I added this a while ago: http://wiki.openstack.org/AddingAlarmsToCeilometer Regards Angus * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
Hi Yawei Wu, The root of the confusion is the fact the cpu meter is reporting the cumlative cpu_time stat from libvirt. This libvirt counter is reset when the associated qemu process is restarted (an artifact of how cpuacct works). So when you stop/start or suspend/resume, a fresh qemu process is sparked up, then the cumulative time is reset. Thanks for bringing this up, as it has implications as to how we meter CPU time and utilization[1]. We may need to start metering the delta between CPU times on subsequent polling cycles, instead of using a cumulative meter (dealing with the edge case where the instance has been restarted within a polling period). Cheers, Eoghan [1] https://review.openstack.org/14921 I am still testing ceilometer now. I am confused about the meter volume in the mongodb. Let's talk about cpu usage. After I create and boot a vm named vm_1, meter data record about cpu usage will be inserted into db in cycle(default 10 minutes). For example,the 'counter_volume' of the first record is '5206000',and the second one is '12389000'. 1) '12389000' nanoseconds means '123.89' seconds or two minutes,it seem like to be 1238.9 seconds actually, is there something wrong ? 2) If I never reboot or suspend vm_1, will the 'counter_volume' of cpu usage record increase all the time ? Just like '8 minutes' - '18 minutes' - '28 minutes' ? 3) If I reboot or suspend vm_1, I find that the 'counter_volume' of cpu usage record will count from zero. Just like '8 minutes' - '18 minutes' - '28 minutes' [- '0 minutes'] -'5 minutes' - '15 minutes'. Does it mean that 'counter_volume' just represents how long has vm_1 been booted up ? 4) This one is about Web API. I find that GET /v1/resources/(resource)/meters/(meter)/volume/sum just return the sum value of all the cpu 'counter_volume', like '8 minutes' + '18 minutes'. Is it reduplicate ? 5) If I want to know how long has vm_1's cpu been used yesterday, how can I do ? It seems like that I have too many questions.. Thank you very much ! --- Yawei Wu Dalian Hi-Think Computer Technology,Corp. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
Thanks for putting that together Sandy. Very nice! Thanks! From my perspective, there are two major things that are undesirable for us: 1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data sinks from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation. Makes sense. You mentioned at the summit about json structured log files, which would help. But additionally the existing notifier driver can easily be used to write to - a file - the logs (I think one exists) - a different rabbit queue (something we're considering) so it need never hit the production rabbit. We'd need a different worker mechanism for parsing the log files (keeping track of what's been done, etc). It might be hard to horizontally scale that though. Also, I'm not sure if latency of writing the log over the network or aggregating the local log files would be any different than writing to another rabbit (which is largely in memory). I'll update the doc to reflect this. (I assume we're talking monitoring here, I would never put instrumentation stuff in the queues) 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc. Yup ... Tach has a driver based notifier mechanism. Easy to do. Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different sinks. Hmm, really not a fan of the decorator approach. Makes the code really ugly. They'd be everywhere. Not sure if I can get over it. :D -S On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks, Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Blueprint proposal: Drop setuptools_git for including data/config files
On 12/18/2012 05:29 PM, Sascha Peilicke wrote: On 12/17/2012 04:47 PM, Thomas Goirand wrote: This means that absolutely all of our packages have to embed a patch in debian/patches to fix the wrong MANIFEST.in. We've spent quite some time on that. Or rather, should I say: it's a real time waster. While I do agree that the MANIFEST.in should be generated automatically, I don't think it should be stored in a wrong way on github. So it should either contain something meaningful or be removed. In their current state, these files are just worthless. Exactly! On 12/18/2012 07:50 PM, Mark McLoughlin wrote: 1) You don't build packages from the tarballs produced by upstream. I don't understand why you don't use tarballs, but I'm willing to assume it's not something you can (or want to) change. In many ways, it's very convenient to do what we do. We could go back to use tarballs, but if it is avoidable, I want to keep the current system. 2) Instead you use git-buildpackage which I know nothing about. Shortly, git-buildpackage uses an upstream branch (often called master) containing upstream code, and a Debian branch containing that, plus the debian folder. Typing git-buildpackage does all the magic needed to build a Debian package, with the added bonus that you don't have to build where your package is stored (usually, we build in ../build-area), which means no risk to modify any files in your Git repository. 3) You work from git repositories forked from upstream e.g. http://anonscm.debian.org/gitweb/?p=openstack/python-novaclient.git;a=shortlog;h=refs/heads/debian/experimental which AFAICT just add a debian/ directory to the source tree Yes. 4) 'get-vcs-source' somehow generates a tarball from this tree. I'm guessing it does 'git archive' rather than 'python setup.py sdist' but I'm not sure. Some more digging turns up: http://anonscm.debian.org/gitweb/?p=openstack/openstack-pkg-tools.git;a=blob_plain;f=pkgos.make;hb=HEAD and it seems that yes, you're using 'git archive' That's correct, you've found out quite well. Also, get-vcs-source fetches the master branch from upstream (git add remote, git fetch...). The repository is either git://github.com/openstack/package-name.git, or we just override the UPSTREAM_GIT variable before including /usr/share/openstack-pkg-tools/pkgos.make (that's needed for python modules which are not on the github.com/openstack repo). 5) I'm guessing there are two issues with these 'git archive' generated tarballs - (a) there's no versioninfo file so setuptools doesn't know have a version number and (b) there's no .git directory so setuptools doesn't have an accurate way of building a manifest I'm not completely clear what setuptools commands are failing because of issue (b) All the above is exactly right! One of the reasons we are happy to use git archive is that this produces a Debian orig.tar.xz (notice the xz, and not just gz) from *any* commit, if we want to. Let me explain how it works. Let's say I want to make a snapshot release of Ceilometer for the commit hash 23ff2f9bbfc14e435c4c04fba473cf2a829b (this is an actual real life example, in fact...). Then, to do that, I just do: git checkout master git log # find out what's the name of the tag... git tag 2013.1_g0.4+23ff2f9bbf 23ff2f9bbfc14e435c4c04fba473cf2a829b git checkout debian/experimental git merge -X theirs 23ff2f9bbfc14e435c4c04fba473cf2a829b Then I edit my debian/changelog so it matches the tag: example debian/changelog ceilometer (2013.1~g0.4+23ff2f9bbf-1) experimental; urgency=low * Initial release (Closes: #693406). -- Thomas Goirand z...@debian.org Wed, 14 Nov 2012 14:41:52 + end of example debian/changelog (the important bit is of course the version number above) Then I generate the orig.tar.xz: ./debian/rules get-vcs-source cp ../ceilometer_2013.1~g0.4+23ff2f9bbf.orig.tar.xz ../build-area Then I just build: git-buildpackage Another reason why git is convenient, is that we can cherry pick -x stuff from upstream, do branching, etc. Well, do I really have to convince people of this list that git is convenient? :) Probably not. 6) However, you seem to be saying that issue (b) isn't an issue but rather inaccurate MANIFEST.in files are the problem. How exactly are they causing a problem. If we delete those files from upstream git, does that work for you? Well, it be better if the MANIFEST.in could be stored correctly in upstream Git repositories, so we wouldn't have to deal with them. Currently, we embed a patch in debian/patches to fix them. 7) You're also describing some issue where 'clean targets' (I don't know what they are? Similar to 'make clean'?) are causing commands like 'git fetch origin' to be run? What exactly is going on here? Is this because the versioninfo file gets deleted by the cleanup and our setup.py
Re: [Openstack] New code name for networks
Anne Gentle wrote: I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I promise, it wasn't the world's most fun session. :) I'm sure. :) I think I don't have much regret but do feel sorry that I don't know more. The Etherpad is here: https://etherpad.openstack.org/ProjectsReNaming I think there is much more value to codenames than just avoiding the cost of a rename when the project becomes OpenStack. This was captured in the session: Codenames drawbacks and benefits (-) Lack of trademark protection (-) Confusing to newcomers (-) Shadow their more official counterparts (+) Short names are highly-convenient and efficient, often less ambiguous (in conversations, executables, modules...) (+) Help building project and team identity (+) Separate the project itself from its functional scope (so they remain valid even if that scope evolves) Those last two bits are pretty essential. There is a reason why a functional description cannot be used as a project name. The project (as in, the code repository and the community of contributors around it) is *distinct* from the functional scope of what its code does. Take Ceilometer (OpenStack Metering). What happens when they grow to cover Monitoring ? You rename the project to OpenStack Metering and Monitoring ? Or you keep the partial functional description ? I'd rather avoid to rename everything every time a project evolves. Those renames are *extremely* costly, as we'll soon enough realize. I find the confusing argument pretty weak myself. Brands are used everywhere, so we are used to make the translation between a name and a function. Microsoft named its desktop environment Windows, rather than Operating system or Desktop environment, and it took people about 5 minutes to get used to it. Go with kumquat, but don't call the CLI kumquat. Call your team kumquat and your repo kumquat. If you call the CLI os-metering, you'll have to rename it when the scope expands, or live with a name that looks like a functional description but is not an accurate one. I very much prefer to call it ceilometer. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Problems with query fields in filters
Hi Angus
without the metadata.size filter I get all samples, with sizeo and with size=0.
Same thing with the metadata.size filter using the following operators:
gt,lt,ne,le,ge
The most strange thing is that if I use the operator eq I get an empty
response!
Thanks
Alex
-Original Message-
From: Angus Salkeld [mailto:asalk...@redhat.com]
Sent: venerdì 12 luglio 2013 12:55
To: openstack@lists.launchpad.net
Cc: Alessandro Barabesi
Subject: Re: [Openstack] [Ceilometer] Problems with query fields in filters
On 12/07/13 11:33 +0200, Julien Danjou wrote:
On Fri, Jul 12 2013, Alessandro Barabesi wrote:
I have the following problems with query fields in filters.
1) Filtering by metadata.size in v2/meters/image apparently is not
working. The following request returns also samples with metadata.size=0.
GET http://10.10.10.10:8777/v2/meters/image
{
q: [{
field: project_id,
op: eq,
value: 77b461539c8542909f67b29939ec87dd
},
{
field: timestamp,
op: ge,
value: 2013-07-11T13:36:00
},
{
field: timestamp,
op: lt,
value: 2013-07-11T13:39:00
},
{
field: metadata.size,
op: gt,
value: 0
I suspect the gt operator is not working (it's probably using eq
given what you are getting). But certainly a bug.
I'd just remove this last query and see what you get with the first 3 queries.
-Angus
}]
}
I am probably doing something wrong but I can't figure out what.
That seems correct from the top of my head. If that really doesn't
work, feel free to open a bug.
2) Filtering by counter_volume returns the folloving error message:
error_message={debuginfo: null, faultcode: Client,
faultstring: Unknown argument: \counter_volume\: unrecognized
query field}
Is this a bug or filtering by counter-volume is not allowed?
Try `volume' instead of `counter_volume'. But I am not sure it's
currently allowed -- in that case opening a bug can be good idea too.
--
Julien Danjou
// Free Software hacker / freelance consultant //
http://julien.danjou.info
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
A question/comment about the scope of the schema or maybe the architecture. Assuming the services will provide the instrumentation to populate the raw metric data, it seems likely that you will need to define an interface between the services/agents that are providing the data and the metering system which stores the generated metric data in the database (as opposed to having the services write directly to the DB). Is the schema intended to be this kind of interop format between the services and the meter's datastore or just the end result of the storage? Thanks, Dan Dyer On Thu, May 3, 2012 at 11:10 AM, Loic Dachary l...@enovance.com wrote: On 05/03/2012 02:22 PM, Loic Dachary wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTChttp://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. I propose an agenda based on the discussions we had on this list. http://wiki.openstack.org/Meetings/MeteringAgenda Topic : schema and counter definitions * counter definitions * Proposed http://wiki.openstack.org/EfficientMetering#Counters * schema definition * Proposed http://wiki.openstack.org/EfficientMetering#Storage * discuss storage assumptions * the storage will store all events * no aggregated value is permanently stored * discuss API assumptions * the API provide a sum() function to aggregate values * the API may transparently store results of the sum function in a cache * discuss event collection * events are collected from a components when possible * ceilometer agent is installed on a node when the a component does not provide the value * contribute to the component instead of developping a ceilometer agent plugin * engaging discussions with core components * nova * cinder * glance * swift * quantum * open discussion For the record, the first two points used all the time but that was the goal of the meeting. The other points would have been nice to discuss but can each be turned into a mailing list thread ;-) == #openstack-meeting Meeting == Meeting started by dachary at 16:00:16 UTC. The full logs are available athttp://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html . Meeting summary --- * actions from previous meetings (dachary, 16:00:36) * creation of the ceilometer project (dachary, 16:00:36) * The repository for the ceilometer project has been created (dachary, 16:00:36) * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) * and the first commit was successfully reviewed and merged today https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) * meeting organisation (dachary, 16:01:03) * This is 1/5 meetings to decide the architecture of the Metering project https://launchpad.net/ceilometer (dachary, 16:01:03) * Today's focus is on the definition of the counters / meters and the associated schema for the storage (dachary, 16:01:03) * It is the conclusion of the discussions held on the mailing list and the goal is to make a final choice that will then be implemented. (dachary, 16:01:03) * The meeting is time boxed and there will not be enough time to introduce inovative ideas and research for solutions. (dachary, 16:01:03) * The debate will be about the pro and cons of the options already discussed on the mailing list. (dachary, 16:01:03) * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, 16:01:03) * counter definitions (dachary, 16:02:10) * Proposed http://wiki.openstack.org/EfficientMetering#Counters (dachary, 16:02:10) * ACTION: dachary fix the note for net_float still talks about number of floating IPs (dachary, 16:09:18) * ACTION: jd___ include Number of object in Swift, Number of containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table (dachary, 16:10:11) * ACTION: dachary add note about the fact that the resource_id for the object count is the container_id (dachary, 16:21:44) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. (dachary, 16:25:35) * ACTION: jd___ document the resource_id for each counter (dachary, 16:30:33) * ACTION: jd___ describes the general table schema and then something that says for each counter exactly what goes in the fields of that table and show how secondary field counters are recorded in the in the schema too (dachary, 16:33:27) * AGREED: s/counter/meter/ (dachary, 16:37:11) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. ? (dachary, 16:37:38) * AGREED: http
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Daniel Dyer dan.dye...@gmail.com wrote: A question/comment about the scope of the schema or maybe the architecture. Assuming the services will provide the instrumentation to populate the raw metric data, it seems likely that you will need to define an interface between the services/agents that are providing the data and the metering system which stores the generated metric data in the database (as opposed to having the services write directly to the DB). Is the schema intended to be this kind of interop format between the services and the meter's datastore or just the end result of the storage? Just the end result, we have a discussion and decision on May 24th regarding the internal API for the agents to use when communicating on the queue. http://wiki.openstack.org/Meetings/MeteringAgenda#Meeting%20topics Thanks, Dan Dyer On Thu, May 3, 2012 at 11:10 AM, Loic Dachary l...@enovance.com wrote: On 05/03/2012 02:22 PM, Loic Dachary wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTChttp://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. I propose an agenda based on the discussions we had on this list. http://wiki.openstack.org/Meetings/MeteringAgenda Topic : schema and counter definitions * counter definitions * Proposed http://wiki.openstack.org/EfficientMetering#Counters * schema definition * Proposed http://wiki.openstack.org/EfficientMetering#Storage * discuss storage assumptions * the storage will store all events * no aggregated value is permanently stored * discuss API assumptions * the API provide a sum() function to aggregate values * the API may transparently store results of the sum function in a cache * discuss event collection * events are collected from a components when possible * ceilometer agent is installed on a node when the a component does not provide the value * contribute to the component instead of developping a ceilometer agent plugin * engaging discussions with core components * nova * cinder * glance * swift * quantum * open discussion For the record, the first two points used all the time but that was the goal of the meeting. The other points would have been nice to discuss but can each be turned into a mailing list thread ;-) == #openstack-meeting Meeting == Meeting started by dachary at 16:00:16 UTC. The full logs are available athttp://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html . Meeting summary --- * actions from previous meetings (dachary, 16:00:36) * creation of the ceilometer project (dachary, 16:00:36) * The repository for the ceilometer project has been created (dachary, 16:00:36) * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) * and the first commit was successfully reviewed and merged today https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) * meeting organisation (dachary, 16:01:03) * This is 1/5 meetings to decide the architecture of the Metering project https://launchpad.net/ceilometer (dachary, 16:01:03) * Today's focus is on the definition of the counters / meters and the associated schema for the storage (dachary, 16:01:03) * It is the conclusion of the discussions held on the mailing list and the goal is to make a final choice that will then be implemented. (dachary, 16:01:03) * The meeting is time boxed and there will not be enough time to introduce inovative ideas and research for solutions. (dachary, 16:01:03) * The debate will be about the pro and cons of the options already discussed on the mailing list. (dachary, 16:01:03) * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, 16:01:03) * counter definitions (dachary, 16:02:10) * Proposed http://wiki.openstack.org/EfficientMetering#Counters (dachary, 16:02:10) * ACTION: dachary fix the note for net_float still talks about number of floating IPs (dachary, 16:09:18) * ACTION: jd___ include Number of object in Swift, Number of containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table (dachary, 16:10:11) * ACTION: dachary add note about the fact that the resource_id for the object count is the container_id (dachary, 16:21:44) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. (dachary, 16:25:35) * ACTION: jd___ document the resource_id for each counter (dachary, 16:30:33) * ACTION: jd___ describes the general table schema and then something that says for each counter exactly what goes in the fields of that table and show how secondary field counters
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
On Thu, May 10, 2012 at 12:17 AM, Daniel Dyer dan.dye...@gmail.com wrote: A question/comment about the scope of the schema or maybe the architecture. Assuming the services will provide the instrumentation to populate the raw metric data, it seems likely that you will need to define an interface between the services/agents that are providing the data and the metering system which stores the generated metric data in the database (as opposed to having the services write directly to the DB). Is the schema intended to be this kind of interop format between the services and the meter's datastore or just the end result of the storage? It may be both, at first, but we also may find some benefit to letting them diverge later so I don't think we need to make it a hard requirement. Thanks, Dan Dyer On Thu, May 3, 2012 at 11:10 AM, Loic Dachary l...@enovance.com wrote: On 05/03/2012 02:22 PM, Loic Dachary wrote: Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTChttp://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. Everyone is welcome. I propose an agenda based on the discussions we had on this list. http://wiki.openstack.org/Meetings/MeteringAgenda Topic : schema and counter definitions * counter definitions * Proposed http://wiki.openstack.org/EfficientMetering#Counters * schema definition * Proposed http://wiki.openstack.org/EfficientMetering#Storage * discuss storage assumptions * the storage will store all events * no aggregated value is permanently stored * discuss API assumptions * the API provide a sum() function to aggregate values * the API may transparently store results of the sum function in a cache * discuss event collection * events are collected from a components when possible * ceilometer agent is installed on a node when the a component does not provide the value * contribute to the component instead of developping a ceilometer agent plugin * engaging discussions with core components * nova * cinder * glance * swift * quantum * open discussion For the record, the first two points used all the time but that was the goal of the meeting. The other points would have been nice to discuss but can each be turned into a mailing list thread ;-) == #openstack-meeting Meeting == Meeting started by dachary at 16:00:16 UTC. The full logs are available athttp://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html . Meeting summary --- * actions from previous meetings (dachary, 16:00:36) * creation of the ceilometer project (dachary, 16:00:36) * The repository for the ceilometer project has been created (dachary, 16:00:36) * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) * and the first commit was successfully reviewed and merged today https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) * meeting organisation (dachary, 16:01:03) * This is 1/5 meetings to decide the architecture of the Metering project https://launchpad.net/ceilometer (dachary, 16:01:03) * Today's focus is on the definition of the counters / meters and the associated schema for the storage (dachary, 16:01:03) * It is the conclusion of the discussions held on the mailing list and the goal is to make a final choice that will then be implemented. (dachary, 16:01:03) * The meeting is time boxed and there will not be enough time to introduce inovative ideas and research for solutions. (dachary, 16:01:03) * The debate will be about the pro and cons of the options already discussed on the mailing list. (dachary, 16:01:03) * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, 16:01:03) * counter definitions (dachary, 16:02:10) * Proposed http://wiki.openstack.org/EfficientMetering#Counters (dachary, 16:02:10) * ACTION: dachary fix the note for net_float still talks about number of floating IPs (dachary, 16:09:18) * ACTION: jd___ include Number of object in Swift, Number of containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table (dachary, 16:10:11) * ACTION: dachary add note about the fact that the resource_id for the object count is the container_id (dachary, 16:21:44) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. (dachary, 16:25:35) * ACTION: jd___ document the resource_id for each counter (dachary, 16:30:33) * ACTION: jd___ describes the general table schema and then something that says for each counter exactly what goes in the fields of that table and show how secondary field counters are recorded in the in the schema too (dachary, 16:33:27
Re: [Openstack] [metering] Where can I consume messages from metering system?
On 25/05/12 10:55 +0200, Szymon Grzybowski wrote:
Hi,
Hi Szymon,
I have very simerlar needs, see comments inline...
(I am working on the heat project https://github.com/heat-api/heat;)
We had a plan to write some plugin which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.
My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:
1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification Alert less than 15% free ram on host}
2. HostA hits less than 15% of free ram and sends notification Alert less
than 15% free ram on HostA.
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.
I am trying to implement AWS::CloudWatch::Alarm
http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html
This defines really simerlar rules to what you are describing, and also
allows users to post (rest) statistics back to a statistics Collecting
server. We could then do fun things like autoscaling and HA recovery.
This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.
Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.
I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data) and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:
- write plugin for agent to send proper alert when metering data hits
proper rules (just like in above scenario with 15% free ram)
- write service which will consume this alert (listen proper topic in
queue) and fetch data from ceilometer collector?? (or it's backend via API)
and find out which VM should be migrated and where (or delegate Where to
nova-scheduler)
I could help out with work on rules and receiving guest statistics if
ceilometer guys are ok with this extension.
Another idea. Are there any Host states in Openstack? For example
Maintenance state? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to maintenance.
2. There goes alert HostA maintenance.
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to active.
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
Everything should be event-driven (alert-notification, host-state-change,
administrator's action).
I'm also wondering what about this
ResourceMonitorAlertsandNotificationshttp://wiki.openstack.org/ResourceMonitorAlertsandNotifications,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).
Me too (I also have been wondering about it), I was quite keen on this
and contacted the authors, but no response...
Regards
Angus Salkeld
First question: Does it make sense? Are such mechanisms implemented in
Openstack right now? Or is it worth to implement?
Second question (if first's answer isn't negative): How and when can I use
Re: [Openstack] Billing in Openstack
On 08/30/2012 03:40 AM, Emilien Macchi wrote: Hi, You should read Ceilometer Project [1] and try it into DevStack directly in editing localrc before to run devstack and add : enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector Enjoy ! [1] http://wiki.openstack.org/EfficientMetering If fully second Emilien's suggestions. As a quick note, I'd like to level set your expectation as Ceilometer is metering tool, not a full billing tool. A full billing solution needs 3 components: - Metering: to know what has happened on the cloud - Rating: which transforms what has happened into price to bill (line items) - Billing: which accumulates line items and create a bill to send to customer and collect money Which means that you will still need component 2 and 3 to do billing after you have deployed Ceilometer. Hope this helps. Nick On Thu, Aug 30, 2012 at 12:21 PM, Rajesh Avula avula.rajes...@gmail.com mailto:avula.rajes...@gmail.com wrote: Greetings...!! I would like to generate billing of used resources (CPU, RAM, Disk) in openstack. Below are the details of my setup (Please ask me for more details if require regarding setup) Mainly I have used 2 system to make my setup, below are the details... 1. *On First Laptop *(Dell - Latitude, with 4 GB RAM, Intel Core i5 vpro Processor, 500 GB Hard disk) I have installed a *virtual machine with CentOS (6.2)* on *Xenserver (6.0.2)* and on that *virtual machine I installed openstack* by following below links and some other links from google https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova http://wiki.openstack.org/XenServer 2. *On Second Laptop* (same as above configuration) I have installed *openfiler and made NFS and iSCSI targets* for glance, swift, and for instances. 3. For network I am using BELKIN wireless adapter 5 port L2-switch) I am able to launch Instances from AMI's, instances are getting IP addresses, able to create volumes and attaching to the instances. All I need is, Is there any possibility in openstack where I can define the values / prices for the resources so that users will get the bills accordingly ? Below are the packages installed in my setup for dashboard python-django-horizon-2012.1.1-1.el6.noarch openstack-dashboard-2012.1.1-1.el6.noarch Is there any other packages should be installed to get billing option on dashboard? Any specific configuration required ? Please let me know how to get billing information of the used / using resources. -- Thanks Regards, Rajesh Avula 09831138115 07259024701. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Emilien Macchi *System Engineer* **www.stackops.com* http://www.stackops.com |*emilien.mac...@stackops.com mailto:emilien.mac...@stackops.com *|*skype:emilien.macchi* * http://www.stackops.com * * ADVERTENCIA LEGAL Le informamos, como destinatario de este mensaje, que el correo electrónico y las comunicaciones por medio de Internet no permiten asegurar ni garantizar la confidencialidad de los mensajes transmitidos, así como tampoco su integridad o su correcta recepción, por lo que STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales circunstancias. Si no consintiese en la utilización del correo electrónico o de las comunicaciones vía Internet le rogamos nos lo comunique y ponga en nuestro conocimiento de manera inmediata. Este mensaje va dirigido, de manera exclusiva, a su destinatario y contiene información confidencial y sujeta al secreto profesional, cuya divulgación no está permitida por la ley. En caso de haber recibido este mensaje por error, le rogamos que, de forma inmediata, nos lo comunique mediante correo electrónico remitido a nuestra atención y proceda a su eliminación, así como a la de cualquier documento adjunto al mismo. Asimismo, le comunicamos que la distribución, copia o utilización de este mensaje, o de cualquier documento adjunto al mismo, cualquiera que fuera su finalidad, están prohibidas por la ley. * PRIVILEGED AND CONFIDENTIAL We hereby inform you, as addressee of this message, that e-mail and Internet do not guarantee the confidentiality, nor the completeness or proper reception of the messages sent and, thus, STACKOPS TECHNOLOGIES S.L. does not assume any liability for those circumstances. Should you not agree to the use of e-mail or to communications via Internet, you are kindly requested to notify us immediately
Re: [Openstack] grizzly swift keystone, http to 8080/8888 wont work
Thanks for your quick reply, Simon,
The role ResellerAdmin does exists and looks good, does it?
root@ns-proxy01:/etc/swift# keystone user-get ceilometer
+--+--+
| Property | Value |
+--+--+
| email | |
| enabled | True |
|id| cde44fe9c6d446da99ea370b88ec7d63 |
| name |ceilometer|
| tenantId | 054ca85bca2e44c29cf4730e1450517f |
+--+--+
root@ns-proxy01:/etc/swift# keystone user-role-list --user-id
cde44fe9c6d446da99ea370b88ec7d63 --tenant-id
054ca85bca2e44c29cf4730e1450517f
+--+---+--+--+
|id| name | user_id
|tenant_id |
+--+---+--+--+
| c2df2bc0fd6f404794565f10cc0e5e7a | ResellerAdmin |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
| 9fe2ff9ee4384b1894a90878d3e92bab |_member_ |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
+--+---+--+--+
And i can see ceilometer log entrys, counting bytes. So that looks good.
My issue it, that with the old swauth setup there was a real simple web
based user manager.
surfing to http://my.swift.proxy:/auth/; was the entry url to this
sort of user manager. But now, after the change to keystone, i get http
result codes like 412 or 401.
Since i inherit this setup i even do not know for sure if this
swift-user-manager it actually a part of swift. i believe so.
Can please one confirm which urls do work on swift-proxy http port
8080/ (proxy-server.conf - [DEFAULT] - bind_port). Should /auth/
return a page?
Thank you. Axel
Am 16.04.13 12:41, schrieb Simon Pasquier:
Hi,
I'm not sure to understand exactly your issue but since your setup
includes ceilometer, I can just give you a hint for the ceilometer/swift
integration.
You have to create a 'ResellerAdmin' role and assign that role to your
ceilometer user. Alternatively you can define the 'reseller_admin_role'
parameter (default value=ResellerAdmin) in the [filter:authtoken]
section of /etc/swift/proxy-server.conf.
Cheers,
Simon
Le 16/04/2013 12:04, Axel Christiansen a écrit :
Dear List,
i got stuck with a setup of openstack grizzly. This setup consists of:
- swift proxy 1.0.8.1
- swift storage nodes 1.0.8.1
- keystone
- ceilometer
I kept browsing the web and reading openstack docs for days now and
can't just get it working right. Because of openstacks diversity a
wasn't able to find something really similar to my situation.
The thing is, i changed swift-proxy from using swauth to keystone.
Keystone and swift-proxy do interact all right as fare as i can say.
What i can't get working is that simple webpage which gave the ability
to log in as superuser, adding new user and so on. It is that webpart
that connects to the proxy on port 8080, respectively port .
Thx o lot for taking a look into this.
Axel
Theses are the browser urls i try:
(delay_auth_decision = 1)
http://the.swift.proxy:/auth/
bad url
Apr 16 11:49:31 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
txcfde073b9ffe4f379da392056e2176de)
Apr 16 11:49:31 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
deflate', 'Host': 'backend', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0)
Gecko/20100101 Firefox/20.0', 'Connection': 'close', 'Content-Type':
None}, 'environ': {'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET',
'PATH_INFO': '/auth/', 'SERVER_PROTOCOL': 'HTTP/1.0', 'HTTP_USER_AGENT':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101
Firefox/20.0', 'HTTP_CONNECTION': 'close', 'eventlet.posthooks': [],
'SERVER_NAME': '10.42.44.101', 'REMOTE_ADDR': '10.42.44.5',
'eventlet.input': eventlet.wsgi.Input object at 0x1d93f10,
'wsgi.url_scheme': 'http', 'SERVER_PORT': '', 'wsgi.input':
swift.common.utils.InputProxy object at 0x2691050, 'HTTP_HOST':
'backend', 'swift.cache': swift.common.memcached.MemcacheRing object at
0x268a750, 'wsgi.multithread': True, 'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'wsgi.version': (1, 0), 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once':
False, 'wsgi.errors': swift.common.utils.LoggerFileObject object at
0x1656190, 'wsgi.multiprocess': False, 'HTTP_ACCEPT_LANGUAGE':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Re: [Openstack] grizzly swift keystone, http to 8080/8888 wont work
The mystery seems solved. There it a webadmin for swauth.
https://github.com/gholt/swauth#web-admin-install
Does there exists is similar thing for keystone?
Regards, Axel
Am 16.04.13 14:53, schrieb Axel Christiansen:
Thanks for your quick reply, Simon,
The role ResellerAdmin does exists and looks good, does it?
root@ns-proxy01:/etc/swift# keystone user-get ceilometer
+--+--+
| Property | Value |
+--+--+
| email | |
| enabled | True |
|id| cde44fe9c6d446da99ea370b88ec7d63 |
| name |ceilometer|
| tenantId | 054ca85bca2e44c29cf4730e1450517f |
+--+--+
root@ns-proxy01:/etc/swift# keystone user-role-list --user-id
cde44fe9c6d446da99ea370b88ec7d63 --tenant-id
054ca85bca2e44c29cf4730e1450517f
+--+---+--+--+
|id| name | user_id
|tenant_id |
+--+---+--+--+
| c2df2bc0fd6f404794565f10cc0e5e7a | ResellerAdmin |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
| 9fe2ff9ee4384b1894a90878d3e92bab |_member_ |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
+--+---+--+--+
And i can see ceilometer log entrys, counting bytes. So that looks good.
My issue it, that with the old swauth setup there was a real simple web
based user manager.
surfing to http://my.swift.proxy:/auth/; was the entry url to this
sort of user manager. But now, after the change to keystone, i get http
result codes like 412 or 401.
Since i inherit this setup i even do not know for sure if this
swift-user-manager it actually a part of swift. i believe so.
Can please one confirm which urls do work on swift-proxy http port
8080/ (proxy-server.conf - [DEFAULT] - bind_port). Should /auth/
return a page?
Thank you. Axel
Am 16.04.13 12:41, schrieb Simon Pasquier:
Hi,
I'm not sure to understand exactly your issue but since your setup
includes ceilometer, I can just give you a hint for the ceilometer/swift
integration.
You have to create a 'ResellerAdmin' role and assign that role to your
ceilometer user. Alternatively you can define the 'reseller_admin_role'
parameter (default value=ResellerAdmin) in the [filter:authtoken]
section of /etc/swift/proxy-server.conf.
Cheers,
Simon
Le 16/04/2013 12:04, Axel Christiansen a écrit :
Dear List,
i got stuck with a setup of openstack grizzly. This setup consists of:
- swift proxy 1.0.8.1
- swift storage nodes 1.0.8.1
- keystone
- ceilometer
I kept browsing the web and reading openstack docs for days now and
can't just get it working right. Because of openstacks diversity a
wasn't able to find something really similar to my situation.
The thing is, i changed swift-proxy from using swauth to keystone.
Keystone and swift-proxy do interact all right as fare as i can say.
What i can't get working is that simple webpage which gave the ability
to log in as superuser, adding new user and so on. It is that webpart
that connects to the proxy on port 8080, respectively port .
Thx o lot for taking a look into this.
Axel
Theses are the browser urls i try:
(delay_auth_decision = 1)
http://the.swift.proxy:/auth/
bad url
Apr 16 11:49:31 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
txcfde073b9ffe4f379da392056e2176de)
Apr 16 11:49:31 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
deflate', 'Host': 'backend', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0)
Gecko/20100101 Firefox/20.0', 'Connection': 'close', 'Content-Type':
None}, 'environ': {'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET',
'PATH_INFO': '/auth/', 'SERVER_PROTOCOL': 'HTTP/1.0', 'HTTP_USER_AGENT':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101
Firefox/20.0', 'HTTP_CONNECTION': 'close', 'eventlet.posthooks': [],
'SERVER_NAME': '10.42.44.101', 'REMOTE_ADDR': '10.42.44.5',
'eventlet.input': eventlet.wsgi.Input object at 0x1d93f10,
'wsgi.url_scheme': 'http', 'SERVER_PORT': '', 'wsgi.input':
swift.common.utils.InputProxy object at 0x2691050, 'HTTP_HOST':
'backend', 'swift.cache': swift.common.memcached.MemcacheRing object at
0x268a750, 'wsgi.multithread': True, 'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application
Re: [Openstack] grizzly swift keystone, http to 8080/8888 wont work
Hi,
I'm not sure to understand exactly your issue but since your setup
includes ceilometer, I can just give you a hint for the ceilometer/swift
integration.
You have to create a 'ResellerAdmin' role and assign that role to your
ceilometer user. Alternatively you can define the 'reseller_admin_role'
parameter (default value=ResellerAdmin) in the [filter:authtoken]
section of /etc/swift/proxy-server.conf.
Cheers,
Simon
Le 16/04/2013 12:04, Axel Christiansen a écrit :
Dear List,
i got stuck with a setup of openstack grizzly. This setup consists of:
- swift proxy 1.0.8.1
- swift storage nodes 1.0.8.1
- keystone
- ceilometer
I kept browsing the web and reading openstack docs for days now and
can't just get it working right. Because of openstacks diversity a
wasn't able to find something really similar to my situation.
The thing is, i changed swift-proxy from using swauth to keystone.
Keystone and swift-proxy do interact all right as fare as i can say.
What i can't get working is that simple webpage which gave the ability
to log in as superuser, adding new user and so on. It is that webpart
that connects to the proxy on port 8080, respectively port .
Thx o lot for taking a look into this.
Axel
Theses are the browser urls i try:
(delay_auth_decision = 1)
http://the.swift.proxy:/auth/
bad url
Apr 16 11:49:31 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
txcfde073b9ffe4f379da392056e2176de)
Apr 16 11:49:31 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
deflate', 'Host': 'backend', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0)
Gecko/20100101 Firefox/20.0', 'Connection': 'close', 'Content-Type':
None}, 'environ': {'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET',
'PATH_INFO': '/auth/', 'SERVER_PROTOCOL': 'HTTP/1.0', 'HTTP_USER_AGENT':
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101
Firefox/20.0', 'HTTP_CONNECTION': 'close', 'eventlet.posthooks': [],
'SERVER_NAME': '10.42.44.101', 'REMOTE_ADDR': '10.42.44.5',
'eventlet.input': eventlet.wsgi.Input object at 0x1d93f10,
'wsgi.url_scheme': 'http', 'SERVER_PORT': '', 'wsgi.input':
swift.common.utils.InputProxy object at 0x2691050, 'HTTP_HOST':
'backend', 'swift.cache': swift.common.memcached.MemcacheRing object at
0x268a750, 'wsgi.multithread': True, 'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'wsgi.version': (1, 0), 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once':
False, 'wsgi.errors': swift.common.utils.LoggerFileObject object at
0x1656190, 'wsgi.multiprocess': False, 'HTTP_ACCEPT_LANGUAGE':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'swift.trans_id':
'txcfde073b9ffe4f379da392056e2176de', 'CONTENT_TYPE': None,
'HTTP_ACCEPT_ENCODING': 'gzip, deflate'}}
Apr 16 11:49:31 ns-proxy01 swift-proxy Authorizing as anonymous (txn:
txcfde073b9ffe4f379da392056e2176de)
Apr 16 11:49:31 ns-proxy01 swift-proxy 10.42.44.5 10.42.44.5
16/Apr/2013/09/49/31 GET /auth/ HTTP/1.0 412 -
Mozilla/5.0%20%28Macintosh%3B%20Intel%20Mac%20OS%20X%2010.8%3B%20rv%3A20.0%29%20Gecko/20100101%20Firefox/20.0
- - 7 - txcfde073b9ffe4f379da392056e2176de - 0.0003 -
(delay_auth_decision = 0)
http://the.swift.proxy:/auth/
401 Unauthorized
Apr 16 11:56:35 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
tx508b08866bbc410399543d98cafa2856)
Apr 16 11:56:35 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
deflate', 'Host': 'backend', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0)
Gecko/20100101 Firefox/20.0', 'Connection': 'close', 'Cache-Control':
'max-age=0', 'Content-Type': None}, 'environ': {'SCRIPT_NAME': '',
'REQUEST_METHOD': 'GET', 'PATH_INFO': '/auth/', 'SERVER_PROTOCOL':
'HTTP/1.0', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X
10.8; rv:20.0) Gecko/20100101 Firefox/20.0', 'HTTP_CONNECTION': 'close',
'eventlet.posthooks': [], 'SERVER_NAME': '10.42.44.101', 'REMOTE_ADDR':
'10.42.44.5', 'eventlet.input': eventlet.wsgi.Input object at
0x1fa41d0, 'wsgi.url_scheme': 'http', 'SERVER_PORT': '',
'wsgi.input': swift.common.utils.InputProxy object at 0x1fa40d0,
'HTTP_HOST': 'backend', 'swift.cache':
swift.common.memcached.MemcacheRing object at 0x288e750,
'wsgi.multithread': True, 'HTTP_CACHE_CONTROL': 'max-age=0',
'HTTP_ACCEPT':
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'wsgi.version': (1, 0), 'GATEWAY_INTERFACE': 'CGI/1.1', 'wsgi.run_once':
False, 'wsgi.errors': swift.common.utils.LoggerFileObject object at
0x185e190, 'wsgi.multiprocess': False, 'HTTP_ACCEPT_LANGUAGE':
'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'swift.trans_id':
'tx508b08866bbc410399543d98cafa2856', 'CONTENT_TYPE': None,
'HTTP_ACCEPT_ENCODING': 'gzip, deflate'}}
export
[Openstack-ubuntu-testing-notifications] Build Fixed: utopic_juno_ceilometer_trunk #1489
Title: utopic_juno_ceilometer_trunk General InformationBUILD SUCCESSBuild URL:https://jenkins.qa.ubuntu.com/job/utopic_juno_ceilometer_trunk/1489/Project:utopic_juno_ceilometer_trunkDate of build:Thu, 06 Nov 2014 03:51:23 -0500Build duration:12 minBuild cause:Started by an SCM changeBuilt on:pkg-builderHealth ReportWDescriptionScoreBuild stability: 4 out of the last 5 builds failed.20ChangesNo ChangesConsole Output[...truncated 11302 lines...]INFO:root:Installing build artifacts into /var/lib/jenkins/www/aptDEBUG:root:['reprepro', '--waitforlock', '10', '-Vb', '/var/lib/jenkins/www/apt', 'include', 'utopic-juno', 'ceilometer_2014.2+git201411060351~utopic-0ubuntu1_amd64.changes']Exporting indices... looking for changes in 'utopic-juno|main|amd64'... replacing '/var/lib/jenkins/www/apt/dists/utopic-juno/main/binary-amd64/Packages' (uncompressed,gzipped)Successfully created '/var/lib/jenkins/www/apt/dists/utopic-juno/Release.gpg.new'Successfully created '/var/lib/jenkins/www/apt/dists/utopic-juno/InRelease.new'Deleting files no longer referenced...deleting and forgetting pool/main/c/ceilometer/ceilometer-agent-central_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-compute_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-ipmi_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-notification_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-alarm-evaluator_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-alarm-notifier_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-api_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-collector_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-common_2014.2+git201410081838~utopic-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/python-ceilometer_2014.2+git201410081838~utopic-0ubuntu1_all.debINFO:root:Storing current commit for next build: b1fca33e5f4c5443389d131a212a058d2306ac23INFO:root:Complete command log:INFO:root:Destroying schroot.apt-get -y install python-setuptools gitpython setup.py sdistbzr branch lp:~ubuntu-server-dev/ceilometer/juno ceilometerdch -b -D utopic --newversion 1:2014.2+git201411060351~utopic-0ubuntu1 Automated Ubuntu testing build:git log -n1 --no-merges --pretty=format:%Hgit log f8f63d4b15ce68797d6e16943bd85efb19a77752..HEAD --no-merges --pretty=format:[%h] %sdch -a [b1fca33] Handle poorly formed individual sensor readingsdch -a [4683a15] Opening stable/junodch -a [9899b6f] Fix recording failure for system pollsterdch -a [619291b] Add oslo.db to config generatordch -a [5b966ba] Manually updated translationsdch -a [aa15b2d] Fix neutron client to catch 404 exceptionsdch -a [fc22a04] Fix OrderedDict usage for Python 2.6dch -a [cac2141] Include a 'node' key and value in ipmi metadatadch -a [e607892] clean path in swift middlewaredch -a [0e499f8] Updated from global requirementsdebcommitapt-get -y install equivs devscripts bzr bzr-builddeb git quilt gnupg piupartsbzr builddeb -S -- -sa -us -ucbzr builddeb -S -- -sa -us -ucdebsign -k9935ACDC ceilometer_2014.2+git201411060351~utopic-0ubuntu1_source.changesmk-build-deps -i -r -t apt-get -y /tmp/tmptbqIMV/ceilometer/debian/controlsbuild -d utopic-juno -n -A ceilometer_2014.2+git201411060351~utopic-0ubuntu1.dscdput ppa:openstack-ubuntu-testing/juno ceilometer_2014.2+git201411060351~utopic-0ubuntu1_source.changesreprepro --waitforlock 10 -Vb /var/lib/jenkins/www/apt include utopic-juno ceilometer_2014.2+git201411060351~utopic-0ubuntu1_amd64.changesEmail was triggered for: FixedTrigger Success was overridden by another trigger and will not send an email.Sending email for trigger: Fixed-- Mailing list: https://launchpad.net/~openstack-ubuntu-testing-notifications Post to : openstack-ubuntu-testing-notifications@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-ubuntu-testing-notifications More help : https://help.launchpad.net/ListHelp
[Openstack-ubuntu-testing-notifications] Build Fixed: vivid_kilo_ceilometer_trunk #33
Title: vivid_kilo_ceilometer_trunk General InformationBUILD SUCCESSBuild URL:https://jenkins.qa.ubuntu.com/job/vivid_kilo_ceilometer_trunk/33/Project:vivid_kilo_ceilometer_trunkDate of build:Sat, 06 Dec 2014 20:55:51 -0500Build duration:16 minBuild cause:Started by user anonymousBuilt on:pkg-builderHealth ReportWDescriptionScoreBuild stability: 3 out of the last 5 builds failed.40ChangesNo ChangesConsole Output[...truncated 10210 lines...] Uploading ceilometer_2014.3+git201412062055~vivid-0ubuntu1_source.changes: done.Successfully uploaded packages.INFO:root:Installing build artifacts into /var/lib/jenkins/www/aptDEBUG:root:['reprepro', '--waitforlock', '10', '-Vb', '/var/lib/jenkins/www/apt', 'include', 'vivid-kilo', 'ceilometer_2014.3+git201412062055~vivid-0ubuntu1_amd64.changes']Exporting indices... looking for changes in 'vivid-kilo|main|amd64'... replacing '/var/lib/jenkins/www/apt/dists/vivid-kilo/main/binary-amd64/Packages' (uncompressed,gzipped)Successfully created '/var/lib/jenkins/www/apt/dists/vivid-kilo/Release.gpg.new'Successfully created '/var/lib/jenkins/www/apt/dists/vivid-kilo/InRelease.new'Deleting files no longer referenced...deleting and forgetting pool/main/c/ceilometer/ceilometer-agent-central_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-compute_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-ipmi_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-agent-notification_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-alarm-evaluator_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-alarm-notifier_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-api_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-collector_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/ceilometer-common_2014.3+git201412040630~vivid-0ubuntu1_all.debdeleting and forgetting pool/main/c/ceilometer/python-ceilometer_2014.3+git201412040630~vivid-0ubuntu1_all.debINFO:root:Storing current commit for next build: 3dba1b1bda683d9aeecdf877712300997c0b0c1bINFO:root:Complete command log:INFO:root:Destroying schroot.apt-get -y install python-setuptools gitpython setup.py sdistbzr branch lp:~ubuntu-server-dev/ceilometer/kilo ceilometerdch -b -D vivid --newversion 1:2014.3+git201412062055~vivid-0ubuntu1 Automated Ubuntu testing build:git log -n1 --no-merges --pretty=format:%Hgit log 6a45cf514e23caaa9aa45753368aaef714385adf..HEAD --no-merges --pretty=format:[%h] %sdch -a [3dba1b1] Updated from global requirementsdch -a [180b97b] Rely on VM UUID to fetch metrics in libvirtdch -a [6737e0b] Initializing a longer resource id in DB2 nosql backenddch -a [1707533] Sync oslo-incubator code to latestdch -a [f379b96] [MongoDB] Fix bug with 'bad' chars in metadatas keysdch -a [db5b19e] Override retry_interval in MongoAutoReconnectTestdch -a [cb3b962] add sahara and heat eventsdch -a [54548f0] add keystone events to definitionsdebcommitapt-get -y install equivs devscripts bzr bzr-builddeb git quilt gnupg piupartsbzr builddeb -S -- -sa -us -ucbzr builddeb -S -- -sa -us -ucdebsign -k9935ACDC ceilometer_2014.3+git201412062055~vivid-0ubuntu1_source.changesmk-build-deps -i -r -t apt-get -y /tmp/tmpP3idN3/ceilometer/debian/controlsbuild -d vivid-kilo -n -A ceilometer_2014.3+git201412062055~vivid-0ubuntu1.dscdput ppa:openstack-ubuntu-testing/kilo ceilometer_2014.3+git201412062055~vivid-0ubuntu1_source.changesreprepro --waitforlock 10 -Vb /var/lib/jenkins/www/apt include vivid-kilo ceilometer_2014.3+git201412062055~vivid-0ubuntu1_amd64.changesEmail was triggered for: FixedTrigger Success was overridden by another trigger and will not send an email.Sending email for trigger: Fixed-- Mailing list: https://launchpad.net/~openstack-ubuntu-testing-notifications Post to : openstack-ubuntu-testing-notifications@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-ubuntu-testing-notifications More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On Tue, May 22, 2012 at 4:05 AM, Endre Karlson endre.karl...@gmail.comwrote: If I'm understanding this correctly, the Collector is kind of like a Agent in Qantum (It sits on a machine doing stuff and passing info upstream). If you look at the approach they have now in Quantum Agent it's writing directly to the DB. But looking at the next version they seem to be moving to having the Agent send data upstream to the Plugin in Quantum. Why not do something similar? I mean if you have a MQ cluster in a deployment I think it makes more sense to have 1 thing that handles the db stuff then having each Collector connect to the db.. That was the goal, but I may have swapped the terminology around. For ceilometer, the agent runs on the compute node and writes only to the message queue. The collector runs in a central location and writes to the database. The number of collectors you need will depend on the number of messages being generated, but the architecture supports running several in parallel in a way that each instance does not need to be aware of the others. Doug Endre. 2012/5/22 Nick Barcet nick.bar...@canonical.com On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
My point of concern. \ If an agent is being built into the compute nodes, that would best be a split out project. Two major reasons. First and foremost sub projects should not be spinning up their own agents. Secondly, there is a use case of agents outside of metering. If an agent is to be built it is a not insignificant change in architecture for openstack. -Matt On Tue, May 22, 2012 at 7:30 AM, Doug Hellmann doug.hellm...@dreamhost.comwrote: On Tue, May 22, 2012 at 4:05 AM, Endre Karlson endre.karl...@gmail.comwrote: If I'm understanding this correctly, the Collector is kind of like a Agent in Qantum (It sits on a machine doing stuff and passing info upstream). If you look at the approach they have now in Quantum Agent it's writing directly to the DB. But looking at the next version they seem to be moving to having the Agent send data upstream to the Plugin in Quantum. Why not do something similar? I mean if you have a MQ cluster in a deployment I think it makes more sense to have 1 thing that handles the db stuff then having each Collector connect to the db.. That was the goal, but I may have swapped the terminology around. For ceilometer, the agent runs on the compute node and writes only to the message queue. The collector runs in a central location and writes to the database. The number of collectors you need will depend on the number of messages being generated, but the architecture supports running several in parallel in a way that each instance does not need to be aware of the others. Doug Endre. 2012/5/22 Nick Barcet nick.bar...@canonical.com On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https
Re: [Openstack] [metering] high-level design proposal
On Tue, May 22, 2012 at 12:53 PM, Matt Joyce matt.jo...@cloudscaling.comwrote: My point of concern. \ If an agent is being built into the compute nodes, that would best be a split out project. Two major reasons. First and foremost sub projects should not be spinning up their own agents. Secondly, there is a use case of agents outside of metering. If an agent is to be built it is a not insignificant change in architecture for openstack. This agent will run on the compute host, next to nova-compute, not inside the VM. Is that still a concern? -Matt On Tue, May 22, 2012 at 7:30 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Tue, May 22, 2012 at 4:05 AM, Endre Karlson endre.karl...@gmail.comwrote: If I'm understanding this correctly, the Collector is kind of like a Agent in Qantum (It sits on a machine doing stuff and passing info upstream). If you look at the approach they have now in Quantum Agent it's writing directly to the DB. But looking at the next version they seem to be moving to having the Agent send data upstream to the Plugin in Quantum. Why not do something similar? I mean if you have a MQ cluster in a deployment I think it makes more sense to have 1 thing that handles the db stuff then having each Collector connect to the db.. That was the goal, but I may have swapped the terminology around. For ceilometer, the agent runs on the compute node and writes only to the message queue. The collector runs in a central location and writes to the database. The number of collectors you need will depend on the number of messages being generated, but the architecture supports running several in parallel in a way that each instance does not need to be aware of the others. Doug Endre. 2012/5/22 Nick Barcet nick.bar...@canonical.com On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack
[Openstack] [metering] resource metadata changes and billing
tl;dr: Ceilometer should ignore resource metadata when computing sums or maximum values for counters through the API. One of the things we discussed early during the design meetings was the need to track metadata along with resources so providers could use the metadata to determine the rate to charge for using the resource (for example, flavor or availability group of an instance). While working on the mongodb driver, I've been thinking about how that requirement changes what the API we defined needs to do. At first I thought we would need to try to return multiple values from the sum and max calls so the values could be associated with the resource metadata and a given time range. I've decided that implementing that inside the ceilometer API will be very complex, and unlikely to produce the correct result. I want to work through my reasoning here in case someone else can find a fault in it or propose a solution I wasn't able to find. First, the scenario: A user boots an instance, lets it run for some time (period 1), then changes the metadata in some way that does not result in the instance being recreated but does result in something the provider would decide introduces a different charge structure. For example, the amount of RAM allocated to the instance might be increased. After running with the new settings for a period of time (period 2), the user changes them back to their original value and the instance continues to run (period 3). The specific change to the metadata doesn't matter (RAM is just an example), except that the metadata change should not require an instance to be recreated because when that happens the user actually gets a new instance (at least I believe they do based on feedback during an early meeting, please correct me if I'm wrong). Getting a new instance simplifies things immensely, since the new instance is a completely new resource and so we can ignore those cases for the rest of this discussion. Another important point in the way the scenario is constructed is that the metadata values go from value A to B and then back to their original value A. That means any signature we calculate for the metadata will be the same during periods 1 and 3. Now we would like for v1/USERS/USER_ID/RESOURCES/instance_id/cpu/VOLUME to return the amount of CPU time used by the instance. However, if that time is billed at a different rate depending on the size of the server then the RAM change will cause a difference in billing rates. We therefore need to return a sequence of 3-tuples containing the total for the counter, the resource metadata, and the time range during which the metadata was in effect. There are two problems implementing this in a generic way in the API server. First, it turns out to be surprisingly difficult to write an efficient query to compute that time range in the case described because it is not easy to recognize the ranges for period 1 and period 3 as being interrupted by period 2 (finding min or max for a value is easy, but finding the endpoint of period 1 is not because the signature for the metadata is the same for period 1 and 3). Second, while calculating ranges is difficult in itself, what is even more difficult is recognizing *important* changes in the metadata that actually imply a change in the billing rate. The logic for that is up to the deployer and their billing rules. There are ways to compute the ranges by using multiple queries [1], and we could create some sort of way for the query to specify which fields in the metadata for a given type of resource are important. Both calculations would be expensive to apply in the API server, though, and I think they can be solved more efficiently on the client-side. If the client grabs all (or a large portion) of the data and processes it sequentially, it is easy to test the metadata fields to find changes and treat that condition as the boundary between the time ranges. My conclusion from all of this (over-)thinking is that the ceilometer API should assume the simple case and ignore the metadata changes when computing the sum or maximum value for a counter over a range of time. More complex processing will be left up to the caller, who can ask for raw metering data in manageable chunks and process them outside of the API. I could be persuaded to do something more complicated if the problems described above can be solved in a relatively simple way, but even then I think we should push that to the v2 API. Thoughts? There-and-back-again-ly, Doug [1] Find the min time for a (resource, metadata) pair; find min time for any different (resource, metadata) pair greater than the first time; find max for original pair with timestamp less than second min; use the max as starting point to determine the next range; loop. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help
[Openstack] OpenStack Community Weekly Newsletter (Oct 19-26)
Highlights of the week More coverage of OpenStack Summit We've all been catching some air this week, it seems. Some more reports from the community: * SDKs and an OpenStack Grizzly Summit Wrap Up http://blog.phymata.com/2012/10/22/sdks-and-an-openstack-grizzly-summit-wrap-up/ * Back from San Diego OpenStack Summit http://maffulli.net/2012/10/20/back-from-san-diego-openstack-summit/ * OpenStack Design Summit -- wrap-up and links http://www.rhonabwy.com/wp/2012/10/20/openstack-design-summit-wrap-up-and-links/ * Swift @ OpenStack Summit 2012 http://www.zmanda.com/blogs/?p=971 * Feedback from Design Summit in the first part of the Project meeting log http://eavesdrop.openstack.org/meetings/project/2012/project.2012-10-23-21.02.log.html * OpenStack Summit Beach Clean Up http://www.openstack.org/blog/2012/10/openstack-summit-beach-clean-up/ with great pictures Inside Synaps, a CloudWatch-like implementation for OpenStack http://julien.danjou.info/blog/2012/openstack-synaps-exploration A few days ago, Samsung http://www.samsung.com/ released the source code of Synaps https://github.com/spcs/synaps, an implementation of the Amazon Web Service CloudWatch API http://aws.amazon.com/cloudwatch/ for OpenStack http://openstack.org/. Julien Danjou http://julien.danjou.info/blog/, a contributor to the Ceilometer http://launchpad.net/ceilometer project, gives a look at this project and how it could overlap with Ceilometer or other projects like Heat http://www.heat-api.org/. Why OpenStack doesn't need a Linus Torvalds http://fnords.wordpress.com/2012/10/25/why-openstack-doesnt-need-a-linus-torvalds/ As comparing OpenStack with Linux becomes an increasingly popular exercise http://www.devx.com/blog/is-openstack-the-linux-of-cloud.html, it's only natural that people and press articles start to ask where the Linus of OpenStack is, or who the Linus of OpenStack http://www.networkworld.com/news/2012/102412-openstack-linus-263659.html?hpg1=bn should be. This assumes that technical leaders could somehow be appointed in OpenStack. This assumes that the single dictator model is somehow reproducible or even desirable. And this assumes that the current technical leadership in OpenStack is somehow lacking. Thierry Carrez thinks all those three assumptions are wrong. Preauthorization in Keystone http://adam.younglogic.com/2012/10/preauthorization-in-keystone/ Sometimes you need to authorize a service to perform an action on your behalf. Often, that action takes place long after any authentication token you can provide would have expired. Currently, the only mechanism in Keystone that people can use is to share credentials. Adam Young http://adam.younglogic.com/ argues: We can do better. New wiki page: Software Development Kits http://wiki.openstack.org/SDKs SDKs are a vital part of any ecosystem and we need to start treating them as such in OpenStack. To do so we need to raise the profile and legitimacy of SDKs that support OpenStack. Heat version 7 released http://markmail.org/message/teb4mvtru6ismk7a Heat allows you to launch AWS CloudFormation templates on OpenStack. CloudFormation is a programmable interface and templating system for orchestrating multiple cloud applications. This version adds an OpenStack-native ReST API. Tips and tricks * By Grid Dynamics OpenStack Team http://openstackgd.wordpress.com/: OpenStack Migration from Diablo to Essex http://openstackgd.wordpress.com/2012/10/20/435/ * By Mirantis http://www.mirantis.com/: Making the most of your application performance on OpenStack Cloud http://www.mirantis.com/blog/making-most-of-openstack-compute-performance/ Upcoming Events * OpenStack China Tour http://hui.csdn.net/MeetingInfo.aspx?mid=141 Oct 27, 2012 -- Chengdu Details http://hui.csdn.net/MeetingInfo.aspx?mid=141 * Swiss OpenStack user group meeting http://www.meetup.com/zhgeeks/events/82917082/ Nov 15, 2012 -- Zürich, CH Details http://www.meetup.com/zhgeeks/events/82917082/ * OpenStack in action! http://openstackinaction3theopenrevolution.eventbrite.com/ Nov 29, 2012 -- Paris, France Register http://openstackinaction3theopenrevolution.eventbrite.com/ * EMEA OpenStack Day http://www.openstack.org/ Dec 05, 2012 -- London Details http://www.openstack.org/ Other news * OpenStack Security Group http://wiki.openstack.org/GrizzlyReleaseSchedule * Grizzly Release Schedule http://wiki.openstack.org/GrizzlyReleaseSchedule published * OpenStack Project Meeting: summary http://eavesdrop.openstack.org/meetings/project/2012/project.2012-10-23-21.02.html and full logs http://eavesdrop.openstack.org/meetings/project/2012/project.2012-10-23-21.02.log.html. /The weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content
[Openstack-ubuntu-testing-notifications] Build Still Failing: trusty_juno_ceilometer_trunk #172
Title: trusty_juno_ceilometer_trunk General InformationBUILD FAILUREBuild URL:https://jenkins.qa.ubuntu.com/job/trusty_juno_ceilometer_trunk/172/Project:trusty_juno_ceilometer_trunkDate of build:Tue, 02 Sep 2014 22:30:44 -0400Build duration:10 minBuild cause:Started by an SCM changeBuilt on:pkg-builderHealth ReportWDescriptionScoreBuild stability: All recent builds failed.0ChangesImported Translations from Transifexby openstack-infraeditceilometer/locale/ceilometer-log-warning.poteditceilometer/locale/it/LC_MESSAGES/ceilometer-log-info.poaddceilometer/locale/pt_BR/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/ceilometer.poteditceilometer/locale/en_GB/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/fr/LC_MESSAGES/ceilometer-log-info.poaddceilometer/locale/de/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/en_AU/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/ceilometer-log-error.poteditceilometer/locale/en_US/LC_MESSAGES/ceilometer.poaddceilometer/locale/en_AU/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/fr/LC_MESSAGES/ceilometer-log-warning.poeditceilometer/locale/en_GB/LC_MESSAGES/ceilometer-log-warning.poeditceilometer/locale/pt_BR/LC_MESSAGES/ceilometer-log-info.poaddceilometer/locale/es/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/de/LC_MESSAGES/ceilometer-log-warning.poeditceilometer/locale/en_GB/LC_MESSAGES/ceilometer-log-info.poaddceilometer/locale/zh_CN/LC_MESSAGES/ceilometer-log-error.poaddceilometer/locale/ko_KR/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/de/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/zh_CN/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/vi_VN/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/vi_VN/LC_MESSAGES/ceilometer-log-error.poeditceilometer/locale/ceilometer-log-info.poteditceilometer/locale/fr/LC_MESSAGES/ceilometer-log-error.poaddceilometer/locale/te_IN/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/es/LC_MESSAGES/ceilometer-log-info.poeditceilometer/locale/ja/LC_MESSAGES/ceilometer-log-error.poConsole Output[...truncated 4101 lines...]dch -a [a4ad787] Fix E265 viINFO:root:Destroying schroot.olations and re-enable gatingdch -a [831c303] Fix E251 violations and re-enable gatingdch -a [c3c4839] Fix E128 violations and re-enable gatingdch -a [7b2bea5] Fix E126,H104 violations and re-enable gatingdch -a [0600d38] Bump hacking to 0.9.xdch -a [acf7f51] Fixed various import issues exposed by unittestdch -a [5418ee4] use urlparse from sixdch -a [a9dc29d] clean up sample indexdch -a [2ec87b0] Fix HBase available capabilities listdch -a [d006f5a] Updated from global requirementsdch -a [d9a804b] VMware:Update the ceilometer doc with VMware optsdch -a [1fcdca0] Handle non-ascii character in meter namedch -a [6403c85] Add log output of "x-openstack-request-id" from novadch -a [f40f935] Imported Translations from Transifexdch -a [c0702b2] fix StringIO errors in unit testdch -a [18eec72] Fix hacking rule 302 and enable itdch -a [5565715] Imported Translations from Transifexdch -a [36638dd] sync oslo codedch -a [ff5c133] Fixes ceilometer-compute service start failuredch -a [59e647f] Avoid reading real config files in unit testdch -a [78423c3] Use hacking from test-requirementsdch -a [7fc5579] Splits hbase storage code basedch -a [3ae3d27] Fix method mocked in a testdebcommitapt-get -y install equivs devscripts bzr bzr-builddeb git quilt gnupg piupartsbzr builddeb -S -- -sa -us -ucbzr builddeb -S -- -sa -us -ucdebsign -k9935ACDC ceilometer_2014.2+git201409022230~trusty-0ubuntu1_source.changesmk-build-deps -i -r -t apt-get -y /tmp/tmpwI36NQ/ceilometer/debian/controlsbuild -d trusty-juno -n -A ceilometer_2014.2+git201409022230~trusty-0ubuntu1.dscTraceback (most recent call last): File "/var/lib/jenkins/tools/openstack-ubuntu-testing/bin/build-package", line 139, in raise esubprocess.CalledProcessError: Command '['sbuild', '-d', 'trusty-juno', '-n', '-A', 'ceilometer_2014.2+git201409022230~trusty-0ubuntu1.dsc']' returned non-zero exit status 3Error in sys.excepthook:Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 67, in apport_excepthookbinary = os.path.realpath(os.path.join(os.getcwd(), sys.argv[0]))OSError: [Errno 2] No such file or directoryOriginal exception was:Traceback (most recent call last): File "/var/lib/jenkins/tools/openstack-ubuntu-testing/bin/build-package", line 139, in raise esubprocess.CalledProcessError: Command '['sbuild', '-d', 'trusty-juno', '-n', '-A', 'ceilometer_2014.2+git201409022230~trusty-0ubuntu1.dsc']' returned non-zero exit status 3Build step 'Execute shell' marked build as failureEmail was triggered for: FailureSending email for trigger: Failure-- Mailing list: https://launchpad.net/~openstack-ubuntu-testing-notifications Post to : openstack-ubuntu-testing-notifications@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstac
Re: [Openstack] [Metering] API Extensibility
On 05/10/2012 05:54 PM, Doug Hellmann wrote:
On Thu, May 10, 2012 at 9:22 AM, Loic Dachary l...@enovance.com
mailto:l...@enovance.com wrote:
Another item that we need to discuss is extensibility of this API.
Hi,
Here is a proposal, which we could discuss further during the meeting.
GET extension=param1=fooparam2=bar
The API looks up /usr/share/ceilometer/extensions/.py and loads it.
The module defines a query function that takes the following arguments:
Andrew Bogott is doing some work with a standardized plugin mechanism for
Nova which will eventually be put in the common lib for all of the projects.
We should look at his work and use it, rather than inventing something else.
I think it will eventually use setuptools entrypoints, which eliminates the
need to worry about search paths.
I agree.
Why would the extension be a query parameter, rather than a URL component?
That is, why wouldn't the extension just add new endpoints that could be
queried directly using their own API? Maybe I don't understand the types of
extensions you are thinking of.
No specific reason, we can just forget about it. I'm grateful jaypipes
suggested it during the last meeting, that will save us time.
Cheers
* QUERY_STRING (i.e. extension=param1=fooparam2=bar )
* a handler to the storage
* a pointer to the configuration (assuming there is a /etc/ceilometer.ini
file, for instance)
The query function would return the result. For instance { 'in': 20001,
'out': 489324 } if asked for aggregated network usage.
Multiple extensions directories could be specified and searched, allowing
a mixture of extensions provided in ceilometer and custom extensions to
address specific needs or to mature an new extension.
The primary benefit of defining extensions in this way is to avoid
complex conventions for aggregations or other advanced operations. If the API
was to impose a syntax or conventions to say sum this field and this one and
display the result ordered in this way and grouped by this field and this
one, it would be redundant with the query language of the underlying data.
For instance, if using mongodb, it would be difficult to expose all the
features provided by http://www.mongodb.org/display/DOCS/Aggregation or
http://www.mongodb.org/display/DOCS/MapReduce
Cheers
--
Loïc Dachary Chief Research Officer
// eNovance labs http://labs.enovance.com
// ✉ l...@enovance.com mailto:l...@enovance.com ☎ +33 1 49 70 99 82
tel:%2B33%201%2049%2070%2099%2082
___
Mailing list: https://launchpad.net/~openstack
https://launchpad.net/%7Eopenstack
Post to : openstack@lists.launchpad.net
mailto:openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
https://launchpad.net/%7Eopenstack
More help : https://help.launchpad.net/ListHelp
--
Loïc Dachary Chief Research Officer
// eNovance labs http://labs.enovance.com
// ✉ l...@enovance.com ☎ +33 1 49 70 99 82
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
If I'm understanding this correctly, the Collector is kind of like a Agent in Qantum (It sits on a machine doing stuff and passing info upstream). If you look at the approach they have now in Quantum Agent it's writing directly to the DB. But looking at the next version they seem to be moving to having the Agent send data upstream to the Plugin in Quantum. Why not do something similar? I mean if you have a MQ cluster in a deployment I think it makes more sense to have 1 thing that handles the db stuff then having each Collector connect to the db.. Endre. 2012/5/22 Nick Barcet nick.bar...@canonical.com On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. Thanks again, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
On Tue, May 22, 2012 at 3:40 AM, Nick Barcet nick.bar...@canonical.comwrote: On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. Yes, that's right. I have updated the wiki page to be more explicit on that point. http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1?action=diffrev2=8rev1=7 * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery exchange on the queue to retrieve values and set defaults if it does not exist yet. I was referring to the config object created by nova.flags (although I think that module is moving to the common library, if it hasn't already). * Metering messages are signed using the hmac module in Python's standard library. A shared secret value can be provided in the ceilometer configuration settings. The messages are signed by feeding the message key names and values into the signature generator in sorted order. Non-string values are converted to unicode and then encoded as UTF-8. The message signature is included in the message for verification by the collector. - The signature is also kept in the database for future audit processes, maybe worth mentioning it here. Yes, good point. http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1?action=diffrev2=9rev1=8 - In addition to a signature, I think we would need a sequence number to be embedded by the agent for each message sent, so that loss of messages, or forgery of messages, can be detected by the collector and further audit process. OK. We have a message id, but I assumed those would be used to eliminate duplicates so this sounds like something different or new. It implies that the agent knows its own id (not hard) and keeps up with a sequence counter (more difficult, though not impossible). Did you have something in mind for how to implement that? Thanks again, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] cfg usage - option registration, global objects
On 06/05/2012 05:35 PM, Russell Bryant wrote: On 06/05/2012 05:25 PM, Mark Washenberger wrote: Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 12:21 -0400, Mark Washenberger wrote: http://wiki.openstack.org/CommonLibrary#Incubation Once an api is in incubation, if you make a change to it, you are expected to update all the other openstack projects (not just core projects?) to make them work with the new api. Am I understanding this requirement correctly? Yes, pretty much. The alternative is that you don't make backwards incompatible API changes. ... What alternative strategy are you suggesting? That if glance, quantum, cinder and ceilometer want to re-use Nova's RPC code, they should copy-and-paste it and hack it to their needs? I don't think our only options here are immediate adoption and relative chaos. Here's what I would like to see: Quantum, cinder, and ceilometer get together, recognize a shared need for rpc, acknowledge the successes and failures of the nova.rpc library, and create a better implementation with eventual adoption by Nova kept in mind. Doesn't that sound better? This approach seems much nicer to me, because I believe that code reuse is likely to be detrimental unless the code we're talking about was created with reuse and generality in mind. Since I find that suggestion implausible regarding nova.rpc in particular, I think we will do better overall avoiding its wider adoption. Please, forgive me if I'm being drawn in falsely by the allure of better code. Code structure and quality *is* something I obsess about. But I appreciate the need to be practical in the short term as well, if I am not always the best at articulating it. The agreement from various quarters about the problems with the current nova.rpc gave me hope that we could craft a better rpc library without too much delay, even if I myself can only contribute in a small way to that effort. That all sounds nice in theory, but like you alluded to here, who specifically is going to sign up to do that? I can finish the move of the current code (I have it ready to propose, actually), but I have other things I should work on after that. How about the stakeholders interested in using this code in other projects? Do you want the current code in now (with the likelihood of having to adapt to some API changes later), or would you like to get together and work on something new? If so, I'll just toss the proposal of the current code for common and let someone else drive the new thing in instead. By the way, the move of the current code is ready if we agree to proceed with it: https://review.openstack.org/#/c/8206/ -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On 24/10/12 23:35 +0200, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate table and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable product/service In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Kind of the same here, if you don't want to really bill the vm, just don't meter them (or ignore the meters) and emit your own meter via your PaaS platform to bill your customer. Or is there a limitation I miss? If you do auto scaling you will have a similar problem. Here you want to monitor the group (with instances comming and going) as a logical unit. One way would be to tag the instances and then extract the tag and send it with the metadata associated with the meter. Then you could query the ceilometer db for that group. (In CloudWatch this is just another Dimension). -Angus -- Julien Danjou -- Free Software hacker freelance -- http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] new mailing list for bare-metal provisioning
subscribe to the Nova topic at the link above. When I write a message about nova I have to add '[Nova]' (or 'Nova') anywhere in the subject (Sidenote: does 'Nova' actually work? Looks to me like the regex in mailman requires the square braces...) Personally I'm fine with either model, but just to call out the common complaint I hear about topics: I think a lot of folks feel that what you just pointed out is actually a major weakness. The topic functionality requires that *each individual sender* be aware of that functionality and format his subject lines accordingly. That often doesn't happen even with folks who've been active in the community for some time, let alone folks who are new. When someone doesn't format his subject properly, it either causes clutter when email sorting filters break or causes folks to miss messages about things they care about if they've subscribed only to particular topics. I know there have been at least a couple of threads about Quantum in the past couple of weeks (one pertaining to Ceilometer integration, one pertaining to the Folsom stable branch come to mind) that didn't have [Quantum] (or [Ceilometer] for that matter) in the subject line. Thus, I'd have missed those messages if I were only subscribed to the Quantum topic. Personally I like to see everything so it doesn't much matter to me other than in how I set up my email filters, but I think perhaps this is one reason why we've had the discussion about more vs fewer mailing lists more than once. At Your Service, Mark T. Voelker On 10/29/2012 09:01 AM, Stefano Maffulli wrote: On 10/29/2012 12:32 PM, Thierry Carrez wrote: We really shouldn't go in that direction: the openstack-dev list is already an aggregator of topics, since we use mailman topics on it. Indeed, mailman topics are very powerful. The current topics for openstack-dev are listed on each subscriber's personal page: http://lists.openstack.org/cgi-bin/mailman/options/openstack-dev As a subscriber of openstack-dev I can decide to receive only messages tagged for a topic by selecting the ones I'm interested in. As a writer of a message I can tag it by adding the topic to the subject line of the message. For example, if I want to receive only messages for Nova, I can subscribe to the Nova topic at the link above. When I write a message about nova I have to add '[Nova]' (or 'Nova') anywhere in the subject. Creating a topic of bare-metal is easy, using topics is a matter of habit. I believe that we should not create more lists unless strictly necessary. I also understood from David that the baremetal group felt very strongly against using any of the existing list, even when I suggested to use topics. I'm glad we're having this conversation now and I'm open to any outcome. /stef ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 10:23 AM, Eoghan Glynn egl...@redhat.com wrote: Hi Yawei Wu, The root of the confusion is the fact the cpu meter is reporting the cumlative cpu_time stat from libvirt. This libvirt counter is reset when the associated qemu process is restarted (an artifact of how cpuacct works). So when you stop/start or suspend/resume, a fresh qemu process is sparked up, then the cumulative time is reset. Thanks for bringing this up, as it has implications as to how we meter CPU time and utilization[1]. We may need to start metering the delta between CPU times on subsequent polling cycles, instead of using a cumulative meter (dealing with the edge case where the instance has been restarted within a polling period). Good idea. We need to capture this issue to make sure we get it onto the roadmap for this cycle. Is there a bug or blueprint for it yet? Doug Cheers, Eoghan [1] https://review.openstack.org/14921 I am still testing ceilometer now. I am confused about the meter volume in the mongodb. Let's talk about cpu usage. After I create and boot a vm named vm_1, meter data record about cpu usage will be inserted into db in cycle(default 10 minutes). For example,the 'counter_volume' of the first record is '5206000',and the second one is '12389000'. 1) '12389000' nanoseconds means '123.89' seconds or two minutes,it seem like to be 1238.9 seconds actually, is there something wrong ? 2) If I never reboot or suspend vm_1, will the 'counter_volume' of cpu usage record increase all the time ? Just like '8 minutes' - '18 minutes' - '28 minutes' ? 3) If I reboot or suspend vm_1, I find that the 'counter_volume' of cpu usage record will count from zero. Just like '8 minutes' - '18 minutes' - '28 minutes' [- '0 minutes'] -'5 minutes' - '15 minutes'. Does it mean that 'counter_volume' just represents how long has vm_1 been booted up ? 4) This one is about Web API. I find that GET /v1/resources/(resource)/meters/(meter)/volume/sum just return the sum value of all the cpu 'counter_volume', like '8 minutes' + '18 minutes'. Is it reduplicate ? 5) If I want to know how long has vm_1's cpu been used yesterday, how can I do ? It seems like that I have too many questions.. Thank you very much ! --- Yawei Wu Dalian Hi-Think Computer Technology,Corp. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New code name for networks
Since networks are moving quite rapidly, can be quite messy and its pretty cold out there my suggestion is: Blizzard -Udi -Original Message- From: Openstack [mailto:openstack-bounces+udi.margolin=alcatel-lucent@lists.launchpad.ne t] On Behalf Of Thierry Carrez Sent: Monday, May 13, 2013 12:47 PM To: openstack@lists.launchpad.net Subject: Re: [Openstack] New code name for networks Anne Gentle wrote: I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I promise, it wasn't the world's most fun session. :) I'm sure. :) I think I don't have much regret but do feel sorry that I don't know more. The Etherpad is here: https://etherpad.openstack.org/ProjectsReNaming I think there is much more value to codenames than just avoiding the cost of a rename when the project becomes OpenStack. This was captured in the session: Codenames drawbacks and benefits (-) Lack of trademark protection (-) Confusing to newcomers (-) Shadow their more official counterparts (+) Short names are highly-convenient and efficient, often less ambiguous (in conversations, executables, modules...) (+) Help building project and team identity (+) Separate the project itself from its functional scope (so they remain valid even if that scope evolves) Those last two bits are pretty essential. There is a reason why a functional description cannot be used as a project name. The project (as in, the code repository and the community of contributors around it) is *distinct* from the functional scope of what its code does. Take Ceilometer (OpenStack Metering). What happens when they grow to cover Monitoring ? You rename the project to OpenStack Metering and Monitoring ? Or you keep the partial functional description ? I'd rather avoid to rename everything every time a project evolves. Those renames are *extremely* costly, as we'll soon enough realize. I find the confusing argument pretty weak myself. Brands are used everywhere, so we are used to make the translation between a name and a function. Microsoft named its desktop environment Windows, rather than Operating system or Desktop environment, and it took people about 5 minutes to get used to it. Go with kumquat, but don't call the CLI kumquat. Call your team kumquat and your repo kumquat. If you call the CLI os-metering, you'll have to rename it when the scope expands, or live with a name that looks like a functional description but is not an accurate one. I very much prefer to call it ceilometer. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Documentation for openstack-java-sdk
Surely Luis can help you, I've used openstack-java-sdk in one of my
projects, and this is the example Luis gave to me
private static final File TEST_FILE = new File(pom.xml);
private static final String KEYSTONE_AUTH_URL =
https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0;;
private static final String KEYSTONE_USERNAME = ;
private static final String KEYSTONE_PASSWORD = ;
/**
* @param args
*/
public static void main(String[] args) throws Exception {
KeystoneClient keystone = new KeystoneClient(KEYSTONE_AUTH_URL);
//access with unscoped token
Access access = keystone.execute(Authenticate.withPasswordCredentials(
KEYSTONE_USERNAME, KEYSTONE_PASSWORD));
//use the token in the following requests
keystone.setToken(access.getToken().getId());
Tenants tenants = keystone.execute(new ListTenants());
//try to exchange token using the first tenant
if(tenants.getList().size() 0) {
access =
keystone.execute(Authenticate.withToken(access.getToken().getId()).withTenantId(tenants.getList().get(0).getId()));
SwiftClient swiftClient =
newSwiftClient(KeystoneUtils.findEndpointURL(access.getServiceCatalog(),
object-store, null, public), access.getToken().getId());
//swiftClient.execute(new DeleteContainer(navidad2));
swiftClient.execute(new CreateContainer(navidad2));
System.out.println(swiftClient.execute(new ListContainers()));
ObjectForUpload upload = new ObjectForUpload();
upload.setContainer(navidad2);
upload.setName(example2);
upload.setInputStream(new FileInputStream(TEST_FILE));
swiftClient.execute(new UploadObject(upload));
System.out.println(swiftClient.execute(new ListObjects(navidad2,
new HashMapString,
String() {{
put(path, );
}})).get(0).getContentType());
}
}
On Thu, Jul 11, 2013 at 11:31 AM, Endre Karlson endre.karl...@gmail.comwrote:
I think Luis can answer that?
-- Videresendt melding --
Fra: Jobin Raju George jobin...@gmail.com
Dato: 11. juli 2013 14:38
Emne: [Openstack] Documentation for openstack-java-sdk
Til: openstack lista openstack@lists.launchpad.net
Kopi:
I am trying to query ceilometer using
openstack-java-sdkhttps://github.com/woorea/openstack-java-sdkfor meters of
VM's running on KVM. I am able to get the CPU meters via curl
on the command line but unfortunately I don't find good documentation for
the SDK's for ceilometer.
I have seen this example
programhttps://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java
but
most of it is commented(probably because it is deprecated).
Where can I find good documentation/examples or java programs/snippets?
--
Thanks and regards,
Jobin Raju George
Third Year, Information Technology
College of Engineering Pune
Alternate e-mail: georgejr10...@coep.ac.in
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
*guilherme* \n
\t *maluf*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] External API definition
Hello, The 3rd meeting of the Ceilometer project will be held on May 10th at 1600 UTC in the #openstack-meeting channel on Freenode. The main subject we will be tackling this week is the external API definition and I'd like to gather you opinions on the subject prior to the meeting. The current proposal which can be found on the blueprint [1] states the following: * Database can only be queried via a REST API (i.e. the database schema is not a supported API and can change in a non backward compatible way from one version to the other). * Requests must be authenticated (separate from keystone, or only linked to accounting type account) * API Server must be able to be redundant * Requests allow to GET account_id list GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime Note: the aggregation of values is done by the API and is not stored in the database. It may be cached for performance reasons but the caching strategy is outside of the scope of this blueprint. Any comments, suggestion, etc... are very welcome. [1] http://wiki.openstack.org/EfficientMetering#API Thanks a lot, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] External API definition
On Tue, May 8, 2012 at 11:27 AM, Nick Barcet nick.bar...@canonical.comwrote: Hello, The 3rd meeting of the Ceilometer project will be held on May 10th at 1600 UTC in the #openstack-meeting channel on Freenode. The main subject we will be tackling this week is the external API definition and I'd like to gather you opinions on the subject prior to the meeting. The current proposal which can be found on the blueprint [1] states the following: * Database can only be queried via a REST API (i.e. the database schema is not a supported API and can change in a non backward compatible way from one version to the other). * Requests must be authenticated (separate from keystone, or only linked to accounting type account) What is the motivation for authenticating with a service other than keystone? * API Server must be able to be redundant * Requests allow to GET account_id list GET list of counter_type GET list of events per account optional start and end for counter_datetime optional counter_type GET sum of (counter_volume, counter_duration) for counter_type and account_id optional start and end for counter_datetime Note: the aggregation of values is done by the API and is not stored in the database. It may be cached for performance reasons but the caching strategy is outside of the scope of this blueprint. Any comments, suggestion, etc... are very welcome. [1] http://wiki.openstack.org/EfficientMetering#API Thanks a lot, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Choice of a messaging queue
Hello everyone, Next week's irc meeting will have for goal to choose a reference messaging queue service for the ceilometer project. For this meeting to be able to be successful, a discussion on the choices that we have to make need to occur first right here. To open the discussion here are a few requirements that I would consider important for the queue to support: a) the queue must guaranty the delivery of messages. To the contrary of monitoring, loss of events may have important billing impacts, it therefore cannot be an option that message be lost. b) the client should be able to store and forward. As the load of system or traffic increases, or if the client is temporarily disconnected, client element of the queue should be able to hold messages in a local queue to be emitted as soon as condition permit. c) client must authenticate Only client which hold a shared private key should be able to send messages on the queue. d) queue may support client signing of individual messages Each message should be individually signed by the agent that emits it in order to guaranty non repudiability. This function can be done by the queue client or by the agent prior to en-queuing of messages. d) queue must be highly available the queue servers must be able to support multiple instances running in parallel in order to support continuation of operations with the loss of one server. This should be achievable without the need to use complex fail over systems and shared storage. e) queue should be horizontally scalable The scalability of queue servers should be achievable by increasing the number of servers. Not sure this list is exhaustive or viable, feel free to comment on it, but the real question is: which queue should we be using here? Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] high-level design proposal
[copying the list] On Tue, May 22, 2012 at 5:02 PM, Matt Joyce matt.jo...@cloudscaling.comwrote: If the agent is simply passively passing data up stream to the collectors I really don't care. As long as it is never accepting commands remotely. Once we do that it becomes something else. Either it's tied into an API or it's acting as an independent agent of execution. Netstack / quantum folks NEED a highly dynamic remote agent they can pass execution parameters to. And they will build one. But if we keep spinning off these types of directly addressable agents of execution we are creating a lot of new things we need to maintain and vet. If we have a single agent of execution project we can pool development efforts and focus on hardening it for all. That would be my primary concern in the agent. That makes sense. In our case we have not identified a need for the agent to receive commands. We are using the nova.service module to run the daemon right now, so although we have created a new daemon we haven't done a lot of work to invent anything new. If there is other work going on to create a single daemon to run on the compute node and if we can tap into that daemon and ensure that periodic tasks are triggered at regular (and relatively dependable) intervals then we could move the agent portion of ceilometer into the common agent. Is there a project for creating that? Or a blueprint I could subscribe to? -Matt On Tue, May 22, 2012 at 11:08 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Tue, May 22, 2012 at 12:53 PM, Matt Joyce matt.jo...@cloudscaling.com wrote: My point of concern. \ If an agent is being built into the compute nodes, that would best be a split out project. Two major reasons. First and foremost sub projects should not be spinning up their own agents. Secondly, there is a use case of agents outside of metering. If an agent is to be built it is a not insignificant change in architecture for openstack. This agent will run on the compute host, next to nova-compute, not inside the VM. Is that still a concern? -Matt On Tue, May 22, 2012 at 7:30 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Tue, May 22, 2012 at 4:05 AM, Endre Karlson endre.karl...@gmail.com wrote: If I'm understanding this correctly, the Collector is kind of like a Agent in Qantum (It sits on a machine doing stuff and passing info upstream). If you look at the approach they have now in Quantum Agent it's writing directly to the DB. But looking at the next version they seem to be moving to having the Agent send data upstream to the Plugin in Quantum. Why not do something similar? I mean if you have a MQ cluster in a deployment I think it makes more sense to have 1 thing that handles the db stuff then having each Collector connect to the db.. That was the goal, but I may have swapped the terminology around. For ceilometer, the agent runs on the compute node and writes only to the message queue. The collector runs in a central location and writes to the database. The number of collectors you need will depend on the number of messages being generated, but the architecture supports running several in parallel in a way that each instance does not need to be aware of the others. Doug Endre. 2012/5/22 Nick Barcet nick.bar...@canonical.com On 05/21/2012 10:52 PM, Doug Hellmann wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 Thanks a lot for putting this together Doug. A few questions: * The collector runs on one or more central management servers to monitor the message queues (for notifications and for metering data coming from the agent). Notification messages are processed and turned into metering messages and sent back out onto the message bus using the appropriate topic. Metering messages are written to the data store without modification. - Is the reason behind why collectors do not write directly to the database a way to allow db less implementations as Francis suggested earlier? In this case it may be useful to say it explicitly. * Plugins may require configuration options, so when the plugin is loaded it is asked to add options to the global flags object, and the results are made available to the plugin before it is asked to do any work. - I am not sure where the global flags object resides and how option are populated. I think it would make sense for this to be globally controlled, and therefore may require for a simple discovery
Re: [Openstack] [metering] high-level design proposal
I've converted the polling agent to use plugins, too. That code is up for review at https://review.stackforge.org/59 On Tue, May 22, 2012 at 9:33 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: A version of the code demonstrating using plugins in this way is up for review at https://review.stackforge.org/#/c/45/ On Tue, May 22, 2012 at 5:59 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: After experimenting with some of the implementation today, I modified the way the notification plugins list the event_types they want to see. http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1?action=diffrev2=10rev1=9 On Mon, May 21, 2012 at 5:52 PM, Doug Hellmann doug.hellm...@dreamhost.com wrote: I have written up some of my thoughts on a proposed design for ceilometer in the wiki [1]. I'm sure there are missing details, but I wanted to start getting ideas into writing so they could be discussed here on the list, since I've talked about different parts with a couple of you separately. Let me know what you think, and especially if I am not clear or have left out any details. Thanks, Doug [1] http://wiki.openstack.org/EfficientMetering/ArchitectureProposalV1 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Agent configuration mechanism
On 06/06/2012 10:35 AM, Julien Danjou wrote: On Tue, Jun 05 2012, Nick Barcet wrote: The main idea is that all agents of a given type should be sending similarly formatted information in order for the information to be usable, hence the need to ensure that configuration info is centrally stored and retrieved. This would rule out, in my mind, the idea that we could use the global flags object, as distribution of the configuration file is left to the cloud implementor and does not lend for easy and synchronized updates of agent config. IMHO this is solving a problem that already exists for all other OpenStack components. A problem that no other OpenStack components tried to resolved, AFAIK. So I don't see why we should try to resolve configuration deployment in ceilometer when it's a much larger issue in the project. Maybe the problem of discrepancy of configuration is more an issue for metering than for other components? At least that's my belief. In fact, I may very well want to have differences in configuration of different compute nodes in a single zone, for very valid reasons (ie use different hypervisor settings, balance usage of components, etc...). However if my metering agents are not all set to report the same things, I think I'll be in trouble to correlate anything, hence the need for metering configuration information to be provided centrally, for anything that sets the behavior of meters and which meters to use. Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
