Re: [Openstack] [Ceilometer] Problems with query fields in filters
On 12/07/13 11:33 +0200, Julien Danjou wrote:
On Fri, Jul 12 2013, Alessandro Barabesi wrote:
I have the following problems with query fields in filters.
1) Filtering by metadata.size in v2/meters/image apparently is not working.
The following request returns also samples with metadata.size=0.
GET http://10.10.10.10:8777/v2/meters/image
{
q: [{
field: project_id,
op: eq,
value: 77b461539c8542909f67b29939ec87dd
},
{
field: timestamp,
op: ge,
value: 2013-07-11T13:36:00
},
{
field: timestamp,
op: lt,
value: 2013-07-11T13:39:00
},
{
field: metadata.size,
op: gt,
value: 0
I suspect the gt operator is not working (it's probably using eq
given what you are getting). But certainly a bug.
I'd just remove this last query and see what you get with the first 3
queries.
-Angus
}]
}
I am probably doing something wrong but I can't figure out what.
That seems correct from the top of my head. If that really doesn't work,
feel free to open a bug.
2) Filtering by counter_volume returns the folloving error message:
error_message={debuginfo: null, faultcode: Client, faultstring: Unknown argument:
\counter_volume\: unrecognized query field}
Is this a bug or filtering by counter-volume is not allowed?
Try `volume' instead of `counter_volume'. But I am not sure it's
currently allowed -- in that case opening a bug can be good idea too.
--
Julien Danjou
// Free Software hacker / freelance consultant
// http://julien.danjou.info
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer][Healthnmon] Monitoring Alarm in openstack
On 02/07/13 11:29 +0200, Emanuel Marzini wrote: Thank you! It's true that in Havana disappear Healthnmon that will be merge with Ceilometer? Not that I am aware of, Ceilometer is just getting Alarming functionality. -Angus 2013/7/2 Angus Salkeld asalk...@redhat.com On 01/07/13 13:42 +0200, Emanuel Marzini wrote: Hi, I want to retrive Vm information from Openstack. I am interested of CPU RAM utilization. I known that someone use collectd, libvirt ecc.. or product like Ceilometer or Healthnmon. I am also interested to receive alarm information from the cloud provider if the CPU or RAM value exceeds a threshold. I read that from Havana, Ceilometer and Healthnmon will be merged, it's true? Might be a good solution to my problems? Ceilometer is getting alarms this cycle. Heat will start using them for autoscaling (cpu/mem stats). https://blueprints.launchpad.**net/ceilometer/+spec/alarminghttps://blueprints.launchpad.net/ceilometer/+spec/alarming Regards Angus Thanks in advance Emanuel __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp __**_ Mailing list: https://launchpad.net/~**openstackhttps://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~**openstackhttps://launchpad.net/~openstack More help : https://help.launchpad.net/**ListHelphttps://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer][Healthnmon] Monitoring Alarm in openstack
On 01/07/13 13:42 +0200, Emanuel Marzini wrote: Hi, I want to retrive Vm information from Openstack. I am interested of CPU RAM utilization. I known that someone use collectd, libvirt ecc.. or product like Ceilometer or Healthnmon. I am also interested to receive alarm information from the cloud provider if the CPU or RAM value exceeds a threshold. I read that from Havana, Ceilometer and Healthnmon will be merged, it's true? Might be a good solution to my problems? Ceilometer is getting alarms this cycle. Heat will start using them for autoscaling (cpu/mem stats). https://blueprints.launchpad.net/ceilometer/+spec/alarming Regards Angus Thanks in advance Emanuel ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer][Ceilometer-API] Ceilometer-API Error 401 Unauthorized
On 27/05/13 11:14 -0300, Bruno Oliveira wrote: Hello stackers, I'm having a really hard time setting up ceilometer-api so I thought if I could ask you guys for some enlightment. I can clearly see data being pulled in the screens that are running /ceilometer-collector, ./ceilometer-agent-compute ,./ceilometer-agent-central Even the screen running ceilometer-api-server starts with no problem. But I cannot reach the api at all via curl. Neither by using its actual port (8777) nor using the port set in the virtual host of apache. All I'm getting is auth error $ curl http://127.0.0.1:8777 OR $ curl http://127.0.0.1:9090 = html head title401 Unauthorized/title /head body h1401 Unauthorized/h1 This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.br /br / Authentication required = Right, Authentication is required by the client, but you are not passing it any credentials. I'd suggest using python-ceilometerclient to do the auth for you: So use it like any other openstack client. try something like this: asalkeld@elf python-ceilometerclient (master)$ . ../devstack/openrc admin admin asalkeld@elf python-ceilometerclient (master)$ ceilometer resource-list +--++-+--+ | Resource ID | Source | User ID | Project ID | +--++-+--+ | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f || None| 1076d9bd669d422bbd74e1e2f54d1510 | +--++-+--+ asalkeld@elf python-ceilometerclient (master)$ ceilometer meter-list +--+---+---+--+-+--+ | Name | Type | Unit | Resource ID | User ID | Project ID | +--+---+---+--+-+--+ | image| gauge | image | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f | None | 1076d9bd669d422bbd74e1e2f54d1510 | | image.size | gauge | B | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f | None | 1076d9bd669d422bbd74e1e2f54d1510 | | image.update | delta | image | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f | None | 1076d9bd669d422bbd74e1e2f54d1510 | | image.upload | delta | image | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f | None | 1076d9bd669d422bbd74e1e2f54d1510 | +--+---+---+--+-+--+ asalkeld@elf python-ceilometerclient (master)$ ceilometer sample-list -m image.update +--+--+---++---++ | Resource ID | Name | Type | Volume | Unit | Timestamp | +--+--+---++---++ | a8ce423c-c1a1-41e3-af7c-b38d92f5e36f | image.update | delta | 1.0| image | 2013-05-28T01:14:40.238000 | +--+--+---++---++ Remember you can only see the samples/meter/resources that you own or all if you are admin. -Angus On top of that, the only thing I had to do in a non-standard basis, was to setup ceilometer virtual host to answer request on port 9090 of apache instead of the default 80 (since horizon is bind to it). Here's a copy of my running ceilometer.conf = /etc/ceilometer/ceilometer.conf = [DEFAULT] os_username=ceilometer os_password=MYSECRET os_tenant_name=admin os_auth_url=http://localhost:5000/v2.0 signing_dirname = /tmp/keystone-signing-ceilometer metering_api_port=8777 auth_strategy=keystone nova_control_exchange=nova hypervisor_inspector=libvirt libvirt_type=kvm glance_control_exchange=glance quantum_control_exchange=quantum debug=true verbose=true (...) *logging writing parameters here* (...) log_dir=/var/log/ceilometer rpc_backend=ceilometer.openstack.common.rpc.impl_kombu rabbit_host=localhost rabbit_port=5672 rabbit_userid=guest rabbit_password=ficrowstran02 rabbit_retry_backoff=2 rabbit_max_retries=0 database_connection=mongodb://localhost:27017/ceilometer sql_connection_debug=0 cinder_control_exchange=cinder enable_v1_api=true [rpc_notifier2] [matchmaker_redis] [publisher_meter] metering_secret=METERING_SECRET [keystone_authtoken] auth_host = localhost auth_port = 5000 admin_user = ceilometer admin_password = MYSECRET
Re: [Openstack] [HeatQuantum] Quantum Template ROLLBACK_FAILED Error
On 28/02/13 15:30 +0800, 蒋闻天 wrote: I have heat and quantum in my devstack, just like: ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng ENABLED_SERVICES+=,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta Then I restart my compute, ./rejoin-stack.sh All Service OK. Then I run some thing like: heat stack-create -f /opt/stack/heat/templates/Quantum.template susu heat stack-list ROLLBACK_FAILED I See The Error Happened , Is there anyone can help me with this problem. Maybe some config i did not known. did you get any errors out of heat-engine? Are you sure your Quantum is working? I made a super quick attempt with your config and Quantum didn't start. Then Heat had the error as you reported (with an exception in the log about no endpoint) since Quantum didn't start. -Angus ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
On 01/11/12 13:58 -0700, Jeffrey Budzinski wrote: Thanks for putting that together Sandy. Very nice! From my perspective, there are two major things that are undesirable for us: 1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data sinks from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation. 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc. Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different sinks. Hi I Agree with what you said above, but lets not get bogged down with monkey patching/modifing the code as I think this is more a question for each ptl. Lets just make it possible to do both. btw: Has anyone started work on this? Is there a repo setup yet? -A On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks, Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ OpenStack-dev mailing list openstack-...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Autoscalar
On 02/11/12 10:47 -0500, Paras pradhan wrote: Well, What I am trying to find is if an instance in nova is low in resources, then another instance should be automatically created and started. Is this possible or any tools are out for this? You can give the heat project a go. https://github.com/heat-api/heat See this template that uses the autoscaling feature: https://github.com/heat-api/heat/blob/master/templates/AutoScalingMultiAZSample.template -Angus (#heat on freenode) Paras. On Fri, Oct 26, 2012 at 3:23 PM, Debo Dutta ddu...@gmail.com wrote: That is a very good idea IMO Sent from my iPhone On Oct 26, 2012, at 1:08 PM, Paras pradhan pradhanpa...@gmail.com wrote: Hi, Can we use auto scalar like Haizea (http://opennebula.org/software:ecosystem:haizea) with openstack compute or there is some other projects/tools similar to this. Thanks! Paras. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Scaling PaaS in OpenStack
On 26/10/12 13:07 +0700, Frans Thamura wrote: Yes, we use it here, but still finding to configure with OpenStack, to bring scale in this case communicate with openstack nova controller, we just use it now here.. You could use the heat project to provide autoscaling. The way this would work is you: 1 create an CloudFormations style template with your application (OpenShift/CloudFoundry) 2 you setup an autoscale group and alarm resource in the template 3 you post the metric of interest in your application to our Cloudwatch (see the calls to cfn-push-stats) as an example look at: https://github.com/heat-api/heat/blob/master/templates/AutoScalingMultiAZSample.template What happens is you setup a threshold that triggers a scale up and scale down action. also see: https://github.com/heat-api/heat/blob/master/templates/OpenShift.template https://github.com/heat-api/heat/wiki -Angus On Fri, Oct 26, 2012 at 1:00 PM, Ray Sun qsun01...@cienet.com.cn wrote: Have you hearad BOSH, a deployment tool for CloudFoundry on cloud(including AWS and openstack)? https://github.com/cloudfoundry/bosh - Ray Yours faithfully, Kind regards. CIeNET Technologies (Beijing) Co., Ltd Email: qsun01...@cienet.com.cn Office Phone: +86-01081470088-7079 Mobile Phone: +86-13581988291 On Fri, Oct 26, 2012 at 1:46 PM, Frans Thamura fr...@meruvian.org wrote: Hi All Anyone can give me reference, related to scaling PaaS system in OpenStack? how (more basic better) scalable is implementing PaaS in OpenStack? right now, we create virtual machine and install ubuntu inside, and run CloudFoundry or OpenShift to make it PaaS enable. my target for PaaS is to run our Java apps inside cloud environment. in another world, we have Liquid VM, but it is not opensource yet, part of Java VE Virtual Edition. The JVM can boot direct from the hypervisor. I still researching the theory behind scalability of cloud esp in openstack + cloudfoundry. F ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On 25/10/12 17:04 -0400, Doug Hellmann wrote: On Thu, Oct 25, 2012 at 10:22 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Oct 25 2012, Doug Hellmann wrote: That would be one way, but adding dimensions to the meters also makes sense because it reduces the need to collect the data more than once. In case of group, the other problem is how to emit instance counter with group metadata (assuming this group implementation is not part of Nova but Heat). Good point. I was assuming the values would be available as metadata of the underlying resource, but that may not always be the case. Yea, we need a consistent way of doing this. That should work on different resource types. We could use the tags or a similar mechanism. -A For instance, if flavor was a dimension of the instance meter I wouldn't need the separate meter instance:flavor. These sorts of use cases were part of the original motivation for collecting all of the metadata about a resource, but what we have now isn't structured enough to let the API user query into it. IIUC, what's need here is a GROUP BY operator in the API. Correct me if I'm wrong, but this is still doable via the API if you request /users/user/meters/instance and treats the events in the client, no? It is possible, but very very inefficient. How, then, do we define the dimensions for a given meter in a more structured way? Some built-in values (like flavor) can be pulled automatically based on the resource type, but what about settings controlled by the deployer and end-user (for purposes other than billing)? Do we have to define dimensions explicitely, or isn't what's needed just ways to filter and/or group events by metadata fields? Querying against arbitrary metadata fields is easy in the MongoDB driver, but not in the SQLAlchemy driver. Adding explicit handling for dimensions would let us implement it in SQL and improve performance with indexes in Mongo. Doug -- Julien Danjou // Free Software hacker freelance // http://julien.danjou.info g ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On 24/10/12 23:35 +0200, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate table and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable product/service In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Kind of the same here, if you don't want to really bill the vm, just don't meter them (or ignore the meters) and emit your own meter via your PaaS platform to bill your customer. Or is there a limitation I miss? If you do auto scaling you will have a similar problem. Here you want to monitor the group (with instances comming and going) as a logical unit. One way would be to tag the instances and then extract the tag and send it with the metadata associated with the meter. Then you could query the ceilometer db for that group. (In CloudWatch this is just another Dimension). -Angus -- Julien Danjou -- Free Software hacker freelance -- http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems every time I turn around there's a new metric/instrumentation effort, which is good I guess :) Here is some fun I have been having with a bit of tach+ceilometer code. https://github.com/asalkeld/statgen Is there a single location I can place my feedback? If not, should we create one? I've got lots of suggestions/ideas and would hate to have to duplicate the threads or leave other groups out. I'll add some links here that I am aware of: https://bugs.launchpad.net/ceilometer/+bug/1071061 https://etherpad.openstack.org/grizzly-common-instrumentation https://etherpad.openstack.org/grizzly-ceilometer-actions https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring 3. I'm wrapping up the packaging / cleanup of StackTach v2 with Stacky and hope to make a more formal announcement on this by the end of the week. Lots of great changes to make it easier to use/deploy based on the Summit feedback! Unifying the stacktach worker (consumer of events) into ceilometer should be a first step to integration (or agree upon a common YAGI-based consumer?) 4. If you're looking at Tach, you should also consider looking at Scrutinize (my replacement effort) https://github.com/SandyWalsh/scrutinize (needs packaging/docs and some notifier tweaks on the cprofiler to be called done for now) Looks great! I like the monkey patching for performance as you have done here, but we also need a nice clean way of manually inserting instrumentation too (that is what I have been experimenting with in statgen). Can we chat in #openstack-metering so we are a bit more aware what we are all up to? -Angus Looking forward to moving ahead on this ... Cheers, -S ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings time change?
On 07/09/12 12:39 +0200, Nick Barcet wrote: On 09/05/2012 11:51 AM, Nick Barcet wrote: Thanks for asking, I was just about to come up with this. So based on the poll, it seems that the 3-4pm UTC time slot received the most favors with 9 yes, 1 if need be and 1 no. So I guess we'll have to do without Angus, unless we want to do alternating meetings to satisfy both sides of the planet. In the later case we could do one week at 3pm UTC, and the other at 9PM. What would the others think about this? In any case, the next meeting will be tomorrow at 3pm UTC. I'll be sending a formal invite later today. Based on yesterday's meeting, we decided to try alternating time. Since 9PM UTC is taken on thursdays, I am proposing to move it on Wednesdays. If Nobody objects, this means that starting next week we'll have our meetings on: Wednesday at 9PM UTC for odd weeks (September 12, 26) Cool thanks, good idea! -Angus Thursday at 3PM UTC for even weeks (September 20 and October 4) Does this work for everyone? Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 05/09/12 12:19 +0200, Nick Barcet wrote: PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER The metering project team holds a meeting in #openstack-meeting, Thursdays at 1500 UTC http://www.timeanddate.com/worldclock/fixedtime.html?hour=16min=0sec=0. It's at 1am now (I might make it periodically). Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list - nijaba to open a bug for cookbook and assign to jaypipes - nijaba to add architecture image to project /doc rather than link from google - nijaba to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later * Discussion on session proposal for the summit http://wiki.openstack.org/EfficientMetering/GrizzlySummit * Discussion on alternating meeting time to allow presence from down under Well thanks for trying. I guess I am the only one down this end. The main things I'd like to achieve is to add alarms and monitoring to ceilometer. To achieve this we would need: - auth on api so normal users can access it. - ablility to post custom data via a rest api - ablility to increase the polling frequency on selected resources (this could it's self be metered and charged for) - add alarms and alarm history - add a notification mechanism - could just be rpc / http hook I am happy to wait for summit and discuss it there. I see you have added a wiki entry for it (let my know if you need me to do anything more to make this happen): http://wiki.openstack.org/EfficientMetering/GrizzlySummit/BeyondMetering I added this a while ago: http://wiki.openstack.org/AddingAlarmsToCeilometer Regards Angus * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings day and time change?
On 29/08/12 19:21 -0700, Nick Barcet wrote: Hello, As we are observing that the current meeting time is not very convenient for a few contributors, we would like to propose to change the date and time to something that would suit a larger number. I propose that we do this in a two phase process: 1/ Each project member that care should send a list of days and times that would be convenient for them as a reply to this message. Please try to avoid a day an time already used by another project as listed on [1] 2/ We then open a poll with a compiled list of proposals and get members vote on it. If you are not a contributor to the project, we are happy to receive your suggestions too, but since this meeting is meant to coordinate between contributors, the result of this poll will be heavily skewed toward active members preferences. [1] http://wiki.openstack.org/Meetings/ Hi I'd like to attend but am in Australia. I am quite flexible so it might be easier to say what times don't suit. Basically midnight - 6am which in UTC is 14:00 to 20:00 Regards Angus Thanks, -- Nick Barcet nick.bar...@canonical.com aka: nijaba, nicolas ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] Announcing proof-of-concept Load Balancing as a Service project
On 25/07/12 13:33 -0700, Eugene Kirpichov wrote: Hi Dan, On Tue, Jul 24, 2012 at 8:30 PM, Dan Wendlandt d...@nicira.com wrote: Hi Eugene, Angus, Adding openstack-dev (probably the more appropriate mailing list for discussion a new openstack feature) and some folks from Radware and F5 who had previously also contacted me about Quantum + Load-balancing as a service. I'm probably leaving out some other people who have contacted me about this as well, but hopefully they are on the ML and can speak up. On Tue, Jul 24, 2012 at 7:51 PM, Angus Salkeld asalk...@redhat.com wrote: On 24/07/12 18:33 -0700, Eugene Kirpichov wrote: Hello community, We at Mirantis have had a number of clients request functionality to control various load balancer devices (software and hardware) via an OpenStack API and horizon. So, in collaboration with Cisco OpenStack team and a number of other community members, we’ve started socializing the blueprints for an elastic load balancer API service. At this point we’d like to share where we are and would very much appreciate anyone participate and provide input. Yes, I definitely think LB is one of the key items that we'll want to tackle during Grizzly in terms of L4-L7 services. Great to hear! The current vision is to allow cloud tenants to request and provision virtual load balancers on demand and allow cloud administrators to manage a pool of available LB devices. Access is provided under a unified interface to different kinds of load balancers, both software and hardware. It means that API for tenants is abstracted away from the actual API of underlying hardware or software load balancers, and LBaaS effectively bridges this gap. That's the openstack way, no arguments there :) POC level support for Cisco ACE and HAproxy is currently implemented in the form of plug-ins to LBaaS called “drivers”. We also started some work on F5 drivers. Would appreciate hearing input on what other drivers may be important at this point…nginx? haproxy is the most common non-vendor solution I hear mentioned. Another question we have is if this should be a standalone module or a Quantum plugin… Based on discussions during the PPB meeting about quantum becoming core, there was a push for having a single network service and API, which would tend to suggest it being a sub-component of Quantum that is independently loadable. I also tend to think that its likely to be a common set of developers working across all such networking functionality, so it wouldn't seem like keeping different core-dev teams, repos, tarballs, docs, etc. probably doesn't make sense. I think this is generally inline with the plan of allowing Quantum to load additional portions of the API as needed for additional services like LB, WAN-bridging, but this is probably a call for the PPB in general. So, if I'm understanding correctly, you're suggesting LBaaS to be usable in 2 ways: * Independently * As a quantum plugin Is this right? In order not to reinvent the wheel, we decided to base our API on Atlas-LB (http://wiki.openstack.org/Atlas-LB). Seems like a good place to start. Here are all the pointers: * Project overview: http://goo.gl/vZdei * Screencast: http://www.youtube.com/watch?v=NgAL-kfdbtE * API draft: http://goo.gl/gFcWT * Roadmap: http://goo.gl/EZAhf * Github repo: https://github.com/Mirantis/openstack-lbaas Will take a look.. I'm getting a permission error on the overview. The code is written in Python and based on the OpenStack service template. We’ll be happy to give a walkthrough over what we have to anyone who may be interested in contributing (for example, creating a driver to support a particular LB device). I made a really simple loadbancer (using HAproxy) in Heat (https://github.com/heat-api/heat/blob/master/heat/engine/loadbalancer.py) to implement the AWS::ElasticLoadBalancing::LoadBalancer but it would be nice to use a more complete loadbancer solution. When I get a moment I'll see if I can integrate. One issue is I need latency statistics to trigger autoscaling events. See the statistics types here: http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/US_MonitoringLoadBalancerWithCW.html Anyways, nice project. Integration with Heat would be great regardless of the above decisions. Yes, sounds like a good idea indeed. Is Heat mature enough and used enough to warrant doing this in the near future, or is this better postponed until G at least? Angus? Well I have only just add my simple loadbancer implementation. So don't think there is a great need to rush it (unless a user is asking for it). It would also need to wait until we can get statistics from the lb. -Angus dan Regards Angus Salkeld All of the documents and code are not set in stone and we’re writing here specifically to ask for feedback and collaboration from the community. We would like to start holding weekly IRC meetings at #openstack-meeting
Re: [Openstack] VM High Availability and Floating IP
On 24/07/12 12:12 -0700, Steven Dake wrote: On 07/24/2012 10:08 AM, Alessandro Tagliapietra wrote: But i don't see any part (except the future plans) talking about HA at instance level, that seems more to an application level Currently heat's healthchecking is application only, but we do intend to improve instance healthchecking in v6. We are currently in our v5 development which concludes on July 30th. We have 4-5 week development windows. I'll make certain vm healthchecking is solid for v6 around August. However, we may be able to fit into v5 - I'll have to check with the main author of the HA feature (Angus Salkeld). I have just added a template demonstating an instance sending healthchecks and getting restarted if it fails to send a healthcheck within a specified period. https://github.com/heat-api/heat/blob/master/templates/WordPress_Single_Instance_With_IHA.template (just to give you another option). -Angus The HA feature HOWTO is described a bit here: https://github.com/heat-api/heat/wiki/Using-HA An example HA application template is here: https://github.com/heat-api/heat/blob/master/templates/WordPress_Single_Instance_With_HA.template This template launches a wordpress cloud application. If httpd or mysql fail, they are restarted. If they fail 3 times in 5 minutes, the entire VM is restarted (this is called escalation) under the assumption that the VM container is defective in some way. Regards -steve Il giorno 24/lug/2012, alle ore 18:56, Jay Pipes ha scritto: On 07/24/2012 12:52 PM, Alessandro Tagliapietra wrote: Thank you Jay, never read about that. Seems something like scalr/chef? WHich handles application and keeps a minimum number of vm running? Yeah, kinda.. just one more way of doing things... :) -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Announcing proof-of-concept Load Balancing as a Service project
On 24/07/12 18:33 -0700, Eugene Kirpichov wrote: Hello community, We at Mirantis have had a number of clients request functionality to control various load balancer devices (software and hardware) via an OpenStack API and horizon. So, in collaboration with Cisco OpenStack team and a number of other community members, we’ve started socializing the blueprints for an elastic load balancer API service. At this point we’d like to share where we are and would very much appreciate anyone participate and provide input. The current vision is to allow cloud tenants to request and provision virtual load balancers on demand and allow cloud administrators to manage a pool of available LB devices. Access is provided under a unified interface to different kinds of load balancers, both software and hardware. It means that API for tenants is abstracted away from the actual API of underlying hardware or software load balancers, and LBaaS effectively bridges this gap. POC level support for Cisco ACE and HAproxy is currently implemented in the form of plug-ins to LBaaS called “drivers”. We also started some work on F5 drivers. Would appreciate hearing input on what other drivers may be important at this point…nginx? Another question we have is if this should be a standalone module or a Quantum plugin… Dan – any feedback on this (and BTW congrats on the acquisition =). In order not to reinvent the wheel, we decided to base our API on Atlas-LB (http://wiki.openstack.org/Atlas-LB). Here are all the pointers: * Project overview: http://goo.gl/vZdei * Screencast: http://www.youtube.com/watch?v=NgAL-kfdbtE * API draft: http://goo.gl/gFcWT * Roadmap: http://goo.gl/EZAhf * Github repo: https://github.com/Mirantis/openstack-lbaas The code is written in Python and based on the OpenStack service template. We’ll be happy to give a walkthrough over what we have to anyone who may be interested in contributing (for example, creating a driver to support a particular LB device). I made a really simple loadbancer (using HAproxy) in Heat (https://github.com/heat-api/heat/blob/master/heat/engine/loadbalancer.py) to implement the AWS::ElasticLoadBalancing::LoadBalancer but it would be nice to use a more complete loadbancer solution. When I get a moment I'll see if I can integrate. One issue is I need latency statistics to trigger autoscaling events. See the statistics types here: http://docs.amazonwebservices.com/ElasticLoadBalancing/latest/DeveloperGuide/US_MonitoringLoadBalancerWithCW.html Anyways, nice project. Regards Angus Salkeld All of the documents and code are not set in stone and we’re writing here specifically to ask for feedback and collaboration from the community. We would like to start holding weekly IRC meetings at #openstack-meeting; we propose 19:00 UTC on Thursdays (this time seems free according to http://wiki.openstack.org/Meetings/ ), starting Aug 2. -- Eugene Kirpichov http://www.linkedin.com/in/eugenekirpichov ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Auto scaling / monitoring features
On 16/07/12 18:36 -0500, Miguel Alejandro González wrote: Hello, Last time I deployed OpenStack, I recall it didn't have any performance monitoring of VM instances or any auto scaling (up/down like Amazon ec2) features. I would like to know if any incubated projects support or plan to support these features. I'm specially interesting in auto scaling for academic research purposes. I suppose this algorithm would need to focus in maximizing performance and not minimizing costs (maybe like Amazon). I haven't delved too much into this topic yet! Anyway if there is any efforts toward these features in the community, I would like to know more information about them. Maybe I can join and help them in some way? Hi I am currently working on these two features for heat. I have some functionality in there already but more to come: https://github.com/heat-api/heat/issues/164 https://github.com/heat-api/heat/issues/165 see: https://github.com/heat-api/heat -Angus Regards!! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Where can I consume messages from metering system?
On 25/05/12 10:55 +0200, Szymon Grzybowski wrote:
Hi,
Hi Szymon,
I have very simerlar needs, see comments inline...
(I am working on the heat project https://github.com/heat-api/heat;)
We had a plan to write some plugin which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.
My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:
1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification Alert less than 15% free ram on host}
2. HostA hits less than 15% of free ram and sends notification Alert less
than 15% free ram on HostA.
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.
I am trying to implement AWS::CloudWatch::Alarm
http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html
This defines really simerlar rules to what you are describing, and also
allows users to post (rest) statistics back to a statistics Collecting
server. We could then do fun things like autoscaling and HA recovery.
This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.
Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.
I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data) and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:
- write plugin for agent to send proper alert when metering data hits
proper rules (just like in above scenario with 15% free ram)
- write service which will consume this alert (listen proper topic in
queue) and fetch data from ceilometer collector?? (or it's backend via API)
and find out which VM should be migrated and where (or delegate Where to
nova-scheduler)
I could help out with work on rules and receiving guest statistics if
ceilometer guys are ok with this extension.
Another idea. Are there any Host states in Openstack? For example
Maintenance state? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to maintenance.
2. There goes alert HostA maintenance.
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to active.
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
Everything should be event-driven (alert-notification, host-state-change,
administrator's action).
I'm also wondering what about this
ResourceMonitorAlertsandNotificationshttp://wiki.openstack.org/ResourceMonitorAlertsandNotifications,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).
Me too (I also have been wondering about it), I was quite keen on this
and contacted the authors, but no response...
Regards
Angus Salkeld
First question: Does it make sense? Are such mechanisms implemented in
Openstack right now? Or is it worth to implement?
Second question (if first's answer isn't negative): How and when can I use
Re: [Openstack] [Openstack-operators] Health Monitoring Blueprints
On 13/04/12 18:48 -0700, Stefano Maffulli wrote: On 04/11/2012 02:52 PM, Stefano Maffulli wrote: We're not yet sure to be able to offer this. Wait for an official communication. Here is a brief update regarding the live streaming situation. Here is the good news: Cisco kindly offered http://openstack.webex.com. If you get there you'll see some meetings setup for the days of the Summit, one meeting per each day, per room. Zareason kindly offered 9 computers to be put in each room, connected to the audio equipment available there. Each of Zareason's PC will act as a 'robot-presenter' for Webex. The good news stop here. There is bad news: at least 6 of the zareason PCs run 64bit systems, and 64bit Java on Linux is not compatible with Webex (audio issues). Two of the remaining computers may be too slow to run Webex (I haven't tested them). Possible solutions: - offer live streaming only from 3 rooms (at best, only one at worst) (which ones?) - spend a few hours on Sunday wiping out the 6 machines and put 32 bit systems on - is there a simpler hack to run 32bit Java on 64bit Ubuntu 11.10 systems? - these hours will have to be on top of the hours necessary to setup the rooms, make sure that audio capture really works with the hotel's audio system etc - give to session leaders the honor/burden to run Webex sessions from their own laptops (given how Webex conferencing works, it may be nearly impossible) Other ideas? I'm inclined to go with the simplest option, #1. Unless many (more than 10) people reply to this message asking me to provide full coverage of the event, I'd go with it. For the other options, I'd ask for somebody else to take the lead: that means, helping wipe and install the machines, install java, test them and install them. I'll be glad to provide all support needed. Hi I'de really like to watch these if possible. I am in Australia so recordings would be great too (I have to sleep at some point:). Any ways I really appreciate any work you do in this area. -Angus Let me know asap, so I can plan the weekend. stef ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Announcing project Heat
Hi I'd like to announce a new project that we have been working on, we have named it Heat (heat raises the clouds:). Our goal is to make it possible to manage multiple instance cloud applications with one template/specification file. Initially we want to implement as much as we can of the AWS CloudFormation API and later look at other API like TOSCA. We are attempting to write this in the style of an openstack project so as many people as possible can help out (come join in!). I know this is a busy topic area with several other interesting projects like Burrow, Donabe, Kanyun, Dough, Reddwarf and PlatformLayer. My hope is there we can form an exciting community at this layer and integrate where it makes sense. For instance many of the above projects implement (or could) some of the AWS resource types [1]. Some things that could be done together: - monitoring (Kanyun) and using the results to provide auto scaling. - AWS::SQS::Queue (Burrow) - AWS::RDS::DBInstance (Reddwarf/PlatformLayer) Unfortunately I won't be at the summit but Mark McLoughlin will give a lightening talk about it. Our project pages: http://heat-api.org/ http://wiki.openstack.org/Heat https://github.com/heat-api/heat Drop in at #heat on freenode or email our mailing list (http://lists.heat-api.org/mailman/listinfo). Regards Angus Salkeld [1] http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Metadata and File Injection (code summit session?)
On 10/04/12 15:36 -0700, Justin Santa Barbara wrote: The immediate use case I have in mind is to support this: http://wiki.openstack.org/**PackageConfigForNovahttp://wiki.openstack.org/PackageConfigForNova. That design requires periodic checkins between an instance agent and a nova driver. It certainly /could/ be implemented using ssh, but I originally wrote the design imagining there was a ready-made, standard communication service, and I still think it would be convenient. My concern with that proposal is that it starts simple enough, but then when you want to know e.g. was the package installed successfully? is the service healthy? then you need more and more complexity i.e. you end up with PlatformLayer, RedDwarf, Heat, Puppet, Chef or Juju. So putting a small piece of the required functionality into nova doesn't address your actual use case, which is I want configured machines, not just the stock images. It's probably easier to put that logic into your management system of choice, so nova shouldn't do it. Am I off base here? Yea, that seem odd. Isn't that what your tool above + cloud-init is for? Looks like a layering violation of some kind. -Angus Justin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
