Re: [Openstack] [ceilometer] *NEW TIME* Metering meeting agenda for Thursday at 15:00 UTC (Sept 5th, 2012)
On 09/05/2012 12:19 PM, Nick Barcet wrote: > PLEASE NOTE THE NEW MEETING TIME, ONE HOUR EARLIER > > The metering project team holds a meeting in #openstack-meeting, > Thursdays at 1500 UTC > <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. > > Everyone is welcome. > > Agenda: > http://wiki.openstack.org/Meetings/MeteringAgenda > > * Review last week's actions > - dhellmann to move summit proposal outlines to the wiki and > email links to the dev mailing list > - nijaba to open a bug for cookbook and assign to jaypipes > - nijaba to add architecture image to project /doc rather than link > from google > - nijaba to include a comment in the rst file linking to the google > document where you build the image, so we can remember where it is > later > * Discussion on session proposal for the summit > http://wiki.openstack.org/EfficientMetering/GrizzlySummit > * Discussion on alternating meeting time to allow presence from down > under > * Open discussion > > If you are not able to attend or have additional topic you would like to > cover, please update the agenda on the wiki. The meeting just took place and here is the summary: == #openstack-meeting: Ceilometer == Meeting started by nijaba at 15:00:36 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/ceilometer/2012/ceilometer.2012-09-06-15.00.log.html . Meeting summary --- * LINK: http://wiki.openstack.org/Meetings/MeteringAgenda (nijaba, 15:00:36) * actions from previous meeting (nijaba, 15:01:28) * dhellmann to move summit proposal outlines to the wiki and email links to the dev mailing list (nijaba, 15:01:49) * nijaba to open a bug for cookbook and assign to jaypipes (nijaba, 15:02:08) * nijaba to add architecture image to project /doc rather than link from google & to include a comment in the rst file linking to the google document where you build the image, so we can remember where it is later (nijaba, 15:03:04) * Discussion on session proposal for the summit (nijaba, 15:04:04) * LINK: http://wiki.openstack.org/EfficientMetering/GrizzlySummit (nijaba, 15:04:16) * ACTION: nijaba to see with ttx how our sessions can be shown on official program (nijaba, 15:14:04) * Discussion on alternating meeting time to allow presence from down under (nijaba, 15:14:16) * VOTE: Voted on "should we hold our meetings at alternating times on even and odd weeks?" Results are, Yes: 6 (nijaba, 15:22:48) * ACTION: nijaba to update meeting page to note new alternating meeting time (nijaba, 15:23:58) * Release management (nijaba, 15:24:14) * ACTION: gmb to act as release manager (nijaba, 15:27:32) * LINK: https://launchpad.net/~heut2008/+archive/ppa (dachary, 15:30:15) * ACTION: gmb to a plan on the ml with fixed dates (nijaba, 15:48:39) * VOTE: Voted on "use 0.1 as our first release?" Results are, yes: 5 (nijaba, 15:50:33) * VOTE: Voted on "let's see if voting is case sensitive?" Results are, YeS: 1, nO: 4 (nijaba, 15:51:43) * Open Discusssion (nijaba, 15:52:44) * ACTION: dhellmann make sure flask is listed as a dependency of ceilometer (dhellmann, 15:56:26) Meeting ended at 15:59:07 UTC. Action items, by person --- * dhellmann * dhellmann make sure flask is listed as a dependency of ceilometer * gmb * gmb to act as release manager * gmb to a plan on the ml with fixed dates * nijaba * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * ttx * nijaba to see with ttx how our sessions can be shown on official program People present (lines said) --- * nijaba (121) * dhellmann (61) * zigo (31) * spn (21) * openstack (20) * jd___ (18) * gmb (14) * dachary (13) * eglynn__ (9) * ttx (7) * asalkeld (7) * zul (4) * uvirtbot (3) * jaypipes (2) -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
On 25/10/12 11:13 +, Sandy Walsh wrote: grizzly-common-instrumentation seems to be the best choice ... hopefully the other groups will use this etherpad too. We need a proper blueprint to nail down the approach. IRC is great, but doesn't retain history for other groups. I think we need to get a plan for translating the etherpad into something concise and nailed down. Agree. statgen should really just be a new notifier in Tach (or Scrutinize) ... vs copy-pasting the code into yet-another repo. Hopefully that's the plan? Tach should remain a generic tool and not pegged to OpenStack. Well that was just an "ideas play pen" not serious code. I might be coming at this from a slightly different angle... I was looking at a library that can be used to generate trace, monitoring and metering data (kind of like log levels for logging). Currently both Tach and Scrutinize don't have enough fields (of course that could be changed). Also I think we should be able to insert instrumentation into the code as well as have the function level performance metrics monkey patched. Then we could have a config that directed metric data to different notifiers like how you do it in Scrutinize perhaps. Also enforcing data rate limits and possible data aggregation could be neat configurable features. Anyway more at the meeting... -Angus -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Angus Salkeld [asalk...@redhat.com] Sent: Thursday, October 25, 2012 1:00 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. Yip. So, to that end ... 1. To those that asked, I'm going to get my slides / video presentation made available via the list. Stay tuned. 2. I'm having a hard time following all the links to various efforts going on (seems every time I turn around there's a new metric/instrumentation effort, which is good I guess :) Here is some fun I have been having with a bit of tach+ceilometer code. https://github.com/asalkeld/statgen Is there a single location I can place my feedback? If not, should we create one? I've got lots of suggestions/ideas and would hate to have to duplicate the threads or leave other groups out. I'll add some links here that I am aware of: https://bugs.launchpad.net/ceilometer/+bug/1071061 https://etherpad.openstack.org/grizzly-common-instrumentation https://etherpad.openstack.org/grizzly-ceilometer-actions https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring 3. I'm wrapping up the packaging / cleanup of StackTach v2 with Stacky and hope to make a more formal announcement on this by the end of the week. Lots of great changes to make it easier to use/deploy based on the Summit feedback! Unifying the stacktach worker (consumer of events) into ceilometer should be a first step to integration (or agree upon a common YAGI-based consumer?) 4. If you're looking at Tach, you should also consider looking at Scrutinize (my replacement effort) https://github.com/SandyWalsh/scrutinize (needs packaging/docs and some notifier tweaks on the cprofiler to be called "done for now") Looks great! I like the monkey patching for performance as you have done here, but we also need a nice clean way of manually inserting instrumentation too (that is what I have been experimenting with in statgen). Can we chat in #openstack-metering so we are a bit more aware what we are all up to? -Angus Looking forward to moving ahead on this ... Cheers, -S ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Fwd: Documentation for openstack-java-sdk
Okay, thanks Gui! Lets see how things go, but still I'm awaiting a little
help with respect to the documentation and examples :/
On Thu, Jul 11, 2013 at 8:31 PM, Gui Maluf wrote:
> I know nothing about ceilometer.
> I think the best thing is to checkout the classes on github and make a lot
> of tests. Probably to functioning of objects and methods are the same in
> ceilometer.
> CHeckout the methods and try to workout with it.
> :)
> Good luck!
>
>
> On Thu, Jul 11, 2013 at 11:59 AM, Jobin Raju George wrote:
>
>> Thanks a log, Gui! This helps but would be more useful if you could point
>> me to some *ceilometer-specific *guides/examples.
>>
>>
>> On Thu, Jul 11, 2013 at 8:25 PM, Gui Maluf wrote:
>>
>>> Surely Luis can help you, I've used openstack-java-sdk in one of my
>>> projects, and this is the example Luis gave to me
>>>
>>>
>>> private static final File TEST_FILE = new File("pom.xml");
>>>
>>> private static final String KEYSTONE_AUTH_URL = "
>>> https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0";;
>>>
>>> private static final String KEYSTONE_USERNAME = "";
>>>
>>> private static final String KEYSTONE_PASSWORD = "";
>>>
>>>
>>> /**
>>>
>>> * @param args
>>>
>>> */
>>>
>>> public static void main(String[] args) throws Exception {
>>>
>>> KeystoneClient keystone = new KeystoneClient(KEYSTONE_AUTH_URL);
>>>
>>> //access with unscoped token
>>>
>>> Access access = keystone.execute(Authenticate.withPasswordCredentials(
>>> KEYSTONE_USERNAME, KEYSTONE_PASSWORD));
>>>
>>> //use the token in the following requests
>>>
>>> keystone.setToken(access.getToken().getId());
>>>
>>> Tenants tenants = keystone.execute(new ListTenants());
>>>
>>> //try to exchange token using the first tenant
>>>
>>> if(tenants.getList().size() > 0) {
>>>
>>>access =
>>> keystone.execute(Authenticate.withToken(access.getToken().getId()).withTenantId(tenants.getList().get(0).getId()));
>>>
>>>SwiftClient swiftClient =
>>> newSwiftClient(KeystoneUtils.findEndpointURL(access.getServiceCatalog(),
>>> "object-store", null, "public"), access.getToken().getId());
>>>
>>>//swiftClient.execute(new DeleteContainer("navidad2"));
>>>
>>>swiftClient.execute(new CreateContainer("navidad2"));
>>>
>>>System.out.println(swiftClient.execute(new ListContainers()));
>>>
>>>ObjectForUpload upload = new ObjectForUpload();
>>>
>>> upload.setContainer("navidad2");
>>>
>>> upload.setName("example2");
>>>
>>> upload.setInputStream(new FileInputStream(TEST_FILE));
>>>
>>> swiftClient.execute(new UploadObject(upload));
>>>
>>>System.out.println(swiftClient.execute(new ListObjects("navidad2",
>>> new HashMap() {{
>>>
>>> put("path", "");
>>>
>>> }})).get(0).getContentType());
>>>
>>>}
>>>
>>>
>>> }
>>>
>>>
>>>
>>> On Thu, Jul 11, 2013 at 11:31 AM, Endre Karlson >> > wrote:
>>>
>>>> I think Luis can answer that?
>>>> -- Videresendt melding --
>>>> Fra: "Jobin Raju George"
>>>> Dato: 11. juli 2013 14:38
>>>> Emne: [Openstack] Documentation for openstack-java-sdk
>>>> Til: "openstack lista"
>>>> Kopi:
>>>>
>>>> I am trying to query ceilometer using
>>>> openstack-java-sdk<https://github.com/woorea/openstack-java-sdk>for meters
>>>> of VM's running on KVM. I am able to get the CPU meters via curl
>>>> on the command line but unfortunately I don't find good documentation for
>>>> the SDK's for ceilometer.
>>>>
>>>> I have seen this example
>>>> program<https://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java>
>>>> but
>>>> most of it is commented(probably because it is deprecated).
>>>>
>>>> Where can I find good documentation/examples or java programs/snippets?
>>>>
>>>> --
&
Re: [Openstack] [Metering] Metering repository in stackforge
Hey! On 04/27/2012 06:20 PM, Loic Dachary wrote: > Hi, > > I would like to create a repository ceilometer in > https://github.com/stackforge to host the code for the newborn Metering > project ( https://launchpad.net/ceilometer , first meeting held this thursday > http://wiki.openstack.org/Meetings/MeteringAgenda ). We would be more than thrilled to get you set up. I'm including Andrew here, as he's been doing most of the StackForge work. I'll also file a bug on openstack-ci to make sure we don't lose this. > I'm not sure how to proceed, could you please guide me ? My github account > name is dachary > > Thanks in advance > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 16th, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of componet responsibility - nijaba to do a second thouroughness check on API and report next week * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 29rd, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to link schema in the doc - nijaba to start a thread on meeting time * Discuss session proposal for Grizzly summit * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Oct 25 2012, Angus Salkeld wrote: > If you do auto scaling you will have a similar problem. Here you > want to monitor the group (with instances comming and going) as > a logical unit. One way would be to tag the instances and then > extract the tag and send it with the metadata associated with > the meter. Then you could query the ceilometer db for that group. What about sending meters where the resource is the group and not the instances? (What do you meter exactly on such a group? I'm not really familiar with CW yet) -- Julien Danjou /* Free Software hacker & freelance http://julien.danjou.info */ pgp5ZEGys3WWJ.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
> > I don't think (max - min) would suffice to give an accurate > > measure of the actual CPU time used, as the counter may have > > reset multiple times in the course of the requested duration. > > It is, because /max in the API should be aware of the fact a > reset can occur and computes accordingly. We started to discuss > this a bit in: > > https://bugs.launchpad.net/ceilometer/+bug/1061817 A-ha, OK, so not so much (max - min) as: (\Sigma local maxima) - first Sounds computationally expensive to produce on the fly, but maybe the local maxima can be efficiently recorded as the data is being ingested. Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
Agreed! Code good. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Jeffrey Budzinski [jeffr...@yahoo-inc.com] Sent: Friday, November 02, 2012 10:11 PM To: Angus Salkeld Cc: openstack-...@lists.openstack.org; openstack@lists.launchpad.net Subject: Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ... Agreed on the point about not getting bogged down. I think Sandy is on the same page too from our last exchange (but he can correct me if not). Tim here is working on some code for us to review. I'll ask him to put it in a repo we can all access. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] How does ceilometer work with multi-DC scenario
On Fri, Nov 23 2012, shengjie_...@dell.com wrote: >> Two regions case: >> If you want to get the meters for TenantA from 'swift-cluster-1' >> specifically. According to the API design we have now, there is no API >> can do that directly unless you do some complex API calls combination. > >>>I think you can use: >>>GET /v1/sources/swift-cluster-1/projects//meters/ > >>>in that case, no? > > I actually didn't see this API you mentioned above defined anywhere. We can probably add that in the API anyway. :) -- Julien Danjou // Free Software hacker & freelance // http://julien.danjou.info pgpLWvbNocQJb.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Wiki content imported into MediaWiki - please check
On Wed, Dec 19, 2012 at 10:57 AM, Ryan Lane wrote:
> My vote is to edit the pages to fix them. The conversion script is a giant
> hideous mess of perl. Overall I think we'll be able to fix most issues
> quickly in a doc sprint by editing the pages.
>
> for <> we can use:
>
> 1. An extension: http://www.mediawiki.org/wiki/Extension:SubPageList
> 2. A built-in feature: {{Special:PrefixIndex/Ceilometer}}
>
> SubPageList is more flexible in how the output is displayed and doesn't
> disable page cache like PrefIndex. I'll add that extension today.
>
>
I just added the extension, and used the Ceilometer page as an example of
enabling it.
- Ryan
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
I think Statistics should be find in Ceilometer. Ceilometer may provide with enough information you need. Best regards, Haomai Wang, UnitedStack Inc. 在 2013-7-14，上午8:09，Ray Sun 写道： > In nova, we have a period task to report the usage of the physical server, > including CPU, Memory and Local Disk, but I don't think I can find the same > strategy in cinder service. Is there any way to do this or is there any > blueprint for this? > > Thanks. > > Best Regards > -- Ray > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Implemented three methods in Ceilometer
That simply appears that you have a network issue… Perhaps a firewall at work blocking that port number, or a DNS issue, or a temporary issue with review.stackforge.org? John Postlethwait Nebula, Inc. 206-999-4492 On Thursday, June 21, 2012 at 11:37 PM, Kobagana Kumar wrote: > Hi Julien Danjou, > > I followed the steps given in http://wiki.openstack.org/GerritWorkflow this > site. But I am getting some error while executing the command "git review". > > While I am executing that command I am getting following error: > > Problem running 'git remote update gerrit' > Fetching gerrit > ssh: connect to host review.stackforge.org (http://review.stackforge.org) > port 29418: Connection timed out > fatal: The remote end hung up unexpectedly > error: Could not fetch gerrit > > Can you please tell me the way to resolve this error. > > Thanks & Regards, > > Bharath Kumar > > -Original Message- > From: Julien Danjou [mailto:julien.dan...@enovance.com] > Sent: Wednesday, June 20, 2012 2:13 PM > To: Kobagana Kumar > Cc: openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) > Subject: Re: [Openstack] [Metering] Implemented three methods in Ceilometer > > On Wed, Jun 20 2012, Kobagana Kumar wrote: > > Hi Kobagana, > > > I have implemented three more classes in Ceilometer plug-in module. > > Added those classes in libvirt.py file in compute. > > The classes which I have added are counters to find out the following: > > > > 1. Number of CPUs used > > > > 2. Memory used > > > > 3. Maximum memory used > > I am also ready with test cases for those methods. > > > > Please let me know the procedure to contribute my code to ceilometer > > code base. > > > > > You need to send your patches to https://review.stackforge.org/ using > git-review. > > You can find some documentation about this here: > > http://wiki.openstack.org/GerritWorkflow > > -- > Julien Danjou > // eNovance http://enovance.com > // ✉ julien.dan...@enovance.com (mailto:julien.dan...@enovance.com) ☎ +33 1 > 49 70 99 81 > > DISCLAIMER > == > This e-mail may contain privileged and confidential information which is the > property of Persistent Systems Ltd. It is intended only for the use of the > individual or entity to which it is addressed. If you are not the intended > recipient, you are not authorized to read, retain, copy, print, distribute or > use this message. If you have received this communication in error, please > notify the sender and delete all copies of this message. Persistent Systems > Ltd. does not accept any liability for virus infected mails. > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Fri, Nov 2, 2012 at 3:07 AM, Patrick Petit < patrick.michel.pe...@gmail.com> wrote: > Folks, > I'd like to add to this that physical server metering shouldn't be treated > differently in Ceilometer now that bare metal provisioning framework enters > into Grizzly. Physical servers will just become billable resources much > like VMs. I am not speaking of physical server monitoring here. Just > extending Ceilometer agent to also report usage data out of the physical > box. > Thanks for posting this, Patrick. I understood "physical devices" as "host server" not as "bare-metal server". I suspect we could use the same agent framework, but almost all of the pollsters would need to be different because they would be running inside the guest OS rather than on a host VM, so the APIs they will use to collect the same data will be different. Doug > Cheers > Patrick > > Envoyé de mon iPad > > Le 1 nov. 2012 à 19:13, Julien Danjou a écrit : > > > On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: > > > >> My goal is to offer monitored data to the admin and customers. The > admin is > >> interested in the utilization of the physical components and the virtual > >> machines and the customer is interested to know what his VMs do or can > do. > >> It would be nice to get the data from a single point. I thought I can > >> enhance the Ceilometer compute agent to get this data out. Does this > make > >> sense or is it better to use another monitoring tool for the physical > >> components? > > > > I think the pollster implementation can be done. I wouldn't implement > > this in the compute agent, but probably in some hardware agent, because > > it's likely it would be used in different kinds of environment and not > > only on compute node, i.e. you may also want to meter hardware usage for > > you cinder or glance node anyway. > > > > About the 10 minutes polling interval Doug mentionned, this can be a > > problem indeed, but it's still solvable later and would be easy to solve > > if this in a different agent, since you could change the periodic > > interval for pollster runs to something like 1 or 5 minutes. > > > > -- > > Julien Danjou > > // Free Software hacker & freelance > > // http://julien.danjou.info > > ___ > > Mailing list: https://launchpad.net/~openstack > > Post to : openstack@lists.launchpad.net > > Unsubscribe : https://launchpad.net/~openstack > > More help : https://help.launchpad.net/ListHelp > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Problem with nova resize events
Hi Jay and Julien thanks very much for your help. I am not terribly familiar with github and code reviews, I am not a developer :(, so maybe I am not using the correct terms here. We are using version 2013.1.2 of ceilometer, so we have no "ComputeInstanceNotificationBase" class in notifications.py, instead we have " class _Base(plugin.NotificationBase) ", which handles events with get_event_types(): return ['compute.instance.create.end', 'compute.instance.exists', 'compute.instance.delete.start', 'compute.instance.finish_resize.end', 'compute.instance.resize.revert.end'] like ComputeInstanceNotificationBase does. What is your suggestion in order to enable handlings of more compute events, considering we will go live in 2-3 weeks time? 1- We wait for Julien's patch to be approved and we upgrade to the patched version. 2- We modify ourselves get_event_types() on our installation 3- ? Thanks Alex -Original Message- From: Julien Danjou [mailto:jul...@danjou.info] Sent: mercoledì 24 luglio 2013 17:39 To: Jay Lau Cc: Alessandro Barabesi; openstack@lists.launchpad.net Subject: Re: [Openstack] [Ceilometer] Problem with nova resize events On Wed, Jul 24 2013, Jay Lau wrote: > Hi Alex, > > Not sure if it is a bug, but you are right, ceiloemeter did not send > the notification that you required. Perhaps you can log a bug or else > you can patch your cluster directly to enable this. > > class ComputeInstanceNotificationBase(ComputeNotificationBase): > """Convert compute.instance.* notifications into Counters > """ > @staticmethod > def get_event_types(): > return ['compute.instance.create.start', > 'compute.instance.create.end', > 'compute.instance.exists', > 'compute.instance.update', > 'compute.instance.delete.start', > 'compute.instance.delete.end', > 'compute.instance.finish_resize.end', > 'compute.instance.resize.revert.end'] > Thanks, Not really a bug, but we know we could do better. I've just cooked a patch that should improve that: https://review.openstack.org/38485 -- Julien Danjou # Free Software hacker # freelance consultant # http://julien.danjou.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Changes to the ceilometer Jenkins job
Hi Doug, On 30/05/12 13:14, Doug Hellmann wrote: > I started working on setting one up yesterday, and plan to continue > working on it later today. I would appreciate any feedback you could > give me on: > > https://github.com/dhellmann/ceilometer/commit/d2855c656c5442e46a4a891ff91d6be0abff9706 I'm not an expert but it looks good to me. Best thing to do is to propose a change via. gerrit. This will run a check test with the new tox.ini. You can then amend the commit and review again to upload further patchsets if you find some things need changing. Once this is in your other commits can simply be rebased to use it. Kind Regards -- Andrew Hutchings - LinuxJedi - http://www.linuxjedi.co.uk/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Metering meeting agenda for Thursday at 16:00 UTC (Aug 23rd, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions - jaypipes to create ceilometer cookbook - nijaba to write description of component responsibility - dhellmann and nijaba to work on sessions for summit via email - dhellmann to ask jtrans about interest in reviewer status - nijaba to give core reviewer rights to gmb * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift]A design draft of Storage Quota
Hi Kevin, On Wed, Feb 20, 2013 at 5:26 PM, Kevin L. Mitchell wrote: > I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and > https://github.com/klmitch/boson with some initial work. Unfortunately, > I'm not able to work on Boson at the moment due to higher-priority > tasks… >From a quick look of it why can't we do the same as Boson without synaps[1]+ceilometer+swift_container_update. I don't know very well those but from the look of it you could have synaps generating alerts based on resources collection from ceilometer and set the enforcement using the native service mechanism? Chmouel, [1] when properly working with a proper API. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] ceilometer and instance snapshots/glance images
I have created instance snapshot. When I am listing them through ceilometer API
call, then I am getting userId: null for them.
I remember I used to get userId(who have created that snapshot) in response but
I am not getting it now.
Is there any change in behaviour?
or there is any extra setting required in configuration?
API call : http://10.102.153.71:8777/v2/resources
response :
* {
* "resource_id": "52f814e5-fa6b-4a3c-acb7-80d13ced88da",
* "project_id": "f57362577d3b4c7c82b6bc713d0dcc25",
* "user_id": null,
* "metadata":
* {
* "status": "active",
* "name": "nanovmsnap",
* "deleted": "False",
* "container_format": "ami",
* "created_at": "2013-07-09T09:43:47",
* "disk_format": "ami",
* "updated_at": "2013-07-09T09:44:08",
* "protected": "False",
* "checksum": "5d8fa241f393bb8ef6eea6576671a187",
* "min_disk": "0",
* "is_public": "False",
* "deleted_at": "None",
* "min_ram": "0",
* "size": "9830400"
* }
* }
Thanks,
Anshul
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Documentation for openstack-java-sdk
I am trying to query ceilometer using openstack-java-sdk<https://github.com/woorea/openstack-java-sdk>for meters of VM's running on KVM. I am able to get the CPU meters via curl on the command line but unfortunately I don't find good documentation for the SDK's for ceilometer. I have seen this example program<https://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java> but most of it is commented(probably because it is deprecated). Where can I find good documentation/examples or java programs/snippets? -- Thanks and regards, Jobin Raju George Third Year, Information Technology College of Engineering Pune Alternate e-mail: georgejr10...@coep.ac.in ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] First meeting for the Metering project
On 12-04-27 10:50 AM, Nick Barcet wrote: > The meeting occurred as planned, and overran... Nevertheless a lot of > good decisions were made, the obviously most important one being > the project name: ceilometer. > > For a more complete meeting summary, go visit: > http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary > > Next week we will have for objective to find an agreement on "schema and > counter definitions". If anyone does not do it before me I'll fire an > introduction email on the subject soon (sometime this we), as we agreed > to start the discussion via mailing list and use the irc meeting to > finalize the choices. The Launchpad project was created: https://launchpad.net/ceilometer Cheers -- Francis J. Lacoste francis.laco...@canonical.com signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Changes to the ceilometer Jenkins job
Hi, We noticed that currently, on Stackforge, the ceilometer project has only one Jenkins job, checking that the merge occurs correctly. We'd like to add jobs to run unit tests, pep8, etc… I took a look at the openstack-ci-puppet repo, but I'm not sure of the changes, so I prefer to ask here for someone that understand this to help. From what I see, it's likely we would like to use the default "python_jobs" template. If anything is missing on our side to use the standard set of checks, we'll do what is necessary to be able to use them. :) Thanks in advance! Cheers, -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Get vcpu hours per instance
On 06/15/2012 10:21 PM, Guillermo Alvarado wrote: > Hi everyone! > > I want to know the vcpu hours per instance, in horizon In nova > dashboard I can see the total hours for the tenant, I want to see the > hours for each instance, which together must be equal to total. How can > I do that? > > I think the api does not have some method like that, What do you propose? Hello Guillermo, While it is not ready yet for consumption, this is part of the dataset that we are targeting for the ceilometer project. Feel free to join us and help moving this effort forward. https://launchpad.net/ceilometer -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [infra] Stackforge issues: tests & mails
On Mon, Jun 25 2012, Andrew Hutchings wrote: > I can see the jobs on Jenkins just fine and puppet confirms there was no > problem with deployment. The Zuul service, however, was not running. > This has been fixed. Things changed indeed, but something is still wrong I think. I just sent a review request: https://review.stackforge.org/#/c/232/ and Jenkins checked it. But we lost the link to Jenkins in the review. Looking for the check manually here: https://jenkins.stackforge.org/view/Ceilometer/job/gate-ceilometer-merge/ seems to show that build #73 was related to this request but it has been successful. But it reviews and reports as "LOST". Any hint? :) -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Oct 25 2012, Dan Dyer wrote: > I don't think its just a matter of adding more meters or events for a couple > of reasons: > 1. In many cases the metadata I am referring to comes from a different > source than the base usage data. Nova is still emitting its normal events, > but we get the service/user mapping from a different source. I would not > characterize this data as usage metrics but more data about the system > relationships. > 2. in the multiple VM case, we need to have the relationships specified so > that we can ignore the proper VM's. There has also been talk of hybrid > billing models that charge for some part of the VM usage as well as other > metrics. Once again we need a way to characterize the relationships so that > processing can associate and filter correctly. Sure, I am not stating you don't need to specify the relationship. I'm just saying I don't see why existing counter in ceilometer should be modified to store the relationship and why it should be aware of it. Maybe I don't understand properly what you want to do, so I'll try to give a concrete example. Please correct me and amend the example if it doesn't match what you've in mind. Let's say you have a PaaS platform built on instances. This PaaS platform has a user P in keystone and launches instances with that user. Now you have to spawn the platform, or a new instance of it, doesn't matter, and launch let's 3 VM instances X, Y and Z. Ceilometer records that 3 VMs instances are running for user P the entire lifetime of the platform. At this point, what I'm saying is that your PaaS platform also needs to send meters too like: "Customer C is using the platform running on VM X, Y and Z". The VM the platform is using can be stored in the metadata of the meter the platform emits. You don't need to modify the existing meters. With that, it's possible when billing customer C to request all meters concerning the PaaS platform (you filter by counter name and/or source) and bill the PaaS platform to C. If you want detail usage, or charge back your platform owner, you also can bill the platform using the meter on the VM (and here bill the IaaS part). So the relationship is stored in Ceilometer (via metadata) and you can exploit it to build a more complex billing system with more layers. How does that sound? -- Julien Danjou ;; Free Software hacker & freelance ;; http://julien.danjou.info pgp8irt5IFg7F.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
On Tue, Jul 16, 2013 at 12:47 PM, Doug Hellmann wrote: > When I said, "we", I meant "the ceilometer team". If the auditing app > isn't finding any volumes, it's not going to notify us. > > If you just want to know how much data is being used by cinder, there may > be a way to get that from their admin API, but I'm not sure. > > > On Mon, Jul 15, 2013 at 7:08 PM, Ray Sun wrote: > >> D >> oug, >> Thanks. I tried it in grizzly, here's the return: >> sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit >> Starting volume usage audit >> Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00 >> Found 0 volumes >> Volume usage audit completed >> >> Actually, I want to get some data like this: >> Total Cinder Storage on Physical Machine: 100G >> Used Cinder Storage on Physical Machine: 10G >> >> Is there any way to get this? >> >> >> Best Regards >> -- Ray >> >> >> On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann < >> doug.hellm...@dreamhost.com> wrote: >> >>> We rely on a similar audit program to get the "exists" notifications >>> about cinder volumes. Look for "cinder-volume-usage-audit". >>> >>> Doug >>> >>> >>> On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun wrote: >>> >>>> Yes, it should be, but seems not at least in grizzly. Any update of >>>> Ceilometer? >>>> >>>> Best Regards >>>> -- Ray >>>> >>>> >>>> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang >>>> wrote: >>>> >>>>> I think Statistics should be find in Ceilometer. Ceilometer may >>>>> provide with >>>>> enough information you need. >>>>> >>>>> Best regards, >>>>> Haomai Wang, UnitedStack Inc. >>>>> >>>>> 在 2013-7-14，上午8:09，Ray Sun 写道： >>>>> >>>>> In nova, we have a period task to report the usage of the physical >>>>> server, including CPU, Memory and Local Disk, but I don't think I can find >>>>> the same strategy in cinder service. Is there any way to do this or is >>>>> there any blueprint for this? >>>>> >>>>> Thanks. >>>>> >>>>> Best Regards >>>>> -- Ray >>>>> ___ >>>>> Mailing list: https://launchpad.net/~openstack >>>>> Post to : openstack@lists.launchpad.net >>>>> Unsubscribe : https://launchpad.net/~openstack >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>> >>>>> >>>> >>>> ___ >>>> Mailing list: https://launchpad.net/~openstack >>>> Post to : openstack@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~openstack >>>> More help : https://help.launchpad.net/ListHelp >>>> >>>> >>> >> > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > there is an os-hosts extension that gives things like volume-count and GB/used on a cinder volume-service node, however it's not currently exposed from the client. Not sure if that's the sort of thing you're looking for or not. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
Yes, I see a general need to be able to represent meta data the identifies associations between monitored systems and between usage stored in the datastore. There could be a variety of ways that we need to relate event data, so the mechanism should be relatively generic. In addition to your use case, we would want to be able to map instances to other tenants, group VM's together to represent some kind of shared identity or behavior, map instances to some kind of special service type. I think there are big advantages to keeping the collection and storage of this data separate: 1. It does not require Nova to be aware of the details of the VM internals. 2.It allows for deferral of processing until later in the rating process, which helps on scalability 3. makes it easier to extract and report on this data for other use cases besides the base billing 4. it more extensible in the sense that you can add arbitrary metadata without affecting the core usage data generation. We use this information as part of our rating process to determine the correct charges to apply, so we will need to be able to query for it. Dan On 11/5/2012 5:04 AM, Doug Hellmann wrote: On Fri, Nov 2, 2012 at 4:42 PM, Dan Dyer <mailto:dan.dye...@gmail.com>> wrote: Yes, I am assuming the service controller provides a different stream of data from the lower level VM events. So the question is how to represent and store this additional meta data in ceilometer. Note that there doesn't necessarily need to be a linkage/grouping between the resources since the association is what is actually contained in the metadata that is provided by the service controller. As a summary Nova provides its normal events for usage Service controller provides a mapping of nova instances to service type and actual end user So the problem isn't necessarily that you want to measure something different, but that the "ownership" in the existing data is not "correct" from the perspective of the billing system. We have a similar issue at DreamHost. Our existing user database has account ids that need to be mapped to tenant ids from keystone. Rather than putting that information in keystone, or ceilometer, we decided to store it in our system and have the DreamHost billing system drive the ceilometer API. Does it make sense to do something similar here? If we definitely want ceilometer to hold the metadata, then I could also see adding an API to let an outside system add metadata to a resource. That would let the PaaS code, which knows about each VM, store extra data that would be returned with the VM metadata when a caller visits /resources/. Would you expect to be able to query using the metadata? For example, "provide the total instance hours for all instances with paas_tag=foo"? Doug Dan On 11/1/2012 11:25 AM, Doug Hellmann wrote: On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer mailto:dan.dye...@gmail.com>> wrote: In some cases, the service controller is actually running inside a VM. It would not have access to the internals of the VM's. It maintains its metadata separately from the Nova infrastructure. It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or "join" it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug DD On 10/25/2012 2:25 AM, Nick Barcet wrote: Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each "Unit" of the service such as : If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specifi
[Openstack] [ceilometer] Release status 2012-10-05
Hi all,
Here's a better-late-than-never release status update for Ceilometer 0.9. We're
now a week away from release.
Release in numbers
--
Bugs in progress: 5
Fixes committed: 39
Triaged bugs: 15
Confirmed bugs: 21
New bugs: 1
These numbers were generated using a launchpadlib API script (attached). We
haven't set up a milestone for the 0.9 release; I suggest that we do and will
take care of it today if no-one objects.
The full list of bugs by status is attached for completeness.
Status of roadmap
-
According to http://wiki.openstack.org/EfficientMetering/RoadMap, there's one
bug still in progress that should be targeted for Folsom.0:
1021775: Assignee: jdanjou; Listen Quantum notifications
Julien, what's the situation with this? I know we're technically in feature
freeze but we've got a week left until release and if we can get this committed
and QA'd in time, that would be lovely.
QA status
-
I've no information about the QA status of any of the above bugs. Is this
recorded anywhere? If not, I'd suggest that we record it as a tag on the bug,
thus (liberally stolen from the Launchpad project):
- Not QA'd yet: qa-needstesting
- QA'd; all good: qa-good
- QA'd; bad: qa-bad (can be updated to
qa-needstesting or -good once problems are fixed)
Objections, comments?
Further questions
-
Are there any bugs not listed on the roadmap that are essential for the 0.9
release?
#! /usr/bin/env python
from launchpadlib.launchpad import Launchpad
from launchpadlib.uris import LPNET_SERVICE_ROOT
project = "ceilometer"
lp = Launchpad.login_anonymously('grabber', LPNET_SERVICE_ROOT)
project_bugtasks = lp.projects[project].searchTasks()
by_status = {}
for bug_task in project_bugtasks:
if bug_task.status not in by_status:
by_status[bug_task.status] = set([bug_task])
else:
by_status[bug_task.status].add(bug_task)
bug_line = " {bug}: Assignee: {assignee}; {title}"
for status, tasks in by_status.items():
print status
print "-" * len(status)
print ""
print "Total bugs: %i" % len(tasks)
for task in tasks:
bug = task.bug
bug_dict = {
'bug': bug.id,
'assignee': task.assignee or "None",
'title': bug.title,
}
print bug_line.format(**bug_dict)
print "\n"
In Progress
---
Total bugs: 5
1012242: Assignee: https://api.launchpad.net/1.0/~jtran; remove database
access from agent pollsters
1024563: Assignee: https://api.launchpad.net/1.0/~jtran; Problems starting
ceilometer-collector due to missing notifications_topics flag
1046404: Assignee: https://api.launchpad.net/1.0/~jaypipes; Ceilometer would
welcome a chef cookbook
1039069: Assignee: https://api.launchpad.net/1.0/~jdanjou; remove "duration"
field
1021350: Assignee: https://api.launchpad.net/1.0/~jsimms; add self-
enable/disable check on plugin load
Fix Committed
-
Total bugs: 39
1024093: Assignee: None; remove dependency on nova.service
1055319: Assignee: https://api.launchpad.net/1.0/~spn; Flask package is
missed since tools/pip-requires is not used
1004200: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; make
collector listen on metering queue
1004127: Assignee: https://api.launchpad.net/1.0/~jdanjou; convert to
openstack-common cfg
1023969: Assignee: https://api.launchpad.net/1.0/~jdanjou; Enhance/implement
counter types
1053515: Assignee: https://api.launchpad.net/1.0/~jtran; pep8 not checking
tests subdirectory
1060918: Assignee: https://api.launchpad.net/1.0/~jdanjou; Misleading network
meter names(net_in_int, net_out_int)
1004130: Assignee: https://api.launchpad.net/1.0/~jdanjou; fix logging
configuration
1004198: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; make agent
publish counters to metering queue(s)
1022679: Assignee: https://api.launchpad.net/1.0/~jtran; add authentication
options to mongodb driver
1018443: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; set up doc
build
1004560: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; add listener
for instance "exists" notifications
1023054: Assignee: https://api.launchpad.net/1.0/~nijaba; Need to add a link
to the roadmap in the docs
1005944: Assignee: https://api.launchpad.net/1.0/~jdanjou; need nova to tell
us when it is deleting instances
1006989: Assignee: https://api.launchpad.net/1.0/~heut2008; add emitting host
field to meter messages
1059765: Assignee: https://api.launchpad.net/1.0/~jdanjou; define constants
for counter types
1060939: Assignee: https://api.launchpad.net/1.0/~jdanjou; Wrong meter name
for disk IO
1006120: Assignee: https://api.launchpad.net/1.0/~doug-hellmann; add me
Re: [Openstack] cfg usage - option registration, global objects
On Wed, Jun 6, 2012 at 10:58 AM, Mark McLoughlin wrote: > On Tue, 2012-06-05 at 17:25 -0400, Mark Washenberger wrote: > > > > "Mark McLoughlin" said: > > > > > On Tue, 2012-06-05 at 12:21 -0400, Mark Washenberger wrote: > > >> > http://wiki.openstack.org/CommonLibrary#Incubation > > >> > > >> Once an api is in incubation, if you make a change to it, you are > > >> expected to update all the other openstack projects (not just core > > >> projects?) to make them work with the new api. Am I understanding this > > >> requirement correctly? > > > > > > Yes, pretty much. > > I should clarify this - I don't think someone improving an API in > openstack-common absolutely must update all projects that use it, but I > think he/she often will update the major users to make sure the new API > works. > > But the reality is that projects which use openstack-common need to be > prepared that someday they might re-sync with latest openstack-common > using update.py and find the API has changed. > > > > The alternative is that you don't make backwards > > > incompatible API changes. > > > > ... > > > > > > > > What alternative strategy are you suggesting? That if glance, quantum, > > > cinder and ceilometer want to re-use Nova's RPC code, they should > > > copy-and-paste it and hack it to their needs? > > > > I don't think our only options here are immediate adoption and relative > > chaos. > > > > Here's what I would like to see: > > > > Quantum, cinder, and ceilometer get together, recognize a shared need > > for rpc, acknowledge the successes and failures of the nova.rpc library, > > and create a better implementation with eventual adoption by Nova kept > > in mind. > > > > Doesn't that sound better? This approach seems much nicer to me, > > because I believe that code reuse is likely to be detrimental unless > > the code we're talking about was created with reuse and generality > > in mind. Since I find that suggestion implausible regarding nova.rpc > > in particular, I think we will do better overall avoiding its wider > > adoption. > > > > Please, forgive me if I'm being drawn in falsely by the allure of better > > code. Code structure and quality *is* something I obsess about. But I > > appreciate the need to be practical in the short term as well, if I am > > not always the best at articulating it. The agreement from various > > quarters about the problems with the current nova.rpc gave me hope > > that we could craft a better rpc library without too much delay, even > > if I myself can only contribute in a small way to that effort. > > I think the summary is that you'd like (someone else?) to start from > scratch on a new RPC API in openstack-common, whereas Russell has gone > for the approach of taking Nova's RPC API and improving it iteratively. > > In the absence of someone appearing with that new idealised RPC API, I > think it's reasonable for Russell to proceed with his approach. > Yes, please, keep going! We're too close now to stop now. I do have some enhancements to propose, but I think I can make them in a backwards compatible way once the existing code is in the common lib. Ceilometer is currently importing bits we need either directly from nova or openstack-common (by "importing" I mean literally using the "import" statement in our code, not copying the required modules into the ceilometer code base). I was under the impression that this is how we wanted (new) projects to use openstack-common, but maybe I misunderstood? Should we be planning to copy code out of openstack-common into the ceilometer repository? Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Glance usage data retrieval
Hi Julien and Stuart! Comments inline... On 06/19/2012 07:12 AM, stuart.mcla...@hp.com wrote: Brian, Jay, I'll give you a chance to reply to Julien first, but I have a follow on query... It doesn't seem like right now Glance produces enough records for full metering of operations. Eg if you want to charge a specific user for every successful image upload/image download this information doesn't 'fall out' of the current logs. For example the eventlet output such as: Jun 19 09:18:39 az1-gl-api-0001 DEBUG 3795 [eventlet.wsgi.server] 127.0.0.1 - - [19/Jun/2012 09:18:39] "GET /v1/images/4921 HTTP/1.1" 200 104858310 2.098967 doesn't tie the GET operation to a particular user. As Julien mentions there is an image_upload notification, but I don't see an equivalent image_download notification. Yeah, there is one :) It's called image.send: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L892 (Note I'm using old Diablo code at the moment, so I'm going on code inspection of the latest code -- apologies if I missed anything.) Is the creation of records of operations in a well-defined format which can be consumed by eg a Metering and Billing team something we'd like to have? (I'm ignoring Data at Rest metering for now.) Not sure about this one, but I believe the message format is the same as Nova: https://github.com/openstack/glance/blob/master/glance/notifier/__init__.py#L55 With the exception of the request ID. The payload is structured here for downloads: https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L882 If so, would something along the following lines be worth considering? A wsgi filter, using a mechanism similar to Swift's posthooklogger, which has a routine which is called when operations are about to complete. It should have access to the context (http return status, user, operation etc) so would be able to raise a notification if appropriate. Using a standard notifier would mean output could be to a log file or, perhaps in time, the notifications could be consumed by ceilometer. LOL, that's actually how it currently works :) https://github.com/openstack/glance/blob/master/glance/api/v1/images.py#L917 All the best, -jay -Stuart On Tue, 19 Jun 2012, Julien Danjou wrote: Hello, As part of the ceilometer project¹, we're working on usage data retrieval from various OpenStack components. One of them is Glance. We're targeting Folsom for the first release, therefore it seems important for both projects to be able to work together, this is why we're bringing ceilometer to your attention and asking for advices. :) What we want is to retrieve the maximum amount of data, so we can meter things, to bill them in the end. For now and for Glance, this only includes the number of image uploaded (per user/tenant), but we'll need probaly more in a near future. At this point we plan to plug into the notification system, since it seems to be what Glance provides to accomplish this. And so far, the notifications provided seem enough to accomplish what we want to do. Do you have any advice regarding integration of Ceilometer and Glance together? Is this something a stable interface we can rely on, or is there a better way to do so? Thanks in advance, Regards, ¹ http://launchpad.net/ceilometer -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Where can I consume messages from metering system?
On Fri, May 25, 2012 at 4:55 AM, Szymon Grzybowski wrote:
> Hi,
>
> We had a plan to write some "plugin" which will collect metering data from
> VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
> via libvirt (mayby later for xen etc), but i've found ceilometer project,
> which is doing what I want or will be doing what I want in near future :).
> I was thinking about collectd/ganglia/munin as source of data and I wanted
> to write local agents which will poll those daemons (collectd/ganglia) and
> push data to Rabbit, so it's pretty close to ceilometer's guys.
>
The idea with metering is that by standardizing on a message format you can
tie in at a couple of different points. For publishing data, you could
write a plugin for our agent that does the polling and lets the agent
publish, or you could have your agent publish messages directly on the
metering queue in the right format. However, our focus right now is on
capturing information about utilization so we can document it and charge
the customer. For the sort of features you are talking about, you probably
want more data collected more frequently.
> My main purpose is to consume those messages and write algorithm to manage
> cloud from administrative perspective. I'd like to create something like
> DRM in VmWare - I was on presentation about this topic recently and from
> that brief I learnt that VmWare has something like rules to manage VMs. I
> think, that example (use case) from presentation would be the best way to
> describe what I'm trying to point out:
>
> 1. Administrator defined a rule: { If free ram on host is less than 15%,
> send notification "Alert less than 15% free ram on host"}
> 2. HostA hits less than 15% of free ram and sends notification "Alert less
> than 15% free ram on HostA".
> 3. Central collector or central daemon receives Alert about free ram.
> 4. Central collector pick VM from HostA and migrate it to HostB. - How he
> decides which VM and to which Host migrate is part of algorithm/policy.
>
> This is the basic idea, how does it work - this is what I've found out
> during presentation. And now my whole idea about this mechanism and
> openstack: there could be different policies(algorithms/plugins) how, where
> and when migrate VMs and this could be implemented in external plugins.
>
> Someone can say that there is nova-scheduler. This scenario isn't exactly
> what nova-scheduler is doing, but I see, that in this solution
> nova-scheduler can play his role as service, which picks best host to
> migrate VM.
>
It sounds like an updated scheduler class could use this new data to make
its decision about which host is least busy, at least.
> I'm trying to find out what can I use from ceilometer as metering system.
> I've read few pages on wiki and posts on mailing list about ceilometer's
> architecture, messages etc.
> Right now I know there are Agents and Collectors. Agents get data from
> hypervisor (metering data) and push them in queue (nova notification
> system). Collectors listens to queue's topic and receives those messages.
> I'd like to:
>
The agent will publish to a new queue using a pattern similar to
notification, but the messages won't be in quite the same format or on the
same exchange.
>
>- write plugin for agent to send proper alert when metering data hits
>proper rules (just like in above scenario with 15% free ram)
>
>
>- write service which will consume this alert (listen proper topic in
>queue) and fetch data from ceilometer collector?? (or it's backend via API)
>and find out which VM should be migrated and where (or delegate "Where" to
>nova-scheduler)
>
>
>
>-
>
> Another idea. Are there any Host states in Openstack? For example
> "Maintenance state"? Besides scenarios with free ram, cpu and other
> resources there is also typical example of how useful this mechanism could
> be with maintenance state.
> *Scenario:*
> 1. Administrator changes HostA state to "maintenance".
> 2. There goes alert "HostA maintenance".
> 3. Central collector or central daemon receives Alert about maintenance.
> 4. Central collector gets list of VMs on HostA.
> 5. Central collectors start migration all VMs from HostA to other Hosts
> (or invoke nova-scheduler to reassign VM from HostA to other hosts, but in
> this case we need to change or implement new filter in nova-scheduler)
> 6. Administrator changes HostA state to "active".
> 7. Openstack starts new machines on this node (this is now done via
> nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
>
> Everything should be event-d
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
As for statgen, I think that¹s just a temp repo, it'd be nice to have the end result of this be a library that provides somewhat generic metrics and plugins and such so that stacktech could use the outputs of it, ceilometer could the outputs and other systems could use the outputs (where an output goes would be configurable so that each system can configure its outputs as the operator desires, ie I want my MONITOR metrics to go to MQ in ceilomter and stacktech consumable formats, or to files or to...). I think when this gets going we should have some repo/project name that goes on stackforge so that even while this is being developed it can go through the normal review process and such (and not in someones special github repo). But we have to start somewhere, something we can discuss as to what a good solution is @ the meeting. -Josh On 10/25/12 5:47 PM, "Jeffrey Budzinski" wrote: > >Yes, I think support for metrics objects that can be leveraged both by >monkey patches and decorators was what we'd been thinking along the lines >of. The metrics would be controlled via config both in what scopes are >active (e.g. on|off for a package, module, etc.) and also the outlet for >the metrics (file, datagram, rpc). Support for instrumentation levels >would also be nice so that metric flow could be controlled (i.e. >instrumentation point is annotated as METRIC, MONITOR, PROFILE and then >level to actually emit can be set in conjunction with a metric >outlet/sink). With this approach, folks could control both the volume of >metrics and also the outlet for the metrics. Ceilometer would also be an >outlet that could be leveraged via RPC flow for metrics -- though I'd >expect some would want to send via datagram to statsd or file for offline >log aggregation. > >I'll post a diagram tomorrow for review and comment. > >Oh btw, I updated the spec with most of what was in the etherpad. We can >update the spec to reflect whatever we decide is the preferred approach. > >-jeff > >On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: > >> On 25/10/12 11:13 +, Sandy Walsh wrote: >>> grizzly-common-instrumentation seems to be the best choice ... >>>hopefully the other groups will use this etherpad too. >>> >>> We need a proper blueprint to nail down the approach. IRC is great, >>>but doesn't retain history for other groups. I think we need to get a >>>plan for translating the etherpad into something concise and nailed >>>down. >> >> Agree. >> >>> >>> statgen should really just be a new notifier in Tach (or Scrutinize) >>>... vs copy-pasting the code into yet-another repo. Hopefully that's >>>the plan? Tach should remain a generic tool and not pegged to OpenStack. >> >> Well that was just an "ideas play pen" not serious code. >> >> I might be coming at this from a slightly different angle... >> I was looking at a library that can be used to generate trace, >>monitoring >> and metering data (kind of like log levels for logging). Currently both >> Tach and Scrutinize don't have enough fields (of course that could be >>changed). >> >> Also I think we should be able to insert instrumentation into the code >>as well >> as have the function level performance metrics monkey patched. >> >> Then we could have a config that directed metric data to different >>notifiers >> like how you do it in Scrutinize perhaps. Also enforcing data rate >>limits >> and possible data aggregation could be neat configurable features. >> >> Anyway more at the meeting... >> >> -Angus >> >>> >>> -S >>> >>> From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net >>>[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on >>>behalf of Angus Salkeld [asalk...@redhat.com] >>> Sent: Thursday, October 25, 2012 1:00 AM >>> To: openstack@lists.launchpad.net >>> Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, >>>CloudWatch integration ... Summit followup >>> >>> On 24/10/12 23:35 +, Sandy Walsh wrote: >>>> Hey y'all, >>>> >>>> Great to chat during the summit last week, but it's been a crazy few >>>>days of catch-up since then. >>>> >>>> The main takeaway for me was the urgent need to get some common >>>>libraries under these efforts. >>> >>> Yip. >>> >>>> >>>> So, to that end ... >>>> >>>> 1. T
Re: [Openstack] [ceilometer] Monitoring physical devices
On Nov 6, 2012, at 6:45 AM, Tim Bell wrote: > > There is a use case for base metal hardware metering in the private cloud > where the user allocated the machine does not have root access to kill the > metering. How can we detect that special case? > > Being able to create a single metering infrastructure for the entire private > cloud, virtual or bare-metal allocation, is a need technically, it is > not clear how to guarantee it but it is worth exploring. I agree, it would be good to have an answer. Ceilometer can already hold the data, even if the agent to collect it is a custom solution. Doug > > Tim > >> -Original Message- >> From: openstack-bounces+tim.bell=cern...@lists.launchpad.net >> [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net] On Behalf >> Of Robert Collins >> Sent: 06 November 2012 11:00 >> To: Graf Lucas (graflu0); Zehnder Toni (zehndton); >> openstack@lists.launchpad.net; Doug Hellmann >> Subject: Re: [Openstack] [ceilometer] Monitoring physical devices >> >> On Tue, Nov 6, 2012 at 10:53 PM, Julien Danjou wrote: >>> On Tue, Nov 06 2012, Graf Lucas (graflu0) wrote: >>> >>>> I'm a little confused now... ;) Is the bare metal run by the platform >>>> operator the physical machine? What do you mean with the bare metal >>>> run to replace virtual instances for any project? >>> >>> AFAIU, bare-metal provisionning is about using hardware (bare-metal) >>> rather than virtual instances as a flavor in Nova. >>> For such case, we won't be able to poll anything about the hardware. >>> >>> For hardware ran by the operator, this will be doable. >> >> We can still talk to the IPMI controller for the machine, which will get > us lots >> of info, without running agents in the host os. >> >> -Rob >> >> >> >> -- >> Robert Collins >> Distinguished Technologist >> HP Cloud Services >> >> ___ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Puppet modules for OpenStack?
Hi Dennis, The modules mentioned above are in a great maintained state (just one known issue related to nova_config that should be merged soon), and I hope to see additional modules added for quantum and ceilometer added in the near future. I've heard reports that people have been happy with the ceilometer module from enovance: https://github.com/enovance/puppet-ceilometer I expect that whatever ends up on stackforge will be based on this. As for quantum, the puppetlabs, bodepd, enovance, and ciscosystems module (from github) are all loosely based on each other. I rencently dug through the various branches of each and decided to base my work on the folsom branch from ciscosystems. My work from this branch can be found here: https://github.com/bodepd/puppet-quantum Unless people protest, I expect this will eventually become the quantum module. On Sat, Apr 13, 2013 at 5:29 AM, Dennis Jacobfeuerborn < denni...@conversis.de> wrote: > Hi, > i'm currently looking into deploying my first "real" (as in "not local > devstack") OpenStack infrastructure and I intend to use Puppet to automate > things. > My questions is where can I find good modules for this purpose? I found > something on puppetforge but there doesn't seem to be any quantum support > and there are also modules available at the CiscoSystems github but these > modules haven't been touched in several months. > > Is there a set of modules out there that have gained a certain momentum > within the community so that they could be called inofficially endorsed by > the (puppet using) openstack community? > > Regards, > Dennis > > __**_ > Mailing list: > https://launchpad.net/~**openstack<https://launchpad.net/~openstack> > Post to : openstack@lists.launchpad.net > Unsubscribe : > https://launchpad.net/~**openstack<https://launchpad.net/~openstack> > More help : > https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp> > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
D oug, Thanks. I tried it in grizzly, here's the return: sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit Starting volume usage audit Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00 Found 0 volumes Volume usage audit completed Actually, I want to get some data like this: Total Cinder Storage on Physical Machine: 100G Used Cinder Storage on Physical Machine: 10G Is there any way to get this? Best Regards -- Ray On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann wrote: > We rely on a similar audit program to get the "exists" notifications about > cinder volumes. Look for "cinder-volume-usage-audit". > > Doug > > > On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun wrote: > >> Yes, it should be, but seems not at least in grizzly. Any update of >> Ceilometer? >> >> Best Regards >> -- Ray >> >> >> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang wrote: >> >>> I think Statistics should be find in Ceilometer. Ceilometer may provide >>> with >>> enough information you need. >>> >>> Best regards, >>> Haomai Wang, UnitedStack Inc. >>> >>> 在 2013-7-14，上午8:09，Ray Sun 写道： >>> >>> In nova, we have a period task to report the usage of the physical >>> server, including CPU, Memory and Local Disk, but I don't think I can find >>> the same strategy in cinder service. Is there any way to do this or is >>> there any blueprint for this? >>> >>> Thanks. >>> >>> Best Regards >>> -- Ray >>> ___ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >>> >>> >>> >> >> ___ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> >> > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] First meeting for the Metering project
On 04/27/2012 05:37 PM, Francis J. Lacoste wrote: > On 12-04-27 10:50 AM, Nick Barcet wrote: >> The meeting occurred as planned, and overran... Nevertheless a lot of >> good decisions were made, the obviously most important one being >> the project name: ceilometer. >> >> For a more complete meeting summary, go visit: >> http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary >> >> Next week we will have for objective to find an agreement on "schema and >> counter definitions". If anyone does not do it before me I'll fire an >> introduction email on the subject soon (sometime this we), as we agreed >> to start the discussion via mailing list and use the irc meeting to >> finalize the choices. > The Launchpad project was created: > > https://launchpad.net/ceilometer Thank you :-) I'm not familiar with the launchpad ways. Should I wait for the git repository to be created ? Or can I help with that ? Chers ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Changes to the ceilometer Jenkins job
On Wed, May 30, 2012 at 8:42 AM, Andrew Hutchings wrote: > Hi Doug, > > On 30/05/12 13:14, Doug Hellmann wrote: > > I started working on setting one up yesterday, and plan to continue > > working on it later today. I would appreciate any feedback you could > > give me on: > > > > > https://github.com/dhellmann/ceilometer/commit/d2855c656c5442e46a4a891ff91d6be0abff9706 > > I'm not an expert but it looks good to me. Best thing to do is to > propose a change via. gerrit. This will run a check test with the new > tox.ini. You can then amend the commit and review again to upload > further patchsets if you find some things need changing. > > Once this is in your other commits can simply be rebased to use it. > It looks like we've got it worked out. Thanks, Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Bandwidth limit
On 07/10/2012 03:55 AM, Guillermo Alvarado wrote: > Hi Everyone! > > I want to know if there is a table or something that contains > information about the bandwidth of each tenant, to billing purposes. This is not complete yet, but is part of the data that we are trying to collect [1] in the Ceilometer project [2]. We now have a nice guide explaining how to contribute additional plugins [3], help is alsways welcome. [1] http://wiki.openstack.org/EfficientMetering#Meters [2] https://launchpad.net/ceilometer [3] http://wiki.openstack.org/EfficientMetering#Contributing_to_Ceilometer > There are a way to limit the bandwith of the tenants?? This would be a request for Quantum, which I don't think has been fully addressed yet, based on this thread [4]. [4] http://www.mail-archive.com/openstack@lists.launchpad.net/msg13611.html Hope this helps, -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)
PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC The metering project team will hold its next meeting at alternate time on *Wednesday* at 9PM UTC <http://www.timeanddate.com/worldclock/fixedtime.html?hour=21&min=0&sec=0>. Everyone is welcome. Agenda: http://wiki.openstack.org/Meetings/MeteringAgenda * Review last week's actions * nijaba to see with ttx how our sessions can be shown on official program * nijaba to update meeting page to note new alternating meeting time * gmb to a plan on the ml with fixed dates * dhellmann make sure flask is listed as a dependency of ceilometer * Open discussion If you are not able to attend or have additional topic you would like to cover, please update the agenda on the wiki. Cheers, -- Nick Barcet aka: nijaba, nicolas signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] billing for openstack
interesting.. still learning how to implement best billing for openstack. in private this is good to make "revenue center" for IT division F On Tue, Oct 9, 2012 at 11:05 PM, Nick Barcet wrote: > On 10/09/2012 05:33 PM, Frans Thamura wrote: >> hi all >> >> my member asks related to billing for openstack, so we can bill anyone >> use openstack's services >> >> are one of this? WHMCS, Clientexec, Blesta, Hostbill >> >> or anyone can help? > > Hello Frans, > > We are just about to release a first version of Ceilometer[1][2] which > provides a central point for a billing implementation to fetch its usage > data from. > > Not sure if that fully answers your question though... > > [1]http://launchpad.net/ceilometer > [2]http://ceilometer.readthedocs.org > > Best, > -- > Nick Barcet > aka: nijaba, nicolas > > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Ceilometer API Glossary
> I am testing ceilometer in my devstack virtual machine. Although I > can see the meter data model in the mongodb, I am confused about > some glossary when I test its Web API. I am not very clear about the > " resource" in the " GET /v1/resources",either "source" in the "GET > /v1/sources/(source)/meters/(meter)". > Can someone explain these two concept? Thanks a lot. Hi Yawei, Good question! My understanding is that the resources collection refers to the set of UUIDs identifying the openstack entities (e.g. instance, volume, image ... etc.) being metered, whereas the source refers to the origin of the metering data and is effectively unused right now as there is only a single source currently (though we could in the future for example distinguish between metric and metering data in this way). Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
> If your pollster is not running to compute delta and you have > no state stored, you'll miss a part of what has been used. Would we have also have some 'misses' with the cumulative approach when the ceilometer agent was down? If I understood the (\Sigma local maxima)-first idea correctly, the usage up to the first polling cycle would always be discounted from any duration. Similarly, calculating the time delta as a gauge measure would discount only the usage up to the first libvirt poll after each ceilo agent restart. As long as the ceilo compute agent was restarted only rarely, I'm not sure the under-reporting would be a huge issue in either case. A more pernicious problem would occur if the instance was being regularly restarted with a higher frequency than the polling period. Cheers, Eoghan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Commong Page for IRC meeting schedules
http://wiki.openstack.org/Meetings lists all the scheduled meetings. Thanks, Kiall On Wed, Oct 31, 2012 at 9:09 PM, Sriram Subramanian wrote: > Is there a central location where project meeting schedules are tracked. > Even if not, could you please help build one by adding to this threa or at > http://wiki.openstack.org/IRC > > *#openstack* (general discussion, support) *:* > > *#openstack-cinder* (cinder team discussions) *:* > > *#openstack-swift* (swift team discussions) *:* > > *#openstack-nova* (nova team discussions) *:* > > *#openstack-dev*(development discussion) *:* > > *#openstack-infra *(infrastructure team discussion, ci, and other > infrastructure) *:* > > *#openstack-meeting* (team meetings) *:* > > *#openstack-metering* (ceilometer team discussions) *:* > > *#openstack-packaging* (packaging discussions) *:* > > *#openstack-chef* (deployment and operating > OpenStack<http://wiki.openstack.org/OpenStack>with Chef) > *:* > *Hyper-V :* > ** > *CeiloMeter :* > ** > *Heat :* > ** > I will update the page appropriately. > ** > Thanks, > -Sriram > ** > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Havana-1 development milestone available
Hi everyone, The first milestone of the Havana development cycle, "havana-1" is now available for Keystone, Glance, Nova, Horizon, Networking, Cinder, Ceilometer, and Heat. It contains all the new features that have been added since the Grizzly pre-release Feature Freeze in March. You can see the full list of new features and fixed bugs, as well as tarball downloads, at: https://launchpad.net/keystone/havana/havana-1 https://launchpad.net/glance/havana/havana-1 https://launchpad.net/nova/havana/havana-1 https://launchpad.net/horizon/havana/havana-1 https://launchpad.net/quantum/havana/havana-1 https://launchpad.net/cinder/havana/havana-1 https://launchpad.net/ceilometer/havana/havana-1 https://launchpad.net/heat/havana/havana-1 Including the oslo libraries, 63 blueprints were implemented and 671 bugs were fixed during this milestone. The next development milestone, havana-2, is scheduled for July 18th. Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Ceilometer getting a "Connection refused" from
I have a controller node and a compute node on two different hosts. When I query ceilometer from the compute node, I get a "Connection refused" errror from port no 27017 of controller node, which I suppose is the port no where mongo listens. I haven't installed mongodb in the compute node, though the error points to the controller node's mongodb port. This is the query is posted: curl -X GET -H 'X-Auth-Token:' " http://127.0.0.1:8777/v2/meters/instance?q.field=metadata.event_type&q.value=compute.instance.delete.start " The result I get: The output <http://pastebin.ubuntu.com/5835477/> (Since the output was too long, I considered not pasting it here, apologies for inconvenience). What is the reason behind this and how should I correct this? Please feel free to ask for clarification. -- Thanks and regards, Jobin Raju George Third Year, Information Technology College of Engineering Pune Alternate e-mail: georgejr10...@coep.ac.in ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Havana-2 development milestone available
Hi everyone, The second milestone of the Havana development cycle, "havana-2" is now available for Keystone, Glance, Nova, Horizon, Neutron, Cinder, Ceilometer, and Heat. In the last 7 weeks, more than 100 features were added and more than 650 bugs fixed. You can see the full list of new features and fixed bugs, as well as tarball downloads, at: https://launchpad.net/keystone/havana/havana-2 https://launchpad.net/glance/havana/havana-2 https://launchpad.net/nova/havana/havana-2 https://launchpad.net/horizon/havana/havana-2 https://launchpad.net/neutron/havana/havana-2 https://launchpad.net/cinder/havana/havana-2 https://launchpad.net/ceilometer/havana/havana-2 https://launchpad.net/heat/havana/havana-2 The next (and last) development milestone of the Havana cycle, havana-3, is scheduled for September 6th (final release is planned October 17th). Regards, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup
Yes, I think support for metrics objects that can be leveraged both by monkey patches and decorators was what we'd been thinking along the lines of. The metrics would be controlled via config both in what scopes are active (e.g. on|off for a package, module, etc.) and also the outlet for the metrics (file, datagram, rpc). Support for instrumentation levels would also be nice so that metric flow could be controlled (i.e. instrumentation point is annotated as METRIC, MONITOR, PROFILE and then level to actually emit can be set in conjunction with a metric outlet/sink). With this approach, folks could control both the volume of metrics and also the outlet for the metrics. Ceilometer would also be an outlet that could be leveraged via RPC flow for metrics -- though I'd expect some would want to send via datagram to statsd or file for offline log aggregation. I'll post a diagram tomorrow for review and comment. Oh btw, I updated the spec with most of what was in the etherpad. We can update the spec to reflect whatever we decide is the preferred approach. -jeff On Oct 25, 2012, at 5:30 PM, Angus Salkeld wrote: > On 25/10/12 11:13 +, Sandy Walsh wrote: >> grizzly-common-instrumentation seems to be the best choice ... hopefully the >> other groups will use this etherpad too. >> >> We need a proper blueprint to nail down the approach. IRC is great, but >> doesn't retain history for other groups. I think we need to get a plan for >> translating the etherpad into something concise and nailed down. > > Agree. > >> >> statgen should really just be a new notifier in Tach (or Scrutinize) ... vs >> copy-pasting the code into yet-another repo. Hopefully that's the plan? >> Tach should remain a generic tool and not pegged to OpenStack. > > Well that was just an "ideas play pen" not serious code. > > I might be coming at this from a slightly different angle... > I was looking at a library that can be used to generate trace, monitoring > and metering data (kind of like log levels for logging). Currently both > Tach and Scrutinize don't have enough fields (of course that could be > changed). > > Also I think we should be able to insert instrumentation into the code as well > as have the function level performance metrics monkey patched. > > Then we could have a config that directed metric data to different notifiers > like how you do it in Scrutinize perhaps. Also enforcing data rate limits > and possible data aggregation could be neat configurable features. > > Anyway more at the meeting... > > -Angus > >> >> -S >> >> From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net >> [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf >> of Angus Salkeld [asalk...@redhat.com] >> Sent: Thursday, October 25, 2012 1:00 AM >> To: openstack@lists.launchpad.net >> Subject: Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, >> CloudWatch integration ... Summit followup >> >> On 24/10/12 23:35 +, Sandy Walsh wrote: >>> Hey y'all, >>> >>> Great to chat during the summit last week, but it's been a crazy few days >>> of catch-up since then. >>> >>> The main takeaway for me was the urgent need to get some common libraries >>> under these efforts. >> >> Yip. >> >>> >>> So, to that end ... >>> >>> 1. To those that asked, I'm going to get my slides / video presentation >>> made available via the list. Stay tuned. >>> >>> 2. I'm having a hard time following all the links to various efforts going >>> on (seems every time I turn around there's a new metric/instrumentation >>> effort, which is good I guess :) >> >> Here is some fun I have been having with a bit of tach+ceilometer code. >> https://github.com/asalkeld/statgen >> >>> >>> Is there a single location I can place my feedback? If not, should we >>> create one? I've got lots of suggestions/ideas and would hate to have to >>> duplicate the threads or leave other groups out. >> >> I'll add some links here that I am aware of: >> https://bugs.launchpad.net/ceilometer/+bug/1071061 >> https://etherpad.openstack.org/grizzly-common-instrumentation >> https://etherpad.openstack.org/grizzly-ceilometer-actions >> https://blueprints.launchpad.net/nova/+spec/nova-instrumentation-metrics-monitoring >> >> >>> >>> 3. I'm wrapping up the packaging / cleanup of StackTach v2 with Stacky and >&
Re: [Openstack] [Metering] Adding a source notion to the schema
On Fri, May 4, 2012 at 12:29 PM, Nick Barcet wrote: > Following up on the discussion on IRC yesterday during the metering > meeting, I'd like to explain my proposal to add the notion of source to the > schema of our event. The current goal we have for ceilometer is to provide > a common way to accumulate counters/meters from various openstack component > in a central repository that can be then used by other tools to produce > bills in fine. One thing we can currently assume in Essex is that all > components share a common repository for identity management: Keystone. > This is the assumption we made when defining the existing schema. However, > I think we need to plan a little further ahead here, and allow for this not > to necessarily remain the same in some complex deployment cases. > > For example, it could be envisioned that some software, outside of the > OpenStack project, maybe deployed and used on top of an OpenStack > deployment. This software may need to be billed to customers, and > collecting meters for it maybe as important for the provider than doing it > for OpenStack components. It cannot be assumed that the identity management > used by the software will be keystone. The software may not provide a > metering interface built in, and it would make sense for the provider to be > able to implement this using existing tool present in the underlying > OpenStack deployment: ceilometer. > > In order to allow for this I think it would make sense to: > * extend the current schema to allow for an additional "source" field in > the event record. This should be a short identifier. > That makes sense. It seems like the identifier(s) used are meant to be defined by the deployer. Is that your intent? > * add another record definition that maps source to identity managment URL > location > * Collecting agent would be in charge to specify the source they map to > (keystone by default). > It is not clear why the identity management location needs to be recorded in ceilometer. If the deployer controls the source strings, they know what each value means. The only thing that needs to translate the (source, project, user) values to a billing identity is the end-consumer of all of this data, which is also under the control of the deployer. Couldn't the mapping of the source token to the identity location be handled in that layer? > > I think this would allow for easy extension of ceilometer to support other > software, but I'd like to know if you share the usefulness of this > extension from your experiences. > > Thanks, > > Nick > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] adding a "worker pool" notion for RPC consumers
ceilometer is going to need to subscribe several worker processes to the notifications.info topic for the other services like nova, glance, and quantum. The pool of workers needs to be assured of receiving all messages, without interference from other clients listening for notifications (such as metering, audit logging, monitoring, etc.). The drivers implement create_consumer() for topic consumers by always making the topic and queue name the same. That supports the consumer patterns we have encountered so far, but does not allow for load balancing between workers as we want for the ceilometer collector. The fanout argument is used by the existing drivers to control the breadth of distribution. With fanout=True, every consumer receives a copy of the event because every consumer has its own queue. This gives us no load balancing benefit for having multiple consumers. With fanout=False only one consumer *at all* will get a copy of the event, since they all listen on the same queue. This means if metering, audit logging, and monitoring are all listening for notifications they will each see only some of the events. The way to achieve load balancing is to have the ceilometer consumers connect to the same exchange and topic using a shared, well-known, queue name that is different from the name used by non-ceilometer consumers. Unfortunately, the only parameter that cannot be controlled by the caller of create_consumer() is the queue name. I have a patch ready for review [1] to add a new method, create_worker() to the RPC Connection class, to allow a group of consumer to share a queue to manage load. The worker pool allows multiple consumers to receive a given message (by subscribing to separate queues), but it also allows several consumers to declare that they are collaborating so that only one of the subset receives a copy (by subscribing to the same queue). That means that multiple "types" of consumers can be listening to notifications and each type of consumer can have a load balanced pool of workers so that messages are only processed once for that type (once for metering and once for logging, for example). Two separate implementations were discussed: Adding a "queue_name" argument to create_consumer() and creating a new method "create_worker()". After considering both options, I chose to add a new method because it clarified the right way to combine the inputs to set up workers (fanout must always be true and the queue name must always be provided). The code is up for review, so please have a look and let me know what you think. Doug [1] https://review.openstack.org/#/c/7590/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Billing in Openstack
On 08/30/2012 03:40 AM, Emilien Macchi wrote: > Hi, > > You should read Ceilometer Project [1] and try it into DevStack directly > in editing localrc before to run devstack and add : > > enable_service ceilometer-acompute,ceilometer-acentral, ceilometer-collector > > Enjoy ! > > > [1] http://wiki.openstack.org/EfficientMetering If fully second Emilien's suggestions. As a quick note, I'd like to level set your expectation as Ceilometer is metering tool, not a full billing tool. A full billing solution needs 3 components: - Metering: to know what has happened on the cloud - Rating: which transforms what has happened into price to bill (line items) - Billing: which accumulates line items and create a bill to send to customer and collect money Which means that you will still need component 2 and 3 to do billing after you have deployed Ceilometer. Hope this helps. Nick > On Thu, Aug 30, 2012 at 12:21 PM, Rajesh Avula <mailto:avula.rajes...@gmail.com>> wrote: > > Greetings...!! > > I would like to generate billing of used resources (CPU, RAM, Disk) > in openstack. Below are the details of my setup (Please ask me for > more details if require regarding setup) > > Mainly I have used 2 system to make my setup, below are the details... > > 1. *On First Laptop *(Dell - Latitude, with 4 GB RAM, Intel Core i5 > vpro Processor, 500 GB Hard disk) I have installed a *virtual > machine with CentOS (6.2)* on *Xenserver (6.0.2)* and on that > *virtual machine I installed openstack* by following below links and > some other links from google > > https://fedoraproject.org/wiki/Getting_started_with_OpenStack_Nova > http://wiki.openstack.org/XenServer > > 2. *On Second Laptop* (same as above configuration) I have installed > *openfiler and made NFS and iSCSI targets* for glance, swift, and > for instances. > > 3. For network I am using BELKIN wireless adapter 5 port L2-switch) > > I am able to launch Instances from AMI's, instances are getting IP > addresses, able to create volumes and attaching to the instances. > > All I need is, Is there any possibility in openstack where I can > define the values / prices for the resources so that users will get > the bills accordingly ? > > Below are the packages installed in my setup for dashboard > python-django-horizon-2012.1.1-1.el6.noarch > openstack-dashboard-2012.1.1-1.el6.noarch > > Is there any other packages should be installed to get billing > option on dashboard? > Any specific configuration required ? > > Please let me know how to get billing information of the used / > using resources. > > > -- > Thanks & Regards, > Rajesh Avula > 09831138115 > 07259024701. > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > <mailto:openstack@lists.launchpad.net> > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > > > > -- > Emilien Macchi > *System Engineer* > **www.stackops.com* <http://www.stackops.com> > > |*emilien.mac...@stackops.com <mailto:emilien.mac...@stackops.com> > *|*skype:emilien.macchi* > * <http://www.stackops.com> > * > > > > * > > ADVERTENCIA LEGAL > Le informamos, como destinatario de este mensaje, que el correo > electrónico y las comunicaciones por medio de Internet no permiten > asegurar ni garantizar la confidencialidad de los mensajes transmitidos, > así como tampoco su integridad o su correcta recepción, por lo que > STACKOPS TECHNOLOGIES S.L. no asume responsabilidad alguna por tales > circunstancias. Si no consintiese en la utilización del correo > electrónico o de las comunicaciones vía Internet le rogamos nos lo > comunique y ponga en nuestro conocimiento de manera inmediata. Este > mensaje va dirigido, de manera exclusiva, a su destinatario y contiene > información confidencial y sujeta al secreto profesional, cuya > divulgación no está permitida por la ley. En caso de haber recibido este > mensaje por error, le rogamos que, de forma inmediata, nos lo comunique > mediante correo electrónico remitido a nuestra atención y proceda a su > eliminación, así como a la de cualquier documento adjunto al mismo. > Asimismo, le comunicamos que la distribución, copia o utilización de > este mensaje, o de cualquier documento adjunto al mismo, cualquiera que > fuera su finalidad, están prohibidas por la ley. > >
Re: [Openstack] grizzly swift keystone, http to 8080/8888 wont work
Thanks for your quick reply, Simon,
The role ResellerAdmin does exists and looks good, does it?
root@ns-proxy01:/etc/swift# keystone user-get ceilometer
+--+--+
| Property | Value |
+--+--+
| email | |
| enabled | True |
|id| cde44fe9c6d446da99ea370b88ec7d63 |
| name |ceilometer|
| tenantId | 054ca85bca2e44c29cf4730e1450517f |
+--+--+
root@ns-proxy01:/etc/swift# keystone user-role-list --user-id
cde44fe9c6d446da99ea370b88ec7d63 --tenant-id
054ca85bca2e44c29cf4730e1450517f
+--+---+--+--+
|id| name | user_id
|tenant_id |
+--+---+--+--+
| c2df2bc0fd6f404794565f10cc0e5e7a | ResellerAdmin |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
| 9fe2ff9ee4384b1894a90878d3e92bab |_member_ |
cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
+--+---+--+--+
And i can see ceilometer log entrys, counting bytes. So that looks good.
My issue it, that with the old swauth setup there was a real simple web
based user manager.
surfing to "http://my.swift.proxy:/auth/"; was the entry url to this
sort of user manager. But now, after the change to keystone, i get http
result codes like 412 or 401.
Since i inherit this setup i even do not know for sure if this
swift-user-manager it actually a part of swift. i believe so.
Can please one confirm which urls do work on swift-proxy http port
8080/ (proxy-server.conf -> [DEFAULT] -> bind_port). Should "/auth/"
return a page?
Thank you. Axel
Am 16.04.13 12:41, schrieb Simon Pasquier:
> Hi,
> I'm not sure to understand exactly your issue but since your setup
> includes ceilometer, I can just give you a hint for the ceilometer/swift
> integration.
> You have to create a 'ResellerAdmin' role and assign that role to your
> ceilometer user. Alternatively you can define the 'reseller_admin_role'
> parameter (default value=ResellerAdmin) in the [filter:authtoken]
> section of /etc/swift/proxy-server.conf.
> Cheers,
> Simon
>
> Le 16/04/2013 12:04, Axel Christiansen a écrit :
>> Dear List,
>>
>>
>> i got stuck with a setup of openstack grizzly. This setup consists of:
>>
>> - swift proxy 1.0.8.1
>> - swift storage nodes 1.0.8.1
>> - keystone
>> - ceilometer
>>
>>
>> I kept browsing the web and reading openstack docs for days now and
>> can't just get it working right. Because of openstacks diversity a
>> wasn't able to find something really similar to my situation.
>>
>>
>> The thing is, i changed swift-proxy from using swauth to keystone.
>> Keystone and swift-proxy do interact all right as fare as i can say.
>> What i can't get working is that simple webpage which gave the ability
>> to log in as superuser, adding new user and so on. It is that webpart
>> that connects to the proxy on port 8080, respectively port .
>>
>>
>> Thx o lot for taking a look into this.
>> Axel
>>
>>
>>
>>
>> Theses are the browser urls i try:
>>
>> (delay_auth_decision = 1)
>> http://the.swift.proxy:/auth/
>> bad url
>> Apr 16 11:49:31 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
>> txcfde073b9ffe4f379da392056e2176de)
>> Apr 16 11:49:31 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
>> 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
>> deflate', 'Host': 'backend', 'Accept':
>> 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
>> 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0)
>> Gecko/20100101 Firefox/20.0', 'Connection': 'close', 'Content-Type':
>> None}, 'environ': {'SCRIPT_NAME': '', 'REQUEST_METHOD': 'GET',
>> 'PATH_INFO': '/auth/', 'SERVER_PROTOCOL': 'HTTP/1.0', 'HTTP_USER_AGENT':
>> 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101
>> Firefox/20.0', 'HTTP_CONNE
Re: [Openstack] grizzly swift keystone, http to 8080/8888 wont work
The mystery seems solved. There it a webadmin for swauth.
https://github.com/gholt/swauth#web-admin-install
Does there exists is similar thing for keystone?
Regards, Axel
Am 16.04.13 14:53, schrieb Axel Christiansen:
>
>
> Thanks for your quick reply, Simon,
>
>
> The role ResellerAdmin does exists and looks good, does it?
>
> root@ns-proxy01:/etc/swift# keystone user-get ceilometer
> +--+--+
> | Property | Value |
> +--+--+
> | email | |
> | enabled | True |
> |id| cde44fe9c6d446da99ea370b88ec7d63 |
> | name |ceilometer|
> | tenantId | 054ca85bca2e44c29cf4730e1450517f |
> +--+--+
> root@ns-proxy01:/etc/swift# keystone user-role-list --user-id
> cde44fe9c6d446da99ea370b88ec7d63 --tenant-id
> 054ca85bca2e44c29cf4730e1450517f
> +--+---+--+--+
> |id| name | user_id
> |tenant_id |
> +--+---+--+--+
> | c2df2bc0fd6f404794565f10cc0e5e7a | ResellerAdmin |
> cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
> | 9fe2ff9ee4384b1894a90878d3e92bab |_member_ |
> cde44fe9c6d446da99ea370b88ec7d63 | 054ca85bca2e44c29cf4730e1450517f |
> +--+---+------+--+
>
> And i can see ceilometer log entrys, counting bytes. So that looks good.
>
>
>
>
> My issue it, that with the old swauth setup there was a real simple web
> based user manager.
>
> surfing to "http://my.swift.proxy:/auth/"; was the entry url to this
> sort of user manager. But now, after the change to keystone, i get http
> result codes like 412 or 401.
>
>
> Since i inherit this setup i even do not know for sure if this
> swift-user-manager it actually a part of swift. i believe so.
>
>
> Can please one confirm which urls do work on swift-proxy http port
> 8080/ (proxy-server.conf -> [DEFAULT] -> bind_port). Should "/auth/"
> return a page?
>
>
> Thank you. Axel
>
>
>
>
> Am 16.04.13 12:41, schrieb Simon Pasquier:
>> Hi,
>> I'm not sure to understand exactly your issue but since your setup
>> includes ceilometer, I can just give you a hint for the ceilometer/swift
>> integration.
>> You have to create a 'ResellerAdmin' role and assign that role to your
>> ceilometer user. Alternatively you can define the 'reseller_admin_role'
>> parameter (default value=ResellerAdmin) in the [filter:authtoken]
>> section of /etc/swift/proxy-server.conf.
>> Cheers,
>> Simon
>>
>> Le 16/04/2013 12:04, Axel Christiansen a écrit :
>>> Dear List,
>>>
>>>
>>> i got stuck with a setup of openstack grizzly. This setup consists of:
>>>
>>> - swift proxy 1.0.8.1
>>> - swift storage nodes 1.0.8.1
>>> - keystone
>>> - ceilometer
>>>
>>>
>>> I kept browsing the web and reading openstack docs for days now and
>>> can't just get it working right. Because of openstacks diversity a
>>> wasn't able to find something really similar to my situation.
>>>
>>>
>>> The thing is, i changed swift-proxy from using swauth to keystone.
>>> Keystone and swift-proxy do interact all right as fare as i can say.
>>> What i can't get working is that simple webpage which gave the ability
>>> to log in as superuser, adding new user and so on. It is that webpart
>>> that connects to the proxy on port 8080, respectively port .
>>>
>>>
>>> Thx o lot for taking a look into this.
>>> Axel
>>>
>>>
>>>
>>>
>>> Theses are the browser urls i try:
>>>
>>> (delay_auth_decision = 1)
>>> http://the.swift.proxy:/auth/
>>> bad url
>>> Apr 16 11:49:31 ns-proxy01 swift-proxy Calling Swift3 Middleware (txn:
>>> txcfde073b9ffe4f379da392056e2176de)
>>> Apr 16 11:49:31 ns-proxy01 swift-proxy {'headers': {'Accept-Language':
>>> 'de-de,de;q=0.8,en-us;q=0.5,en;q=0.3', 'Accept-Encoding': 'gzip,
>>> deflate', 'Host&
Re: [Openstack] [metering] Changes to the ceilometer Jenkins job
Hi Julien, On 29/05/12 16:46, Julien Danjou wrote: > We noticed that currently, on Stackforge, the ceilometer project has > only one Jenkins job, checking that the merge occurs correctly. > > We'd like to add jobs to run unit tests, pep8, etc… I took a look at the > openstack-ci-puppet repo, but I'm not sure of the changes, so I prefer > to ask here for someone that understand this to help. > From what I see, it's likely we would like to use the default > "python_jobs" template. > > If anything is missing on our side to use the standard set of checks, > we'll do what is necessary to be able to use them. :) Within an hour of this landing it will be done automatically for you: https://review.openstack.org/7884 We can then see if there are any jobs not working right for you. Please let me know if there are any problems. Kind Regards -- Andrew Hutchings - LinuxJedi - http://www.linuxjedi.co.uk/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Changes to the ceilometer Jenkins job
On Tue, May 29, 2012 at 5:45 PM, Andrew Hutchings wrote: > Hi Julien, > > On 29/05/12 19:12, Andrew Hutchings wrote: > >> If anything is missing on our side to use the standard set of checks, > >> we'll do what is necessary to be able to use them. :) > > > We can then see if there are any jobs not working right for you. Please > > let me know if there are any problems. > > All done on the Jenkins side. The one thing you are currently missing > is a tox.ini file. I can help you create this tomorrow or if you want > to do it in the mean time look at the examples in other projects. > I started working on setting one up yesterday, and plan to continue working on it later today. I would appreciate any feedback you could give me on: https://github.com/dhellmann/ceilometer/commit/d2855c656c5442e46a4a891ff91d6be0abff9706 Thanks for all your help, Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Changes to the ceilometer Jenkins job
On Tue, May 29 2012, Julien Danjou wrote: > We noticed that currently, on Stackforge, the ceilometer project has > only one Jenkins job, checking that the merge occurs correctly. > > We'd like to add jobs to run unit tests, pep8, etc… I took a look at the > openstack-ci-puppet repo, but I'm not sure of the changes, so I prefer > to ask here for someone that understand this to help. > From what I see, it's likely we would like to use the default > "python_jobs" template. Is there a way to add more checks without touching the openstack-ci-puppet repository? I'd like to run tests with tox on the same py26/27 envs but using a different set of pip-requires. I found out how to write the tox.ini part, but I am not sure how the envlist is controlled on the Jenkins side. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
On 25/10/12 10:16 +0200, Julien Danjou wrote: On Thu, Oct 25 2012, Angus Salkeld wrote: If you do auto scaling you will have a similar problem. Here you want to monitor the group (with instances comming and going) as a logical unit. One way would be to tag the instances and then extract the tag and send it with the metadata associated with the meter. Then you could query the ceilometer db for that group. What about sending meters where the resource is the group and not the instances? (What do you meter exactly on such a group? I'm not really familiar with CW yet) So we need normal stuff like cpu/mem usage but aggregated over the instances in the group. So this is not easy to do externally. -Angus -- Julien Danjou /* Free Software hacker & freelance http://julien.danjou.info */ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] not getting cinder volume meters/resources from ceilometer
I have created the openstack setup with devstack. I have created the volumes and instances. When I am listing resources using http://10.102.153.37:8777/v2/resources then I am getting only instnaces not volumes. Similarly I am not getting any meters. I tried setting notification_driver=cinder.openstack.common.notifier.rabbit_notifier notification_driver=cinder.openstack.common.notifier.rpc_notifier in cinder.conf and restarting cinder-volume service and ceilometer-collector service. Still I am not getting any volume related meters. Neither they are mentioned in resources. In c-vol screen logs I can see below kind of logs when creating and deleting volumes. f4-8ba8-8cdac08c7b21', 'project_id': u'39587e1865d14ea991b4ddf82c1dc1ab', 'read_deleted': u'no', 'tenant': u'39587e1865d14ea991b4ddf82c1dc1ab'} 2013-05-31 16:54:24 INFO [cinder.volume.manager] volume volume-147adb04-be41-4949-9fbf-52c871593de4: deleting 2013-05-31 16:54:24DEBUG [cinder.openstack.common.rpc.amqp] Sending volume.delete.start on notifications.info 2013-05-31 16:54:24DEBUG [cinder.openstack.common.rpc.amqp] UNIQUE_ID is fd2593f309ca45579955afa7eb20d6ca. 2013-05-31 16:54:24 INFO [cinder.volume.manager] Clear capabilities Thanks, Anshul___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] ceilometer client
Hi Stackers
I've installed the requirements from requirements.txt with pip, and then
tried to install the ceilometer client, and I get this error:
root@control:~/python-ceilometerclient# python setup.py install
running install
Traceback (most recent call last):
File "setup.py", line 21, in
d2to1=True)
File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 310,
in run
_pip_install(links, self.distribution.install_requires, self.root)
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 95,
in _pip_install
" ".join(_wrap_in_quotes(_missing_requires(requires,
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py", line 83,
in _missing_requires
pkg_resources.Requirement.parse(r))]
File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2515, in parse
try:
ValueError: ('No requirements found', '# This file is managed by
openstack-depends')
Does anyone got something like this?
Thanks in advance
Claudio Marques
clau...@onesource.pt
http://www.onesource.pt/
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Fwd: Documentation for openstack-java-sdk
I think Luis can answer that? -- Videresendt melding -- Fra: "Jobin Raju George" Dato: 11. juli 2013 14:38 Emne: [Openstack] Documentation for openstack-java-sdk Til: "openstack lista" Kopi: I am trying to query ceilometer using openstack-java-sdk<https://github.com/woorea/openstack-java-sdk>for meters of VM's running on KVM. I am able to get the CPU meters via curl on the command line but unfortunately I don't find good documentation for the SDK's for ceilometer. I have seen this example program<https://github.com/woorea/openstack-java-sdk/blob/master/openstack-examples/src/main/java/com/woorea/openstack/examples/metering/v2/TestAll.java> but most of it is commented(probably because it is deprecated). Where can I find good documentation/examples or java programs/snippets? -- Thanks and regards, Jobin Raju George Third Year, Information Technology College of Engineering Pune Alternate e-mail: georgejr10...@coep.ac.in ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Havana-2 development milestone available
On Jul 18, 2013, at 12:05 PM, Thierry Carrez wrote: > Hi everyone, > > The second milestone of the Havana development cycle, "havana-2" is now > available for Keystone, Glance, Nova, Horizon, Neutron, Cinder, > Ceilometer, and Heat. In the last 7 weeks, more than 100 features were > added and more than 650 bugs fixed. > > You can see the full list of new features and fixed bugs, as well as > tarball downloads, at: > > https://launchpad.net/keystone/havana/havana-2 > https://launchpad.net/glance/havana/havana-2 > https://launchpad.net/nova/havana/havana-2 > https://launchpad.net/horizon/havana/havana-2 > https://launchpad.net/neutron/havana/havana-2 > https://launchpad.net/cinder/havana/havana-2 > https://launchpad.net/ceilometer/havana/havana-2 > https://launchpad.net/heat/havana/havana-2 > > The next (and last) development milestone of the Havana cycle, havana-3, > is scheduled for September 6th (final release is planned October 17th). And let us not forget our incubated program, Trove, and its milestone!! https://launchpad.net/trove/havana/havana-2 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] Adding a source notion to the schema
Following up on the discussion on IRC yesterday during the metering meeting, I'd like to explain my proposal to add the notion of source to the schema of our event. The current goal we have for ceilometer is to provide a common way to accumulate counters/meters from various openstack component in a central repository that can be then used by other tools to produce bills in fine. One thing we can currently assume in Essex is that all components share a common repository for identity management: Keystone. This is the assumption we made when defining the existing schema. However, I think we need to plan a little further ahead here, and allow for this not to necessarily remain the same in some complex deployment cases. For example, it could be envisioned that some software, outside of the OpenStack project, maybe deployed and used on top of an OpenStack deployment. This software may need to be billed to customers, and collecting meters for it maybe as important for the provider than doing it for OpenStack components. It cannot be assumed that the identity management used by the software will be keystone. The software may not provide a metering interface built in, and it would make sense for the provider to be able to implement this using existing tool present in the underlying OpenStack deployment: ceilometer. In order to allow for this I think it would make sense to: * extend the current schema to allow for an additional "source" field in the event record. This should be a short identifier. * add another record definition that maps source to identity managment URL location * Collecting agent would be in charge to specify the source they map to (keystone by default). I think this would allow for easy extension of ceilometer to support other software, but I'd like to know if you share the usefulness of this extension from your experiences. Thanks, Nick ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ceilometer (java implementation)
That's fantastic! What I mind is that the counters only should gather event info. Maybe multiple counters listening, collecting info about the system. But only one persist the event. Other process agreggates the data and creates different views/reports/billing, this should be outside of openstack (since it depends on the business rules, not depends on the infrastructure) The agreggator, also should offer an api for polling or can be configured throught drivers to push the the 3rd systems Now, in my implementation i'm putting all the stuff on a directory but it can be as well: - AMQP - SQL / NoSQL (the persistent layer shold be interchangeable) So as response to your duration question, this think is out scope for the counter tasks. On Wed, May 9, 2012 at 5:30 PM, Doug Hellmann wrote: > > > On Tue, May 8, 2012 at 7:19 PM, Luis Gervaso wrote: > >> Hi, >> >> I have uploaded a toy version of ceilometer (java implementation). >> >> It does implement the first two counters (instance : rabbitmq listener >> and cpu : polling from libvirt) >> >> i need more clarification on the meaining: >> >> counter_volume >> counter_duration >> counter_datetaime >> >> I hope this helps to figure out how to agreggate these data. >> >> http://github.com/woorea/ceilometer-java > > > Nice! > > I have also been experimenting. I have some Python code in > https://github.com/dhellmann/metering-prototype that listens for > notifications related to instances (create, delete, exists) and converts > them to counter output (see spy.py). There is also a pair of scripts for > recording an event stream and playing it back (useful for testing, see > recorder.py and player.py). I don't have any libvirt polling, yet, though. > > In the course of looking at the notifications being generated for > different scenarios, I discovered that the instance delete messages do not > have any duration information right now. I don't know if that is a bug, or > if the idea is to figure out the durations from looking at the most recent > "exists" event. What do other people think? > > Doug > -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO & CTO mobile: (+34) 627983344 luis@ woorea.es ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Puppet modules for OpenStack?
Mirantis folks released FUEL days ago, its tool for production deployments www.mirantis.com/company/press-release/mirantis-fuel-openstack-secret-sauce/ Regards, JuanFra El 13/04/2013 21:01, "Dan Bode" escribió: > Hi Dennis, > > The modules mentioned above are in a great maintained state (just one > known issue related to nova_config that should be merged soon), and I hope > to see additional modules added for quantum and ceilometer added in the > near future. > > I've heard reports that people have been happy with the ceilometer module > from enovance: > > https://github.com/enovance/puppet-ceilometer > > I expect that whatever ends up on stackforge will be based on this. > > As for quantum, the puppetlabs, bodepd, enovance, and ciscosystems module > (from github) are all loosely based on each other. I rencently dug through > the various branches of each and decided to base my work on the folsom > branch from ciscosystems. My work from this branch can be found here: > > https://github.com/bodepd/puppet-quantum > > Unless people protest, I expect this will eventually become the quantum > module. > > > > On Sat, Apr 13, 2013 at 5:29 AM, Dennis Jacobfeuerborn < > denni...@conversis.de> wrote: > >> Hi, >> i'm currently looking into deploying my first "real" (as in "not local >> devstack") OpenStack infrastructure and I intend to use Puppet to automate >> things. >> My questions is where can I find good modules for this purpose? I found >> something on puppetforge but there doesn't seem to be any quantum support >> and there are also modules available at the CiscoSystems github but these >> modules haven't been touched in several months. >> >> Is there a set of modules out there that have gained a certain momentum >> within the community so that they could be called inofficially endorsed by >> the (puppet using) openstack community? >> >> Regards, >> Dennis >> >> __**_ >> Mailing list: >> https://launchpad.net/~**openstack<https://launchpad.net/~openstack> >> Post to : openstack@lists.launchpad.net >> Unsubscribe : >> https://launchpad.net/~**openstack<https://launchpad.net/~openstack> >> More help : >> https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp> >> > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
John, Thanks. I will look into that extension today. The requirement is as an administrator, I want to know how many real resources I have in my cloud pool. If we don't have such interface in client side, I would be a contributor to add the code in cinder client. Best Regards -- Ray On Wed, Jul 17, 2013 at 2:50 AM, John Griffith wrote: > > > > On Tue, Jul 16, 2013 at 12:47 PM, Doug Hellmann < > doug.hellm...@dreamhost.com> wrote: > >> When I said, "we", I meant "the ceilometer team". If the auditing app >> isn't finding any volumes, it's not going to notify us. >> >> If you just want to know how much data is being used by cinder, there may >> be a way to get that from their admin API, but I'm not sure. >> >> >> On Mon, Jul 15, 2013 at 7:08 PM, Ray Sun wrote: >> >>> D >>> oug, >>> Thanks. I tried it in grizzly, here's the return: >>> sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit >>> Starting volume usage audit >>> Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00 >>> Found 0 volumes >>> Volume usage audit completed >>> >>> Actually, I want to get some data like this: >>> Total Cinder Storage on Physical Machine: 100G >>> Used Cinder Storage on Physical Machine: 10G >>> >>> Is there any way to get this? >>> >>> >>> Best Regards >>> -- Ray >>> >>> >>> On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann < >>> doug.hellm...@dreamhost.com> wrote: >>> >>>> We rely on a similar audit program to get the "exists" notifications >>>> about cinder volumes. Look for "cinder-volume-usage-audit". >>>> >>>> Doug >>>> >>>> >>>> On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun wrote: >>>> >>>>> Yes, it should be, but seems not at least in grizzly. Any update of >>>>> Ceilometer? >>>>> >>>>> Best Regards >>>>> -- Ray >>>>> >>>>> >>>>> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang >>>>> wrote: >>>>> >>>>>> I think Statistics should be find in Ceilometer. Ceilometer may >>>>>> provide with >>>>>> enough information you need. >>>>>> >>>>>> Best regards, >>>>>> Haomai Wang, UnitedStack Inc. >>>>>> >>>>>> 在 2013-7-14，上午8:09，Ray Sun 写道： >>>>>> >>>>>> In nova, we have a period task to report the usage of the physical >>>>>> server, including CPU, Memory and Local Disk, but I don't think I can >>>>>> find >>>>>> the same strategy in cinder service. Is there any way to do this or is >>>>>> there any blueprint for this? >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Best Regards >>>>>> -- Ray >>>>>> ___ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>>> >>>>>> >>>>>> >>>>> >>>>> ___ >>>>> Mailing list: https://launchpad.net/~openstack >>>>> Post to : openstack@lists.launchpad.net >>>>> Unsubscribe : https://launchpad.net/~openstack >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>> >>>> >>> >> >> ___ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp >> >> there is an os-hosts extension that gives things like volume-count and > GB/used on a cinder volume-service node, however it's not currently exposed > from the client. Not sure if that's the sort of thing you're looking for > or not. > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
Yes, I am assuming the service controller provides a different stream of data from the lower level VM events. So the question is how to represent and store this additional meta data in ceilometer. Note that there doesn't necessarily need to be a linkage/grouping between the resources since the association is what is actually contained in the metadata that is provided by the service controller. As a summary Nova provides its normal events for usage Service controller provides a mapping of nova instances to service type and actual end user Dan On 11/1/2012 11:25 AM, Doug Hellmann wrote: On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer <mailto:dan.dye...@gmail.com>> wrote: In some cases, the service controller is actually running inside a VM. It would not have access to the internals of the VM's. It maintains its metadata separately from the Nova infrastructure. It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or "join" it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug DD On 10/25/2012 2:25 AM, Nick Barcet wrote: Let's imagine that the service that launch instances can tag the instance with: a) a common service identifier (constant) b) a uuid unique for each "Unit" of the service such as : If that tag is passed onto the events which ceilometer stores in its entirety as meta, I do not see what the difficulty would be for the rating engine to be able to reconcile the information to handle your 2 use cases. Am I missing something? Nick On 10/25/2012 12:03 AM, Dan Dyer wrote: I don't think its just a matter of adding more meters or events for a couple of reasons: 1. In many cases the metadata I am referring to comes from a different source than the base usage data. Nova is still emitting its normal events, but we get the service/user mapping from a different source. I would not characterize this data as usage metrics but more data about the system relationships. 2. in the multiple VM case, we need to have the relationships specified so that we can ignore the proper VM's. There has also been talk of hybrid billing models that charge for some part of the VM usage as well as other metrics. Once again we need a way to characterize the relationships so that processing can associate and filter correctly. Dan On 10/24/2012 3:35 PM, Julien Danjou wrote: On Wed, Oct 24 2012, Dan Dyer wrote: Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed I think that for this, you just need to add more meters on top of the existing one with your own user and project id information. As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate "table" and used to determine the real user at aggregation time. This is probably where you should emit the meters you need. Use Case 2 Multple Instances combine to make a billable "product/service" In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed
[Openstack] OpenStack Community Weekly Newsletter (Jan 11 – 18)
Highlights of the week My first week at OpenStack <http://vmartinezdelacruz.com/my-first-week-at-openstack/> Victoria Martínez de la Cruz <http://vmartinezdelacruz.com/> is one of the three interns working on OpenStack under the Outreach Program for Women (OPW –Anne Gentle shared some details about the program before.) She will be working on Tenant Deletion Workflow <https://blueprints.launchpad.net/horizon/+spec/tenant-deletion> in the next months. This first blog post about OpenStack contains lots of good advice for any developer joining the community: a must read, for experienced developers, hiring managers and newcomers to this great community. Ceilometer Grizzly 2 Milestone Available <http://blog.doughellmann.com/2013/01/ceilometer-grizzly-2-milestone-available.html> The Ceilometer team is proud to announce the first synchronous milestone delivery with the OpenStack <http://openstack.org/> project. Grizzly-2 <https://launchpad.net/ceilometer/+milestone/grizzly-2> is also the last Folsom compatible version of Ceilometer <https://launchpad.net/ceilometer> as we are planning to introduce some breaking changes very soon in our trunk to enable a totally new set of features bringing Ceilometer beyond basic metering into monitoring and alerting. How many people does one need to build a multi-region cloud? <http://freedomhui.com/2013/01/stacklabhow-many-people-it-needs-to-build-a-multi-regions-cloud/> Hui Cheng describes in details the StackLab project. Spearheaded by Sina, Intel, Gamewave, Xi’an Jiaotong University, South China University of Technology and others, it’s a non profit platform to try and test OpenStack. Sina’s OpenStack development team was responsible for the development, operations, and go online initially. More volunteers have joined the effort and are actively being recruited to get involved to this great career, which will accelerate OpenStack popularizing in China. An Image Transfers Service For OpenStack <https://tropicaldevel.wordpress.com/2013/01/11/an-image-transfers-service-for-openstack/> John Bresnahan <https://tropicaldevel.wordpress.com/> makes the case for a new transfer service component in OpenStack. IaaS clouds must transfer VM images from the repositories in which they reside to compute nodes where they are booted. In the current state of OpenStack images download via HTTP to a nova-compute client speaking to the Glance image service. In the future proposed by John a transfers service would provide more predictable quality of service with horizontal scalability. Check his idea. Ceilomenter is looking for volunteers to take on unassigned blueprints <http://lists.openstack/> The team is about to start implementing the blueprints for the g3 milestone, there are few blueprints which still don’t have anyone assigned to them. If you are looking for something useful to code, head over to see the list of things that you could be working on. Remember that contributions during the Grizzly lifecycle will get you a free ticket to the OpenStack Summit. Tips and tricks * By Patrick McGarry <http://ceph.com/author/scuttlemonkey/>: Building a Public AMI with Ceph and OpenStack <http://ceph.com/howto/building-a-public-ami-with-ceph-and-openstack/> * By Davide Guerry: Using JuJu on OpenStack-based UniCloud <http://youtu.be/K5-iJ37q23k> * By John Bresnaha <https://tropicaldevel.wordpress.com/>: How to configure OpenStack Glance And Nova Backed By Red Hat Storage <https://tropicaldevel.wordpress.com/2013/01/16/openstack-glance-and-nova-backed-by-red-hat-storage/> * By Loïc Dachary <http://dachary.org/>: Installing OpenStack Folsom on Debian GNU/Linux wheezy <http://dachary.org/?p=1791> * By Robert Collins <http://rbtcollins.wordpress.com/>: Multi-machine parallel testing of nova with testrepository <http://rbtcollins.wordpress.com/2013/01/14/multi-machine-parallel-testing-of-nova-with-testrepository/> * By Kyle Mestery <http://www.siliconloons.com/>: Multi-node OpenStack Folsom devstack <http://www.siliconloons.com/?p=395> Upcoming Events * OpenStack Users January 2013 meetup <http://www.meetup.com/Indian-OpenStack-User-Group/events/93144352/> Jan 19, 2013 – Bangalore, India Details <http://www.meetup.com/Indian-OpenStack-User-Group/events/93144352/> * Openstack Developers Meetup <http://wiki.openstack.org/DeveloperMeetupRaanana> Jan 20, 2013 – Raanana, Israel Details <http://wiki.openstack.org/DeveloperMeetupRaanana> * Chef for OpenStack Hack Day <http://www.eventbrite.com/event/4395344594> Jan 22, 2013 – Boston, MA Details <http://www.eventbrite.com/event/4395344594> * oVirt Workshop <http://www.ovirt.org/NetApp_Workshop_January_2
Re: [Openstack] [ceilometer] Potential New Use Cases
On Thu, Nov 1, 2012 at 10:21 AM, Dan Dyer wrote: > In some cases, the service controller is actually running inside a VM. > It would not have access to the internals of the VM's. It maintains its > metadata separately from the Nova infrastructure. > It doesn't need internal access to the VM, but something has to share the metadata with ceilometer (or "join" it to the data ceilometer has) at some point. If it would be too difficult to get the data into the events, then it could be done by the app that uses the ceilometer API to query for usage. For example, the app that loads data from ceilometer to your real billing system could be driven by data saved by the service controller in whatever database it uses. Doug > > > DD > > > On 10/25/2012 2:25 AM, Nick Barcet wrote: > > Let's imagine that the service that launch instances can tag the > instance with: > a) a common service identifier (constant) > b) a uuid unique for each "Unit" of the service > such as : > > If that tag is passed onto the events which ceilometer stores in its > entirety as meta, I do not see what the difficulty would be for the > rating engine to be able to reconcile the information to handle your 2 > use cases. Am I missing something? > > Nick > > On 10/25/2012 12:03 AM, Dan Dyer wrote: > > I don't think its just a matter of adding more meters or events for a > couple of reasons: > 1. In many cases the metadata I am referring to comes from a different > source than the base usage data. Nova is still emitting its normal > events, but we get the service/user mapping from a different source. I > would not characterize this data as usage metrics but more data about > the system relationships. > 2. in the multiple VM case, we need to have the relationships specified > so that we can ignore the proper VM's. There has also been talk of > hybrid billing models that charge for some part of the VM usage as well > as other metrics. Once again we need a way to characterize the > relationships so that processing can associate and filter correctly. > > Dan > > On 10/24/2012 3:35 PM, Julien Danjou wrote: > > On Wed, Oct 24 2012, Dan Dyer wrote: > > > Use Case 1 > Service Owned Instances > There are a set of use cases where a service is acting on behalf of a > user, > the service is the owner of the VM but billing needs to be attributed > to the > end user of the system.This scenario drives two requirements: > 1. Pricing is similar to base VM's but with a premium. So the type of > service for a VM needs to be identifiable so that the appropriate > pricing > can be applied. > 2. The actual end user of the VM needs to be identified so usage can be > properly attributed > > I think that for this, you just need to add more meters on top of the > existing one with your own user and project id information. > > > As an example, in some of our PAAS use cases, there is a service > controller > running on top of the base VM that maintains the control and and > manages the > customer experience. The idea is to expose the service and not have the > customer have to (or even be able to) manipulate the virtual machine > directly. So in this case, from a Nova perspective, the PAAS service > owns > the VM and it's tenantID is what is reported back in events. The way we > resolve this is to query the service controller for meta data about that > instances they own. This is stored off in a separate "table" and used to > determine the real user at aggregation time. > > This is probably where you should emit the meters you need. > > > Use Case 2 > Multple Instances combine to make a billable "product/service" > In this use case, a service might consist of several VM's, but the > actual > number does not directly drive the billing. An example of this might > be a > redundant service that has a primary and two backup VM's that make up a > deployment. The customer is charged for the service, not the fact > that there > are 3 VM's running. Once again, we need meta data that is able to > describe > this relationship so that when the billing records are processed, this > relationship can be identified and billed properly. > > Kind of the same here, if you don't want to really bill the vm, just > don't meter them (or ignore the meters) and emit your own meter via your > PaaS platform to bill your customer. > > Or is there a limitation I miss? > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack >
[Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
Hi, The metering project team holds a meeting in #openstack-meeting, Thursdays at 1600 UTC <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. Everyone is welcome. I propose an agenda based on the discussions we had on this list. http://wiki.openstack.org/Meetings/MeteringAgenda Topic : schema and counter definitions * counter definitions * Proposed http://wiki.openstack.org/EfficientMetering#Counters * schema definition * Proposed http://wiki.openstack.org/EfficientMetering#Storage * discuss storage assumptions * the storage will store all events * no aggregated value is permanently stored * discuss API assumptions * the API provide a sum() function to aggregate values * the API may transparently store results of the sum function in a cache * discuss event collection * events are collected from a components when possible * ceilometer agent is installed on a node when the a component does not provide the value * contribute to the component instead of developping a ceilometer agent plugin * engaging discussions with core components * nova * cinder * glance * swift * quantum * open discussion Cheers -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ? l...@enovance.com ? +33 1 49 70 99 82 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack] Ceilometer API Glossary
On Tue, Oct 23, 2012 at 5:34 AM, Eoghan Glynn wrote: > > > > I am testing ceilometer in my devstack virtual machine. Although I > > can see the meter data model in the mongodb, I am confused about > > some glossary when I test its Web API. I am not very clear about the > > " resource" in the " GET /v1/resources",either "source" in the "GET > > /v1/sources/(source)/meters/(meter)". > > Can someone explain these two concept? Thanks a lot. > > Hi Yawei, > > Good question! > > My understanding is that the resources collection refers to the set > of UUIDs identifying the openstack entities (e.g. instance, volume, > image ... etc.) being metered, whereas the source refers to the > origin of the metering data and is effectively unused right now > as there is only a single source currently (though we could in the > future for example distinguish between metric and metering data > in this way). > That's exactly right. I'll open a ticket to add a glossary to the documentation. Doug > > Cheers, > Eoghan > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31 2012, Eoghan Glynn wrote: > Would that total usage be much more apparent if we started > metering the delta between CPU times on subsequent polling > periods as a gauge measure? (As opposed to treating it as > a cumulative measure) I'm rather against the idea of transforming all cumulative counters to delta, for the simple reason that this imply to lose information if your system is not launched to compute delta, or that you have to maintaint a previous value accross restart. The API will be capable to do the operation you need, no matter what the type of counter is (delta or cumulative). > I don't think (max - min) would suffice to give an accurate > measure of the actual CPU time used, as the counter may have > reset multiple times in the course of the requested duration. It is, because /max in the API should be aware of the fact a reset can occur and computes accordingly. We started to discuss this a bit in: https://bugs.launchpad.net/ceilometer/+bug/1061817 -- Julien Danjou # Free Software hacker & freelance # http://julien.danjou.info pgpRVR7qdZlI9.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] ceilometer not getting nova notifications
I am not getting disk.root.size meter notifications from nova to ceilometer. I am getting all ther pollster meters but not notification meters. Do I need to add any cron job like cinder to get notifications? I am using devstack and below are the contents of my nova.conf. firewall_driver = nova.virt.libvirt.firewall.IptablesFirewallDriver compute_driver = libvirt.LibvirtDriver flat_interface = eth0 flat_network_bridge = br100 vlan_interface = eth0 public_interface = br100 network_manager = nova.network.manager.FlatDHCPManager glance_api_servers = 10.102.153.5:9292 rabbit_password = freebsd rabbit_host = localhost rpc_backend = nova.openstack.common.rpc.impl_kombu ec2_dmz_host = 10.102.153.5 vncserver_proxyclient_address = 127.0.0.1 vncserver_listen = 127.0.0.1 vnc_enabled = true xvpvncproxy_base_url = http://10.102.153.5:6081/console novncproxy_base_url = http://10.102.153.5:6080/vnc_auto.html notification_driver = nova.openstack.common.notifier.rpc_notifier,ceilometer.compute.nova_notifier instance_usage_audit_period = hour instance_usage_audit = True logging_context_format_string = %(asctime)s.%(msecs)03d %(levelname)s %(name)s [%(request_id)s %(user_name)s %(project_name)s] %(instance)s%(message)s use_syslog = True instances_path = /opt/stack/data/nova/instances lock_path = /opt/stack/data/nova state_path = /opt/stack/data/nova volume_api_class = nova.volume.cinder.API enabled_apis = ec2,osapi_compute,metadata instance_name_template = instance-%08x libvirt_cpu_mode = none libvirt_type = qemu sql_connection = mysql://root:freebsd@localhost/nova?charset=utf8 my_ip = 10.102.153.5 osapi_compute_extension = nova.api.openstack.compute.contrib.standard_extensions s3_port = s3_host = 10.102.153.5 default_floating_pool = public fixed_range = force_dhcp_release = True dhcpbridge_flagfile = /etc/nova/nova.conf Thanks, Anshul 1,1 Top ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Adding a source notion to the schema
On Fri, May 4, 2012 at 6:03 PM, Nick Barcet wrote: > On 05/04/2012 02:25 PM, Doug Hellmann wrote: > > > > > > On Fri, May 4, 2012 at 12:29 PM, Nick Barcet > <mailto:nick.bar...@canonical.com>> wrote: > > > > Following up on the discussion on IRC yesterday during the metering > > meeting, I'd like to explain my proposal to add the notion of source > > to the schema of our event. The current goal we have for ceilometer > > is to provide a common way to accumulate counters/meters from > > various openstack component in a central repository that can be then > > used by other tools to produce bills in fine. One thing we can > > currently assume in Essex is that all components share a common > > repository for identity management: Keystone. This is the > > assumption we made when defining the existing schema. However, I > > think we need to plan a little further ahead here, and allow for > > this not to necessarily remain the same in some complex deployment > > cases. > > > > For example, it could be envisioned that some software, outside of > > the OpenStack project, maybe deployed and used on top of an > > OpenStack deployment. This software may need to be billed to > > customers, and collecting meters for it maybe as important for the > > provider than doing it for OpenStack components. It cannot be > > assumed that the identity management used by the software will be > > keystone. The software may not provide a metering interface built > > in, and it would make sense for the provider to be able to implement > > this using existing tool present in the underlying OpenStack > > deployment: ceilometer. > > > > In order to allow for this I think it would make sense to: > > * extend the current schema to allow for an additional "source" > > field in the event record. This should be a short identifier. > > > > > > That makes sense. It seems like the identifier(s) used are meant to be > > defined by the deployer. Is that your intent? > > Correct. We'll just need a sane default for Keystone. > You're focusing on source as the origin for the identity information, but it seems like a layer sitting on top of OpenStack may well use Keystone for identity but generate different billable events. Should "source" be the origin of the metric data being sent to ceilometer? > > > > > * add another record definition that maps source to identity > > managment URL location > > * Collecting agent would be in charge to specify the source they map > > to (keystone by default). > > > > > > It is not clear why the identity management location needs to be > > recorded in ceilometer. If the deployer controls the source strings, > > they know what each value means. The only thing that needs to translate > > the (source, project, user) values to a billing identity is the > > end-consumer of all of this data, which is also under the control of the > > deployer. Couldn't the mapping of the source token to the identity > > location be handled in that layer? > > True. It might be over-engineering to try to keep the info in the same > place as well, but the impact on the system would be negligible. I'd be > happy either ways :) It will be easier to add it later if we really need it than to take it out if we decide we don't. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [ceilometer] Potential New Use Cases
Based on a discussion with Doug at the Summit, I would like to propose a couple of new use cases for Ceilometer. As background, up until now, the usage data that Ceilometer collects could be considered atomic in the sense that everything needed to understand/process the information could be contained in a single generated event. We have identified some use cases that will require additional meta data about the source and/or type of the event so that later processing can be performed. Use Case 1 Service Owned Instances There are a set of use cases where a service is acting on behalf of a user, the service is the owner of the VM but billing needs to be attributed to the end user of the system.This scenario drives two requirements: 1. Pricing is similar to base VM's but with a premium. So the type of service for a VM needs to be identifiable so that the appropriate pricing can be applied. 2. The actual end user of the VM needs to be identified so usage can be properly attributed As an example, in some of our PAAS use cases, there is a service controller running on top of the base VM that maintains the control and and manages the customer experience. The idea is to expose the service and not have the customer have to (or even be able to) manipulate the virtual machine directly. So in this case, from a Nova perspective, the PAAS service owns the VM and it's tenantID is what is reported back in events. The way we resolve this is to query the service controller for meta data about that instances they own. This is stored off in a separate "table" and used to determine the real user at aggregation time.Note that in theory you could do this in the agent as part of collection, but we have found that this is very expensive and scales best if the actual substitution is delayed until the latest point possible (which at that point potentially means there are less records to process or can be better handled with parallel processing using something like MapReduce.From a billing perspective these instances will have unique pricing (i.e. premium on top of the base VM cost). Part of the aggregation process is to substitute the billable account for the service account and identify the service type so that proper billing can be applied. We would like to see the Ceilometer data model expanded to store this kind of metadata. Use Case 2 Multple Instances combine to make a billable "product/service" In this use case, a service might consist of several VM's, but the actual number does not directly drive the billing. An example of this might be a redundant service that has a primary and two backup VM's that make up a deployment. The customer is charged for the service, not the fact that there are 3 VM's running. Once again, we need meta data that is able to describe this relationship so that when the billing records are processed, this relationship can be identified and billed properly. Both of these use cases point to a general need to be able to store meta-data that will allow the usage processing logic to identify relationships between VM's and provide additional context for determining billing policy. Dan Dyer HP Cloud Services aka: DanD ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cinder Storage Server Statistics
Understood, and completely agree. I'll look at opening a bug to get this and run it by you when I have a patch to make sure we're meeting your needs here. On Wed, Jul 17, 2013 at 5:04 PM, Ray Sun wrote: > John, > Thanks. I will look into that extension today. The requirement is > as an administrator, I want to know how many real resources I have in my > cloud pool. > > If we don't have such interface in client side, I would be a contributor > to add the code in cinder client. > > Best Regards > -- Ray > > > On Wed, Jul 17, 2013 at 2:50 AM, John Griffith < > john.griff...@solidfire.com> wrote: > >> >> >> >> On Tue, Jul 16, 2013 at 12:47 PM, Doug Hellmann < >> doug.hellm...@dreamhost.com> wrote: >> >>> When I said, "we", I meant "the ceilometer team". If the auditing app >>> isn't finding any volumes, it's not going to notify us. >>> >>> If you just want to know how much data is being used by cinder, there >>> may be a way to get that from their admin API, but I'm not sure. >>> >>> >>> On Mon, Jul 15, 2013 at 7:08 PM, Ray Sun wrote: >>> >>>> D >>>> oug, >>>> Thanks. I tried it in grizzly, here's the return: >>>> sysadmin@demo:/opt/stack/cinder/bin$ cinder-volume-usage-audit >>>> Starting volume usage audit >>>> Creating usages for 2013-06-01 00:00:00 until 2013-07-01 00:00:00 >>>> Found 0 volumes >>>> Volume usage audit completed >>>> >>>> Actually, I want to get some data like this: >>>> Total Cinder Storage on Physical Machine: 100G >>>> Used Cinder Storage on Physical Machine: 10G >>>> >>>> Is there any way to get this? >>>> >>>> >>>> Best Regards >>>> -- Ray >>>> >>>> >>>> On Tue, Jul 16, 2013 at 6:27 AM, Doug Hellmann < >>>> doug.hellm...@dreamhost.com> wrote: >>>> >>>>> We rely on a similar audit program to get the "exists" notifications >>>>> about cinder volumes. Look for "cinder-volume-usage-audit". >>>>> >>>>> Doug >>>>> >>>>> >>>>> On Sun, Jul 14, 2013 at 11:04 AM, Ray Sun wrote: >>>>> >>>>>> Yes, it should be, but seems not at least in grizzly. Any update of >>>>>> Ceilometer? >>>>>> >>>>>> Best Regards >>>>>> -- Ray >>>>>> >>>>>> >>>>>> On Sun, Jul 14, 2013 at 11:04 AM, Haomai Wang >>>>> > wrote: >>>>>> >>>>>>> I think Statistics should be find in Ceilometer. Ceilometer may >>>>>>> provide with >>>>>>> enough information you need. >>>>>>> >>>>>>> Best regards, >>>>>>> Haomai Wang, UnitedStack Inc. >>>>>>> >>>>>>> 在 2013-7-14，上午8:09，Ray Sun 写道： >>>>>>> >>>>>>> In nova, we have a period task to report the usage of the physical >>>>>>> server, including CPU, Memory and Local Disk, but I don't think I can >>>>>>> find >>>>>>> the same strategy in cinder service. Is there any way to do this or is >>>>>>> there any blueprint for this? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Best Regards >>>>>>> -- Ray >>>>>>> ___ >>>>>>> Mailing list: https://launchpad.net/~openstack >>>>>>> Post to : openstack@lists.launchpad.net >>>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>>> More help : https://help.launchpad.net/ListHelp >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> ___ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>>> >>>>>> >>>>> >>>> >>> >>> ___ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >>> >>> there is an os-hosts extension that gives things like volume-count and >> GB/used on a cinder volume-service node, however it's not currently exposed >> from the client. Not sure if that's the sort of thing you're looking for >> or not. >> >> > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Weekly irc meetings day and time change?
On 08/30/2012 09:11 AM, Thomas Goirand wrote: > On 08/30/2012 04:20 PM, Julien Danjou wrote: >> On Thu, Aug 30 2012, Angus Salkeld wrote: >> >>> I'd like to attend but am in Australia. I am quite flexible so it might >>> be easier to say what times don't suit. >>> Basically midnight - 6am which in UTC is 14:00 to 20:00 >> >> That's gonna be a tough one. :) >> >> In the same idea, living in France, I'd exclude UTC 22:00 to 04:00. >> >> So that let us with UTC 04:00 to 14:00 and 20:00 to 22:00 >> >> Doug being in the US in IIRC UTC-7 that would exclude >> UTC 07:00 to UTC 13:00. >> >> Finally letting us with UTC 04:00 to 06:00 and 20:00 to 22:00. >> >> So finally, I would propose to use UTC 21:00: >> - 23:00 for me (and Nick most of the time I guess) in France >> - 14:00 for Doug >> - 7:00 for Angus >> >> or UTC 06:00: >> - 08:00 for me (and Nick most of the time I guess) in France >> - 23:00 for Doug >> - 16:00 for Angus >> >> (Hope I did not get the math wrong) > > If it's the former, I wont ever attend, no way I can get up at 5am > (Shanghai is GMT+8). This has always been at this time, and never, I can > go online. > > The later is ok though. I am about to open a poll to pick from 0600, 1200, 1500 and 2100 UTC on Google. Will reply to this thread when it opens. We'll have to go with what fits most. > Thomas > > P.S: I'd like to contribute to the ceilometer project. I haven't yet > (though I've done quite some work on the Debian packaging of the rest of > Openstack), and I'd like to know in which area I could start implementing. I would recommend that you take a look at the roadmap[1], and assign yourself one of the bugs in there if you feel like it. A good recommended read to start is at [2]. [1] http://wiki.openstack.org/EfficientMetering/RoadMap [2] http://ceilometer.readthedocs.org/ Hope this helps, and welcome to the ceilometer team! Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] First meeting for the Metering project
On 12-04-27 01:24 PM, Loic Dachary wrote: > On 04/27/2012 05:37 PM, Francis J. Lacoste wrote: >> On 12-04-27 10:50 AM, Nick Barcet wrote: >>> The meeting occurred as planned, and overran... Nevertheless a lot of >>> good decisions were made, the obviously most important one being >>> the project name: ceilometer. >>> >>> For a more complete meeting summary, go visit: >>> http://wiki.openstack.org/Meetings/MeteringAgenda/meeting-0-summary >>> >>> Next week we will have for objective to find an agreement on "schema and >>> counter definitions". If anyone does not do it before me I'll fire an >>> introduction email on the subject soon (sometime this we), as we agreed >>> to start the discussion via mailing list and use the irc meeting to >>> finalize the choices. >> The Launchpad project was created: >> >> https://launchpad.net/ceilometer > Thank you :-) I'm not familiar with the launchpad ways. Should I wait for the > git repository to be created ? Or can I help with that ? > Yes, please. I'm not familiar with the github ways :-) So please go ahead, and create the required github repository. Cheers -- Francis J. Lacoste francis.laco...@canonical.com signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] *ALT TIME* Metering meeting agenda for Wed at 21:00 UTC (Sept 12th, 2012)
On Tue, Sep 11, 2012 at 5:22 PM, Nick Barcet wrote: > PLEASE NOTE THE ALTERNATIVE MEETING TIME WED 21:00 UTC > > The metering project team will hold its next meeting at alternate time > on *Wednesday* at 9PM UTC > <http://www.timeanddate.com/worldclock/fixedtime.html?hour=21&min=0&sec=0 > >. > > Everyone is welcome. > > Agenda: > http://wiki.openstack.org/Meetings/MeteringAgenda > > * Review last week's actions > * nijaba to see with ttx how our sessions can be shown on official > program > * nijaba to update meeting page to note new alternating meeting time > * gmb to a plan on the ml with fixed dates > * dhellmann make sure flask is listed as a dependency of ceilometer > In case I can't make it to the meeting, I did check that Flask is listed in the tools/pip-requires file. It is pegged to version 0.9. > * Open discussion > > If you are not able to attend or have additional topic you would like to > cover, please update the agenda on the wiki. > > Cheers, > -- > Nick Barcet > aka: nijaba, nicolas > > > > > > > > > > > > > > > > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Hbase storage backend for Ceilometer - blueprint?
On Thu, Nov 15 2012, shengjie_...@dell.com wrote: > 1. Does it need to be approved first, what's the process to get it > approved/assigned to a release? Btw, I was trying to add Julien in the > approver list as well, didn't manage to do that. I don't think we've been anything formal so far about that. The storage part has been abstracted so everybody can come up with a new storage engine. So you're planning to work a new one, I don't see any reason not to approve such a blueprint. But I've set this blueprint to approved anyway. :) More discussion is likely to be needed if you need to do modifications to the abstraction layer itself. > 2. The community meeting normally is the place to have this kind of > tech discussion or it's for something else? It its, so I suggest you bring this today (in 2 minutes) in the open discussion at the end of the meeting, or add it the agenda for next meeting next week. -- Julien Danjou ;; Free Software hacker & freelance ;; http://julien.danjou.info pgpcjsYblpraH.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] Where can I consume messages from metering system?
On 25/05/12 10:55 +0200, Szymon Grzybowski wrote:
Hi,
Hi Szymon,
I have very simerlar needs, see comments inline...
(I am working on the heat project "https://github.com/heat-api/heat";)
We had a plan to write some "plugin" which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.
My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:
1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification "Alert less than 15% free ram on host"}
2. HostA hits less than 15% of free ram and sends notification "Alert less
than 15% free ram on HostA".
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.
I am trying to implement AWS::CloudWatch::Alarm
http://docs.amazonwebservices.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html
This defines really simerlar rules to what you are describing, and also
allows users to post (rest) statistics back to a statistics Collecting
server. We could then do fun things like autoscaling and HA recovery.
This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.
Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.
I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data) and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:
- write plugin for agent to send proper alert when metering data hits
proper rules (just like in above scenario with 15% free ram)
- write service which will consume this alert (listen proper topic in
queue) and fetch data from ceilometer collector?? (or it's backend via API)
and find out which VM should be migrated and where (or delegate "Where" to
nova-scheduler)
I could help out with work on rules and receiving guest statistics if
ceilometer guys are ok with this extension.
Another idea. Are there any Host states in Openstack? For example
"Maintenance state"? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to "maintenance".
2. There goes alert "HostA maintenance".
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to "active".
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
Everything should be event-driven (alert-notification, host-state-change,
administrator's action).
I'm also wondering what about this
ResourceMonitorAlertsandNotifications<http://wiki.openstack.org/ResourceMonitorAlertsandNotifications>,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).
Me too (I also have been wondering about it), I was quite keen on this
and contacted the authors, but no response...
Regards
Angus Salkeld
Re: [Openstack] Blueprint proposal: Drop setuptools_git for including data/config files
On 12/18/2012 05:29 PM, Sascha Peilicke wrote: > On 12/17/2012 04:47 PM, Thomas Goirand wrote: >> This >> means that absolutely all of our packages have to embed a patch in >> debian/patches to "fix" the "wrong" MANIFEST.in. >> >> We've spent quite some time on that. Or rather, should I say: it's a >> real time waster. >> >> While I do agree that the MANIFEST.in should be generated automatically, >> I don't think it should be stored in a "wrong" way on github. > So it should either contain something meaningful or be removed. In their > current state, these files are just worthless. Exactly! On 12/18/2012 07:50 PM, Mark McLoughlin wrote: > 1) You don't build packages from the tarballs produced by upstream. > I don't understand why you don't use tarballs, but I'm willing > to assume it's not something you can (or want to) change. In many ways, it's very convenient to do what we do. We could go back to use tarballs, but if it is avoidable, I want to keep the current system. > 2) Instead you use git-buildpackage which I know nothing about. Shortly, git-buildpackage uses an upstream branch (often called "master") containing upstream code, and a Debian branch containing that, plus the debian folder. Typing "git-buildpackage" does all the magic needed to build a Debian package, with the added bonus that you don't have to build where your package is stored (usually, we build in ../build-area), which means no risk to modify any files in your Git repository. > 3) You work from git repositories forked from upstream e.g. > > http://anonscm.debian.org/gitweb/?p=openstack/python-novaclient.git;a=shortlog;h=refs/heads/debian/experimental > > which AFAICT just add a debian/ directory to the source tree Yes. > 4) 'get-vcs-source' somehow generates a tarball from this tree. I'm > guessing it does 'git archive' rather than > 'python setup.py sdist' but I'm not sure. Some more digging > turns up: > > http://anonscm.debian.org/gitweb/?p=openstack/openstack-pkg-tools.git;a=blob_plain;f=pkgos.make;hb=HEAD > > and it seems that yes, you're using 'git archive' That's correct, you've found out quite well. Also, "get-vcs-source" fetches the master branch from upstream (git add remote, git fetch...). The repository is either git://github.com/openstack/.git, or we just override the UPSTREAM_GIT variable before including /usr/share/openstack-pkg-tools/pkgos.make (that's needed for python modules which are not on the github.com/openstack repo). > 5) I'm guessing there are two issues with these 'git archive' > generated tarballs - (a) there's no versioninfo file so > setuptools doesn't know have a version number and (b) there's > no .git directory so setuptools doesn't have an accurate way of > building a manifest > > I'm not completely clear what setuptools commands are failing > because of issue (b) All the above is exactly right! One of the reasons we are happy to use git archive is that this produces a Debian orig.tar.xz (notice the xz, and not just gz) from *any* commit, if we want to. Let me explain how it works. Let's say I want to make a snapshot release of Ceilometer for the commit hash 23ff2f9bbfc14e435c4c04fba473cf2a829b (this is an actual real life example, in fact...). Then, to do that, I just do: git checkout master git log # find out what's the name of the tag... git tag 2013.1_g0.4+23ff2f9bbf 23ff2f9bbfc14e435c4c04fba473cf2a829b git checkout debian/experimental git merge -X theirs 23ff2f9bbfc14e435c4c04fba473cf2a829b Then I edit my debian/changelog so it matches the tag: ceilometer (2013.1~g0.4+23ff2f9bbf-1) experimental; urgency=low * Initial release (Closes: #693406). -- Thomas Goirand Wed, 14 Nov 2012 14:41:52 + (the important bit is of course the version number above) Then I generate the orig.tar.xz: ./debian/rules get-vcs-source cp ../ceilometer_2013.1~g0.4+23ff2f9bbf.orig.tar.xz ../build-area Then I just build: git-buildpackage Another reason why git is convenient, is that we can cherry pick -x stuff from upstream, do branching, etc. Well, do I really have to convince people of this list that git is convenient? :) Probably not. > 6) However, you seem to be saying that issue (b) isn't an issue but > rather inaccurate MANIFEST.in files are the problem. How exactly > are they causing a problem. If we delete those files from > upstream git, does that work for you? Well, it be better if the MANIFEST.in could be stored correctly in upstream Git repositories, so we wouldn't have to deal with them. Cur
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31 2012, Eoghan Glynn wrote: > Would we have also have some 'misses' with the cumulative approach > when the ceilometer agent was down? No, unless the counter resets several times while your agent is down. But delta has the same issue. > If I understood the (\Sigma local maxima)-first idea correctly, > the usage up to the first polling cycle would always be > discounted from any duration. No, because if you have: Time | Value 0| 10 1| 30 2| 50 3| 80 4| 100 If your delta-pollster is down at 1 and 2, you restart at 3, therefore at 4 you'll send "20" as usage (100 minus 80). So you miss the delta between 10 (time 0) and 80 (time 3) (therefore 70 for free!). If you send right away 80 at time 3 when restarting, the API will be able to guess that between 0 and 3 the value went from 10 to 80. With delta approach, the API cannot guess that. > A more pernicious problem would occur if the instance was being > regularly restarted with a higher frequency than the polling period. Yes, but in that case, whatever counting method you use, you're screwed. :) -- Julien Danjou -- Free Software hacker & freelance -- http://julien.danjou.info pgpPm0HT9Gwc2.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: > My goal is to offer monitored data to the admin and customers. The admin is > interested in the utilization of the physical components and the virtual > machines and the customer is interested to know what his VMs do or can do. > It would be nice to get the data from a single point. I thought I can > enhance the Ceilometer compute agent to get this data out. Does this make > sense or is it better to use another monitoring tool for the physical > components? I think the pollster implementation can be done. I wouldn't implement this in the compute agent, but probably in some hardware agent, because it's likely it would be used in different kinds of environment and not only on compute node, i.e. you may also want to meter hardware usage for you cinder or glance node anyway. About the 10 minutes polling interval Doug mentionned, this can be a problem indeed, but it's still solvable later and would be easy to solve if this in a different agent, since you could change the periodic interval for pollster runs to something like 1 or 5 minutes. -- Julien Danjou // Free Software hacker & freelance // http://julien.danjou.info pgpDAWYZn0Ulm.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Monitoring physical devices
On Thu, Nov 1, 2012 at 2:13 PM, Julien Danjou wrote: > On Thu, Nov 01 2012, Zehnder Toni (zehndton) wrote: > > > My goal is to offer monitored data to the admin and customers. The admin > is > > interested in the utilization of the physical components and the virtual > > machines and the customer is interested to know what his VMs do or can > do. > > It would be nice to get the data from a single point. I thought I can > > enhance the Ceilometer compute agent to get this data out. Does this make > > sense or is it better to use another monitoring tool for the physical > > components? > > I think the pollster implementation can be done. I wouldn't implement > this in the compute agent, but probably in some hardware agent, because > it's likely it would be used in different kinds of environment and not > only on compute node, i.e. you may also want to meter hardware usage for > you cinder or glance node anyway. > > About the 10 minutes polling interval Doug mentionned, this can be a > problem indeed, but it's still solvable later and would be easy to solve > if this in a different agent, since you could change the periodic > interval for pollster runs to something like 1 or 5 minutes. > Good points. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Availability of metrics from SWIFT - Object Storage
Hi All, I am able to configure Openstack Essex using Ubuntu 12.04 Precise Pangolin for Swift Proxy, and 3 storage nodes. In dashboard while I create a Container it is creating however a error pops up Error Unable to create container. I can upload objects but cant delete objects or container. Any help would be appreciated. Regards, Raghavendra Lad From: Openstack [mailto:openstack-bounces+raghavendra.lad=accenture@lists.launchpad.net] On Behalf Of Narayanan, Krishnaprasad Sent: Wednesday, June 26, 2013 3:10 PM To: openstack@lists.launchpad.net Subject: [Openstack] Availability of metrics from SWIFT - Object Storage Hallo All, Based on the documentation from Ceilometer, I see the metrics from all the components except SWIFT. Can I get to know whether Ceilometer offers any metrics from the SWIFT component? Thanks Krishnaprasad This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. __ www.accenture.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] Potential New Use Cases
+1 for both of these use cases On Oct 24, 2012 5:06 PM, "Dan Dyer" wrote: > Based on a discussion with Doug at the Summit, I would like to propose a > couple of new use cases for Ceilometer. As background, up until now, the > usage data that Ceilometer collects could be considered atomic in the sense > that everything needed to understand/process the information could be > contained in a single generated event. We have identified some use cases > that will require additional meta data about the source and/or type of the > event so that later processing can be performed. > > Use Case 1 > Service Owned Instances > There are a set of use cases where a service is acting on behalf of a > user, the service is the owner of the VM but billing needs to be attributed > to the end user of the system. This scenario drives two requirements: > 1. Pricing is similar to base VM's but with a premium. So the type of > service for a VM needs to be identifiable so that the appropriate pricing > can be applied. > 2. The actual end user of the VM needs to be identified so usage can be > properly attributed > > As an example, in some of our PAAS use cases, there is a service > controller running on top of the base VM that maintains the control and and > manages the customer experience. The idea is to expose the service and not > have the customer have to (or even be able to) manipulate the virtual > machine directly. So in this case, from a Nova perspective, the PAAS > service owns the VM and it's tenantID is what is reported back in events. > The way we resolve this is to query the service controller for meta data > about that instances they own. This is stored off in a separate "table" and > used to determine the real user at aggregation time. Note that in theory > you could do this in the agent as part of collection, but we have found > that this is very expensive and scales best if the actual substitution is > delayed until the latest point possible (which at that point potentially > means there are less records to process or can be better handled with > parallel processing using something like MapReduce. From a billing > perspective these instances will have unique pricing (i.e. premium on top > of the base VM cost). Part of the aggregation process is to substitute > the billable account for the service account and identify the service type > so that proper billing can be applied. We would like to see the Ceilometer > data model expanded to store this kind of metadata. > > > Use Case 2 > Multple Instances combine to make a billable "product/service" > In this use case, a service might consist of several VM's, but the actual > number does not directly drive the billing. An example of this might be a > redundant service that has a primary and two backup VM's that make up a > deployment. The customer is charged for the service, not the fact that > there are 3 VM's running. Once again, we need meta data that is able to > describe this relationship so that when the billing records are processed, > this relationship can be identified and billed properly. > > Both of these use cases point to a general need to be able to store > meta-data that will allow the usage processing logic to identify > relationships between VM's and provide additional context for determining > billing policy. > > Dan Dyer > HP Cloud Services > aka: DanD > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [metering] Where can I consume messages from metering system?
Hi,
We had a plan to write some "plugin" which will collect metering data from
VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts)
via libvirt (mayby later for xen etc), but i've found ceilometer project,
which is doing what I want or will be doing what I want in near future :).
I was thinking about collectd/ganglia/munin as source of data and I wanted
to write local agents which will poll those daemons (collectd/ganglia) and
push data to Rabbit, so it's pretty close to ceilometer's guys.
My main purpose is to consume those messages and write algorithm to manage
cloud from administrative perspective. I'd like to create something like
DRM in VmWare - I was on presentation about this topic recently and from
that brief I learnt that VmWare has something like rules to manage VMs. I
think, that example (use case) from presentation would be the best way to
describe what I'm trying to point out:
1. Administrator defined a rule: { If free ram on host is less than 15%,
send notification "Alert less than 15% free ram on host"}
2. HostA hits less than 15% of free ram and sends notification "Alert less
than 15% free ram on HostA".
3. Central collector or central daemon receives Alert about free ram.
4. Central collector pick VM from HostA and migrate it to HostB. - How he
decides which VM and to which Host migrate is part of algorithm/policy.
This is the basic idea, how does it work - this is what I've found out
during presentation. And now my whole idea about this mechanism and
openstack: there could be different policies(algorithms/plugins) how, where
and when migrate VMs and this could be implemented in external plugins.
Someone can say that there is nova-scheduler. This scenario isn't exactly
what nova-scheduler is doing, but I see, that in this solution
nova-scheduler can play his role as service, which picks best host to
migrate VM.
I'm trying to find out what can I use from ceilometer as metering system.
I've read few pages on wiki and posts on mailing list about ceilometer's
architecture, messages etc.
Right now I know there are Agents and Collectors. Agents get data from
hypervisor (metering data) and push them in queue (nova notification
system). Collectors listens to queue's topic and receives those messages.
I'd like to:
- write plugin for agent to send proper alert when metering data hits
proper rules (just like in above scenario with 15% free ram)
- write service which will consume this alert (listen proper topic in
queue) and fetch data from ceilometer collector?? (or it's backend via API)
and find out which VM should be migrated and where (or delegate "Where" to
nova-scheduler)
Another idea. Are there any Host states in Openstack? For example
"Maintenance state"? Besides scenarios with free ram, cpu and other
resources there is also typical example of how useful this mechanism could
be with maintenance state.
*Scenario:*
1. Administrator changes HostA state to "maintenance".
2. There goes alert "HostA maintenance".
3. Central collector or central daemon receives Alert about maintenance.
4. Central collector gets list of VMs on HostA.
5. Central collectors start migration all VMs from HostA to other Hosts (or
invoke nova-scheduler to reassign VM from HostA to other hosts, but in this
case we need to change or implement new filter in nova-scheduler)
6. Administrator changes HostA state to "active".
7. Openstack starts new machines on this node (this is now done via
nova-scheduler) or migrate VMs from other overloaded hosts to HostA.
Everything should be event-driven (alert-notification, host-state-change,
administrator's action).
I'm also wondering what about this
ResourceMonitorAlertsandNotifications<http://wiki.openstack.org/ResourceMonitorAlertsandNotifications>,
because when I'm looking at schema, it has everything what I need, but
blueprint link is broken. Is this idea obsolete? Or is it implemented?
(this was created 2nd April 2012, so I think it's not implemented yet).
First question: Does it make sense? Are such mechanisms implemented in
Openstack right now? Or is it worth to implement?
Second question (if first's answer isn't negative): How and when can I use
ceilometer to do what I've described? From meeting's
topics<http://wiki.openstack.org/Meetings/MeteringAgenda>I know that
yesterday you were deciding what to use as notification bus,
and in june you will be thinking about storage backend, how is it going on
right now? Is such scenario possible in ceilometer (I mean plugins in
agents)? Collecting data not only from guests but also from hosts.
*Cheers,
*
*
*
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
On 05/03/2012 02:22 PM, Loic Dachary wrote: > Hi, > > The metering project team holds a meeting in #openstack-meeting, Thursdays at > 1600 UTC > <http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. > Everyone is welcome. > I propose an agenda based on the discussions we had on this list. > > http://wiki.openstack.org/Meetings/MeteringAgenda > Topic : schema and counter definitions > > * counter definitions >* Proposed http://wiki.openstack.org/EfficientMetering#Counters > * schema definition >* Proposed http://wiki.openstack.org/EfficientMetering#Storage > * discuss storage assumptions >* the storage will store all events >* no aggregated value is permanently stored > * discuss API assumptions >* the API provide a sum() function to aggregate values >* the API may transparently store results of the sum function in a cache > * discuss event collection >* events are collected from a components when possible >* ceilometer agent is installed on a node when the a component does not > provide the value >* contribute to the component instead of developping a ceilometer agent > plugin > * engaging discussions with core components >* nova >* cinder >* glance >* swift >* quantum > * open discussion > For the record, the first two points used all the time but that was the goal of the meeting. The other points would have been nice to discuss but can each be turned into a mailing list thread ;-) == #openstack-meeting Meeting == Meeting started by dachary at 16:00:16 UTC. The full logs are available at http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html . Meeting summary --- * actions from previous meetings (dachary, 16:00:36) * creation of the ceilometer project (dachary, 16:00:36) * The repository for the ceilometer project has been created (dachary, 16:00:36) * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) * and the first commit was successfully reviewed and merged today https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) * meeting organisation (dachary, 16:01:03) * This is 1/5 meetings to decide the architecture of the Metering project https://launchpad.net/ceilometer (dachary, 16:01:03) * Today's focus is on the definition of the counters / meters and the associated schema for the storage (dachary, 16:01:03) * It is the conclusion of the discussions held on the mailing list and the goal is to make a final choice that will then be implemented. (dachary, 16:01:03) * The meeting is time boxed and there will not be enough time to introduce inovative ideas and research for solutions. (dachary, 16:01:03) * The debate will be about the pro and cons of the options already discussed on the mailing list. (dachary, 16:01:03) * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, 16:01:03) * counter definitions (dachary, 16:02:10) * Proposed http://wiki.openstack.org/EfficientMetering#Counters (dachary, 16:02:10) * ACTION: dachary fix the note for net_float still talks about "number of floating IPs" (dachary, 16:09:18) * ACTION: jd___ include Number of object in Swift, Number of containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift in the table (dachary, 16:10:11) * ACTION: dachary add note about the fact that the resource_id for the object count is the container_id (dachary, 16:21:44) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. (dachary, 16:25:35) * ACTION: jd___ document the resource_id for each counter (dachary, 16:30:33) * ACTION: jd___ describes the general table schema and then something that says for each counter exactly what goes in the fields of that table and show how secondary field counters are recorded in the in the schema too (dachary, 16:33:27) * AGREED: s/counter/meter/ (dachary, 16:37:11) * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. ? (dachary, 16:37:38) * AGREED: http://wiki.openstack.org/EfficientMetering#Counters is agreed on, provided the actions listed above are carried out. ? (dachary, 16:38:52) * schema definition (dachary, 16:39:05) * Proposed http://wiki.openstack.org/EfficientMetering#Storage (dachary, 16:39:05) * ACTION: jd___ clarify / document the fact that the secondary field or the resource_id field carries the IP/VIF for the meters related to network so that they can be "per IP" (dachary, 16:40:42) * ACTION: nijaba describe the source field
[Openstack] OpenStack Community Weekly Newsletter (Feb 22 - Mar 1)
Highlights of the week Important CLA changes coming this weekend (2nd try) <http://markmail.org/message/azrdwianmnt2j5oc> Last weekend the change was blocked at the last minute by a request to amend the CLA text. We're trying again this weekend. Starting on March 3, 2013 1700UTC all contributors MUST review and agree to the new OpenStack Individual Contributor License Agreement and provide updated contact information at https://review.openstack.org/#/settings/agreements. On that day the Gerrit interface will be changing to present the new CLA text referring to the OpenStack Foundation, and will prompt you to agree to it there. Any previous agreement with OpenStack LLC will be marked expired at that time. The text of the new agreement is available for your convenience <https://review.openstack.org/static/cla.html>. You must also sign up for an OpenStack Foundation Individual Membership with the same E-mail address as used for your Gerrit contact information: http://openstack.org/register/. What's new in OpenStack Grizzly ? <http://www.enovance.com/%25postname> Folsom brought us two new core projects : Quantum <https://wiki.openstack.org/wiki/Quantum> (Networking) and Cinder <https://wiki.openstack.org/wiki/Cinder> (Volumes). Two new projects have been incubated : Ceilometer <https://wiki.openstack.org/wiki/Ceilometer> (metering) led by Nicolas Barcet <http://fr.linkedin.com/in/nbarcet> (VP Products at eNovance), and Heat <https://wiki.openstack.org/wiki/Heat> (Cloud Orchestration). Oslo <https://wiki.openstack.org/wiki/Oslo> is a new incubated project which produces a set of python libraries containing infrastructure code shared by all OpenStack projects. More details for each OpenStack project on this post by eNovance. OpenStack Ceilometer and Heat projects graduated <http://julien.danjou.info/blog/2013/openstack-ceilometer-head-graduated> The OpenStack Technical Committee <http://www.openstack.org/foundation/technical-committee/> have voted these last weeks about graduation of Heat <https://launchpad.net/heat> and Ceilometer <http://launchpad.net/ceilometer>, to change their status from /incubation/ to /integrated/*. *Heat and Ceilometer will be released as part as OpenStack for the next release cycle /Havana/, due in October 2013. **Raspberry Pi as a transparent squid caching proxy* <http://blog.stevebaker.org/2013/02/raspberry-pi-as-transparent-squid.html>* How does a hacker improve build times when he has a slow Internet connection? A fascinating use of a tiny sub-$50 computer (RaspberryPi) to build cloud images for OpenStack. By Steve Baker <http://blog.stevebaker.org/search/label/openstack> **Data placement in OpenStack Object Storage Swift* <http://swiftstack.com/blog/2013/02/25/data-placement-in-swift/>* Technical deep dive in how OpenStack Object Storage solves one of the hard problems of all distributed storage system: how to effectively place the data within the storage cluster. Swift has a "unique-as-possible" placement algorithm which ensures that the data is placed efficiently and with as much protection from hardware failure as possible. **The life of an OpenStack libvirt image* <http://www.pixelbeat.org/docs/openstack_libvirt_images/#1361764412>* The main stages of a Virtual Machine disk image as it transfers through OpenStack to be booted under libvirt explained by Pádraig Brady <http://www.pixelbeat.org/>. Security Advisories * OSSA-2013-006] VNC proxy can connect to the wrong VM (CVE-2013-0335) <http://lists.openstack.org/pipermail/openstack-announce/2013-February/82.html> Tips and Tricks * By Daniel P. Berrangé <https://www.berrange.com/>: Installing a 4 node Fedora 18 OpenStack Folsom cluster with PackStack <https://www.berrange.com/posts/2013/03/01/installing-a-4-node-fedora-18-openstack-folsom-cluster-with-packstack/> * By Victoria Martínez de la Cruz <http://vmartinezdelacruz.com/>: Attention newcomers! #openstack-101 is now open at Freenode <http://vmartinezdelacruz.com/attention-newcomers-openstack-101-is-now-open-at-freenode/> * By Anita Kuno <http://anteaya.info/>: The Structure and Habits of Git <http://anteaya.info/blog/2013/02/26/the-structure-and-habits-of-git/> * By Andreas Jaeger <http://jaegerandi.blogspot.com/search/label/OpenStack>: Coordinating efforts to create OpenStack packages for openSUSE and SUSE Linux Enterprise Server <http://jaegerandi.blogspot.com/2013/02/coordinating-efforts-to-create.html> Upcoming Events * OpenStack Sessions @ Barcamp Bangalore13 <http://barcampbangalore.org/bcb/event/bcb13> Mar 02, 2013 -- Bangalore, India Details <http://barcampbangalore.org/bcb/event
Re: [Openstack] [metering] licensing
On 05/12/2012 01:30 AM, Doug Hellmann wrote: > > > On Fri, May 11, 2012 at 5:27 PM, Loic Dachary <mailto:l...@enovance.com>> wrote: > > On 05/11/2012 10:01 PM, Doug Hellmann wrote: > > I was very surprised to see the change to license ceilometer as AGPL > [1]. Why are we not using the same Apache v2 license that all other OpenStack > projects are using? > > > > Doug > > > > [1] https://review.stackforge.org/#/c/29/ > > > > > Hi, > > Of course it will be Apache v2 when it is incubated, otherwise it will be > rejected. eNovance policy is to publish server side code under AGPLv3+, hence > the change. This should have been discussed before though. If this raises any > issues please let me know and I'll make sure they are resolved. > > > As I pointed out privately, it seems like it will be easier to start with the > license we know we will need later than to change the license after we have a > bunch of contributions. I appreciate the rationale behind a "default to > copyleft" policy, but the community already has a standard for licensing and > that should take precedence here. > This is reasonable indeed and we will publish all code to ceilometer under Apache from now on. Thanks for raising this issue. Cheers -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ? l...@enovance.com ? +33 1 49 70 99 82 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering] licensing
On Mon, May 14, 2012 at 3:50 AM, Loic Dachary wrote: > On 05/12/2012 01:30 AM, Doug Hellmann wrote: > > > > On Fri, May 11, 2012 at 5:27 PM, Loic Dachary wrote: > >> On 05/11/2012 10:01 PM, Doug Hellmann wrote: >> > I was very surprised to see the change to license ceilometer as AGPL >> [1]. Why are we not using the same Apache v2 license that all other >> OpenStack projects are using? >> > >> > Doug >> > >> > [1] https://review.stackforge.org/#/c/29/ >> > >> > >> Hi, >> >> Of course it will be Apache v2 when it is incubated, otherwise it will be >> rejected. eNovance policy is to publish server side code under AGPLv3+, >> hence the change. This should have been discussed before though. If this >> raises any issues please let me know and I'll make sure they are resolved. >> > > As I pointed out privately, it seems like it will be easier to start > with the license we know we will need later than to change the license > after we have a bunch of contributions. I appreciate the rationale behind a > "default to copyleft" policy, but the community already has a standard for > licensing and that should take precedence here. > > This is reasonable indeed and we will publish all code to ceilometer > under Apache from now on. Thanks for raising this issue. > Good, thank you! Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Ceilometer API Glossary
On Thu, Oct 25 2012, 吴亚伟 wrote: > If it is just like what Eoghan said that "it is unused right now as there > is only a single source currently", what does the "?" represent? > If I want to establish a customer billing system,do I need it ? It represents the fact we had no idea to use it when we write the code the first time. We discussed this yesterday, now we have a good plan. See https://bugs.launchpad.net/ceilometer/+bug/1070857 > I do mean the cinder volumes , but I don't know how to configure it ,could > you tell me more detailed steps on how to do this ? You need to configure the notifier to use rabbitmq (this is commented out in the default configuration file IIRC), to set the audit interval to something low like daily or hourlay (this should also be in the default configuration file) and to run this program into a cronjob. > Same to cinder , detailed steps will be highly appreciated . Like cinder, you need to configure the notification backend to use rabbitmq. > If I modify the configuration file of service like nova or cinder , how to > restart the related service in devstack ? Just press C-c in the screen window, up arrow and return. -- Julien Danjou /* Free Software hacker & freelance http://julien.danjou.info */ pgpgvSRvt2T8L.pgp Description: PGP signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 11:25 AM, Eoghan Glynn wrote: > > > > > I don't think (max - min) would suffice to give an accurate > > > measure of the actual CPU time used, as the counter may have > > > reset multiple times in the course of the requested duration. > > > > It is, because /max in the API should be aware of the fact a > > reset can occur and computes accordingly. We started to discuss > > this a bit in: > > > > https://bugs.launchpad.net/ceilometer/+bug/1061817 > > A-ha, OK, so not so much (max - min) as: > >(\Sigma local maxima) - first > > Sounds computationally expensive to produce on the fly, but maybe > the local maxima can be efficiently recorded as the data is being > ingested. > Is that better than just reporting the data in a more easily digested format in the first place? Julien, I don't understand your comment about losing data "if your system is not launched to compute delta". Can you clarify what you mean there? I do understand that the agent would need to store state about the counter locally in order to track the delta value, but I think we could provide a convenient way for pollsters to do that without complicating them excessively. Doug > > Cheers, > Eoghan > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [ceilometer] meter data volume
On Wed, Oct 31, 2012 at 1:39 PM, Julien Danjou wrote: > On Wed, Oct 31 2012, Eoghan Glynn wrote: > > > Would we have also have some 'misses' with the cumulative approach > > when the ceilometer agent was down? > > No, unless the counter resets several times while your agent is down. > But delta has the same issue. > > > If I understood the (\Sigma local maxima)-first idea correctly, > > the usage up to the first polling cycle would always be > > discounted from any duration. > > No, because if you have: > > Time | Value > 0| 10 > 1| 30 > 2| 50 > 3| 80 > 4| 100 > > If your delta-pollster is down at 1 and 2, you restart at 3, therefore > at 4 you'll send "20" as usage (100 minus 80). So you miss the delta > between 10 (time 0) and 80 (time 3) (therefore 70 for free!). > If you send right away 80 at time 3 when restarting, the API will be > able to guess that between 0 and 3 the value went from 10 to 80. > With delta approach, the API cannot guess that. > Sure it can, you just need to move where the caching is done. Using a local cache to maintain the previous time a value was published you would know at time 3 that the last published value was 10, and so send 70. So the total will be correct. Doug ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...
Thanks for putting that together Sandy. Very nice! >From my perspective, there are two major things that are undesirable for us: 1) Putting this data through the queue is not something that feels right. We'd like to have to option to use other types of data "sinks" from the generation points. Some folks might want to use the queues, but we do not wish to burden the queueing system with this volume of data. In some cases, we will just drop these into log files for later collection and aggregation. 2) We would like a common mechanism for instrumenting but we would like to be able to route data to any number of places: local file, datagram endpoint, etc. Now, getting the lower level measurement library consistent is definitely the right approach. I still think we need to support decorators in addition to monkey patching. And, we should make the gauges or whatever we call them usable with different "sinks". On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: > Hey! > > Here's a first pass at a proposal for unifying StackTach/Ceilometer and other > instrumentation/metering/monitoring efforts. > > It's v1, so bend, spindle, mutilate as needed ... but send feedback! > > http://wiki.openstack.org/UnifiedInstrumentationMetering > > Thanks, > Sandy > > ___ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Ceilometer] Problems with query fields in filters
Hi Julien
thanks for your reply, unfortunately "volume" instead of "counter_volume" is
not working either.
Alex
-Original Message-
From: Julien Danjou [mailto:jul...@danjou.info]
Sent: venerdì 12 luglio 2013 11:34
To: Alessandro Barabesi
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Ceilometer] Problems with query fields in filters
On Fri, Jul 12 2013, Alessandro Barabesi wrote:
> I have the following problems with query fields in filters.
>
> 1) Filtering by "metadata.size" in v2/meters/image apparently is not working.
> The following request returns also samples with metadata.size=0.
>
>
> GET http://10.10.10.10:8777/v2/meters/image
>
>
> {
> "q": [{
> "field": "project_id",
> "op": "eq",
> "value": "77b461539c8542909f67b29939ec87dd"
> },
> {
> "field": "timestamp",
> "op": "ge",
> "value": "2013-07-11T13:36:00"
> },
> {
> "field": "timestamp",
> "op": "lt",
> "value": "2013-07-11T13:39:00"
> },
> {
> "field": "metadata.size",
> "op": "gt",
> "value": "0"
> }]
>
> }
>
> I am probably doing something wrong but I can't figure out what.
That seems correct from the top of my head. If that really doesn't work, feel
free to open a bug.
> 2) Filtering by "counter_volume" returns the folloving error message:
>
> error_message={"debuginfo": null, "faultcode": "Client",
> "faultstring": "Unknown argument: \"counter_volume\": unrecognized
> query field"}
>
> Is this a bug or filtering by "counter-volume" is not allowed?
Try `volume' instead of `counter_volume'. But I am not sure it's currently
allowed -- in that case opening a bug can be good idea too.
--
Julien Danjou
// Free Software hacker / freelance consultant // http://julien.danjou.info
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] [heat] Heat 8-13-2012 Planning and Status Meeting
= #heat Meeting = Meeting started by asalkeld at 21:14:37 UTC. The full logs are available at heat/2012/heat.2012-08-13-21.14.log.html . The full logs are available at: http://heat-api.org/heat-irc-meetings/heat.2012-08-13-21.14.log.html All IRC meeting logs are available at: http://heat-api.org/irclogs.html Meeting summary --- * rollcall (sdake, 21:17:23) * zaneb, sdake, imain, asalkeld, shardy, funzo, jpeeler present (sdake, 21:18:30) * shadower present too (sdake, 21:19:07) * v6 additional features (sdake, 21:19:14) * shardy wrapped up openshift (sdake, 21:23:54) * ACTION: shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort (sdake, 21:25:38) * ACTION: zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code (sdake, 21:26:11) * additional features from returning from pto devs - openstack rest orchestraition API and improved CloudWatch implementation (sdake, 21:28:10) * ACTION: slower to investigate s3 integration support with swift (sdake, 21:29:04) * 9/4 release target for v6 (sdake, 21:33:50) * zaneb to work on rest api for v6, sdake start sorting out quantum integration plans in v6, then v7 shardy,sdake,zaneb work on quantum vpc (sdake, 21:45:33) * open items (sdake, 21:45:53) * ACTION: sdake to find out how tempest expects test cases (sdake, 21:52:35) * LINK: https://github.com/openstack/tempest/ (asalkeld, 21:53:04) * unanimous vote to add tempest integration as roadmap item for future versions (sdake, 21:55:35) * as user base increases, add troubleshooting tips to troubleshooting wiki based upon user feedback (sdake, 21:58:49) Meeting ended at 22:06:14 UTC. Action Items * shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort * zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code * slower to investigate s3 integration support with swift * sdake to find out how tempest expects test cases Action Items, by person --- * sdake * sdake to find out how tempest expects test cases * shardy * shardy working on initial cloudwatch improvements - to potentially merge into ceilometer later without much wasted effort * zaneb * zaneb help wrap up integration testing and start cracking on openstack orchestration rest api code * **UNASSIGNED** * slower to investigate s3 integration support with swift People Present (lines said) --- * sdake (134) * asalkeld (32) * zaneb (26) * shardy (24) * Slower (12) * jpeeler (7) * mheat-bot (5) * shadower (5) * russellb (2) * funzo (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] [Metering] First draft of plugin/agent documentation
I've just committed a first of the plugin/agent documentation to stackforge/ceilometer. Reviews welcome: https://review.stackforge.org/#/c/250/ Thanks, Nick signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Regarding openstack metering
Hi, I am new to openstack and looking for the metering tools. I read about the ceilometer, but not found any proper documentation for this. Please help me understand regarding the available metering tools. Regards, Arumon ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
A question/comment about the scope of the schema or maybe the architecture. Assuming the services will provide the instrumentation to populate the raw metric data, it seems likely that you will need to define an interface between the services/agents that are providing the data and the metering system which stores the generated metric data in the database (as opposed to having the services write directly to the DB). Is the schema intended to be this kind of interop format between the services and the meter's datastore or just the end result of the storage? Thanks, Dan Dyer On Thu, May 3, 2012 at 11:10 AM, Loic Dachary wrote: > On 05/03/2012 02:22 PM, Loic Dachary wrote: > > Hi, > > The metering project team holds a meeting in #openstack-meeting, > Thursdays at 1600 > UTC<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. > Everyone is welcome. > I propose an agenda based on the discussions we had on this list. > > http://wiki.openstack.org/Meetings/MeteringAgenda > Topic : schema and counter definitions > > * counter definitions >* Proposed http://wiki.openstack.org/EfficientMetering#Counters > * schema definition >* Proposed http://wiki.openstack.org/EfficientMetering#Storage > * discuss storage assumptions >* the storage will store all events >* no aggregated value is permanently stored > * discuss API assumptions >* the API provide a sum() function to aggregate values >* the API may transparently store results of the sum function in a cache > * discuss event collection >* events are collected from a components when possible >* ceilometer agent is installed on a node when the a component does not > provide the value >* contribute to the component instead of developping a ceilometer agent > plugin > * engaging discussions with core components >* nova >* cinder >* glance >* swift >* quantum > * open discussion > > For the record, the first two points used all the time but that was the > goal of the meeting. The other points would have been nice to discuss but > can each be turned into a mailing list thread ;-) > > == > #openstack-meeting Meeting > == > > > Meeting started by dachary at 16:00:16 UTC. The full logs are available > athttp://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html > . > > > > Meeting summary > --- > > * actions from previous meetings (dachary, 16:00:36) > * creation of the ceilometer project (dachary, 16:00:36) > * The repository for the ceilometer project has been created > (dachary, 16:00:36) > * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) > * and the first commit was successfully reviewed and merged today > https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) > > * meeting organisation (dachary, 16:01:03) > * This is 1/5 meetings to decide the architecture of the Metering > project https://launchpad.net/ceilometer (dachary, 16:01:03) > * Today's focus is on the definition of the counters / meters and the > associated schema for the storage (dachary, 16:01:03) > * It is the conclusion of the discussions held on the mailing list and > the goal is to make a final choice that will then be implemented. > (dachary, 16:01:03) > * The meeting is time boxed and there will not be enough time to > introduce inovative ideas and research for solutions. (dachary, > 16:01:03) > * The debate will be about the pro and cons of the options already > discussed on the mailing list. (dachary, 16:01:03) > * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, > 16:01:03) > > * counter definitions (dachary, 16:02:10) > * Proposed http://wiki.openstack.org/EfficientMetering#Counters > (dachary, 16:02:10) > * ACTION: dachary fix the note for net_float still talks about "number > of floating IPs" (dachary, 16:09:18) > * ACTION: jd___ include Number of object in Swift, Number of > containers in Swift, Number of GET/HEAD/PUT/POST requests in Swift > in the table (dachary, 16:10:11) > * ACTION: dachary add note about the fact that the resource_id for the > object count is the container_id (dachary, 16:21:44) > * LINK: http://wiki.openstack.org/EfficientMetering#Counters is agreed > on, provided the actions listed above are carried out. (dachary, > 16:25:35) > * ACTION: jd___ document the resource_id for each counter (dachary, > 16:30:33) > * ACTION: jd___ describes the general table schema and then something > that says
Re: [Openstack] [Metering] Meeting agenda for today 16:00 UTC (May 3rd, 2012)
On Thu, May 10, 2012 at 12:17 AM, Daniel Dyer wrote: > A question/comment about the scope of the schema or maybe the > architecture. Assuming the services will provide the instrumentation to > populate the raw metric data, it seems likely that you will need to define > an interface between the services/agents > that are providing the data and the metering system which stores the > generated metric data in the database (as opposed to having the services > write directly to the DB). Is the schema intended to be this kind of > interop format between the services and > the meter's datastore or just the end result of the storage? > It may be both, at first, but we also may find some benefit to letting them diverge later so I don't think we need to make it a hard requirement. > > Thanks, > Dan Dyer > > On Thu, May 3, 2012 at 11:10 AM, Loic Dachary wrote: > >> On 05/03/2012 02:22 PM, Loic Dachary wrote: >> >> Hi, >> >> The metering project team holds a meeting in #openstack-meeting, >> Thursdays at 1600 >> UTC<http://www.timeanddate.com/worldclock/fixedtime.html?hour=16&min=0&sec=0>. >> Everyone is welcome. >> I propose an agenda based on the discussions we had on this list. >> >> http://wiki.openstack.org/Meetings/MeteringAgenda >> Topic : schema and counter definitions >> >> * counter definitions >>* Proposed http://wiki.openstack.org/EfficientMetering#Counters >> * schema definition >>* Proposed http://wiki.openstack.org/EfficientMetering#Storage >> * discuss storage assumptions >>* the storage will store all events >>* no aggregated value is permanently stored >> * discuss API assumptions >>* the API provide a sum() function to aggregate values >>* the API may transparently store results of the sum function in a >> cache >> * discuss event collection >> * events are collected from a components when possible >>* ceilometer agent is installed on a node when the a component does >> not provide the value >>* contribute to the component instead of developping a ceilometer >> agent plugin >> * engaging discussions with core components >>* nova >>* cinder >>* glance >>* swift >>* quantum >> * open discussion >> >> For the record, the first two points used all the time but that was the >> goal of the meeting. The other points would have been nice to discuss but >> can each be turned into a mailing list thread ;-) >> >> == >> #openstack-meeting Meeting >> ====== >> >> >> Meeting started by dachary at 16:00:16 UTC. The full logs are available >> athttp://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-03-16.00.log.html >> . >> >> >> >> Meeting summary >> --- >> >> * actions from previous meetings (dachary, 16:00:36) >> * creation of the ceilometer project (dachary, 16:00:36) >> * The repository for the ceilometer project has been created >> (dachary, 16:00:36) >> * LINK: https://github.com/stackforge/ceilometer (dachary, 16:00:36) >> * and the first commit was successfully reviewed and merged today >> https://review.stackforge.org/#/c/25/ (dachary, 16:00:37) >> >> * meeting organisation (dachary, 16:01:03) >> * This is 1/5 meetings to decide the architecture of the Metering >> project https://launchpad.net/ceilometer (dachary, 16:01:03) >> * Today's focus is on the definition of the counters / meters and the >> associated schema for the storage (dachary, 16:01:03) >> * It is the conclusion of the discussions held on the mailing list and >> the goal is to make a final choice that will then be implemented. >> (dachary, 16:01:03) >> * The meeting is time boxed and there will not be enough time to >> introduce inovative ideas and research for solutions. (dachary, >> 16:01:03) >> * The debate will be about the pro and cons of the options already >> discussed on the mailing list. (dachary, 16:01:03) >> * LINK: https://lists.launchpad.net/openstack/msg10810.html (dachary, >> 16:01:03) >> >> * counter definitions (dachary, 16:02:10) >> * Proposed http://wiki.openstack.org/EfficientMetering#Counters >> (dachary, 16:02:10) >> * ACTION: dachary fix the note for net_float still talks about "number >> of floating IPs" (dachary, 16:09:18) >> * ACTION: jd___ include Number of object in Swift, Num
