Re: [openstack-dev] [ceilometer][horizon] metric-list not complete and show metrics in horizon dashboard
> From: gordon chung [mailto:g...@live.ca] > Sent: Tuesday, April 12, 2016 8:00 AM > > On 12/04/2016 7:47 AM, Safka, JaroslavX wrote: > > > * > > And my question is: How is connected the database table meter and the > command metric-list? > > i assume you mean meter-list. it uses a combination of data from meter table > and resource table[1]. this is because it lists all the meters and the > resources > associated with it. yes, it's complicated (which is why we'd recommend > Gnocchi or your own special solution) > > > Second question how I can propagate the metrics to horizon dashboard > "Resource usage"? > > (I'm able only see cpu metric from started worker image) > > horizon uses ceilometerclient to grab data so i imagine they are using some > combination of resource-list, sample-list, meter-list. as Matthias mentioned, > you probably shouldn't rely on the existing horizon interface as the hope is > to > deprecate the view in horizon since no one is really sure what it's designed > to > show. > > > > > Background: > > I'm writing plugin which connects collectd and ceilometer and I want to see > the collectd metrics in the horizon or at least in the ceilometer shell. > > is this an addition to the work that Emma did[1]? i'm assuming so, given your > locale/company. Yes, Jarek works with me. There's no Ceilometer plugin for horizon, okay, gnocchi will be just fine. We can transform from Ceilometer to Gnocchi, and get those stats into Horizon. Later, we can push directly into gnocchi, and don't need to adjust visauls. Jarek, you'll need to modify the following file to pass the Ceilometer stats to Gnocchi: https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/gnocchi_resources.yaml [1] [1] http://lists.openstack.org/pipermail/openstack-dev/2016-January/085189.html > > [1] > https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/ > impl_sqlalchemy.py#L539-L549 > [1] https://github.com/openstack/collectd-ceilometer-plugin > > -- > gord > > __ > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
> > So, metrics are grouped by the type of resource they use, and each metric > has to be listed. > > Grouping isn't a problem, but creating an exhaustive list might be, since > there are 100+ plugins [1] in collectd which can provide statistics, although > not all of these are useful, and some require extensive configuration. The > plugins each provide multiple metrics, and each metric can be duplicated for > a number of instances, examples: [2]. > > > > Collectd data is minimal: timestamp and volume, so there's little room to > find interesting meta data. > > It would be nice to see this support integrated, but it might be very > > tedious to list all the metric names and group by resource type without any > form of Do the resource definitions support wildcards? Collectd can provide > A LOT of metrics. > > > > Regards, > > Emma > > > > [1] https://collectd.org/wiki/index.php/Table_of_Plugins > > [2] https://collectd.org/wiki/index.php/Naming_schema > > gnocchi is strongly typed when compared to classical ceilometer db where > you can dump anything and everything. we don't support wildcards as is but i > believe it's something we can aim to support? > Mehdi is currently in process of implementing dynamic resources which > would give more flexiblity on what type of data we can store in Gnocchi. > i believe from ceilometer pov, we can add support to allow wildcard support > in regards to adding new metrics. > It makes sense to support wildcards, if someone is introducing a huge source of meters. I can help with that, if needed. Regards, Emma -- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
> On Thu, Feb 04 2016, Foley, Emma L wrote: > > > Question is where it should live. > > Since gnocchi is designed to be standalone, it seem like that's a potential > home for it. > > If not, it also fits in with the existing plugin. > > It it's a collectd plugin that talk statsd protocol, I'd say it should live > near > collectd, no? > If it's in Python, collectd doesn't take it. I'll house it with the existing plugin. Emma __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
> nice, do you have resource to look at this? or maybe something to add to > Gnocchi's potential backlog. existing plugin still seems useful to those who > want to use custom/proprietary storage. I should have resources for this. Question is where it should live. Since gnocchi is designed to be standalone, it seem like that's a potential home for it. If not, it also fits in with the existing plugin. Regards, Emma __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
AFAICT there's no such thing out of the box but it should be fairly straightforward to implement a StatsD writer using the collectd Python plugin. Simon [1] https://collectd.org/documentation/manpages/collectd-python.5.shtml I guess that’ll have to be the plan now: get a prototype in place and have a look at how well it does. The first one is always the most difficult, so it should be fairly quick to get this going. Regards, Emma -- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
Hi Simon, So collectd acts as a statsd server, and the metrics are aggregated and dispatched to the collectd daemon. Collectd’s write plugins then output the stats to wherever we want them to go. In order to interact with gnocchi using statsd, we require collectd to act as a statsd client and dispatch the metrics to gnocchi-statsd service. Regards, Emma From: Simon Pasquier [mailto:spasqu...@mirantis.com] Sent: Monday, February 1, 2016 9:02 AM To: OpenStack Development Mailing List (not for usage questions) ; Foley, Emma L Subject: Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin On Fri, Jan 29, 2016 at 6:30 PM, Julien Danjou mailto:jul...@danjou.info>> wrote: On Fri, Jan 29 2016, Foley, Emma L wrote: > Supporting statsd would require some more investigation, as collectd's > statsd plugin supports reading stats from the system, but not writing > them. I'm not sure what that means? https://collectd.org/wiki/index.php/Plugin:StatsD seems to indicate it can send metrics to a statsd daemon. Nope that is the opposite: collectd can act as a statsd server. The man page [1] is clearer than the collectd Wiki. Simon [1] https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_statsd __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
> So, metrics are grouped by the type of resource they use, and each metric has > to be listed. > Grouping isn't a problem, but creating an exhaustive list might be, > since there are 100+ plugins [1] in collectd which can provide > statistics, although not all of these are useful, and some require > extensive configuration. The plugins each provide multiple metrics, > and each metric can be duplicated for a number of instances, examples: [2]. > > Collectd data is minimal: timestamp and volume, so there's little room to > find interesting meta data. > It would be nice to see this support integrated, but it might be very > tedious to list all the metric names and group by resource type without any > form of Do the resource definitions support wildcards? Collectd can provide A > LOT of metrics. One also has to put into balance the upside of going through Ceilometer, as Gnocchi has direct support for statsd: http://gnocchi.xyz/statsd.html Supporting statsd would require some more investigation, as collectd's statsd plugin supports reading stats from the system, but not writing them. Also, what are the usage figures for gnocchi? How many people use it, and how easy is it to convert existing deployments to use gnocchi? I mean, if someone was upgrading, would their data be preserved? How easy is it to consume gnocchi statistics using an external system/application? I'm not against the idea, but it requires a little more consideration. Regards, Emma -- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
So, metrics are grouped by the type of resource they use, and each metric has to be listed. Grouping isn't a problem, but creating an exhaustive list might be, since there are 100+ plugins [1] in collectd which can provide statistics, although not all of these are useful, and some require extensive configuration. The plugins each provide multiple metrics, and each metric can be duplicated for a number of instances, examples: [2]. Collectd data is minimal: timestamp and volume, so there's little room to find interesting meta data. It would be nice to see this support integrated, but it might be very tedious to list all the metric names and group by resource type without any form of Do the resource definitions support wildcards? Collectd can provide A LOT of metrics. Regards, Emma [1] https://collectd.org/wiki/index.php/Table_of_Plugins [2] https://collectd.org/wiki/index.php/Naming_schema Original Message- From: gordon chung [mailto:g...@live.ca] Sent: Monday, January 25, 2016 5:12 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin you can consider ceilometer database (and api) as an open-ended model designed to capture the full fidelity of a datapoint (something useful for auditing, post processing). alternatively, gnocchi is a strongly type model which captures only required data. in the case of ceilometer -> gnocchi, the measurement data ceilometer collects is sent to gnocchi and mapped to specific resource types[1]. here we define all the resources and the metric mappings available. with regards to collectd, i'm just wondering what additional metrics are added and possibly any interesting metadata? [1] https://github.com/openstack/ceilometer/blob/master/etc/ceilometer/gnocchi_resources.yaml On 25/01/2016 9:21 AM, Foley, Emma L wrote: > I'm not overly familiar with Gnocchi, so I can't answer that off the bat, but > I would be looking for answers to the following questions: > What changes need to be made to gnocchi to accommodate regular data from > ceilometer? > Is there anything additional in Gnocchi's data model that is not part of > Ceilometer? > > Regards, > Emma > > > -Original Message- > From: gord chung [mailto:g...@live.ca] > Sent: Friday, January 22, 2016 2:41 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [telemetry][ceilometer] New project: > collectd-ceilometer-plugin > > nice! thanks Emma! > > i'm wondering if there's an additional metrics/resources we should add to > gnocchi to accommodate the data? > > On 22/01/2016 6:11 AM, Foley, Emma L wrote: >> Hi folks, >> >> A new plug-in for making collectd[1] stats available to Ceilometer [2] is >> ready for use. >> >> The collectd-ceilometer-plugin make system statistics from collectd >> available to Ceilometer. >> These additional statistics make it easier to detect faults and identify >> performance bottlenecks (among other uses). >> >> Regards, >> Emma >> >> [1] https://collectd.org/ >> [2] http://github.com/openstack/collectd-ceilometer-plugin >> >> -- >> Intel Research and Development Ireland Limited Registered in Ireland >> Registered Office: Collinstown Industrial Park, Leixlip, County >> Kildare Registered Number: 308263 >> >> >> _ >> _ OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- > gord > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
I'm not overly familiar with Gnocchi, so I can't answer that off the bat, but I would be looking for answers to the following questions: What changes need to be made to gnocchi to accommodate regular data from ceilometer? Is there anything additional in Gnocchi's data model that is not part of Ceilometer? Regards, Emma -Original Message- From: gord chung [mailto:g...@live.ca] Sent: Friday, January 22, 2016 2:41 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin nice! thanks Emma! i'm wondering if there's an additional metrics/resources we should add to gnocchi to accommodate the data? On 22/01/2016 6:11 AM, Foley, Emma L wrote: > Hi folks, > > A new plug-in for making collectd[1] stats available to Ceilometer [2] is > ready for use. > > The collectd-ceilometer-plugin make system statistics from collectd available > to Ceilometer. > These additional statistics make it easier to detect faults and identify > performance bottlenecks (among other uses). > > Regards, > Emma > > [1] https://collectd.org/ > [2] http://github.com/openstack/collectd-ceilometer-plugin > > -- > Intel Research and Development Ireland Limited Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County > Kildare Registered Number: 308263 > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [telemetry][ceilometer] New project: collectd-ceilometer-plugin
Hi folks, A new plug-in for making collectd[1] stats available to Ceilometer [2] is ready for use. The collectd-ceilometer-plugin make system statistics from collectd available to Ceilometer. These additional statistics make it easier to detect faults and identify performance bottlenecks (among other uses). Regards, Emma [1] https://collectd.org/ [2] http://github.com/openstack/collectd-ceilometer-plugin -- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-infra] Vitrage is missing in the formal OpenStack launchpad
Hi Maty, Have you set up the launchpad project? http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad Regards, Emma From: GROSZ, Maty (Maty) [mailto:maty.gr...@alcatel-lucent.com] Sent: Tuesday, November 17, 2015 8:18 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [openstack-infra] Vitrage is missing in the formal OpenStack launchpad Hello, I have lately added the new Vitrage RCA project for OpenStack. I have followed the instructions for adding a setting a new project, but for some reason, when I enters OpenStack launchpad - https://launchpad.net/openstack I don’t see the Vitrage project in the lists (for example, I don’t see any of our blueprints when I enter the launchpad blueprints: https://blueprints.launchpad.net/openstack) What do I miss? Where did I need to add Vitrage so that it would appear? Thanks, Maty __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
