Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Lu, Lianhao]

I know customers are using the 'pushing sample API', but don’t know whether 
they're applying transformer to those data or not.

-Lianhao Lu

> -Original Message-
> From: Julien Danjou [mailto:jul...@danjou.info]
> Sent: Wednesday, October 05, 2016 12:35 AM
> To: gordon chung
> Cc: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [telemetry] Deprecating the Ceilometer
> API
> 
> On Tue, Oct 04 2016, gordon chung wrote:
> 
> > so one thing we probably do need to keep is the ability push samples
> > (and events?). i know previously people were actually using this
> feature.
> 
> That's debatable.
> 
> Pushing events is IMHO out of scope of Ceilometer – it's related to
> oslo.messaging notification at this point. So if we want to allow that via
> an API endpoint, we should build one in OpenStack over
> oslo.messaging. Good idea or not, I don't know.
> 
> Pushing samples is probably doable with some kind of small API, but I
> am not sure it's worth anything. You could still directly push the data to
> Gnocchi and save some you some load. Unless you have transformers
> to apply, but… really?
> 
> --
> Julien Danjou
> /* Free Software hacker
>https://julien.danjou.info */
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Yarko Tymciurak]

On Tue, Oct 4, 2016 at 11:22 AM, gordon chung  wrote:

>
>
> On 04/10/2016 12:04 PM, Yarko Tymciurak wrote:
> > I won't be at Barcelona, but would like to participate in the telemetry
> > sessions, and particularly this one.
> > Are there plans to enable remote participants (beyond just etherpad),
> >  i.e. hangouts or skype, or something similar?
>
> we can try this. i know some other projects have done this. hangouts
> probably easiest choice (assuming we don't have more than 10 people)?
>
> i would definitely raise topics beforehand though. especially if we have
> morning sessions. i tried waking up for Europe time last year for
> virtual midcycle -- it sucks waking up at 3am.
>

Sounds good - sucks even more @ 2AM...

I'm central time (San Antonio), so 7 hours difference:
09AM - 05PM Barcelona ==
02AM - 10AM San Antonio (CDT)

Note:
01PM == 6AM
so, participation will be simplest for me for after lunch sessions.

Regardless, I plan to participate.
(If I really need to, I'll try sleeping 6PM - 2AM)

Thanks,
Yarko


> cheers,
> --
> gord
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Mehdi Abaakouk]

Le 2016-10-04 18:09, gordon chung a écrit :

On 04/10/2016 11:58 AM, Tim Bell wrote:
What would be the impact for Heat users who are using the Ceilometer 
scaling in their templates?


Tim


pretty big. :/


The use-case itself is still supported.

Using Ceilometer alarming or Aodh alarming is transparent from the Heat 
template point of view, Heat already does the API calls to endpoint 
found in Keystone.


For the storage, Heat users can move their storage to Gnocchi and 
updates their templates to create in Aodh, Gnocchi alarms instead of 
legacy Ceilometer alarms.


I agree with gordc, this is not a light change. But well the old 
Ceilometer storage don't work at scale, each alarm evaluation take a lot 
of times and CPU to retrieve statistics from the old storage, while it's 
very quick when Gnocchi is used.


Keeping the old storage system for the autoscaling use-case doesn't make 
sense to me.


Also we have a integration gating job that tests the 
Heat+Ceilometer+Aodh+Gnocchi since two cycles. While the previous/legacy 
Ceilometer scaling system never had functional/integration tests.



--
Mehdi Abaakouk
mail: sil...@sileht.net
irc: sileht

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[gordon chung]

On 04/10/2016 12:35 PM, Julien Danjou wrote:
> On Tue, Oct 04 2016, gordon chung wrote:
>
>> so one thing we probably do need to keep is the ability push samples
>> (and events?). i know previously people were actually using this feature.
>
> That's debatable.
>
> Pushing events is IMHO out of scope of Ceilometer – it's related to
> oslo.messaging notification at this point. So if we want to allow that
> via an API endpoint, we should build one in OpenStack over
> oslo.messaging. Good idea or not, I don't know.
>
> Pushing samples is probably doable with some kind of small API, but I am
> not sure it's worth anything. You could still directly push the data to
> Gnocchi and save some you some load. Unless you have transformers to
> apply, but… really?

i don't know reasoning but agreed, it's more useful if available 
generically from something else.

i'll throw another idea out there, this relates to what stevelle 
suggested (i think it was him). an api to see live view of services: 
amount of messages it's processing at moment, queue sizes, what 
pollsters are enabled, maybe be able to dynamically load/remove 
pollsters, agent state. if i was not lazy i would look at other pipeline 
services right to see what they provide as 'API'.

cheers,
-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Hague, Darren]

https://github.com/sapcc/openstack-kube - Ceilometer is not yet mentioned in 
the README but it's in there.

Bear in mind this is currently work in progress, and is our team sharing the 
work we are doing in building up OpenStack on Kubernetes - it's not intended to 
be a fully-documented or supported release of any kind, but it serves as a 
real-world implementation that others can use as a starting point.

Here's my boss talking about it and showing a demo: 
https://www.youtube.com/watch?v=4gyeixJLabo 

Cheers,
Darren

-Original Message-
From: Jay Pipes [mailto:jaypi...@gmail.com] 
Sent: 04 October 2016 17:12
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

> we removed the API and storage containers from our Kubernetes deployment of 
> Ceilometer some time ago.
> We use the collector's HTTP Dispatcher to send events and metrics to our 
> billing database.

Cool. Are you going to share the code for your k8s deployment of Ceilometer?

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Julien Danjou]

On Tue, Oct 04 2016, gordon chung wrote:

> so one thing we probably do need to keep is the ability push samples 
> (and events?). i know previously people were actually using this feature.

That's debatable.

Pushing events is IMHO out of scope of Ceilometer – it's related to
oslo.messaging notification at this point. So if we want to allow that
via an API endpoint, we should build one in OpenStack over
oslo.messaging. Good idea or not, I don't know.

Pushing samples is probably doable with some kind of small API, but I am
not sure it's worth anything. You could still directly push the data to
Gnocchi and save some you some load. Unless you have transformers to
apply, but… really?

-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Julien Danjou]

On Tue, Oct 04 2016, gordon chung wrote:

> i think the main issue right now is, no one is maintaining the storage 
> solutions (for a while now if we're talking about metric storage) but 
> people are using it. many people have figured it out and bailed on the 
> storage for their own solutions or Gnocchi. now how do we deal with 
> those who haven't? we definitely don't want to handle any new people on 
> this API.
>
> maybe we can start with just deleting the API and storage install docs.

Yes, I already proposed a patch that does not set the default to
anything with Ocata:

  https://review.openstack.org/#/c/373329/

So that helps not having "wrong" default, at least.

-- 
Julien Danjou
// Free Software hacker
// https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[gordon chung]

On 04/10/2016 12:04 PM, Yarko Tymciurak wrote:
> I won't be at Barcelona, but would like to participate in the telemetry
> sessions, and particularly this one.
> Are there plans to enable remote participants (beyond just etherpad),
>  i.e. hangouts or skype, or something similar?

we can try this. i know some other projects have done this. hangouts 
probably easiest choice (assuming we don't have more than 10 people)?

i would definitely raise topics beforehand though. especially if we have 
morning sessions. i tried waking up for Europe time last year for 
virtual midcycle -- it sucks waking up at 3am.

cheers,
-- 
gord

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[gordon chung]

so one thing we probably do need to keep is the ability push samples 
(and events?). i know previously people were actually using this feature.

On 04/10/2016 11:27 AM, Julien Danjou wrote:
> Hi,
>
> Considering the split of Ceilometer in subprojects (Aodh and Panko)
> during those last cycles, and the increasing usage of Gnocchi, I am
> starting to wonder if it makes sense to maintain the legacy Ceilometer
> API.
>
> As of today, it's still barely useful, unusable at large scale, and
> barely maintained.
>
> I've proposed a session¹ about that for the next Barcelona summit, but
> feel free to share your thoughts on this thread before.
>
> Cheers,
>
> ¹  https://etherpad.openstack.org/p/ocata-telemetry-summit-planning
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

-- 
gord

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Jay Pipes]

On 10/04/2016 11:53 AM, Hague, Darren wrote:

On Tue, 4 Oct 2016, Chris Dent wrote:
On Tue, 4 Oct 2016, Julien Danjou wrote:



Considering the split of Ceilometer in subprojects (Aodh and Panko)
during those last cycles, and the increasing usage of Gnocchi, I am
starting to wonder if it makes sense to maintain the legacy Ceilometer
API.


No surprise, as I've been saying this for a long time, but yeah, I think
the API and storage should be deprecated.

The data gathering (and to some extent transforming) parts are the only
parts of ceilometer that are particularly unique so it would be good
for those to be preserved.


That works for me - we removed the API and storage containers from our 
Kubernetes deployment of Ceilometer some time ago.
We use the collector's HTTP Dispatcher to send events and metrics to our 
billing database.


Cool. Are you going to share the code for your k8s deployment of Ceilometer?

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[gordon chung]

On 04/10/2016 11:58 AM, Tim Bell wrote:
> What would be the impact for Heat users who are using the Ceilometer scaling 
> in their templates?
>
> Tim

pretty big. :/

i don't really have anything else to add to this answer. jd__, add it to 
discussion :P

i think the main issue right now is, no one is maintaining the storage 
solutions (for a while now if we're talking about metric storage) but 
people are using it. many people have figured it out and bailed on the 
storage for their own solutions or Gnocchi. now how do we deal with 
those who haven't? we definitely don't want to handle any new people on 
this API.

maybe we can start with just deleting the API and storage install docs.

cheers,
-- 
gord
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Yarko Tymciurak]

On Tue, Oct 4, 2016 at 10:27 AM, Julien Danjou  wrote:

> Hi,
>
> Considering the split of Ceilometer in subprojects (Aodh and Panko)
> during those last cycles, and the increasing usage of Gnocchi, I am
> starting to wonder if it makes sense to maintain the legacy Ceilometer
> API.
>
> As of today, it's still barely useful, unusable at large scale, and
> barely maintained.
>
> I've proposed a session¹ about that for the next Barcelona summit, but
> feel free to share your thoughts on this thread before.
>

Hi,

I think this is inline with what I intuitively suggested at Austin Summit
(ie. about ceilometer doing too much, should be just a queue or transport
channel manager).

I won't be at Barcelona, but would like to participate in the telemetry
sessions, and particularly this one.
Are there plans to enable remote participants (beyond just etherpad),  i.e.
hangouts or skype, or something similar?

Thanks,
Yarko


>
> Cheers,
>
> ¹  https://etherpad.openstack.org/p/ocata-telemetry-summit-planning
>
> --
> Julien Danjou
> -- Free Software hacker
> -- https://julien.danjou.info
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Tim Bell]

> On 4 Oct 2016, at 17:36, Chris Dent  wrote:
> 
> On Tue, 4 Oct 2016, Julien Danjou wrote:
> 
>> Considering the split of Ceilometer in subprojects (Aodh and Panko)
>> during those last cycles, and the increasing usage of Gnocchi, I am
>> starting to wonder if it makes sense to maintain the legacy Ceilometer
>> API.
> 
> No surprise, as I've been saying this for a long time, but yeah, I think
> the API and storage should be deprecated. I think at some of the mid-
> cycles and summits we've had discussions about the parts of ceilometer
> that should be preserved, including any pollsters for which a
> notification does not or cannot exist.
> 
> The data gathering (and to some extent transforming) parts are the only
> parts of ceilometer that are particularly unique so it would be good
> for those to be preserved.
> 
> 

What would be the impact for Heat users who are using the Ceilometer scaling in 
their templates?

Tim

> -- 
> Chris Dent   ┬─┬ノ( º _ ºノ)https://anticdent.org/
> freenode: cdent tw: 
> @anticdent__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Hague, Darren]

On Tue, 4 Oct 2016, Chris Dent wrote:
On Tue, 4 Oct 2016, Julien Danjou wrote:
> 
> > Considering the split of Ceilometer in subprojects (Aodh and Panko)
> > during those last cycles, and the increasing usage of Gnocchi, I am
> > starting to wonder if it makes sense to maintain the legacy Ceilometer
> > API.
> 
> No surprise, as I've been saying this for a long time, but yeah, I think
> the API and storage should be deprecated. 
> 
> The data gathering (and to some extent transforming) parts are the only
> parts of ceilometer that are particularly unique so it would be good
> for those to be preserved.

That works for me - we removed the API and storage containers from our 
Kubernetes deployment of Ceilometer some time ago.
We use the collector's HTTP Dispatcher to send events and metrics to our 
billing database.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] Deprecating the Ceilometer API

[Chris Dent]

On Tue, 4 Oct 2016, Julien Danjou wrote:


Considering the split of Ceilometer in subprojects (Aodh and Panko)
during those last cycles, and the increasing usage of Gnocchi, I am
starting to wonder if it makes sense to maintain the legacy Ceilometer
API.


No surprise, as I've been saying this for a long time, but yeah, I think
the API and storage should be deprecated. I think at some of the mid-
cycles and summits we've had discussions about the parts of ceilometer
that should be preserved, including any pollsters for which a
notification does not or cannot exist.

The data gathering (and to some extent transforming) parts are the only
parts of ceilometer that are particularly unique so it would be good
for those to be preserved.


--
Chris Dent   ┬─┬ノ( º _ ºノ)https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev