Re: [openstack-dev] [oslo] Should we drop kafka driver ?

[Nadya Shakhat]

Hi all,

This one [1] is used very intensively actually :) It is not merged because
we cannot just upgrade python-kafka version because of Monasca. We had some
talks in their channel, but no results still. I suggest the following:
1. On our side we will update this change
2. I will ping Monasca guys once more and we will proceed with discussions
in the CR

[1] https://review.openstack.org/#/c/332105/

Nadya

On Wed, Nov 30, 2016 at 7:48 PM, Mehdi Abaakouk  wrote:

>
>
> Le 2016-11-30 17:45, Joshua Harlow a écrit :
>
>> IMHO, not just yet, dims and I have been trying to use this driver
>> recently (for notifications only in my case) and I am more than
>> willing to try to get the changes needed to get this into a healthy
>> state (from my understanding dims and friends have been working
>> through this as well).
>>
>> One of the places for gate testing that is still being worked on is
>> the following: https://github.com/jd/pifpaf/pull/28
>>
>> That will aid with some of the gate testing.
>>
>
> awesome !
>
> --
> Mehdi Abaakouk
> mail: sil...@sileht.net
> irc: sileht
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer][panko] ElasticSearch support is broken

[Nadya Shakhat]

Hi Julien,

Thank you for notifying! I've assigned this bug to me. Will work on that
today.

Nadya

On Fri, Jul 29, 2016 at 12:11 PM, Julien Danjou  wrote:

> Hi there,
>
> This is a reminder that ElasticSearch support for event in Ceilometer
> (now Panko) is broken for more than a month¹. If nothing is done by the
> time we need to release RC1, as I don't see the point of releasing
> broken code, I might suggest that we remove this driver altogether.
>
> If you care, I'd suggest to fix it before it's too late.
>
> ¹  https://bugs.launchpad.net/ceilometer/+bug/1596988
>
> --
> Julien Danjou
> -- Free Software hacker
> -- https://julien.danjou.info
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] [Ceilometer][Architecture] Transformers in Kilo vs Liberty(and Mitaka)

[Nadya Shakhat]

Hi Gordon,

I'd like to add some clarifications and comments.

this is not entirely accurate pre-polling change, the polling agents
> publish one message per sample. not the polling agents publish one
> message per interval (multiple samples).

Looks like there is some misunderstanding here. In the code, there is
"batch_polled_samples" option. You can switch it off and get the result you
described, but it's True by default.  See
https://github.com/openstack/ceilometer/blob/master/ceilometer/agent/manager.py#L205-L211
.

You wrote:

> the polling change is not related to coordination work in notification.
> the coordination work was to handle HA / multiple notification agents.
> regardless polling change, this must exist.

and

> transformers are already optional. they can be removed from
> pipeline.yaml if not required (and thus coordination can be disabled).


So, coordination is needed only to support transformations. Polling change
does relate to this because it has brought additional transformations on
notification agent side. I suggest to pay attention to the existing use
cases. In real life, people use transformers for polling-based metrics
only. The most important use case for transformation is Heat autoscaling.
It usually based on cpu_util. Before Liberty, we were able not to use
coordination for notification agent to support the autoscaling usecase. In
Liberty we cannot support it without Redis. Now "transformers are already
optional", that's true. But I think it's better to add some restrictions
like "we don't support transformations for notifications" and have
transformers optional on polling-agent only instead of introducing such a
comprehensive coordination.

> IPC is one of the
> standard use cases for message queues. the concept of using queues to
> pass around and distribute work is essentially what it's designed for.
> if rabbit or any message queue service can't provide this function, it
> does worry me.


I see your point here, but Ceilometer aims to take care of the OpenStack,
monitor it's state. Now it is known as a "Rabbit killer". We cannot ignore
that if we want anybody uses Ceilometer.


Also, I'd like to copy-paste Chris's ideas from the previous message:

Are the options the following?
> * Do what you suggest and pull transformers back into the pollsters.

  Basically revert the change. I think this is the wrong long term
>   solution but might be the best option if there's nobody to do the
>   other options.
> * Implement a pollster.yaml for use by the pollsters and consider
>   pipeline.yaml as the canonical file for the notification agents as
>   there's where the actual _pipelines_ are. Somewhere in there kill
>   interval as a concept on pipeline side.
>   This of course doesn't address the messaging complexity. I admit
>   that I don't understand all the issues there but it often feels
>   like we are doing that aspect of things completely wrong, so I
>   would hope that before we change things there we consider all the
>   options.

I think that two types of agents should have two different pipeline
descriptions, but I still think that "pipeline" should be described and
fully applied on the both types of agents. On polling ones it should be the
same as it is now, on notification: remove interval and refuse from
transformations at all. Chris, I see your point about "long term", but I'm
afraid that "long term" may not happen...


> What else?
> One probably crazy idea: What about figuring out the desired end-meters
> of common transformations and making them into dedicated pollsters?
> Encapsulating that transformation not at the level of the polling
> manager but at the individual pollster.


Your "crazy idea" may work at least for restoring autoscaling functionality
indeed.

Thanks,
Nadya

On Wed, Apr 13, 2016 at 9:25 PM, gordon chung  wrote:

> hi Nadya,
>
> copy/pasting full original message with comments inline to clarify some
> comments.
>
> i think a lot of the confusion is because we use pipeline.yaml across
> both polling and notification agents when really it only applies to
> latter. just an fyi, we've had an open work item to create a
> polling.yaml file... just the issue of 'resources'.
>
> > Hello colleagues,
> >
> > I'd like to discuss one question with you. Perhaps, you remember that
> > in Liberty we decided to get rid of transformers on polling agents [1].
> I'd
> > like to describe several issues we are facing now because of this
> decision.
> > 1. pipeline.yaml inconsistency.
> > Ceilometer pipeline consists from the two basic things: source and
> > sink. In source, we describe how to get data, in sink - how to deal with
> > the data. After the refactoring described in [1], on polling agents we
> > apply only "source" definition, on notification agents we apply only
> "sink"
> > one. It causes the problems described in the mailing thread [2]: the
> "pipe"
> > concept is actually broken. To make it work more or less correctly, the
> 

Re: [openstack-dev] [telemetry] stepping down as PTL

[Nadya Shakhat]

Gordon,
Thank you very much for your work! I was always amazed by your patience and
proficiency. It seems to me that you answered billion questions during
working on Telemetry. I wish you all the best in the future, you are
awesome :)!

Thanks,
Nadya

On Mon, Mar 14, 2016 at 12:34 PM, Julien Danjou  wrote:

> On Fri, Mar 11 2016, gordon chung wrote:
>
> > as the PTL nominations are open, i just want to note that i won't be
> > running again as the Telemetry PTL for Newton cycle.
> >
> > i've thoroughly enjoyed my time as PTL which afforded me the opportunity
> > to work with and learn from some great individuals who share the same
> > passion to collaborate openly. i have the utmost confidence that the
> > projects will continue to grow and become more refined under the next
> > leadership.
> >
> > personal thanks to everyone in Aodh, Ceilometer and Gnocchi communities
> > as well as the projects that provided valuable feedback by collaborating
> > with us. i thank you for sharing in the decision making so i could
> > spread the blame around. i also thank you for reading through terribly
> > written messages that make you question whether the shift keys on all my
> > keyboards are broken.
>
> I'd like to join others and thank you for your amazing work as a PTL.
>
> You've led the community in an open way that allowed it to share the
> leadership and be one of the most thrilling group that I've been able to
> contribute to in OpenStack.
>
> Cheers,
> --
> Julien Danjou
> // Free Software hacker
> // https://julien.danjou.info
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [aodh] HBase gate job

[Nadya Shakhat]

Hi folks,

Sorry I missed the letter. I'll take a look on this. As I understand, it's
enough just to run functional tests to see the issue, right?

Nadya

On Tue, Nov 17, 2015 at 4:48 PM, gord chung  wrote:

> let's change this gate to experimental for now.
>
> On 17/11/15 05:10 AM, Julien Danjou wrote:
>
> Hi,
>
> So we recently decided to run the functional tests for the HBase driver
> and enabled the gate job. It turned out the gate job didn't worked, so I
> tried to fix it. Now it's almost fixed, and it run the functional tests
> and Gabbi tests against Aodh with HBase as a back-end:
>
>   https://review.openstack.org/#/c/245585/
>
> Obviously, as you know this is not a real HBase back-end, but a
> in-memory backend that is provided inside Aodh. Now, it turns out that
> this driver or this in-memory backend has a bug, as the Gabbi tests
> fail.
>
> I don't have time nor interest to fix that, so the way I see it it's
> either:
> 1. Someone stands up, determines if the issue is in the driver or the
>in-memory implementation and fix it
> 2. Nobody cares and we drop HBase gate and deprecates the driver
>
> If nobody stands up in the next week, I'll start working on 2.
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> gord
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] Aodh has been imported, next steps

[Nadya Shakhat]

I'm afraid user experience will change because of API. Do we have a plan
about it?
Will we interact with Aodh through ceilometer-api first? Or make user to go
to aodh-api service?
So I agree with Gordon that code-cleanup is more preferred option because
we can't maintain two version simultaneously. But we need to think more
about end users: is it appropriate just remove options from
ceilometer-api?

On Mon, Jun 29, 2015 at 10:47 PM, gordon chung g...@live.ca wrote:



 On 29/06/2015 11:40 AM, Chris Dent wrote:

 On Mon, 29 Jun 2015, Julien Danjou wrote:

  Hi team,

 Aodh has been imported and is now available at:

  https://git.openstack.org/cgit/openstack/aodh/


 woot!

  I'm pretty clear about the next steps for Aodh and what we need to
 build, but something is still not clear to me. Do we go ahead and bite
 the bullet and remove ceilometer-alarming from ceilometer in Liberty?


 i think we should follow up with the packagers. if i understand correctly,
 the location of the code is not known from a user pov, it's the packagers
 that build the appropriate packages for them to use.

 if from packagers pov, they just need to work against Aodh, then i would
 lean more to removing alarming from Ceilometer repo asap to avoid
 maintaining duplicate code bases and the eventual diversion of the two.


 This is the big question and is one of the things listed on the
 potential agenda for the mid-cylce. When we do the splits do we
 deprecate or delete the old code. Given the high chance of us
 missing some of potential issues it seems like hasing it some before
 the mid-cylce is a good idea.

 The two big overarching issues (that inform a lot of the details)
 that I'm aware of are:

 * If we delete then we need to make sure we're working hand in hand
   with all of: downstream packagers, tempest, grenade, devstack,
   etc.

 * If we deprecate will people bother to use the new stuff?


 i would think/hope the experience from end user doesn't actually change.
 ie. all the same packaged services remain.


 I'm sure there are plenty of others.


 --
 gord



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] virtual mid-cycle planning

[Nadya Shakhat]

Chris,
What is the workflow of adding a new topic? I suggest to add it to both [1]
and [2]. Also it may be useful to add more items for each topic like
description, open questions, leader. It may help to understand what should
be discussed and who is an initiator. What do you think?

[1] https://etherpad.openstack.org/p/ceilometer-liberty-midcycle-agenda
[2] https://etherpad.openstack.org/p/ceilometer-liberty-midcycle.

On Fri, Jun 26, 2015 at 7:09 PM, Chris Dent chd...@redhat.com wrote:

 On Fri, 26 Jun 2015, Chris Dent wrote:

  Please provide your input soon. We will close the voting at the end
 of Wednesday, July 3rd and will be posting a proposed schedule soon
 thereafter. Note that due to time constraints the mid-cycle could
 happen as soon as the following Monday, July 6th.


 Sigh, I can't read my calendar. The voting will stop at the end
 of Wednesday, July 1st.


 --
 Chris Dent tw:@anticdent freenode:cdent
 https://tank.peermore.com/tanks/cdent

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev