Re: [openstack-dev] [oslo] Should we drop kafka driver ?
Hi all, This one [1] is used very intensively actually :) It is not merged because we cannot just upgrade python-kafka version because of Monasca. We had some talks in their channel, but no results still. I suggest the following: 1. On our side we will update this change 2. I will ping Monasca guys once more and we will proceed with discussions in the CR [1] https://review.openstack.org/#/c/332105/ Nadya On Wed, Nov 30, 2016 at 7:48 PM, Mehdi Abaakouk wrote: > > > Le 2016-11-30 17:45, Joshua Harlow a écrit : > >> IMHO, not just yet, dims and I have been trying to use this driver >> recently (for notifications only in my case) and I am more than >> willing to try to get the changes needed to get this into a healthy >> state (from my understanding dims and friends have been working >> through this as well). >> >> One of the places for gate testing that is still being worked on is >> the following: https://github.com/jd/pifpaf/pull/28 >> >> That will aid with some of the gate testing. >> > > awesome ! > > -- > Mehdi Abaakouk > mail: sil...@sileht.net > irc: sileht > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer][panko] ElasticSearch support is broken
Hi Julien, Thank you for notifying! I've assigned this bug to me. Will work on that today. Nadya On Fri, Jul 29, 2016 at 12:11 PM, Julien Danjou wrote: > Hi there, > > This is a reminder that ElasticSearch support for event in Ceilometer > (now Panko) is broken for more than a month¹. If nothing is done by the > time we need to release RC1, as I don't see the point of releasing > broken code, I might suggest that we remove this driver altogether. > > If you care, I'd suggest to fix it before it's too late. > > ¹ https://bugs.launchpad.net/ceilometer/+bug/1596988 > > -- > Julien Danjou > -- Free Software hacker > -- https://julien.danjou.info > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack] [Ceilometer][Architecture] Transformers in Kilo vs Liberty(and Mitaka)
Hi Gordon, I'd like to add some clarifications and comments. this is not entirely accurate pre-polling change, the polling agents > publish one message per sample. not the polling agents publish one > message per interval (multiple samples). Looks like there is some misunderstanding here. In the code, there is "batch_polled_samples" option. You can switch it off and get the result you described, but it's True by default. See https://github.com/openstack/ceilometer/blob/master/ceilometer/agent/manager.py#L205-L211 . You wrote: > the polling change is not related to coordination work in notification. > the coordination work was to handle HA / multiple notification agents. > regardless polling change, this must exist. and > transformers are already optional. they can be removed from > pipeline.yaml if not required (and thus coordination can be disabled). So, coordination is needed only to support transformations. Polling change does relate to this because it has brought additional transformations on notification agent side. I suggest to pay attention to the existing use cases. In real life, people use transformers for polling-based metrics only. The most important use case for transformation is Heat autoscaling. It usually based on cpu_util. Before Liberty, we were able not to use coordination for notification agent to support the autoscaling usecase. In Liberty we cannot support it without Redis. Now "transformers are already optional", that's true. But I think it's better to add some restrictions like "we don't support transformations for notifications" and have transformers optional on polling-agent only instead of introducing such a comprehensive coordination. > IPC is one of the > standard use cases for message queues. the concept of using queues to > pass around and distribute work is essentially what it's designed for. > if rabbit or any message queue service can't provide this function, it > does worry me. I see your point here, but Ceilometer aims to take care of the OpenStack, monitor it's state. Now it is known as a "Rabbit killer". We cannot ignore that if we want anybody uses Ceilometer. Also, I'd like to copy-paste Chris's ideas from the previous message: Are the options the following? > * Do what you suggest and pull transformers back into the pollsters. Basically revert the change. I think this is the wrong long term > solution but might be the best option if there's nobody to do the > other options. > * Implement a pollster.yaml for use by the pollsters and consider > pipeline.yaml as the canonical file for the notification agents as > there's where the actual _pipelines_ are. Somewhere in there kill > interval as a concept on pipeline side. > This of course doesn't address the messaging complexity. I admit > that I don't understand all the issues there but it often feels > like we are doing that aspect of things completely wrong, so I > would hope that before we change things there we consider all the > options. I think that two types of agents should have two different pipeline descriptions, but I still think that "pipeline" should be described and fully applied on the both types of agents. On polling ones it should be the same as it is now, on notification: remove interval and refuse from transformations at all. Chris, I see your point about "long term", but I'm afraid that "long term" may not happen... > What else? > One probably crazy idea: What about figuring out the desired end-meters > of common transformations and making them into dedicated pollsters? > Encapsulating that transformation not at the level of the polling > manager but at the individual pollster. Your "crazy idea" may work at least for restoring autoscaling functionality indeed. Thanks, Nadya On Wed, Apr 13, 2016 at 9:25 PM, gordon chung wrote: > hi Nadya, > > copy/pasting full original message with comments inline to clarify some > comments. > > i think a lot of the confusion is because we use pipeline.yaml across > both polling and notification agents when really it only applies to > latter. just an fyi, we've had an open work item to create a > polling.yaml file... just the issue of 'resources'. > > > Hello colleagues, > > > > I'd like to discuss one question with you. Perhaps, you remember that > > in Liberty we decided to get rid of transformers on polling agents [1]. > I'd > > like to describe several issues we are facing now because of this > decision. > > 1. pipeline.yaml inconsistency. > > Ceilometer pipeline consists from the two basic things: source and > > sink. In source, we describe how to get data, in sink - how to deal with > > the data. After the refactoring described in [1], on polling agents we > > apply only "source" definition, on notification agents we apply only > "sink" > > one. It causes the problems described in the mailing thread [2]: the > "pipe" > > concept is actually broken. To make it work more or less correctly, the > > user should c
Re: [openstack-dev] [telemetry] stepping down as PTL
Gordon, Thank you very much for your work! I was always amazed by your patience and proficiency. It seems to me that you answered billion questions during working on Telemetry. I wish you all the best in the future, you are awesome :)! Thanks, Nadya On Mon, Mar 14, 2016 at 12:34 PM, Julien Danjou wrote: > On Fri, Mar 11 2016, gordon chung wrote: > > > as the PTL nominations are open, i just want to note that i won't be > > running again as the Telemetry PTL for Newton cycle. > > > > i've thoroughly enjoyed my time as PTL which afforded me the opportunity > > to work with and learn from some great individuals who share the same > > passion to collaborate openly. i have the utmost confidence that the > > projects will continue to grow and become more refined under the next > > leadership. > > > > personal thanks to everyone in Aodh, Ceilometer and Gnocchi communities > > as well as the projects that provided valuable feedback by collaborating > > with us. i thank you for sharing in the decision making so i could > > spread the blame around. i also thank you for reading through terribly > > written messages that make you question whether the shift keys on all my > > keyboards are broken. > > I'd like to join others and thank you for your amazing work as a PTL. > > You've led the community in an open way that allowed it to share the > leadership and be one of the most thrilling group that I've been able to > contribute to in OpenStack. > > Cheers, > -- > Julien Danjou > // Free Software hacker > // https://julien.danjou.info > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [aodh] HBase gate job
Hi folks, Sorry I missed the letter. I'll take a look on this. As I understand, it's enough just to run functional tests to see the issue, right? Nadya On Tue, Nov 17, 2015 at 4:48 PM, gord chung wrote: > let's change this gate to experimental for now. > > On 17/11/15 05:10 AM, Julien Danjou wrote: > > Hi, > > So we recently decided to run the functional tests for the HBase driver > and enabled the gate job. It turned out the gate job didn't worked, so I > tried to fix it. Now it's almost fixed, and it run the functional tests > and Gabbi tests against Aodh with HBase as a back-end: > > https://review.openstack.org/#/c/245585/ > > Obviously, as you know this is not a real HBase back-end, but a > in-memory backend that is provided inside Aodh. Now, it turns out that > this driver or this in-memory backend has a bug, as the Gabbi tests > fail. > > I don't have time nor interest to fix that, so the way I see it it's > either: > 1. Someone stands up, determines if the issue is in the driver or the >in-memory implementation and fix it > 2. Nobody cares and we drop HBase gate and deprecates the driver > > If nobody stands up in the next week, I'll start working on 2. > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > -- > gord > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] Aodh has been imported, next steps
I'm afraid user experience will change because of API. Do we have a plan about it? Will we interact with Aodh through ceilometer-api first? Or make user to go to aodh-api service? So I agree with Gordon that code-cleanup is more preferred option because we can't maintain two version simultaneously. But we need to think more about end users: is it appropriate just remove options from ceilometer-api? On Mon, Jun 29, 2015 at 10:47 PM, gordon chung wrote: > > > On 29/06/2015 11:40 AM, Chris Dent wrote: > >> On Mon, 29 Jun 2015, Julien Danjou wrote: >> >> Hi team, >>> >>> Aodh has been imported and is now available at: >>> >>> https://git.openstack.org/cgit/openstack/aodh/ >>> >> >> woot! >> >> I'm pretty clear about the next steps for Aodh and what we need to >>> build, but something is still not clear to me. Do we go ahead and bite >>> the bullet and remove ceilometer-alarming from ceilometer in Liberty? >>> >> > i think we should follow up with the packagers. if i understand correctly, > the location of the code is not known from a user pov, it's the packagers > that build the appropriate packages for them to use. > > if from packagers pov, they just need to work against Aodh, then i would > lean more to removing alarming from Ceilometer repo asap to avoid > maintaining duplicate code bases and the eventual diversion of the two. > > >> This is the big question and is one of the things listed on the >> potential agenda for the mid-cylce. When we do the splits do we >> deprecate or delete the old code. Given the high chance of us >> missing some of potential issues it seems like hasing it some before >> the mid-cylce is a good idea. >> >> The two big overarching issues (that inform a lot of the details) >> that I'm aware of are: >> >> * If we delete then we need to make sure we're working hand in hand >> with all of: downstream packagers, tempest, grenade, devstack, >> etc. >> >> * If we deprecate will people bother to use the new stuff? >> > > i would think/hope the experience from end user doesn't actually change. > ie. all the same packaged services remain. > > >> I'm sure there are plenty of others. >> >> > -- > gord > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] virtual mid-cycle planning
Chris, What is the workflow of adding a new topic? I suggest to add it to both [1] and [2]. Also it may be useful to add more items for each topic like description, open questions, leader. It may help to understand what should be discussed and who is an initiator. What do you think? [1] https://etherpad.openstack.org/p/ceilometer-liberty-midcycle-agenda [2] https://etherpad.openstack.org/p/ceilometer-liberty-midcycle. On Fri, Jun 26, 2015 at 7:09 PM, Chris Dent wrote: > On Fri, 26 Jun 2015, Chris Dent wrote: > > Please provide your input soon. We will close the voting at the end >> of Wednesday, July 3rd and will be posting a proposed schedule soon >> thereafter. Note that due to time constraints the mid-cycle could >> happen as soon as the following Monday, July 6th. >> > > Sigh, I can't read my calendar. The voting will stop at the end > of Wednesday, July 1st. > > > -- > Chris Dent tw:@anticdent freenode:cdent > https://tank.peermore.com/tanks/cdent > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev