date:20160920

[openstack-dev] [AODH] event-alarm timeout discussion

2016-09-20 Thread Zhai, Edwin


All,

I'd like make some clarification for the event-alarm timeout design as many of 
you have some misunderstanding here. Pls. correct me if any mistakes.


I realized that there are 2 different things, but we mix them sometime:
1. event-timeout-alarm
This is one new type of alarm that bracket *.start and *.end events and get 
alarmed when receive *.start but no *.end in timeout. This new alarm handles one 
type of events/actions, e.g. create one alarm for instance creation, then all 
instances created in future will be handled by this alarm. This is not for real 
time, so it's acceptable that user know one instance creation failure in 5 mins.


This new type of alarm can be implemented by one worker to check the DB 
periodically to do the statistic work. That is, new evaluator works in 'polling' 
mode, something like threshold alarm evaluator.


One BP is @
https://review.openstack.org/#/c/199005/

2. event-alarm timeout
This is one new feature for _existed_ event-alarm evaluator. One alarm becomes 
'UNALARM' when not receive desire event in timeout. This feature just handles 
one specific event, e.g create one alarm for instance ABC's XYZ operation with 
5s, then user is notified in 5s immediately if no XYZ.done event comes. If want 
check for another instance, we need create another alarm.


This is used in telco scenario, where operator want know if operation failure in 
real time.


My patch(https://review.openstack.org/#/c/272028/) is for this purpose only, but 
I feel many guys mistaken them(sometimes even me) as they looks similar. So my 
question is: Do you think this telco usage model of event-alarm timeout is 
valid? If not, we can avoid discussing its implementation and ignore following.



=== event-alarm timeout implementation =
As it's for event-alarm, we need keep it as event-driven. Furthermore, for quick 
response, we need use event for timeout handling. Periodic worker can't meet 
real time requirement.


Separated queue for 'alarm.timeout.end'(indicates timeout expire) leads tricky 
race condition.  e.g.  'XYZ.done' comes in queue1, and 'alarm.timeout.end' comes 
in queue2, so that they are handled in parallel way:


1. In queue1, 'XYZ.done' is checking against alarm(current UNKNOWN), and will be 
set ALARM in next step.
2. In queue2, 'alarm.timeout.end' is checking against same alarm(current 
UNKNOWN), and will be set to OK(UNALARM) in next step.

3. In qeueu1, alarm transition happen: UNKNOWN => ALARM
4. In queue2, another alarm transition happen: ALARM =>OK(UNALARM)

So this alarm has bogus transition: UNKNOWN=>ALARM=>UNALARM, and tells the user: 
required event came, then no required event came;


If put all events in one queue, evaluator handles them one by one(low level oslo 
mesg should be multi-threaded) so that second event would see alarm state as not 
UNKNOWN, and give up its transition.  As Gordc said, it's slow. But only very 
small part of the event-alarm need timeout handling, as it's only for telco 
usage model.


One possible improvement as JD pointed out is to avoid so many spawned thread. 
We can just create one thread inside evaluator, and ask this thread handle all 
timeout requests from evaluator. Is it acceptable for event-alarm timeout 
solution?



Best Rgds,
Edwin

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][stable/liberty] Backport impasse: "virt: set address space & CPU time limits when running qemu-img"

2016-09-20 Thread Tony Breeds

On Tue, Sep 20, 2016 at 11:57:26AM +0100, Daniel P. Berrange wrote:
> On Tue, Sep 20, 2016 at 12:48:49PM +0200, Kashyap Chamarthy wrote:
> > The said patch in question fixes a CVE[x] in stable/liberty.
> > 
> > We currently have two options, both of them have caused an impasse with
> > the Nova upstream / stable maintainers.  We've had two-ish months to
> > mull over this.  I'd prefer to get this out of a limbo, & bring this to
> > a logical conclusion.
> > 
> > The two options at hand:
> > 
> > (1) Nova backport from master (that also adds a check for the presence
> > of 'ProcessLimits' attribute which is only present in
> > oslo.concurrency>=2.6.1; and a conditional check for 'prlimit'
> > parameter in qemu_img_info() method.)
> > 
> > https://review.openstack.org/#/c/327624/ -- "virt: set address space
> > & CPU time limits when running qemu-img"
> > 
> > (2) Or bump global-requirements for 'oslo.concurrency'
> > 
> > https://review.openstack.org/#/c/337277/5 -- Bump
> > 'global-requirements' for 'oslo.concurrency' to 2.6.1
> 
> Actually we have 3 options
> 
>   (3) Do nothing, leave the bug unfixed in stable/liberty
> 
> While this is a security bug, it is one that has existed in every single
> openstack release ever, and it is not a particularly severe bug. Even if
> we fixed in liberty, it would still remain unfixed in every release before
> liberty. We're in the verge of releasing Newton at which point liberty
> becomes less relevant. So I question whether it is worth spending more
> effort on dealing with this in liberty upstream.  Downstream vendors
> still have the option to do either (1) or (2) in their own private
> branches if they so desire, regardless of whether we fix it upstream.

I think 3 is the least worst option.  If we're going to do something else then
it'd need to be (1).  I feel like we need to rule out (2).

I'll hack something up in the requirements repo to show that the try/except
does what is needed which oslo.concurrency is < 2.6.1

Yours Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Security groups - API signature breakage

2016-09-20 Thread Kevin Benton

I didn't realize how many plugins subclassed the mixin and called super
with the default_sg kwarg. We need a different fix.

On Tue, Sep 20, 2016 at 10:00 PM, Gary Kotton  wrote:

> Hi,
>
> The patch https://review.openstack.org/373108 has broken the API
> signature. I have posted a revert for this.
>
> Can we do something like this? I do not think so. This breaks all plugins.
>
> Thanks
>
> Gary
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [osc][keystone] User Project List

2016-09-20 Thread Adrian Turjak

I'm running into what doesn't really seem like a sensible design choice
around how 'openstack project list' handles the user filter.

Because of this here:
https://github.com/openstack/python-openstackclient/blob/master/openstackclient/identity/v3/project.py#L199
and because of the find_resource call, even if you are already passing
in a user_id, the client MUST go to Keystone to get your user to be able
to filter your project list.

The default keystone policy up until Newton doesn't let a user get their
own user, and thus that client call falls over despite the ability to
use this endpoint just fine:
http://developer.openstack.org/api-ref/identity/v3/index.html?expanded=list-projects-for-user-detail#list-projects-for-user

Plus doing: "keystoneclient.projects.list(user='')" works
just fine.

By forcing that resource_find you are making it impossible for anyone to
use the client to actually get that a list of their own projects if
their Keystone policy is still the old one.

To fix this, how do people feel about adding a "user project list"
command or even a "--auth-user" option to the "project list" command. In
both cases we can get the id from the token and bypass the need for a
resource_find. That way it's less API calls, but also compatible with
older policies.

Any preferences as to which solution? I don't mind writing the
blueprint/patch myself, just would like to know the preferred option.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

90 matches

Mail list logo