Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Doug Hellmann]

Excerpts from Renat Akhmerov's message of 2016-05-19 13:19:54 +0700:
> https://github.com/openstack/requirements/blame/master/README.rst#L95-L100 
> 
> 
> Doug, Is it still relevant? I’m just trying to understand how to enforce 
> upper-constraints.txt in the best way for our jobs like py27, py34 etc.

Several projects are doing this already. Have a look at what nova and
neutron do in the [testenv] section of their tox.ini for examples.

Doug

> 
> Renat Akhmerov
> @Nokia
> 
> > On 19 May 2016, at 12:15, Renat Akhmerov  wrote:
> > 
> >> 
> >> On 18 May 2016, at 22:51, Joshua Harlow  >> > wrote:
> >> 
> >> Roman Dobosz wrote:
> >>> On Tue, 17 May 2016 21:41:11 -0700
> >>> Joshua Harlow>  
> >>> wrote:
> >>> 
> >> Options I see:
> >> Constrain oslo.messaging in global-requirements.txt for
> >> stable/mitaka with 4.6.1. Hard to do since it requires wide
> >> cross-project coordination.
> >> Remove that hack in stable/mitaka as we did with master. It may
> >> be bad because this was wanted very much by some of the users
> >> 
> >> Not sure what else we can do.
> > You could set up your test jobs to use the upper-constraints.txt
> > file in
> > the requirements repo.
> > 
> > What was the outcome of the discussion about adding the
> > at-least-once
> > semantics to oslo.messaging?
>  So there are a few options I am seeing so far (there might be more
>  that I don't see also), others can hopefully correct me if they are
>  wrong (which they might be) ;)
>  
>  Option #1
>  
>  Oslo.messaging (and the dispatcher part that does this) stays as is,
>  doing at-most-once for RPC (notifications are in a different
>  category here so let's not discuss them) and doing at-most-once well
>  and battle-hardened (it's current goal) across the various backend
>  drivers it supports.
>  
>  At that point at-least-once will have to done via some other library
>  where this kind of semantics can be placed, that might be tooz via
>  https://review.openstack.org/#/c/260246/ 
>   (which has similar
>  semantics, but is not based on a kind of RPC, instead it's more like
>  a job-queue).
>  
>  Option #2
>  
>  Oslo.messaging (and the dispatcher part that does this) changes
>  (possibly allowing it to be replaced with a different type of
>  dispatcher, ie like in https://review.openstack.org/#/c/314732/ 
>  );
>  the default class continues being great at for RPC (notifications
>  are in a different category here so let's not discuss them) and
>  doing at-most-once well and battle-hardened (it's current goal)
>  across the various backend drivers it supports. If people want to
>  provide an alternate class with different semantics they are
>  somewhat on there own (but at least they can do this).
>  
>  Issues raised: this though may not be wanted, as some of the
>  oslo.messaging folks do not want the dispatcher class to be exposed
>  at all (and would prefer to make it totally private, so exposing it
>  would be against that goal); though people are already 'hacking'
>  this kind of functionality in, so it might be the best we can get at
>  the current time?
>  
>  Option #3
>  
>  Do nothing.
>  
>  Issues raised: everytime oslo.messaging changes this *mostly*
>  internal dispatcher API a project will have to make a new 'hack' to
>  replace it and hope that the semantics that it has 'hacked' in will
>  continue to be compatible with the various drivers in
>  oslo.messaging. Not IMHO a sustainable way to keep on working (and
>  I'd be wary of doing this in a project if I was the owner of one,
>  because it's ummm, 'dirty').
>  
>  My thoughts on what could work:
>  
>  What I'd personally like to see is a mix of option #1 and #2, where
>  we have commitment from folks (besides myself, lol) to work on
>  option #1 and we temporarily move forward with option #2 with a
>  strict-statement that the functionality we would be enabling will
>  only exist for say a single release (and then it will be removed).
>  
>  Thoughts from others?
> >>> 
> >>> Option #4
> >>> 
> >>> (Which might be obvious) Directly use RabbitMQ driver, like
> >>> pika/kombu, which can expose all the message queue features to the
> >>> project.
> >>> 
> >>> Issues raised: Pushback from the community due not using
> >>> oslo.messaging and potential necessity for implementing it for other
> >>> drivers/transports, or forcing to use particular message queue/driver
> >>> in every 

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Roman Dobosz]

On Wed, 18 May 2016 08:51:00 -0700
Joshua Harlow  wrote:

> >> Option #1
> >>
> >> Oslo.messaging (and the dispatcher part that does this) stays as is,
> >> doing at-most-once for RPC (notifications are in a different
> >> category here so let's not discuss them) and doing at-most-once well
> >> and battle-hardened (it's current goal) across the various backend
> >> drivers it supports.
> >>
> >> At that point at-least-once will have to done via some other library
> >> where this kind of semantics can be placed, that might be tooz via
> >> https://review.openstack.org/#/c/260246/ (which has similar
> >> semantics, but is not based on a kind of RPC, instead it's more like
> >> a job-queue).

[..]

> > Option #4
> >
> > (Which might be obvious) Directly use RabbitMQ driver, like
> > pika/kombu, which can expose all the message queue features to the
> > project.
> >
> > Issues raised: Pushback from the community due not using
> > oslo.messaging and potential necessity for implementing it for other
> > drivers/transports, or forcing to use particular message queue/driver
> > in every project.
> 
> Isn't this similar/same as to option #1? Nothing about option #1 says 
> (from my understanding) that it must be implemented via oslo.messaging 
> (and in all likely-hood it can't be).

It is similar, but not the same. Using RabbitMQ/pika alone it is fairly
easy to accomplish the goal for using messaging system to wait for the
result. Difference is, there is no external queue involved.

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Renat Akhmerov]

https://github.com/openstack/requirements/blame/master/README.rst#L95-L100 


Doug, Is it still relevant? I’m just trying to understand how to enforce 
upper-constraints.txt in the best way for our jobs like py27, py34 etc.

Renat Akhmerov
@Nokia

> On 19 May 2016, at 12:15, Renat Akhmerov  wrote:
> 
>> 
>> On 18 May 2016, at 22:51, Joshua Harlow > > wrote:
>> 
>> Roman Dobosz wrote:
>>> On Tue, 17 May 2016 21:41:11 -0700
>>> Joshua Harlow>  wrote:
>>> 
>> Options I see:
>> Constrain oslo.messaging in global-requirements.txt for
>> stable/mitaka with 4.6.1. Hard to do since it requires wide
>> cross-project coordination.
>> Remove that hack in stable/mitaka as we did with master. It may
>> be bad because this was wanted very much by some of the users
>> 
>> Not sure what else we can do.
> You could set up your test jobs to use the upper-constraints.txt
> file in
> the requirements repo.
> 
> What was the outcome of the discussion about adding the
> at-least-once
> semantics to oslo.messaging?
 So there are a few options I am seeing so far (there might be more
 that I don't see also), others can hopefully correct me if they are
 wrong (which they might be) ;)
 
 Option #1
 
 Oslo.messaging (and the dispatcher part that does this) stays as is,
 doing at-most-once for RPC (notifications are in a different
 category here so let's not discuss them) and doing at-most-once well
 and battle-hardened (it's current goal) across the various backend
 drivers it supports.
 
 At that point at-least-once will have to done via some other library
 where this kind of semantics can be placed, that might be tooz via
 https://review.openstack.org/#/c/260246/ 
  (which has similar
 semantics, but is not based on a kind of RPC, instead it's more like
 a job-queue).
 
 Option #2
 
 Oslo.messaging (and the dispatcher part that does this) changes
 (possibly allowing it to be replaced with a different type of
 dispatcher, ie like in https://review.openstack.org/#/c/314732/ 
 );
 the default class continues being great at for RPC (notifications
 are in a different category here so let's not discuss them) and
 doing at-most-once well and battle-hardened (it's current goal)
 across the various backend drivers it supports. If people want to
 provide an alternate class with different semantics they are
 somewhat on there own (but at least they can do this).
 
 Issues raised: this though may not be wanted, as some of the
 oslo.messaging folks do not want the dispatcher class to be exposed
 at all (and would prefer to make it totally private, so exposing it
 would be against that goal); though people are already 'hacking'
 this kind of functionality in, so it might be the best we can get at
 the current time?
 
 Option #3
 
 Do nothing.
 
 Issues raised: everytime oslo.messaging changes this *mostly*
 internal dispatcher API a project will have to make a new 'hack' to
 replace it and hope that the semantics that it has 'hacked' in will
 continue to be compatible with the various drivers in
 oslo.messaging. Not IMHO a sustainable way to keep on working (and
 I'd be wary of doing this in a project if I was the owner of one,
 because it's ummm, 'dirty').
 
 My thoughts on what could work:
 
 What I'd personally like to see is a mix of option #1 and #2, where
 we have commitment from folks (besides myself, lol) to work on
 option #1 and we temporarily move forward with option #2 with a
 strict-statement that the functionality we would be enabling will
 only exist for say a single release (and then it will be removed).
 
 Thoughts from others?
>>> 
>>> Option #4
>>> 
>>> (Which might be obvious) Directly use RabbitMQ driver, like
>>> pika/kombu, which can expose all the message queue features to the
>>> project.
>>> 
>>> Issues raised: Pushback from the community due not using
>>> oslo.messaging and potential necessity for implementing it for other
>>> drivers/transports, or forcing to use particular message queue/driver
>>> in every project.
>>> 
>> 
>> Isn't this similar/same as to option #1? Nothing about option #1 says (from 
>> my understanding) that it must be implemented via oslo.messaging (and in all 
>> likely-hood it can't be).
> 
> 
> We’ll most likely proceed with #1 (special case is #4) for now just for 
> progress sake.
> 
> It seems to me that we as a community may just need to accumulate more 
> data/experience/patterns well realized so that we 

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Renat Akhmerov]

> On 18 May 2016, at 22:51, Joshua Harlow  wrote:
> 
> Roman Dobosz wrote:
>> On Tue, 17 May 2016 21:41:11 -0700
>> Joshua Harlow  wrote:
>> 
> Options I see:
> Constrain oslo.messaging in global-requirements.txt for
> stable/mitaka with 4.6.1. Hard to do since it requires wide
> cross-project coordination.
> Remove that hack in stable/mitaka as we did with master. It may
> be bad because this was wanted very much by some of the users
> 
> Not sure what else we can do.
 You could set up your test jobs to use the upper-constraints.txt
 file in
 the requirements repo.
 
 What was the outcome of the discussion about adding the
 at-least-once
 semantics to oslo.messaging?
>>> So there are a few options I am seeing so far (there might be more
>>> that I don't see also), others can hopefully correct me if they are
>>> wrong (which they might be) ;)
>>> 
>>> Option #1
>>> 
>>> Oslo.messaging (and the dispatcher part that does this) stays as is,
>>> doing at-most-once for RPC (notifications are in a different
>>> category here so let's not discuss them) and doing at-most-once well
>>> and battle-hardened (it's current goal) across the various backend
>>> drivers it supports.
>>> 
>>> At that point at-least-once will have to done via some other library
>>> where this kind of semantics can be placed, that might be tooz via
>>> https://review.openstack.org/#/c/260246/ (which has similar
>>> semantics, but is not based on a kind of RPC, instead it's more like
>>> a job-queue).
>>> 
>>> Option #2
>>> 
>>> Oslo.messaging (and the dispatcher part that does this) changes
>>> (possibly allowing it to be replaced with a different type of
>>> dispatcher, ie like in https://review.openstack.org/#/c/314732/);
>>> the default class continues being great at for RPC (notifications
>>> are in a different category here so let's not discuss them) and
>>> doing at-most-once well and battle-hardened (it's current goal)
>>> across the various backend drivers it supports. If people want to
>>> provide an alternate class with different semantics they are
>>> somewhat on there own (but at least they can do this).
>>> 
>>> Issues raised: this though may not be wanted, as some of the
>>> oslo.messaging folks do not want the dispatcher class to be exposed
>>> at all (and would prefer to make it totally private, so exposing it
>>> would be against that goal); though people are already 'hacking'
>>> this kind of functionality in, so it might be the best we can get at
>>> the current time?
>>> 
>>> Option #3
>>> 
>>> Do nothing.
>>> 
>>> Issues raised: everytime oslo.messaging changes this *mostly*
>>> internal dispatcher API a project will have to make a new 'hack' to
>>> replace it and hope that the semantics that it has 'hacked' in will
>>> continue to be compatible with the various drivers in
>>> oslo.messaging. Not IMHO a sustainable way to keep on working (and
>>> I'd be wary of doing this in a project if I was the owner of one,
>>> because it's ummm, 'dirty').
>>> 
>>> My thoughts on what could work:
>>> 
>>> What I'd personally like to see is a mix of option #1 and #2, where
>>> we have commitment from folks (besides myself, lol) to work on
>>> option #1 and we temporarily move forward with option #2 with a
>>> strict-statement that the functionality we would be enabling will
>>> only exist for say a single release (and then it will be removed).
>>> 
>>> Thoughts from others?
>> 
>> Option #4
>> 
>> (Which might be obvious) Directly use RabbitMQ driver, like
>> pika/kombu, which can expose all the message queue features to the
>> project.
>> 
>> Issues raised: Pushback from the community due not using
>> oslo.messaging and potential necessity for implementing it for other
>> drivers/transports, or forcing to use particular message queue/driver
>> in every project.
>> 
> 
> Isn't this similar/same as to option #1? Nothing about option #1 says (from 
> my understanding) that it must be implemented via oslo.messaging (and in all 
> likely-hood it can't be).


We’ll most likely proceed with #1 (special case is #4) for now just for 
progress sake.

It seems to me that we as a community may just need to accumulate more 
data/experience/patterns well realized so that we could explain value of 
certain patterns more clearly. Through our endeavours and researches hopefully 
we’ll be able to communicate our thoughts better. As far as semantical 
differences RPC vs Messaging vs Jobs vs Notifications vs Concrete 
implementation of any of those, it’s simply a matter of how we want to do it, 
less a matter of what is right. I’m just saying this that it’s OK for now that 
we can’t come to a consensus. We need to keep in touch and exchange our ideas. 
Excuse me, it’s just a lyric digression.

Renat Akhmerov
@Nokia

__
OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Renat Akhmerov]

> On 19 May 2016, at 01:46, Doug Hellmann  wrote:
> 
> Excerpts from Renat Akhmerov's message of 2016-05-18 11:07:29 +0700:
>> 
>>> On 17 May 2016, at 21:50, Doug Hellmann  wrote:
>>> 
>>> Excerpts from Renat Akhmerov's message of 2016-05-17 19:10:55 +0700:
 Team,
 
 Our stable/mitaka branch is now broken by oslo.messaging 5.0.0. Global 
 requirements for stable/mitaka has oslo.messaging>=4.0.0 so it can fetch 
 5.0.0.
 
 Just reminding that it breaks us because we intentionally modified 
 RPCDispatcher like in [1]. It was needed for “at-least-once” delivery. In 
 master we already agreed to remove that hack and work towards having a 
 decent solution (there are options). The patch is [2]. But we need to 
 handle it in mitaka somehow.
 
 Options I see:
 Constrain oslo.messaging in global-requirements.txt for stable/mitaka with 
 4.6.1. Hard to do since it requires wide cross-project coordination.
 Remove that hack in stable/mitaka as we did with master. It may be bad 
 because this was wanted very much by some of the users
 
 Not sure what else we can do.
>>> 
>>> You could set up your test jobs to use the upper-constraints.txt file in
>>> the requirements repo.
>> 
>> Yes, it’s an option. I’m just thinking from a regular user perspective. 
>> There will be a lot of people who don’t know about upper-constraints.txt and 
>> they will be stumbling on it just using our requirements.txt. My question 
>> here is: is upper-constraints.txt something that’s officially promoted and 
>> should be used by everyone or it’s mostly introduced for our internal 
>> OpenStack gating system?
> 
> Anyone installing from PyPI should use the constraints list. Anyone
> installing from system packages won't need it.


Ok, thanks Doug.

Renat Akhmerov
@Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Doug Hellmann]

Excerpts from Renat Akhmerov's message of 2016-05-18 11:07:29 +0700:
> 
> > On 17 May 2016, at 21:50, Doug Hellmann  wrote:
> > 
> > Excerpts from Renat Akhmerov's message of 2016-05-17 19:10:55 +0700:
> >> Team,
> >> 
> >> Our stable/mitaka branch is now broken by oslo.messaging 5.0.0. Global 
> >> requirements for stable/mitaka has oslo.messaging>=4.0.0 so it can fetch 
> >> 5.0.0.
> >> 
> >> Just reminding that it breaks us because we intentionally modified 
> >> RPCDispatcher like in [1]. It was needed for “at-least-once” delivery. In 
> >> master we already agreed to remove that hack and work towards having a 
> >> decent solution (there are options). The patch is [2]. But we need to 
> >> handle it in mitaka somehow.
> >> 
> >> Options I see:
> >> Constrain oslo.messaging in global-requirements.txt for stable/mitaka with 
> >> 4.6.1. Hard to do since it requires wide cross-project coordination.
> >> Remove that hack in stable/mitaka as we did with master. It may be bad 
> >> because this was wanted very much by some of the users
> >> 
> >> Not sure what else we can do.
> > 
> > You could set up your test jobs to use the upper-constraints.txt file in
> > the requirements repo.
> 
> Yes, it’s an option. I’m just thinking from a regular user perspective. There 
> will be a lot of people who don’t know about upper-constraints.txt and they 
> will be stumbling on it just using our requirements.txt. My question here is: 
> is upper-constraints.txt something that’s officially promoted and should be 
> used by everyone or it’s mostly introduced for our internal OpenStack gating 
> system?

Anyone installing from PyPI should use the constraints list. Anyone
installing from system packages won't need it.

Doug

> 
> > What was the outcome of the discussion about adding the at-least-once
> > semantics to oslo.messaging?
> 
> 
> No outcome yet, we’re still discussing. I expect that more people join the 
> thread since some stakeholders were off after the summit.
> 
> Renat Akhmerov
> @Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Joshua Harlow]

Roman Dobosz wrote:

On Tue, 17 May 2016 21:41:11 -0700
Joshua Harlow  wrote:


Options I see:
Constrain oslo.messaging in global-requirements.txt for
stable/mitaka with 4.6.1. Hard to do since it requires wide
cross-project coordination.
Remove that hack in stable/mitaka as we did with master. It may
be bad because this was wanted very much by some of the users

Not sure what else we can do.

You could set up your test jobs to use the upper-constraints.txt
file in
the requirements repo.

What was the outcome of the discussion about adding the
at-least-once
semantics to oslo.messaging?

So there are a few options I am seeing so far (there might be more
that I don't see also), others can hopefully correct me if they are
wrong (which they might be) ;)

Option #1

Oslo.messaging (and the dispatcher part that does this) stays as is,
doing at-most-once for RPC (notifications are in a different
category here so let's not discuss them) and doing at-most-once well
and battle-hardened (it's current goal) across the various backend
drivers it supports.

At that point at-least-once will have to done via some other library
where this kind of semantics can be placed, that might be tooz via
https://review.openstack.org/#/c/260246/ (which has similar
semantics, but is not based on a kind of RPC, instead it's more like
a job-queue).

Option #2

Oslo.messaging (and the dispatcher part that does this) changes
(possibly allowing it to be replaced with a different type of
dispatcher, ie like in https://review.openstack.org/#/c/314732/);
the default class continues being great at for RPC (notifications
are in a different category here so let's not discuss them) and
doing at-most-once well and battle-hardened (it's current goal)
across the various backend drivers it supports. If people want to
provide an alternate class with different semantics they are
somewhat on there own (but at least they can do this).

Issues raised: this though may not be wanted, as some of the
oslo.messaging folks do not want the dispatcher class to be exposed
at all (and would prefer to make it totally private, so exposing it
would be against that goal); though people are already 'hacking'
this kind of functionality in, so it might be the best we can get at
the current time?

Option #3

Do nothing.

Issues raised: everytime oslo.messaging changes this *mostly*
internal dispatcher API a project will have to make a new 'hack' to
replace it and hope that the semantics that it has 'hacked' in will
continue to be compatible with the various drivers in
oslo.messaging. Not IMHO a sustainable way to keep on working (and
I'd be wary of doing this in a project if I was the owner of one,
because it's ummm, 'dirty').

My thoughts on what could work:

What I'd personally like to see is a mix of option #1 and #2, where
we have commitment from folks (besides myself, lol) to work on
option #1 and we temporarily move forward with option #2 with a
strict-statement that the functionality we would be enabling will
only exist for say a single release (and then it will be removed).

Thoughts from others?


Option #4

(Which might be obvious) Directly use RabbitMQ driver, like
pika/kombu, which can expose all the message queue features to the
project.

Issues raised: Pushback from the community due not using
oslo.messaging and potential necessity for implementing it for other
drivers/transports, or forcing to use particular message queue/driver
in every project.



Isn't this similar/same as to option #1? Nothing about option #1 says 
(from my understanding) that it must be implemented via oslo.messaging 
(and in all likely-hood it can't be).


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Roman Dobosz]

On Tue, 17 May 2016 21:41:11 -0700
Joshua Harlow  wrote:

> >> Options I see:
> >> Constrain oslo.messaging in global-requirements.txt for
> >> stable/mitaka with 4.6.1. Hard to do since it requires wide
> >> cross-project coordination.
> >> Remove that hack in stable/mitaka as we did with master. It may
> >> be bad because this was wanted very much by some of the users
> >>
> >> Not sure what else we can do.
> >
> > You could set up your test jobs to use the upper-constraints.txt
> > file in
> > the requirements repo.
> >
> > What was the outcome of the discussion about adding the
> > at-least-once
> > semantics to oslo.messaging?
>
> So there are a few options I am seeing so far (there might be more
> that I don't see also), others can hopefully correct me if they are
> wrong (which they might be) ;)
>
> Option #1
>
> Oslo.messaging (and the dispatcher part that does this) stays as is,
> doing at-most-once for RPC (notifications are in a different
> category here so let's not discuss them) and doing at-most-once well
> and battle-hardened (it's current goal) across the various backend
> drivers it supports.
>
> At that point at-least-once will have to done via some other library
> where this kind of semantics can be placed, that might be tooz via
> https://review.openstack.org/#/c/260246/ (which has similar
> semantics, but is not based on a kind of RPC, instead it's more like
> a job-queue).
>
> Option #2
>
> Oslo.messaging (and the dispatcher part that does this) changes
> (possibly allowing it to be replaced with a different type of
> dispatcher, ie like in https://review.openstack.org/#/c/314732/);
> the default class continues being great at for RPC (notifications
> are in a different category here so let's not discuss them) and
> doing at-most-once well and battle-hardened (it's current goal)
> across the various backend drivers it supports. If people want to
> provide an alternate class with different semantics they are
> somewhat on there own (but at least they can do this).
>
> Issues raised: this though may not be wanted, as some of the
> oslo.messaging folks do not want the dispatcher class to be exposed
> at all (and would prefer to make it totally private, so exposing it
> would be against that goal); though people are already 'hacking'
> this kind of functionality in, so it might be the best we can get at
> the current time?
>
> Option #3
>
> Do nothing.
>
> Issues raised: everytime oslo.messaging changes this *mostly*
> internal dispatcher API a project will have to make a new 'hack' to
> replace it and hope that the semantics that it has 'hacked' in will
> continue to be compatible with the various drivers in
> oslo.messaging. Not IMHO a sustainable way to keep on working (and
> I'd be wary of doing this in a project if I was the owner of one,
> because it's ummm, 'dirty').
>
> My thoughts on what could work:
>
> What I'd personally like to see is a mix of option #1 and #2, where
> we have commitment from folks (besides myself, lol) to work on
> option #1 and we temporarily move forward with option #2 with a
> strict-statement that the functionality we would be enabling will
> only exist for say a single release (and then it will be removed).
>
> Thoughts from others?

Option #4

(Which might be obvious) Directly use RabbitMQ driver, like
pika/kombu, which can expose all the message queue features to the
project.

Issues raised: Pushback from the community due not using
oslo.messaging and potential necessity for implementing it for other
drivers/transports, or forcing to use particular message queue/driver
in every project.

-- 
Cheers,
Roman Dobosz

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Mehdi Abaakouk]

On Tue, May 17, 2016 at 09:41:11PM -0700, Joshua Harlow wrote:

Doug Hellmann wrote:

So there are a few options I am seeing so far (there might be more 
that I don't see also), others can hopefully correct me if they are 
wrong (which they might be) ;)


Option #1

Oslo.messaging (and the dispatcher part that does this) stays as is, 
doing at-most-once for RPC (notifications are in a different category 
here so let's not discuss them) and doing at-most-once well and 
battle-hardened (it's current goal) across the various backend drivers 
it supports.


At that point at-least-once will have to done via some other library 
where this kind of semantics can be placed, that might be tooz via 
https://review.openstack.org/#/c/260246/ (which has similar semantics, 
but is not based on a kind of RPC, instead it's more like a 
job-queue).


This is still my favorite option.


Option #2

Oslo.messaging (and the dispatcher part that does this) changes 
(possibly allowing it to be replaced with a different type of 
dispatcher, ie like in https://review.openstack.org/#/c/314732/); the 
default class continues being great at for RPC (notifications are in a 
different category here so let's not discuss them) and doing 
at-most-once well and battle-hardened (it's current goal) across the 
various backend drivers it supports. If people want to provide an 
alternate class with different semantics they are somewhat on there 
own (but at least they can do this).


Issues raised: this though may not be wanted, as some of the 
oslo.messaging folks do not want the dispatcher class to be exposed at 
all (and would prefer to make it totally private, so exposing it would 
be against that goal); though people are already 'hacking' this kind 
of functionality in, so it might be the best we can get at the current 
time?


Exposing dispatcher will not fix the mistral issue, because since
oslo.msg 5.0, dispatcher does not talk directly to the driver interface
anymore (that was a design issue). All drivers interactions have been
moved into RPCServer and NotificationListener where stuffs are already
private since the beginning. Thanks to Dmitriy Ukhlov, for its amazing
work on this (The server/executor/dispatcher refactoring was a huge
simplication of oslo.messaging internal).


Option #3

Do nothing.

Issues raised: everytime oslo.messaging changes this *mostly* internal 
dispatcher API a project will have to make a new 'hack' to replace it 
and hope that the semantics that it has 'hacked' in will continue to 
be compatible with the various drivers in oslo.messaging. Not IMHO a 
sustainable way to keep on working (and I'd be wary of doing this in a 
project if I was the owner of one, because it's ummm, 'dirty').


Mistral have removed its hack since we break them with oslo.messaging
5.0.0


Cheers,
--
Mehdi Abaakouk
mail: sil...@sileht.net
irc: sileht

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Joshua Harlow]

Doug Hellmann wrote:

Excerpts from Renat Akhmerov's message of 2016-05-17 19:10:55 +0700:

Team,

Our stable/mitaka branch is now broken by oslo.messaging 5.0.0. Global 
requirements for stable/mitaka has oslo.messaging>=4.0.0 so it can fetch 5.0.0.

Just reminding that it breaks us because we intentionally modified 
RPCDispatcher like in [1]. It was needed for “at-least-once” delivery. In 
master we already agreed to remove that hack and work towards having a decent 
solution (there are options). The patch is [2]. But we need to handle it in 
mitaka somehow.

Options I see:
Constrain oslo.messaging in global-requirements.txt for stable/mitaka with 
4.6.1. Hard to do since it requires wide cross-project coordination.
Remove that hack in stable/mitaka as we did with master. It may be bad because 
this was wanted very much by some of the users

Not sure what else we can do.


You could set up your test jobs to use the upper-constraints.txt file in
the requirements repo.

What was the outcome of the discussion about adding the at-least-once
semantics to oslo.messaging?



So there are a few options I am seeing so far (there might be more that 
I don't see also), others can hopefully correct me if they are wrong 
(which they might be) ;)


Option #1

Oslo.messaging (and the dispatcher part that does this) stays as is, 
doing at-most-once for RPC (notifications are in a different category 
here so let's not discuss them) and doing at-most-once well and 
battle-hardened (it's current goal) across the various backend drivers 
it supports.


At that point at-least-once will have to done via some other library 
where this kind of semantics can be placed, that might be tooz via 
https://review.openstack.org/#/c/260246/ (which has similar semantics, 
but is not based on a kind of RPC, instead it's more like a job-queue).


Option #2

Oslo.messaging (and the dispatcher part that does this) changes 
(possibly allowing it to be replaced with a different type of 
dispatcher, ie like in https://review.openstack.org/#/c/314732/); the 
default class continues being great at for RPC (notifications are in a 
different category here so let's not discuss them) and doing 
at-most-once well and battle-hardened (it's current goal) across the 
various backend drivers it supports. If people want to provide an 
alternate class with different semantics they are somewhat on there own 
(but at least they can do this).


Issues raised: this though may not be wanted, as some of the 
oslo.messaging folks do not want the dispatcher class to be exposed at 
all (and would prefer to make it totally private, so exposing it would 
be against that goal); though people are already 'hacking' this kind of 
functionality in, so it might be the best we can get at the current time?


Option #3

Do nothing.

Issues raised: everytime oslo.messaging changes this *mostly* internal 
dispatcher API a project will have to make a new 'hack' to replace it 
and hope that the semantics that it has 'hacked' in will continue to be 
compatible with the various drivers in oslo.messaging. Not IMHO a 
sustainable way to keep on working (and I'd be wary of doing this in a 
project if I was the owner of one, because it's ummm, 'dirty').


My thoughts on what could work:

What I'd personally like to see is a mix of option #1 and #2, where we 
have commitment from folks (besides myself, lol) to work on option #1 
and we temporarily move forward with option #2 with a strict-statement 
that the functionality we would be enabling will only exist for say a 
single release (and then it will be removed).


Thoughts from others?


Doug


Thoughts?

[1] 
https://github.com/openstack/mistral/blob/stable/mitaka/mistral/engine/rpc.py#L38-L88
[2] 
https://review.openstack.org/#/c/316578/

Renat Akhmerov
@Nokia


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Renat Akhmerov]

> On 17 May 2016, at 21:50, Doug Hellmann  wrote:
> 
> Excerpts from Renat Akhmerov's message of 2016-05-17 19:10:55 +0700:
>> Team,
>> 
>> Our stable/mitaka branch is now broken by oslo.messaging 5.0.0. Global 
>> requirements for stable/mitaka has oslo.messaging>=4.0.0 so it can fetch 
>> 5.0.0.
>> 
>> Just reminding that it breaks us because we intentionally modified 
>> RPCDispatcher like in [1]. It was needed for “at-least-once” delivery. In 
>> master we already agreed to remove that hack and work towards having a 
>> decent solution (there are options). The patch is [2]. But we need to handle 
>> it in mitaka somehow.
>> 
>> Options I see:
>> Constrain oslo.messaging in global-requirements.txt for stable/mitaka with 
>> 4.6.1. Hard to do since it requires wide cross-project coordination.
>> Remove that hack in stable/mitaka as we did with master. It may be bad 
>> because this was wanted very much by some of the users
>> 
>> Not sure what else we can do.
> 
> You could set up your test jobs to use the upper-constraints.txt file in
> the requirements repo.

Yes, it’s an option. I’m just thinking from a regular user perspective. There 
will be a lot of people who don’t know about upper-constraints.txt and they 
will be stumbling on it just using our requirements.txt. My question here is: 
is upper-constraints.txt something that’s officially promoted and should be 
used by everyone or it’s mostly introduced for our internal OpenStack gating 
system?

> What was the outcome of the discussion about adding the at-least-once
> semantics to oslo.messaging?


No outcome yet, we’re still discussing. I expect that more people join the 
thread since some stakeholders were off after the summit.

Renat Akhmerov
@Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][oslo.messaging] stable/mitaka is broken by oslo.messaging 5.0.0

[Doug Hellmann]

Excerpts from Renat Akhmerov's message of 2016-05-17 19:10:55 +0700:
> Team,
> 
> Our stable/mitaka branch is now broken by oslo.messaging 5.0.0. Global 
> requirements for stable/mitaka has oslo.messaging>=4.0.0 so it can fetch 
> 5.0.0.
> 
> Just reminding that it breaks us because we intentionally modified 
> RPCDispatcher like in [1]. It was needed for “at-least-once” delivery. In 
> master we already agreed to remove that hack and work towards having a decent 
> solution (there are options). The patch is [2]. But we need to handle it in 
> mitaka somehow.
> 
> Options I see:
> Constrain oslo.messaging in global-requirements.txt for stable/mitaka with 
> 4.6.1. Hard to do since it requires wide cross-project coordination.
> Remove that hack in stable/mitaka as we did with master. It may be bad 
> because this was wanted very much by some of the users
> 
> Not sure what else we can do.

You could set up your test jobs to use the upper-constraints.txt file in
the requirements repo.

What was the outcome of the discussion about adding the at-least-once
semantics to oslo.messaging?

Doug

> 
> Thoughts?
> 
> [1] 
> https://github.com/openstack/mistral/blob/stable/mitaka/mistral/engine/rpc.py#L38-L88
>  
> 
> [2] https://review.openstack.org/#/c/316578/ 
> 
> 
> Renat Akhmerov
> @Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev