Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Davanum Srinivas
Josh,

We can generate options, if folks who need/want it are not here to do
the necessary work, not much we can do :(

-- dims

On Tue, Jun 23, 2015 at 11:30 AM, Joshua Harlow harlo...@outlook.com wrote:
 Flavio Percoco wrote:

 On 22/06/15 12:43 -0700, Clint Byrum wrote:

 Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

 On 06/20/2015 10:28 AM, Flavio Percoco wrote:
 
 
  As promissed: https://review.openstack.org/#/c/193804/
 
  Cheers,
 You can't deprecate a driver without providing a viable alternative.

 Right now, QPID is the only driver that supports Kerberos.

 TO support Kerberos, tyou need support for the GSSAPI library, which is
 usually done via support for SASL. Why is it so
 convoluted...historical...

 We've talked with both teams (I work with Ken) and I think Proton is
 likely going to be the first to have support. The folks working on
 Rabbit have the double hurdle of getting SASL support into Erlang first,
 and then support for SASL into Rabbit. They've indicated a preference
 for getting it in to the AMQP 1.0 driver, and not bothering with the
 exisiting, but, check me on this, the Oso.Messaging code only support
 the pre 1.0 Rabbit.


 So..until we have a viable alternative, please leave QPID alone. I've
 not been bothering people about it, as there seems to be work to get
 ahead, but until either Rabbit or Proton support Kerberos, I need QPID
 as is.


 Adam that is all great information, thank you. However, the policy is
 clear: commit resources for integration testing, or it needs to move
 out of tree.

 It's not a mountain of resources. Just an integration test that passes
 reliably, and a couple of QPID+OpenStack experts who we can contact when
 it breaks. If nobody is willing to put that much effort in, then it is
 not really something we want in our official messaging library tree.

 So please if you can carry that message up to those who want it to
 stay in
 tree, that would be helpful and would put the stops on this deprecation.


 Agreed with the above.

 I'd also like to add that it was also discussed with folks previously
 maintaining the qpid driver what their plans with that work were and
 the agreement of deprecating it was reached with them.


 Just to note, something that may be acceptable for people that need this,
 and don't mind doing a little bit of work to maintain it out of tree. It
 appears the kombu qpid driver does have SASL support (from a quick glance at
 the code):

 - https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1250
 - https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1210

 So until this gets resolved and/or maintained it appears folks could just
 use the one built-in to kombu (assuming it works)? If the oslo.messaging
 'impl_rabbit.py' one was more of a kombu 'wrapper' (and renamed
 'impl_kombu.py'?) than this might have been even easier to support/make
 possible.

 Food for thought :)



 I know this doesn't solve the current problem of not having kerberos
 support but it clears that this discussion has been had already.

 That said, the point being raised is very good and unfortunate.

 Cheers,
 Flavio



 __

 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Flavio Percoco

On 22/06/15 12:43 -0700, Clint Byrum wrote:

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

On 06/20/2015 10:28 AM, Flavio Percoco wrote:


 As promissed: https://review.openstack.org/#/c/193804/

 Cheers,
You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports  Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is
usually done via support for SASL.  Why is it so convoluted...historical...

We've talked with both teams (I work with Ken) and I think Proton is
likely going to be the first to have support.  The folks working on
Rabbit have the double hurdle of getting SASL support into Erlang first,
and then support for SASL into Rabbit. They've indicated a preference
for getting it in to the AMQP 1.0 driver, and not bothering with the
exisiting, but, check me on this, the Oso.Messaging  code only support
the pre 1.0 Rabbit.


So..until we have a viable alternative, please leave QPID alone. I've
not been bothering people about it, as there seems to be work to get
ahead, but until either Rabbit or  Proton support Kerberos, I need QPID
as is.



Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to stay in
tree, that would be helpful and would put the stops on this deprecation.


Agreed with the above.

I'd also like to add that it was also discussed with folks previously
maintaining the qpid driver what their plans with that work were and
the agreement of deprecating it was reached with them.

I know this doesn't solve the current problem of not having kerberos
support but it clears that this discussion has been had already.

That said, the point being raised is very good and unfortunate.

Cheers,
Flavio



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
@flaper87
Flavio Percoco


pgpq_zrNRSEVp.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Joshua Harlow

Flavio Percoco wrote:

On 22/06/15 12:43 -0700, Clint Byrum wrote:

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

On 06/20/2015 10:28 AM, Flavio Percoco wrote:


 As promissed: https://review.openstack.org/#/c/193804/

 Cheers,
You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is
usually done via support for SASL. Why is it so
convoluted...historical...

We've talked with both teams (I work with Ken) and I think Proton is
likely going to be the first to have support. The folks working on
Rabbit have the double hurdle of getting SASL support into Erlang first,
and then support for SASL into Rabbit. They've indicated a preference
for getting it in to the AMQP 1.0 driver, and not bothering with the
exisiting, but, check me on this, the Oso.Messaging code only support
the pre 1.0 Rabbit.


So..until we have a viable alternative, please leave QPID alone. I've
not been bothering people about it, as there seems to be work to get
ahead, but until either Rabbit or Proton support Kerberos, I need QPID
as is.



Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to
stay in
tree, that would be helpful and would put the stops on this deprecation.


Agreed with the above.

I'd also like to add that it was also discussed with folks previously
maintaining the qpid driver what their plans with that work were and
the agreement of deprecating it was reached with them.


Just to note, something that may be acceptable for people that need 
this, and don't mind doing a little bit of work to maintain it out of 
tree. It appears the kombu qpid driver does have SASL support (from a 
quick glance at the code):


- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1250
- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1210

So until this gets resolved and/or maintained it appears folks could 
just use the one built-in to kombu (assuming it works)? If the 
oslo.messaging 'impl_rabbit.py' one was more of a kombu 'wrapper' (and 
renamed 'impl_kombu.py'?) than this might have been even easier to 
support/make possible.


Food for thought :)



I know this doesn't solve the current problem of not having kerberos
support but it clears that this discussion has been had already.

That said, the point being raised is very good and unfortunate.

Cheers,
Flavio



__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Flavio Percoco

On 23/06/15 08:30 -0700, Joshua Harlow wrote:

Flavio Percoco wrote:

On 22/06/15 12:43 -0700, Clint Byrum wrote:

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

On 06/20/2015 10:28 AM, Flavio Percoco wrote:




As promissed: https://review.openstack.org/#/c/193804/

Cheers,

You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is
usually done via support for SASL. Why is it so
convoluted...historical...

We've talked with both teams (I work with Ken) and I think Proton is
likely going to be the first to have support. The folks working on
Rabbit have the double hurdle of getting SASL support into Erlang first,
and then support for SASL into Rabbit. They've indicated a preference
for getting it in to the AMQP 1.0 driver, and not bothering with the
exisiting, but, check me on this, the Oso.Messaging code only support
the pre 1.0 Rabbit.


So..until we have a viable alternative, please leave QPID alone. I've
not been bothering people about it, as there seems to be work to get
ahead, but until either Rabbit or Proton support Kerberos, I need QPID
as is.



Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to
stay in
tree, that would be helpful and would put the stops on this deprecation.


Agreed with the above.

I'd also like to add that it was also discussed with folks previously
maintaining the qpid driver what their plans with that work were and
the agreement of deprecating it was reached with them.


Just to note, something that may be acceptable for people that need 
this, and don't mind doing a little bit of work to maintain it out of 
tree. It appears the kombu qpid driver does have SASL support (from a 
quick glance at the code):


- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1250
- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1210

So until this gets resolved and/or maintained it appears folks could 
just use the one built-in to kombu (assuming it works)? If the 
oslo.messaging 'impl_rabbit.py' one was more of a kombu 'wrapper' (and 
renamed 'impl_kombu.py'?) than this might have been even easier to 
support/make possible.


TBH, I'm more for making the impl_rabbit driver more rabbit-focused
rather than more kombu-focused. Using kombu for it is great and I
wouldn't advice to move away from it (not in the short run at least).
But if there are changes that we can do to make it more rabbit
specific, I'd be all for that.

Cheers,
Flavio




Food for thought :)



I know this doesn't solve the current problem of not having kerberos
support but it clears that this discussion has been had already.

That said, the point being raised is very good and unfortunate.

Cheers,
Flavio



__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
@flaper87
Flavio Percoco


pgpU4Zqgf6Qpy.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Joshua Harlow

Flavio Percoco wrote:

On 23/06/15 08:30 -0700, Joshua Harlow wrote:

Flavio Percoco wrote:

On 22/06/15 12:43 -0700, Clint Byrum wrote:

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

On 06/20/2015 10:28 AM, Flavio Percoco wrote:




As promissed: https://review.openstack.org/#/c/193804/

Cheers,

You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library,
which is
usually done via support for SASL. Why is it so
convoluted...historical...

We've talked with both teams (I work with Ken) and I think Proton is
likely going to be the first to have support. The folks working on
Rabbit have the double hurdle of getting SASL support into Erlang
first,
and then support for SASL into Rabbit. They've indicated a preference
for getting it in to the AMQP 1.0 driver, and not bothering with the
exisiting, but, check me on this, the Oso.Messaging code only support
the pre 1.0 Rabbit.


So..until we have a viable alternative, please leave QPID alone. I've
not been bothering people about it, as there seems to be work to get
ahead, but until either Rabbit or Proton support Kerberos, I need QPID
as is.



Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact
when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to
stay in
tree, that would be helpful and would put the stops on this
deprecation.


Agreed with the above.

I'd also like to add that it was also discussed with folks previously
maintaining the qpid driver what their plans with that work were and
the agreement of deprecating it was reached with them.


Just to note, something that may be acceptable for people that need
this, and don't mind doing a little bit of work to maintain it out of
tree. It appears the kombu qpid driver does have SASL support (from a
quick glance at the code):

-
https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1250
-
https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1210

So until this gets resolved and/or maintained it appears folks could
just use the one built-in to kombu (assuming it works)? If the
oslo.messaging 'impl_rabbit.py' one was more of a kombu 'wrapper' (and
renamed 'impl_kombu.py'?) than this might have been even easier to
support/make possible.


TBH, I'm more for making the impl_rabbit driver more rabbit-focused
rather than more kombu-focused. Using kombu for it is great and I
wouldn't advice to move away from it (not in the short run at least).
But if there are changes that we can do to make it more rabbit
specific, I'd be all for that.

Cheers,
Flavio


Understood, I see the benefit of going both ways and I am fine with 
however it turns out...







Food for thought :)



I know this doesn't solve the current problem of not having kerberos
support but it clears that this discussion has been had already.

That said, the point being raised is very good and unfortunate.

Cheers,
Flavio



__


OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-23 Thread Joshua Harlow

Davanum Srinivas wrote:

Josh,

We can generate options, if folks who need/want it are not here to do
the necessary work, not much we can do :(



True dat, u are very wise :-)


-- dims

On Tue, Jun 23, 2015 at 11:30 AM, Joshua Harlowharlo...@outlook.com  wrote:

Flavio Percoco wrote:

On 22/06/15 12:43 -0700, Clint Byrum wrote:

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:

On 06/20/2015 10:28 AM, Flavio Percoco wrote:

As promissed: https://review.openstack.org/#/c/193804/

Cheers,

You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is
usually done via support for SASL. Why is it so
convoluted...historical...

We've talked with both teams (I work with Ken) and I think Proton is
likely going to be the first to have support. The folks working on
Rabbit have the double hurdle of getting SASL support into Erlang first,
and then support for SASL into Rabbit. They've indicated a preference
for getting it in to the AMQP 1.0 driver, and not bothering with the
exisiting, but, check me on this, the Oso.Messaging code only support
the pre 1.0 Rabbit.


So..until we have a viable alternative, please leave QPID alone. I've
not been bothering people about it, as there seems to be work to get
ahead, but until either Rabbit or Proton support Kerberos, I need QPID
as is.


Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to
stay in
tree, that would be helpful and would put the stops on this deprecation.


Agreed with the above.

I'd also like to add that it was also discussed with folks previously
maintaining the qpid driver what their plans with that work were and
the agreement of deprecating it was reached with them.


Just to note, something that may be acceptable for people that need this,
and don't mind doing a little bit of work to maintain it out of tree. It
appears the kombu qpid driver does have SASL support (from a quick glance at
the code):

- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1250
- https://github.com/celery/kombu/blob/master/kombu/transport/qpid.py#L1210

So until this gets resolved and/or maintained it appears folks could just
use the one built-in to kombu (assuming it works)? If the oslo.messaging
'impl_rabbit.py' one was more of a kombu 'wrapper' (and renamed
'impl_kombu.py'?) than this might have been even easier to support/make
possible.

Food for thought :)



I know this doesn't solve the current problem of not having kerberos
support but it clears that this discussion has been had already.

That said, the point being raised is very good and unfortunate.

Cheers,
Flavio



__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-22 Thread Ken Giusti
On Mon, Jun 22, 2015 at 2:27 PM Adam Young ayo...@redhat.com wrote:

 On 06/20/2015 10:28 AM, Flavio Percoco wrote:
 
 
  As promissed: https://review.openstack.org/#/c/193804/
 
  Cheers,
 You can't deprecate a driver without providing a viable alternative.

 Right now, QPID is the only driver that supports  Kerberos.

 TO support Kerberos, tyou need support for the GSSAPI library, which is
 usually done via support for SASL.  Why is it so convoluted...historical...

 We've talked with both teams (I work with Ken) and I think Proton is
 likely going to be the first to have support.  The folks working on
 Rabbit have the double hurdle of getting SASL support into Erlang first,
 and then support for SASL into Rabbit. They've indicated a preference
 for getting it in to the AMQP 1.0 driver, and not bothering with the
 exisiting, but, check me on this, the Oso.Messaging  code only support
 the pre 1.0 Rabbit.


 So..until we have a viable alternative, please leave QPID alone. I've
 not been bothering people about it, as there seems to be work to get
 ahead, but until either Rabbit or  Proton support Kerberos, I need QPID
 as is.


Re: proton - Kerberos support is in progress upstream [0],[1], but as you
point out it's not yet available.  That blocks kerberos support for the
amqp1.0 driver.

Once proton does release that, and the amqp1.0 driver adopts it, you'll be
able to migrate to the amqp1.0 driver and continue to work with the QPID
broker (as long as the version of the QPID broker supports AMQP 1.0).

That doesn't help you now, but it is something to plan for.

[0]https://issues.apache.org/jira/browse/PROTON-334
[1]https://issues.apache.org/jira/browse/PROTON-911


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-22 Thread Adam Young

On 06/20/2015 10:28 AM, Flavio Percoco wrote:




As promissed: https://review.openstack.org/#/c/193804/

Cheers, 

You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports  Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is 
usually done via support for SASL.  Why is it so convoluted...historical...


We've talked with both teams (I work with Ken) and I think Proton is 
likely going to be the first to have support.  The folks working on 
Rabbit have the double hurdle of getting SASL support into Erlang first, 
and then support for SASL into Rabbit. They've indicated a preference 
for getting it in to the AMQP 1.0 driver, and not bothering with the 
exisiting, but, check me on this, the Oso.Messaging  code only support 
the pre 1.0 Rabbit.



So..until we have a viable alternative, please leave QPID alone. I've 
not been bothering people about it, as there seems to be work to get 
ahead, but until either Rabbit or  Proton support Kerberos, I need QPID 
as is.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-22 Thread Clint Byrum
Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:
 On 06/20/2015 10:28 AM, Flavio Percoco wrote:
 
 
  As promissed: https://review.openstack.org/#/c/193804/
 
  Cheers, 
 You can't deprecate a driver without providing a viable alternative.
 
 Right now, QPID is the only driver that supports  Kerberos.
 
 TO support Kerberos, tyou need support for the GSSAPI library, which is 
 usually done via support for SASL.  Why is it so convoluted...historical...
 
 We've talked with both teams (I work with Ken) and I think Proton is 
 likely going to be the first to have support.  The folks working on 
 Rabbit have the double hurdle of getting SASL support into Erlang first, 
 and then support for SASL into Rabbit. They've indicated a preference 
 for getting it in to the AMQP 1.0 driver, and not bothering with the 
 exisiting, but, check me on this, the Oso.Messaging  code only support 
 the pre 1.0 Rabbit.
 
 
 So..until we have a viable alternative, please leave QPID alone. I've 
 not been bothering people about it, as there seems to be work to get 
 ahead, but until either Rabbit or  Proton support Kerberos, I need QPID 
 as is.
 

Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to stay in
tree, that would be helpful and would put the stops on this deprecation.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-20 Thread Flavio Percoco

On 19/06/15 15:01 +, Ken Giusti wrote:



On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com wrote:

   On 18/06/15 16:37 -0400, Doug Hellmann wrote:
   Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.
   
According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.
   
So, with that, the following is the status of each driver in tree right
now:
   
rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.
   
   +1
   
   
qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.
   
   +1 - I believe Ken and the other folks interested in this indicated that
   the AMQP 1.0 driver should replace this one.

   The qpid driver should be deprecated, I'll be doing so in the next
   couple of days. Look forward to the patch.


+1


As promissed: https://review.openstack.org/#/c/193804/

Cheers,
Flavio


 

   
   Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
   separate from the driver named qpid).

   I'd like to clarify something about the AMQP 1.0 driver. It's not a
   direct replacement for the qpidd one because it uses an entirely
   different protocol that recently became a standard.

   The reason I mention this is because it doesn't really require qpidd -
   not the double d - which is the broker daemon in the qpid family. I
   guess the confusion comes because the library it sits on top off is
   called qpid-proton.

   The qpid family is a set of tools that provide messaging capabilities.
   Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
   library), qpid-dispatch (message router). It's confusing indeed.

   The importance of this distinction is that the amqp1.0 driver in
   oslo.messaging is intended as a protocol based driver and not
   targetting any technology. That is to say, that it could be written
   using a library that is not qpid-proton and it can talk to any message
   router or message broker that has support for amqp 1.0.



+1 - yeah, we really shouldn't be considering the amqp1 driver as simply the
replacement qpid driver - as Flavio points out it has the potential to
provide compatibility with other messaging back ends.

Clint - can you also include separate metrics for the amqp1 driver? 


 

   The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
   plugin enabled) and qpidd.

   Since we're at it, let me share some updates:

   The driver unittests now run on every python2 job and the work on
   python3 is almost done. There's also a functional tests gate like we
   have for other drivers.

   The missing bit is an integration gate, which we'll be start working
   on in the next couple of days.

   Hope the above helps clarifying confusions around this driver.

   
   
zmq - Unit test coverage is 74%. There are no currently active
   integration
tests in OpenStack's infrastructure. Several individuals have self
identified as being willing to work on creating them. We have not had
the conversations yet about ongoing support. I recommend we continue to
support their effort to become policy compliant. If that does not
   solidify
by M3 of Liberty, or if the new zmq driver appears with integration
tests and support manpower, then we can deprecate at that time.
   
   +1 - I know interest has been growing in this, so let's keep it going
   and see where we end up.
   
   
There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?
   
   Normally we would recommend using versionutils from oslo.log, but we've
   been trying to avoid making oslo.log a dependency of the oslo libs
   because it uses some of them and that introduces cycles. Dims had a
   patch recently that just used a DeprecationWarning, and relied on
   oslo.log to redirect the warning to the log file. That seems like a good
   pattern to repeat.

   Can we use debtcollector to decorate the main driver class? A warning
   will be printted every time an 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-20 Thread Flavio Percoco

On 19/06/15 13:43 -0700, Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2015-06-18 23:14:49 -0700:

[snip]

I'd like to clarify something about the AMQP 1.0 driver. It's not a
direct replacement for the qpidd one because it uses an entirely
different protocol that recently became a standard.



Could you clarify where the code actually is, or more specifically, why
the code doesn't follow the same organization as the others? There's no
impl_amqp and the tests don't live in tests/drivers, but in
tests/test_amqp_driver. I left it out because I didn't realize this new
driver lived in tree.


As Ken mentioned in a different email, the amqp driver was created as
a protocol specific driver rather than a technology (as in software)
based driver. Therefore, at the time, it was decided to put it in a
different package to separate it from others.

Now that you mentioned this, we could really move out of protocols and
rename the `amqp` package into `impl_amqp` but still keep it as a
package.

TBH, I prefer using packages to organize these drivers since some of
them (or all?) require more than one module to be implemented.



Anyway, this driver appears to land in the prospective drivers category
in the spec. We failed to call out drivers that are still considered
experimental, however I kind of feel like the driver should be out
of tree until such time as it is usable enough that it _could_ have an
integration test.

So, is it feasible that this driver will fall in-line with the others in
the next 6 months, or should we be looking at either revising the
policy, or moving it out of tree until it is capable of that?


Yes, I'm confident this will happen. It went from having a single gate
running funcitonal tests to running unittests in all gates but python3
(this will be solved in the next week or two) and the original
functional gate.

The team is working on adding an integrated gate for this driver as
well. If by the time the freeze is coming the driver is still not
running integrated tests, I'd agree on removing it.

As it stands right now, it is moving on the right direction.

[snip]



Can we use debtcollector to decorate the main driver class? A warning
will be printted every time an instance of such class is created
(rather than at import time).

If we don't want to add a dependency on that, we could just use a
DeprecationWarning as Doug mentioned.



From a user perspective, as long as it only warns once per invocation of
the consuming process (so it should warn when nova-conductor starts, not
when nova-conductor sends a message), then I think we should just use
DeprecationWarning's because they're relatively simple. Unless I'm
missing something magical that debtcollector does.


I don't think you're missing anything. I'm asking because this is what
debtcollator was created for.

--
@flaper87
Flavio Percoco


pgpo1NrEhDH8v.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-20 Thread Flavio Percoco

On 19/06/15 15:01 +, Ken Giusti wrote:



On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com wrote:

   On 18/06/15 16:37 -0400, Doug Hellmann wrote:
   Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.
   
According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.
   
So, with that, the following is the status of each driver in tree right
now:
   
rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.
   
   +1
   
   
qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.
   
   +1 - I believe Ken and the other folks interested in this indicated that
   the AMQP 1.0 driver should replace this one.

   The qpid driver should be deprecated, I'll be doing so in the next
   couple of days. Look forward to the patch.


+1
 

   
   Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
   separate from the driver named qpid).

   I'd like to clarify something about the AMQP 1.0 driver. It's not a
   direct replacement for the qpidd one because it uses an entirely
   different protocol that recently became a standard.

   The reason I mention this is because it doesn't really require qpidd -
   not the double d - which is the broker daemon in the qpid family. I
   guess the confusion comes because the library it sits on top off is
   called qpid-proton.

   The qpid family is a set of tools that provide messaging capabilities.
   Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
   library), qpid-dispatch (message router). It's confusing indeed.

   The importance of this distinction is that the amqp1.0 driver in
   oslo.messaging is intended as a protocol based driver and not
   targetting any technology. That is to say, that it could be written
   using a library that is not qpid-proton and it can talk to any message
   router or message broker that has support for amqp 1.0.



+1 - yeah, we really shouldn't be considering the amqp1 driver as simply the
replacement qpid driver - as Flavio points out it has the potential to
provide compatibility with other messaging back ends.

Clint - can you also include separate metrics for the amqp1 driver? 



oslo_messaging/_drivers/protocols/amqp/controller 302 46 0 80 18 82%
oslo_messaging/_drivers/protocols/amqp/driver 134 13 0 28 6 88%
oslo_messaging/_drivers/protocols/amqp/drivertasks 52 5 0 8 2 85%
oslo_messaging/_drivers/protocols/amqp/eventloop 195 43 0 64 18 71%

Cheers,
Flavio



 

   The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
   plugin enabled) and qpidd.

   Since we're at it, let me share some updates:

   The driver unittests now run on every python2 job and the work on
   python3 is almost done. There's also a functional tests gate like we
   have for other drivers.

   The missing bit is an integration gate, which we'll be start working
   on in the next couple of days.

   Hope the above helps clarifying confusions around this driver.

   
   
zmq - Unit test coverage is 74%. There are no currently active
   integration
tests in OpenStack's infrastructure. Several individuals have self
identified as being willing to work on creating them. We have not had
the conversations yet about ongoing support. I recommend we continue to
support their effort to become policy compliant. If that does not
   solidify
by M3 of Liberty, or if the new zmq driver appears with integration
tests and support manpower, then we can deprecate at that time.
   
   +1 - I know interest has been growing in this, so let's keep it going
   and see where we end up.
   
   
There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?
   
   Normally we would recommend using versionutils from oslo.log, but we've
   been trying to avoid making oslo.log a dependency of the oslo libs
   because it uses some of them and that introduces cycles. Dims had a
   patch recently that just used a DeprecationWarning, and relied 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Clint Byrum
Excerpts from Ken Giusti's message of 2015-06-19 08:01:46 -0700:
 On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com wrote:
 
  On 18/06/15 16:37 -0400, Doug Hellmann wrote:
  Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
   Hello! I know there's been a lot of churn and misunderstanding over the
   recent devstack changes, so I wanted to make it clear where we're going
   with messaging drivers now that the policy [1] was approved.
  
   According to the policy, drivers need to have at least 60% unit test
   coverage, and an integration test suite with at least 3 popular
   OpenStack projects, with preference for Nova, Cinder, and Glance, and
   individuals who will support said test suite.
  
   So, with that, the following is the status of each driver in tree right
   now:
  
   rabbit - 89% Unit test coverage. Being the default in devstack, and
   the default in nearly every project's config files, this one is heavily
   integration tested. There are multiple individuals who have proven to
   be able to debug failures related to RabbitMQ and are well known in
   the community.
  
  +1
  
  
   qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
   find any integration testing being done, so it fails that. I also have
   not found anyone who will step up and support it. So this should be
   deprecated immediately.
  
  +1 - I believe Ken and the other folks interested in this indicated that
  the AMQP 1.0 driver should replace this one.
 
  The qpid driver should be deprecated, I'll be doing so in the next
  couple of days. Look forward to the patch.
 
  +1
 
  
  Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
  separate from the driver named qpid).
 
  I'd like to clarify something about the AMQP 1.0 driver. It's not a
  direct replacement for the qpidd one because it uses an entirely
  different protocol that recently became a standard.
 
  The reason I mention this is because it doesn't really require qpidd -
  not the double d - which is the broker daemon in the qpid family. I
  guess the confusion comes because the library it sits on top off is
  called qpid-proton.
 
  The qpid family is a set of tools that provide messaging capabilities.
  Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
  library), qpid-dispatch (message router). It's confusing indeed.
 
  The importance of this distinction is that the amqp1.0 driver in
  oslo.messaging is intended as a protocol based driver and not
  targetting any technology. That is to say, that it could be written
  using a library that is not qpid-proton and it can talk to any message
  router or message broker that has support for amqp 1.0.
 
 
 +1 - yeah, we really shouldn't be considering the amqp1 driver as simply
 the replacement qpid driver - as Flavio points out it has the potential
 to provide compatibility with other messaging back ends.
 
 Clint - can you also include separate metrics for the amqp1 driver?
 

It's far less clear to me how to measure the unit test coverage of the
amqp driver. I'll wait for Flavio's answer to my question about where
the code lives, because it is definitely not organized like the others.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Clint Byrum
Excerpts from Flavio Percoco's message of 2015-06-18 23:14:49 -0700:
 On 18/06/15 16:37 -0400, Doug Hellmann wrote:
 Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
  Hello! I know there's been a lot of churn and misunderstanding over the
  recent devstack changes, so I wanted to make it clear where we're going
  with messaging drivers now that the policy [1] was approved.
 
  According to the policy, drivers need to have at least 60% unit test
  coverage, and an integration test suite with at least 3 popular
  OpenStack projects, with preference for Nova, Cinder, and Glance, and
  individuals who will support said test suite.
 
  So, with that, the following is the status of each driver in tree right
  now:
 
  rabbit - 89% Unit test coverage. Being the default in devstack, and
  the default in nearly every project's config files, this one is heavily
  integration tested. There are multiple individuals who have proven to
  be able to debug failures related to RabbitMQ and are well known in
  the community.
 
 +1
 
 
  qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
  find any integration testing being done, so it fails that. I also have
  not found anyone who will step up and support it. So this should be
  deprecated immediately.
 
 +1 - I believe Ken and the other folks interested in this indicated that
 the AMQP 1.0 driver should replace this one.
 
 The qpid driver should be deprecated, I'll be doing so in the next
 couple of days. Look forward to the patch.
 
 
 Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
 separate from the driver named qpid).
 
 I'd like to clarify something about the AMQP 1.0 driver. It's not a
 direct replacement for the qpidd one because it uses an entirely
 different protocol that recently became a standard.
 

Could you clarify where the code actually is, or more specifically, why
the code doesn't follow the same organization as the others? There's no
impl_amqp and the tests don't live in tests/drivers, but in
tests/test_amqp_driver. I left it out because I didn't realize this new
driver lived in tree.

Anyway, this driver appears to land in the prospective drivers category
in the spec. We failed to call out drivers that are still considered
experimental, however I kind of feel like the driver should be out
of tree until such time as it is usable enough that it _could_ have an
integration test.

So, is it feasible that this driver will fall in-line with the others in
the next 6 months, or should we be looking at either revising the
policy, or moving it out of tree until it is capable of that?

 The reason I mention this is because it doesn't really require qpidd -
 not the double d - which is the broker daemon in the qpid family. I
 guess the confusion comes because the library it sits on top off is
 called qpid-proton.
 
 The qpid family is a set of tools that provide messaging capabilities.
 Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
 library), qpid-dispatch (message router). It's confusing indeed.
 
 The importance of this distinction is that the amqp1.0 driver in
 oslo.messaging is intended as a protocol based driver and not
 targetting any technology. That is to say, that it could be written
 using a library that is not qpid-proton and it can talk to any message
 router or message broker that has support for amqp 1.0.
 
 The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
 plugin enabled) and qpidd.
 
 Since we're at it, let me share some updates:
 
 The driver unittests now run on every python2 job and the work on
 python3 is almost done. There's also a functional tests gate like we
 have for other drivers.
 
 The missing bit is an integration gate, which we'll be start working
 on in the next couple of days.
 
 Hope the above helps clarifying confusions around this driver.
 

That definitely clarifies the vision, thanks for sharing.

  There's also the question of _how_ to deprecate them. I figure we should
  deprecate when the driver is loaded. Is there an existing mechanism
  that someone can point me to, or should I look at adding that to
  oslo.messaging/stevedore?
 
 Normally we would recommend using versionutils from oslo.log, but we've
 been trying to avoid making oslo.log a dependency of the oslo libs
 because it uses some of them and that introduces cycles. Dims had a
 patch recently that just used a DeprecationWarning, and relied on
 oslo.log to redirect the warning to the log file. That seems like a good
 pattern to repeat.
 
 Can we use debtcollector to decorate the main driver class? A warning
 will be printted every time an instance of such class is created
 (rather than at import time).
 
 If we don't want to add a dependency on that, we could just use a
 DeprecationWarning as Doug mentioned.
 

From a user perspective, as long as it only warns once per invocation of
the consuming process (so it should warn when nova-conductor starts, not
when 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Ken Giusti
On Fri, Jun 19, 2015 at 4:47 PM Clint Byrum cl...@fewbar.com wrote:

 Excerpts from Ken Giusti's message of 2015-06-19 08:01:46 -0700:
  On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com
 wrote:
 
   On 18/06/15 16:37 -0400, Doug Hellmann wrote:
   Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
Hello! I know there's been a lot of churn and misunderstanding over
 the
recent devstack changes, so I wanted to make it clear where we're
 going
with messaging drivers now that the policy [1] was approved.
   
According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance,
 and
individuals who will support said test suite.
   
So, with that, the following is the status of each driver in tree
 right
now:
   
rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is
 heavily
integration tested. There are multiple individuals who have proven
 to
be able to debug failures related to RabbitMQ and are well known in
the community.
   
   +1
   
   
qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also
 have
not found anyone who will step up and support it. So this should be
deprecated immediately.
   
   +1 - I believe Ken and the other folks interested in this indicated
 that
   the AMQP 1.0 driver should replace this one.
  
   The qpid driver should be deprecated, I'll be doing so in the next
   couple of days. Look forward to the patch.
  
   +1
 
   
   Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
   separate from the driver named qpid).
  
   I'd like to clarify something about the AMQP 1.0 driver. It's not a
   direct replacement for the qpidd one because it uses an entirely
   different protocol that recently became a standard.
  
   The reason I mention this is because it doesn't really require qpidd -
   not the double d - which is the broker daemon in the qpid family. I
   guess the confusion comes because the library it sits on top off is
   called qpid-proton.
  
   The qpid family is a set of tools that provide messaging capabilities.
   Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
   library), qpid-dispatch (message router). It's confusing indeed.
  
   The importance of this distinction is that the amqp1.0 driver in
   oslo.messaging is intended as a protocol based driver and not
   targetting any technology. That is to say, that it could be written
   using a library that is not qpid-proton and it can talk to any message
   router or message broker that has support for amqp 1.0.
  
  
  +1 - yeah, we really shouldn't be considering the amqp1 driver as simply
  the replacement qpid driver - as Flavio points out it has the potential
  to provide compatibility with other messaging back ends.
 
  Clint - can you also include separate metrics for the amqp1 driver?
 

 It's far less clear to me how to measure the unit test coverage of the
 amqp driver. I'll wait for Flavio's answer to my question about where
 the code lives, because it is definitely not organized like the others.


No, it isn't.  At the time we proposed this structure for the amqp1 driver,
there really wasn't much formal structure within the driver directory.  We
had impl_rabbit, which was copied to impl_qpid in the hopes that all the
rest of the code could be shared (it didn't work out very well).  And the
zmq code wasn't working at the time.  And all of it was crammed into one
flat directory.

Rather than heap yet more stuff into that directory, we proposed to put the
amqp1 driver into its own sub-directory.  We chose a 'protocols'
subdirectory, into which the amqp1 driver lives in the 'amqp'
subdirectory.  The hope was that other protocols would put their
implementation bits into that protocols directory rather than lump them in
directly under _drivers.

It looks like the zmq driver will be structured a bit different from that -
there will be a top level impl_zmq file along side a zmq_driver directory.

I like the way zmq has laid out the sources, and maybe that should be the
'official' structure to olso.messaging drivers.  If that's the case I'll be
more that happy to arrange the amqp1 driver in a similar manner.

I've gone off into the weeds, sorry.

The amqp1 driver lives in _drivers/protocols/amqp directory (for now).  And
yes, those unit tests should be in the tests/drivers directory like
everything else - when the tests/drivers directory was created that file
wasn't moved.  Oversight - I'll submit a patch to fix it.

As far as coverage testing is concerned - I'm not sure how best to do this
accurately.  The amqp1 driver doesn't use any of the code in amqpdriver.py,
amqp.py, common.py (I think), 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Joshua Harlow

Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2015-06-18 23:14:49 -0700:

On 18/06/15 16:37 -0400, Doug Hellmann wrote:

Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:

Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.

According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.

So, with that, the following is the status of each driver in tree right
now:

rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.

+1


qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.

+1 - I believe Ken and the other folks interested in this indicated that
the AMQP 1.0 driver should replace this one.

The qpid driver should be deprecated, I'll be doing so in the next
couple of days. Look forward to the patch.


Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
separate from the driver named qpid).

I'd like to clarify something about the AMQP 1.0 driver. It's not a
direct replacement for the qpidd one because it uses an entirely
different protocol that recently became a standard.



Could you clarify where the code actually is, or more specifically, why
the code doesn't follow the same organization as the others? There's no
impl_amqp and the tests don't live in tests/drivers, but in
tests/test_amqp_driver. I left it out because I didn't realize this new
driver lived in tree.

Anyway, this driver appears to land in the prospective drivers category
in the spec. We failed to call out drivers that are still considered
experimental, however I kind of feel like the driver should be out
of tree until such time as it is usable enough that it _could_ have an
integration test.

So, is it feasible that this driver will fall in-line with the others in
the next 6 months, or should we be looking at either revising the
policy, or moving it out of tree until it is capable of that?


The reason I mention this is because it doesn't really require qpidd -
not the double d - which is the broker daemon in the qpid family. I
guess the confusion comes because the library it sits on top off is
called qpid-proton.

The qpid family is a set of tools that provide messaging capabilities.
Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
library), qpid-dispatch (message router). It's confusing indeed.

The importance of this distinction is that the amqp1.0 driver in
oslo.messaging is intended as a protocol based driver and not
targetting any technology. That is to say, that it could be written
using a library that is not qpid-proton and it can talk to any message
router or message broker that has support for amqp 1.0.

The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
plugin enabled) and qpidd.

Since we're at it, let me share some updates:

The driver unittests now run on every python2 job and the work on
python3 is almost done. There's also a functional tests gate like we
have for other drivers.

The missing bit is an integration gate, which we'll be start working
on in the next couple of days.

Hope the above helps clarifying confusions around this driver.



That definitely clarifies the vision, thanks for sharing.


There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?

Normally we would recommend using versionutils from oslo.log, but we've
been trying to avoid making oslo.log a dependency of the oslo libs
because it uses some of them and that introduces cycles. Dims had a
patch recently that just used a DeprecationWarning, and relied on
oslo.log to redirect the warning to the log file. That seems like a good
pattern to repeat.

Can we use debtcollector to decorate the main driver class? A warning
will be printted every time an instance of such class is created
(rather than at import time).

If we don't want to add a dependency on that, we could just use a
DeprecationWarning as Doug mentioned.



 From a user perspective, as long as it only warns once per invocation of
the consuming process (so it should warn when nova-conductor starts, not
when nova-conductor sends a message), then I think we should just use
DeprecationWarning's 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Flavio Percoco

On 18/06/15 16:37 -0400, Doug Hellmann wrote:

Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:

Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.

According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.

So, with that, the following is the status of each driver in tree right
now:

rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.


+1



qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.


+1 - I believe Ken and the other folks interested in this indicated that
the AMQP 1.0 driver should replace this one.


The qpid driver should be deprecated, I'll be doing so in the next
couple of days. Look forward to the patch.



Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
separate from the driver named qpid).


I'd like to clarify something about the AMQP 1.0 driver. It's not a
direct replacement for the qpidd one because it uses an entirely
different protocol that recently became a standard.

The reason I mention this is because it doesn't really require qpidd -
not the double d - which is the broker daemon in the qpid family. I
guess the confusion comes because the library it sits on top off is
called qpid-proton.

The qpid family is a set of tools that provide messaging capabilities.
Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
library), qpid-dispatch (message router). It's confusing indeed.

The importance of this distinction is that the amqp1.0 driver in
oslo.messaging is intended as a protocol based driver and not
targetting any technology. That is to say, that it could be written
using a library that is not qpid-proton and it can talk to any message
router or message broker that has support for amqp 1.0.

The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
plugin enabled) and qpidd.

Since we're at it, let me share some updates:

The driver unittests now run on every python2 job and the work on
python3 is almost done. There's also a functional tests gate like we
have for other drivers.

The missing bit is an integration gate, which we'll be start working
on in the next couple of days.

Hope the above helps clarifying confusions around this driver.





zmq - Unit test coverage is 74%. There are no currently active integration
tests in OpenStack's infrastructure. Several individuals have self
identified as being willing to work on creating them. We have not had
the conversations yet about ongoing support. I recommend we continue to
support their effort to become policy compliant. If that does not solidify
by M3 of Liberty, or if the new zmq driver appears with integration
tests and support manpower, then we can deprecate at that time.


+1 - I know interest has been growing in this, so let's keep it going
and see where we end up.



There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?


Normally we would recommend using versionutils from oslo.log, but we've
been trying to avoid making oslo.log a dependency of the oslo libs
because it uses some of them and that introduces cycles. Dims had a
patch recently that just used a DeprecationWarning, and relied on
oslo.log to redirect the warning to the log file. That seems like a good
pattern to repeat.


Can we use debtcollector to decorate the main driver class? A warning
will be printted every time an instance of such class is created
(rather than at import time).

If we don't want to add a dependency on that, we could just use a
DeprecationWarning as Doug mentioned.



Doug



[1] 
http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
@flaper87
Flavio Percoco


pgpRxjxkSBOzd.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Gordon Sim

On 06/19/2015 07:14 AM, Flavio Percoco wrote:

The qpid family is a set of tools that provide messaging capabilities.
Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
library), qpid-dispatch (message router). It's confusing indeed.


Apache Qpid is an Apache Software Foundation project aiming to help 
promote the AMQP protocol by open, collaborative development of 
libraries, services and tools that use it. It includes many different 
components now. So the 'qpid' in the name of the different components 
denotes the community from which they originated. Hopefully that makes 
it less confusing.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Ken Giusti
On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com wrote:

 On 18/06/15 16:37 -0400, Doug Hellmann wrote:
 Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
  Hello! I know there's been a lot of churn and misunderstanding over the
  recent devstack changes, so I wanted to make it clear where we're going
  with messaging drivers now that the policy [1] was approved.
 
  According to the policy, drivers need to have at least 60% unit test
  coverage, and an integration test suite with at least 3 popular
  OpenStack projects, with preference for Nova, Cinder, and Glance, and
  individuals who will support said test suite.
 
  So, with that, the following is the status of each driver in tree right
  now:
 
  rabbit - 89% Unit test coverage. Being the default in devstack, and
  the default in nearly every project's config files, this one is heavily
  integration tested. There are multiple individuals who have proven to
  be able to debug failures related to RabbitMQ and are well known in
  the community.
 
 +1
 
 
  qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
  find any integration testing being done, so it fails that. I also have
  not found anyone who will step up and support it. So this should be
  deprecated immediately.
 
 +1 - I believe Ken and the other folks interested in this indicated that
 the AMQP 1.0 driver should replace this one.

 The qpid driver should be deprecated, I'll be doing so in the next
 couple of days. Look forward to the patch.

 +1


 
 Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
 separate from the driver named qpid).

 I'd like to clarify something about the AMQP 1.0 driver. It's not a
 direct replacement for the qpidd one because it uses an entirely
 different protocol that recently became a standard.

 The reason I mention this is because it doesn't really require qpidd -
 not the double d - which is the broker daemon in the qpid family. I
 guess the confusion comes because the library it sits on top off is
 called qpid-proton.

 The qpid family is a set of tools that provide messaging capabilities.
 Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
 library), qpid-dispatch (message router). It's confusing indeed.

 The importance of this distinction is that the amqp1.0 driver in
 oslo.messaging is intended as a protocol based driver and not
 targetting any technology. That is to say, that it could be written
 using a library that is not qpid-proton and it can talk to any message
 router or message broker that has support for amqp 1.0.


+1 - yeah, we really shouldn't be considering the amqp1 driver as simply
the replacement qpid driver - as Flavio points out it has the potential
to provide compatibility with other messaging back ends.

Clint - can you also include separate metrics for the amqp1 driver?




 The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
 plugin enabled) and qpidd.

 Since we're at it, let me share some updates:

 The driver unittests now run on every python2 job and the work on
 python3 is almost done. There's also a functional tests gate like we
 have for other drivers.

 The missing bit is an integration gate, which we'll be start working
 on in the next couple of days.

 Hope the above helps clarifying confusions around this driver.

 
 
  zmq - Unit test coverage is 74%. There are no currently active
 integration
  tests in OpenStack's infrastructure. Several individuals have self
  identified as being willing to work on creating them. We have not had
  the conversations yet about ongoing support. I recommend we continue to
  support their effort to become policy compliant. If that does not
 solidify
  by M3 of Liberty, or if the new zmq driver appears with integration
  tests and support manpower, then we can deprecate at that time.
 
 +1 - I know interest has been growing in this, so let's keep it going
 and see where we end up.
 
 
  There's also the question of _how_ to deprecate them. I figure we should
  deprecate when the driver is loaded. Is there an existing mechanism
  that someone can point me to, or should I look at adding that to
  oslo.messaging/stevedore?
 
 Normally we would recommend using versionutils from oslo.log, but we've
 been trying to avoid making oslo.log a dependency of the oslo libs
 because it uses some of them and that introduces cycles. Dims had a
 patch recently that just used a DeprecationWarning, and relied on
 oslo.log to redirect the warning to the log file. That seems like a good
 pattern to repeat.

 Can we use debtcollector to decorate the main driver class? A warning
 will be printted every time an instance of such class is created
 (rather than at import time).

 If we don't want to add a dependency on that, we could just use a
 DeprecationWarning as Doug mentioned.

 
 Doug
 
 
  [1]
 http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html
 
 
 

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Joshua Harlow

Flavio Percoco wrote:

On 18/06/15 16:37 -0400, Doug Hellmann wrote:

Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:

Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.

According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.

So, with that, the following is the status of each driver in tree right
now:

rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.


+1



qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.


+1 - I believe Ken and the other folks interested in this indicated that
the AMQP 1.0 driver should replace this one.


The qpid driver should be deprecated, I'll be doing so in the next
couple of days. Look forward to the patch.



Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
separate from the driver named qpid).


I'd like to clarify something about the AMQP 1.0 driver. It's not a
direct replacement for the qpidd one because it uses an entirely
different protocol that recently became a standard.

The reason I mention this is because it doesn't really require qpidd -
not the double d - which is the broker daemon in the qpid family. I
guess the confusion comes because the library it sits on top off is
called qpid-proton.

The qpid family is a set of tools that provide messaging capabilities.
Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
library), qpid-dispatch (message router). It's confusing indeed.

The importance of this distinction is that the amqp1.0 driver in
oslo.messaging is intended as a protocol based driver and not
targetting any technology. That is to say, that it could be written
using a library that is not qpid-proton and it can talk to any message
router or message broker that has support for amqp 1.0.

The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
plugin enabled) and qpidd.

Since we're at it, let me share some updates:

The driver unittests now run on every python2 job and the work on
python3 is almost done. There's also a functional tests gate like we
have for other drivers.

The missing bit is an integration gate, which we'll be start working
on in the next couple of days.

Hope the above helps clarifying confusions around this driver.





zmq - Unit test coverage is 74%. There are no currently active
integration
tests in OpenStack's infrastructure. Several individuals have self
identified as being willing to work on creating them. We have not had
the conversations yet about ongoing support. I recommend we continue to
support their effort to become policy compliant. If that does not
solidify
by M3 of Liberty, or if the new zmq driver appears with integration
tests and support manpower, then we can deprecate at that time.


+1 - I know interest has been growing in this, so let's keep it going
and see where we end up.



There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?


Normally we would recommend using versionutils from oslo.log, but we've
been trying to avoid making oslo.log a dependency of the oslo libs
because it uses some of them and that introduces cycles. Dims had a
patch recently that just used a DeprecationWarning, and relied on
oslo.log to redirect the warning to the log file. That seems like a good
pattern to repeat.


Can we use debtcollector to decorate the main driver class? A warning
will be printted every time an instance of such class is created
(rather than at import time).


Seems fair to me:

http://docs.openstack.org/developer/debtcollector/api.html#debtcollector.removals.remove 
(which itself uses a deprecation warning...).




If we don't want to add a dependency on that, we could just use a
DeprecationWarning as Doug mentioned.



Doug



[1]
http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html




__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Doug Hellmann
Excerpts from Flavio Percoco's message of 2015-06-19 08:14:49 +0200:
 On 18/06/15 16:37 -0400, Doug Hellmann wrote:
 Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
  Hello! I know there's been a lot of churn and misunderstanding over the
  recent devstack changes, so I wanted to make it clear where we're going
  with messaging drivers now that the policy [1] was approved.
 
  According to the policy, drivers need to have at least 60% unit test
  coverage, and an integration test suite with at least 3 popular
  OpenStack projects, with preference for Nova, Cinder, and Glance, and
  individuals who will support said test suite.
 
  So, with that, the following is the status of each driver in tree right
  now:
 
  rabbit - 89% Unit test coverage. Being the default in devstack, and
  the default in nearly every project's config files, this one is heavily
  integration tested. There are multiple individuals who have proven to
  be able to debug failures related to RabbitMQ and are well known in
  the community.
 
 +1
 
 
  qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
  find any integration testing being done, so it fails that. I also have
  not found anyone who will step up and support it. So this should be
  deprecated immediately.
 
 +1 - I believe Ken and the other folks interested in this indicated that
 the AMQP 1.0 driver should replace this one.
 
 The qpid driver should be deprecated, I'll be doing so in the next
 couple of days. Look forward to the patch.
 
 
 Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
 separate from the driver named qpid).
 
 I'd like to clarify something about the AMQP 1.0 driver. It's not a
 direct replacement for the qpidd one because it uses an entirely
 different protocol that recently became a standard.
 
 The reason I mention this is because it doesn't really require qpidd -
 not the double d - which is the broker daemon in the qpid family. I
 guess the confusion comes because the library it sits on top off is
 called qpid-proton.
 
 The qpid family is a set of tools that provide messaging capabilities.
 Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
 library), qpid-dispatch (message router). It's confusing indeed.
 
 The importance of this distinction is that the amqp1.0 driver in
 oslo.messaging is intended as a protocol based driver and not
 targetting any technology. That is to say, that it could be written
 using a library that is not qpid-proton and it can talk to any message
 router or message broker that has support for amqp 1.0.
 
 The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
 plugin enabled) and qpidd.
 
 Since we're at it, let me share some updates:
 
 The driver unittests now run on every python2 job and the work on
 python3 is almost done. There's also a functional tests gate like we
 have for other drivers.
 
 The missing bit is an integration gate, which we'll be start working
 on in the next couple of days.
 
 Hope the above helps clarifying confusions around this driver.
 
 
 
  zmq - Unit test coverage is 74%. There are no currently active integration
  tests in OpenStack's infrastructure. Several individuals have self
  identified as being willing to work on creating them. We have not had
  the conversations yet about ongoing support. I recommend we continue to
  support their effort to become policy compliant. If that does not solidify
  by M3 of Liberty, or if the new zmq driver appears with integration
  tests and support manpower, then we can deprecate at that time.
 
 +1 - I know interest has been growing in this, so let's keep it going
 and see where we end up.
 
 
  There's also the question of _how_ to deprecate them. I figure we should
  deprecate when the driver is loaded. Is there an existing mechanism
  that someone can point me to, or should I look at adding that to
  oslo.messaging/stevedore?
 
 Normally we would recommend using versionutils from oslo.log, but we've
 been trying to avoid making oslo.log a dependency of the oslo libs
 because it uses some of them and that introduces cycles. Dims had a
 patch recently that just used a DeprecationWarning, and relied on
 oslo.log to redirect the warning to the log file. That seems like a good
 pattern to repeat.
 
 Can we use debtcollector to decorate the main driver class? A warning
 will be printted every time an instance of such class is created
 (rather than at import time).

debtcollector was originally meant to be used for developer-facing
deprecations, but if we can make it emit messages indicating the release
cycle when the driver is going to be fully dropped, it might be the
right answer.

 
 If we don't want to add a dependency on that, we could just use a
 DeprecationWarning as Doug mentioned.

The dependencies are only a concern to me if we introduce a cycle, so as
long as that doesn't happen I would be fine with us using debtcollector.

Doug

 
 
 Doug
 
 
  [1] 
  

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-19 Thread Doug Hellmann
Excerpts from Ken Giusti's message of 2015-06-19 15:01:46 +:
 On Fri, Jun 19, 2015 at 2:15 AM Flavio Percoco fla...@redhat.com wrote:
 
  On 18/06/15 16:37 -0400, Doug Hellmann wrote:
  Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
   Hello! I know there's been a lot of churn and misunderstanding over the
   recent devstack changes, so I wanted to make it clear where we're going
   with messaging drivers now that the policy [1] was approved.
  
   According to the policy, drivers need to have at least 60% unit test
   coverage, and an integration test suite with at least 3 popular
   OpenStack projects, with preference for Nova, Cinder, and Glance, and
   individuals who will support said test suite.
  
   So, with that, the following is the status of each driver in tree right
   now:
  
   rabbit - 89% Unit test coverage. Being the default in devstack, and
   the default in nearly every project's config files, this one is heavily
   integration tested. There are multiple individuals who have proven to
   be able to debug failures related to RabbitMQ and are well known in
   the community.
  
  +1
  
  
   qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
   find any integration testing being done, so it fails that. I also have
   not found anyone who will step up and support it. So this should be
   deprecated immediately.
  
  +1 - I believe Ken and the other folks interested in this indicated that
  the AMQP 1.0 driver should replace this one.
 
  The qpid driver should be deprecated, I'll be doing so in the next
  couple of days. Look forward to the patch.
 
  +1
 
  
  Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
  separate from the driver named qpid).
 
  I'd like to clarify something about the AMQP 1.0 driver. It's not a
  direct replacement for the qpidd one because it uses an entirely
  different protocol that recently became a standard.
 
  The reason I mention this is because it doesn't really require qpidd -
  not the double d - which is the broker daemon in the qpid family. I
  guess the confusion comes because the library it sits on top off is
  called qpid-proton.
 
  The qpid family is a set of tools that provide messaging capabilities.
  Among those you find qpidd (broker daemon), qpid-proton (amqp1.0
  library), qpid-dispatch (message router). It's confusing indeed.
 
  The importance of this distinction is that the amqp1.0 driver in
  oslo.messaging is intended as a protocol based driver and not
  targetting any technology. That is to say, that it could be written
  using a library that is not qpid-proton and it can talk to any message
  router or message broker that has support for amqp 1.0.
 
 
 +1 - yeah, we really shouldn't be considering the amqp1 driver as simply
 the replacement qpid driver - as Flavio points out it has the potential
 to provide compatibility with other messaging back ends.

OK, sorry for my confusion on that and thanks to you and Flavio for
clarifying.

 
 Clint - can you also include separate metrics for the amqp1 driver?
 
  The ones we're targetting for the gate are rabbitmq (with the amqp 1.0
  plugin enabled) and qpidd.
 
  Since we're at it, let me share some updates:
 
  The driver unittests now run on every python2 job and the work on
  python3 is almost done. There's also a functional tests gate like we
  have for other drivers.
 
  The missing bit is an integration gate, which we'll be start working
  on in the next couple of days.
 
  Hope the above helps clarifying confusions around this driver.
 
  
  
   zmq - Unit test coverage is 74%. There are no currently active
  integration
   tests in OpenStack's infrastructure. Several individuals have self
   identified as being willing to work on creating them. We have not had
   the conversations yet about ongoing support. I recommend we continue to
   support their effort to become policy compliant. If that does not
  solidify
   by M3 of Liberty, or if the new zmq driver appears with integration
   tests and support manpower, then we can deprecate at that time.
  
  +1 - I know interest has been growing in this, so let's keep it going
  and see where we end up.
  
  
   There's also the question of _how_ to deprecate them. I figure we should
   deprecate when the driver is loaded. Is there an existing mechanism
   that someone can point me to, or should I look at adding that to
   oslo.messaging/stevedore?
  
  Normally we would recommend using versionutils from oslo.log, but we've
  been trying to avoid making oslo.log a dependency of the oslo libs
  because it uses some of them and that introduces cycles. Dims had a
  patch recently that just used a DeprecationWarning, and relied on
  oslo.log to redirect the warning to the log file. That seems like a good
  pattern to repeat.
 
  Can we use debtcollector to decorate the main driver class? A warning
  will be printted every time an instance of such class is created
  (rather than at 

[openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-18 Thread Clint Byrum
Hello! I know there's been a lot of churn and misunderstanding over the
recent devstack changes, so I wanted to make it clear where we're going
with messaging drivers now that the policy [1] was approved.

According to the policy, drivers need to have at least 60% unit test
coverage, and an integration test suite with at least 3 popular
OpenStack projects, with preference for Nova, Cinder, and Glance, and
individuals who will support said test suite.

So, with that, the following is the status of each driver in tree right
now:

rabbit - 89% Unit test coverage. Being the default in devstack, and
the default in nearly every project's config files, this one is heavily
integration tested. There are multiple individuals who have proven to
be able to debug failures related to RabbitMQ and are well known in
the community.

qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
find any integration testing being done, so it fails that. I also have
not found anyone who will step up and support it. So this should be
deprecated immediately.

zmq - Unit test coverage is 74%. There are no currently active integration
tests in OpenStack's infrastructure. Several individuals have self
identified as being willing to work on creating them. We have not had
the conversations yet about ongoing support. I recommend we continue to
support their effort to become policy compliant. If that does not solidify
by M3 of Liberty, or if the new zmq driver appears with integration
tests and support manpower, then we can deprecate at that time.

There's also the question of _how_ to deprecate them. I figure we should
deprecate when the driver is loaded. Is there an existing mechanism
that someone can point me to, or should I look at adding that to
oslo.messaging/stevedore?

[1] 
http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-18 Thread Doug Hellmann
Excerpts from Clint Byrum's message of 2015-06-18 12:47:21 -0700:
 Hello! I know there's been a lot of churn and misunderstanding over the
 recent devstack changes, so I wanted to make it clear where we're going
 with messaging drivers now that the policy [1] was approved.
 
 According to the policy, drivers need to have at least 60% unit test
 coverage, and an integration test suite with at least 3 popular
 OpenStack projects, with preference for Nova, Cinder, and Glance, and
 individuals who will support said test suite.
 
 So, with that, the following is the status of each driver in tree right
 now:
 
 rabbit - 89% Unit test coverage. Being the default in devstack, and
 the default in nearly every project's config files, this one is heavily
 integration tested. There are multiple individuals who have proven to
 be able to debug failures related to RabbitMQ and are well known in
 the community.

+1

 
 qpid - Unit test coverage is at 77%, so it passes that bar. I cannot
 find any integration testing being done, so it fails that. I also have
 not found anyone who will step up and support it. So this should be
 deprecated immediately.

+1 - I believe Ken and the other folks interested in this indicated that
the AMQP 1.0 driver should replace this one.

Speaking of AMQP 1.0, you don't mention that one (it uses qpid, but is
separate from the driver named qpid).

 
 zmq - Unit test coverage is 74%. There are no currently active integration
 tests in OpenStack's infrastructure. Several individuals have self
 identified as being willing to work on creating them. We have not had
 the conversations yet about ongoing support. I recommend we continue to
 support their effort to become policy compliant. If that does not solidify
 by M3 of Liberty, or if the new zmq driver appears with integration
 tests and support manpower, then we can deprecate at that time.

+1 - I know interest has been growing in this, so let's keep it going
and see where we end up.

 
 There's also the question of _how_ to deprecate them. I figure we should
 deprecate when the driver is loaded. Is there an existing mechanism
 that someone can point me to, or should I look at adding that to
 oslo.messaging/stevedore?

Normally we would recommend using versionutils from oslo.log, but we've
been trying to avoid making oslo.log a dependency of the oslo libs
because it uses some of them and that introduces cycles. Dims had a
patch recently that just used a DeprecationWarning, and relied on
oslo.log to redirect the warning to the log file. That seems like a good
pattern to repeat.

Doug

 
 [1] 
 http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html
 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev