Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-06 Thread Monty Taylor


On 12/06/2013 08:35 AM, John Wood wrote:
 Hello folks,
 
 Just an FYI that I've submitted a pull request [1] to replace Celery
 with oslo.messaging.

wow. That was quick!

/me is impressed

Since you jumped on that - I went ahead and jumped on a pbr-ification
patch for you. It may not work yet - I'm on weird network and having
trouble install python things into virtualenvs:

  https://review.openstack.org/60551
  https://review.openstack.org/60552


 I've tagged it as a work in progress per this note:
 
 Please review this CR, which replaces Celery with oslo.messaging
 components. I've verified that this works in my local environment,
 but I still need to add unit testing. I also need to verify that it
 works correctly with an HA Rabbit MQ cluster, as that is a hard
 requirement for Barbican.
 
 Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me
 to very useful links here [2] and here [3] respectively.
 
 [1] https://review.openstack.org/#/c/60427/ [2]
 https://review.openstack.org/#/c/39929 [3]
 https://review.openstack.org/#/c/57880
 
 Thanks, John
 
  From: Monty Taylor
 [mord...@inaugust.com] Sent: Thursday, December 05, 2013 8:35 PM To:
 Mark McLoughlin; Douglas Mendizabal Cc: OpenStack Development Mailing
 List (not for usage questions); openstack...@lists.openstack.org;
 barbi...@lists.rackspace.com Subject: Re: [openstack-tc]
 [openstack-dev] Incubation Request for Barbican
 
 On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
 On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
 
 I agree that this is concerning. And that what's concerning
 isn't so much that the project did something different, but
 rather that choice was apparently made because the project
 thought it was perfectly fine for them to ignore what other
 OpenStack projects do and go off and do its own thing.
 
 We can't make this growth in the number of OpenStack projects
 work if each project goes off randomly and does its own thing
 without any concern for the difficulties that creates.
 
 Mark.
 
 Hi Mark,
 
 You may have missed it, but barbican has added a blueprint to
 change our queue to use oslo.messaging [1]
 
 I just wanted to clarify that we didn’t choose Celery because we
 thought that “it was perfectly fine to ignore what other
 OpenStack projects do”. Incubation has been one of our goals
 since the project began.  If you’ve taken the time to look at our
 code, you’ve seen that we have been using oslo.config this whole
 time.  We chose Celery because it was
 
 a) Properly packaged like any other python library, so we could
 just pip-install it. b) Well documented c) Well tested in
 production environments
 
 At the time none of those were true for oslo.messaging.  In
 fact, oslo.messgaging still cannot be pip-installed as of today.
 Obviously, had we know that using oslo.messaging is hard
 requirement in advance, we would have chosen it despite its poor
 distribution story.
 
 I do sympathise, but it's also true is that all other projects
 were using the oslo-incubator RPC code at the time you chose
 Celery.
 
 I think all the verbiage in this thread about celery is just to 
 reinforce that we need to be very sure that new projects feel a 
 responsibility to fit closely in with the rest of OpenStack. It's
 not about technical requirements so much as social responsibility.
 
 But look - I think you've reacted well to the concern and hopefully
 if it feels like there was an overreaction that you can understand
 the broader thing we're trying to get at here.
 
 I agree. I think you've done an excellent job in responding to it -
 and I appreciate that. We're trying to be clearer about expectations
 moving forward, which I hope this thread in some part helps with.
 
 Monty
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-06 Thread John Wood
 Just an FYI that I've submitted a pull request [1] to replace Celery
 with oslo.messaging.

 wow. That was quick!

 /me is impressed

Ha! Just trying to put the polish on! Thanks for the feedback and hope it is a 
step in the right direction. I'll look to add testing for it on Monday or 
before to take it out of 'in progress'.


 Since you jumped on that - I went ahead and jumped on a pbr-ification
 patch for you. ...

Thanks for the assist! I think we have some competing CRs out there but we'll 
sort those out once the oslo.messaging CR goes in...then we'll be able to 
remove Celery from the requirements once and for all!

Thanks again,
John



From: Monty Taylor [mord...@inaugust.com]
Sent: Friday, December 06, 2013 10:11 AM
To: John Wood; Mark McLoughlin; Douglas Mendizabal
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack...@lists.openstack.org; barbi...@lists.rackspace.com
Subject: Re: [openstack-tc] [openstack-dev] Incubation Request for Barbican

On 12/06/2013 08:35 AM, John Wood wrote:
 Hello folks,

 Just an FYI that I've submitted a pull request [1] to replace Celery
 with oslo.messaging.

wow. That was quick!

/me is impressed

Since you jumped on that - I went ahead and jumped on a pbr-ification
patch for you. It may not work yet - I'm on weird network and having
trouble install python things into virtualenvs:

  https://review.openstack.org/60551
  https://review.openstack.org/60552


 I've tagged it as a work in progress per this note:

 Please review this CR, which replaces Celery with oslo.messaging
 components. I've verified that this works in my local environment,
 but I still need to add unit testing. I also need to verify that it
 works correctly with an HA Rabbit MQ cluster, as that is a hard
 requirement for Barbican.

 Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me
 to very useful links here [2] and here [3] respectively.

 [1] https://review.openstack.org/#/c/60427/ [2]
 https://review.openstack.org/#/c/39929 [3]
 https://review.openstack.org/#/c/57880

 Thanks, John

  From: Monty Taylor
 [mord...@inaugust.com] Sent: Thursday, December 05, 2013 8:35 PM To:
 Mark McLoughlin; Douglas Mendizabal Cc: OpenStack Development Mailing
 List (not for usage questions); openstack...@lists.openstack.org;
 barbi...@lists.rackspace.com Subject: Re: [openstack-tc]
 [openstack-dev] Incubation Request for Barbican

 On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
 On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:

 I agree that this is concerning. And that what's concerning
 isn't so much that the project did something different, but
 rather that choice was apparently made because the project
 thought it was perfectly fine for them to ignore what other
 OpenStack projects do and go off and do its own thing.

 We can't make this growth in the number of OpenStack projects
 work if each project goes off randomly and does its own thing
 without any concern for the difficulties that creates.

 Mark.

 Hi Mark,

 You may have missed it, but barbican has added a blueprint to
 change our queue to use oslo.messaging [1]

 I just wanted to clarify that we didn’t choose Celery because we
 thought that “it was perfectly fine to ignore what other
 OpenStack projects do”. Incubation has been one of our goals
 since the project began.  If you’ve taken the time to look at our
 code, you’ve seen that we have been using oslo.config this whole
 time.  We chose Celery because it was

 a) Properly packaged like any other python library, so we could
 just pip-install it. b) Well documented c) Well tested in
 production environments

 At the time none of those were true for oslo.messaging.  In
 fact, oslo.messgaging still cannot be pip-installed as of today.
 Obviously, had we know that using oslo.messaging is hard
 requirement in advance, we would have chosen it despite its poor
 distribution story.

 I do sympathise, but it's also true is that all other projects
 were using the oslo-incubator RPC code at the time you chose
 Celery.

 I think all the verbiage in this thread about celery is just to
 reinforce that we need to be very sure that new projects feel a
 responsibility to fit closely in with the rest of OpenStack. It's
 not about technical requirements so much as social responsibility.

 But look - I think you've reacted well to the concern and hopefully
 if it feels like there was an overreaction that you can understand
 the broader thing we're trying to get at here.

 I agree. I think you've done an excellent job in responding to it -
 and I appreciate that. We're trying to be clearer about expectations
 moving forward, which I hope this thread in some part helps with.

 Monty


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-05 Thread Douglas Mendizabal


I agree that this is concerning. And that what's concerning isn't so
much that the project did something different, but rather that choice
was apparently made because the project thought it was perfectly fine
for them to ignore what other OpenStack projects do and go off and do
its own thing.

We can't make this growth in the number of OpenStack projects work if
each project goes off randomly and does its own thing without any
concern for the difficulties that creates.

Mark.

Hi Mark,

You may have missed it, but barbican has added a blueprint to change our
queue to use oslo.messaging [1]

I just wanted to clarify that we didn’t choose Celery because we thought
that “it was perfectly fine to ignore what other OpenStack projects do”.
Incubation has been one of our goals since the project began.  If you’ve
taken the time to look at our code, you’ve seen that we have been using
oslo.config this whole time.  We chose Celery because it was

a) Properly packaged like any other python library, so we could just
pip-install it.
b) Well documented
c) Well tested in production environments

At the time none of those were true for oslo.messaging.  In fact,
oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
we know that using oslo.messaging is hard requirement in advance, we would
have chosen it despite its poor distribution story.

- Doug Mendizabal

[1] 
https://blueprints.launchpad.net/barbican/+spec/queue-use-oslo-messaging

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-05 Thread Mark McLoughlin
On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
 
 I agree that this is concerning. And that what's concerning isn't so
 much that the project did something different, but rather that choice
 was apparently made because the project thought it was perfectly fine
 for them to ignore what other OpenStack projects do and go off and do
 its own thing.
 
 We can't make this growth in the number of OpenStack projects work if
 each project goes off randomly and does its own thing without any
 concern for the difficulties that creates.
 
 Mark.
 
 Hi Mark,
 
 You may have missed it, but barbican has added a blueprint to change our
 queue to use oslo.messaging [1]
 
 I just wanted to clarify that we didn’t choose Celery because we thought
 that “it was perfectly fine to ignore what other OpenStack projects do”.
 Incubation has been one of our goals since the project began.  If you’ve
 taken the time to look at our code, you’ve seen that we have been using
 oslo.config this whole time.  We chose Celery because it was
 
 a) Properly packaged like any other python library, so we could just
 pip-install it.
 b) Well documented
 c) Well tested in production environments
 
 At the time none of those were true for oslo.messaging.  In fact,
 oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
 we know that using oslo.messaging is hard requirement in advance, we would
 have chosen it despite its poor distribution story.

I do sympathise, but it's also true is that all other projects were
using the oslo-incubator RPC code at the time you chose Celery.

I think all the verbiage in this thread about celery is just to
reinforce that we need to be very sure that new projects feel a
responsibility to fit closely in with the rest of OpenStack. It's not
about technical requirements so much as social responsibility.

But look - I think you've reacted well to the concern and hopefully if
it feels like there was an overreaction that you can understand the
broader thing we're trying to get at here.

Thanks,
Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-05 Thread John Wood
Hello folks,

Just an FYI that I've submitted a pull request [1] to replace Celery with 
oslo.messaging.

I've tagged it as a work in progress per this note:

Please review this CR, which replaces Celery with oslo.messaging components. 
I've verified that this works in my local environment, but I still need to add 
unit testing. I also need to verify that it works correctly with an HA Rabbit 
MQ cluster, as that is a hard requirement for Barbican.

Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me to very 
useful links here [2] and here [3] respectively.

[1] https://review.openstack.org/#/c/60427/
[2] https://review.openstack.org/#/c/39929
[3] https://review.openstack.org/#/c/57880

Thanks,
John


From: Monty Taylor [mord...@inaugust.com]
Sent: Thursday, December 05, 2013 8:35 PM
To: Mark McLoughlin; Douglas Mendizabal
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack...@lists.openstack.org; barbi...@lists.rackspace.com
Subject: Re: [openstack-tc] [openstack-dev] Incubation Request for Barbican

On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
 On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:

 I agree that this is concerning. And that what's concerning isn't so
 much that the project did something different, but rather that choice
 was apparently made because the project thought it was perfectly fine
 for them to ignore what other OpenStack projects do and go off and do
 its own thing.

 We can't make this growth in the number of OpenStack projects work if
 each project goes off randomly and does its own thing without any
 concern for the difficulties that creates.

 Mark.

 Hi Mark,

 You may have missed it, but barbican has added a blueprint to change our
 queue to use oslo.messaging [1]

 I just wanted to clarify that we didn’t choose Celery because we thought
 that “it was perfectly fine to ignore what other OpenStack projects do”.
 Incubation has been one of our goals since the project began.  If you’ve
 taken the time to look at our code, you’ve seen that we have been using
 oslo.config this whole time.  We chose Celery because it was

 a) Properly packaged like any other python library, so we could just
 pip-install it.
 b) Well documented
 c) Well tested in production environments

 At the time none of those were true for oslo.messaging.  In fact,
 oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
 we know that using oslo.messaging is hard requirement in advance, we would
 have chosen it despite its poor distribution story.

 I do sympathise, but it's also true is that all other projects were
 using the oslo-incubator RPC code at the time you chose Celery.

 I think all the verbiage in this thread about celery is just to
 reinforce that we need to be very sure that new projects feel a
 responsibility to fit closely in with the rest of OpenStack. It's not
 about technical requirements so much as social responsibility.

 But look - I think you've reacted well to the concern and hopefully if
 it feels like there was an overreaction that you can understand the
 broader thing we're trying to get at here.

I agree. I think you've done an excellent job in responding to it - and
I appreciate that. We're trying to be clearer about expectations moving
forward, which I hope this thread in some part helps with.

Monty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-04 Thread Jarret Raim

 
 I think this conversation has gotten away from our incubation request
and
 into an argument about what makes a good library and when and how
projects
 should choose between oslo and other options. I¹m happy to have the
second
 one in another thread, but that seems like a longer conversation that is
 separate from our request.

It's absolutely about your incubation request.  Part of this process is
looking at the technical fit with the rest of OpenStack, and this is
well within the scope of that discussion.

Oh understood, I’m just saying that the ask was to move away from Celery
to oslo.messaging. We’ve agreed to work on that and I don’t think there is
a problem in getting that done for Icehouse.

It is the larger discussion about whether oslo.messaging is really the
best approach that seems off topic. We’re going to use it, so the argument
is moot for now. If we want to take up that discussion, we can certainly
do that, but it more like a conversation that happens within the oslo
group rather than part of Barbican's request.

I need to make another pass over the added info and go deeper into the
technical bits, so I suspect there will be more feedback still.  It's
only been a day.  :-)

Great. Always happy to have another set of eyes on the code.

Also, note that most of the things brought up so far are based on the
proposed requirements for becoming incubated, not for things to be
worked on during incubation.  Unless the requirements change, so far it
looks like this request should be deferred a bit longer.

We are working on the issues that have been brought up so far. Today, we
submitted a PR to add in pbr and the global requirements [1]. Once that
has been reviewed, we'll land it after the Icehouse-1 release. Note that
celery is still in the list until we complete the oslo.messengaging work.


Jarret




[1] https://review.openstack.org/#/c/59817/1


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Flavio Percoco

On 03/12/13 03:12 +, Jarret Raim wrote:

There are two big parts to this, I think.  One is techincal - a significant
portion
of OpenStack deployments will not work with this because Celery does not
work with their deployed messaging architecture.
 See another reply in this thread for an example of someone that sees the
inability to use Qpid as a roadblock for an example.  This is solvable, but
not
quickly.

The other is somewhat technical, but also a community issue.  Monty
articulated this well in another reply.  Barbican has made a conflicting
library
choice with what every other project using messaging is using.
With the number of projects we have, it is in our best interest to strive
for
consistency where we can.  Differences should not be arbitrary.  The
differences should only be where an exception is well justified.  I don't
see
that as being the case here.  Should everyone using oslo.messaging (or its
predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't know,
but that's the question at hand.  Ideally this would have come up with a
more
broad audience sooner.  If it did, I'm sorry I missed it.


I understand the concern here and I'm happy to have Barbican look at using
oslo.messaging during the Icehouse cycle.

I am a bit surprised at the somewhat strong reactions to our choice. When we
created Barbican, we looked at the messaging frameworks out there for use. At
the time, oslo.messaging was not packaged, not documented, not tested, had no
track record and an unknown level of community support.


But there was oslo-incubator/rpc which all projects where already
using.


Celery is a battle-tested library that is widely deployed with a good track
record, strong community and decent documentation. We made our choice based on
those factors, just as the same as we would for any library inclusion.

As celery has met our needs up to this point, we saw no reason to revisit the
decision until now. In that time oslo.messaging  has moved to a separate repo.
It still has little to no documentation, but the packaging and maintenance
issues seem to be on the way to being sorted.

So in short, in celery we get a reliable library with good docs that is battle
tested, but is limited to the transports supported by Kombu. Both celery and
Kombu are extendable and have many backends including AMQP, Redis, Beanstalk,
Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.

Oslo.messaging seems to have good support in OpenStack, but still lacks
documentation and packaging (though some of that is being sorted out now). It
offers support for qpid which celery seems to lack. It also offers a common
place for message signing and some other nice to have features for OpenStack.


I think there's something else you should take under consideration.
Oslo messaging is not just an OpenStack library. It's the RPC library
that all projects are relying on and one of the strong goals we have
in OpenStack is to reduce code and efforts duplications. We'd love to
have more people testing and contributing to oslo.messaging in order
to make it as battle tested as celery is.

Please, don't get me wrong. I don't mean to say you didn't considered
it, I just want to add another reason why we should always try to
re-use the libraries that other projects are using - unless there's a
strong technical reason ;).


Based on the commonality in OpenStack (and the lack of anyone else using
Celery), I think looking to move to oslo.messaging is a good goal. This will
take some time, but I think doing it by Icehouse seems reasonable. I think
that is what you and Monty are asking for?




I have added the task to our list on
https://wiki.openstack.org/wiki/Barbican/Incubation.



Thanks a lot for this, really!

Cheers,
FF

--
@flaper87
Flavio Percoco


pgpznZ7d3tIAe.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Doug Hellmann
On Mon, Dec 2, 2013 at 10:12 PM, Jarret Raim jarret.r...@rackspace.comwrote:

  There are two big parts to this, I think.  One is techincal - a
 significant
  portion
  of OpenStack deployments will not work with this because Celery does not
  work with their deployed messaging architecture.
   See another reply in this thread for an example of someone that sees the
  inability to use Qpid as a roadblock for an example.  This is solvable,
 but
  not
  quickly.
 
  The other is somewhat technical, but also a community issue.  Monty
  articulated this well in another reply.  Barbican has made a conflicting
  library
  choice with what every other project using messaging is using.
  With the number of projects we have, it is in our best interest to strive
  for
  consistency where we can.  Differences should not be arbitrary.  The
  differences should only be where an exception is well justified.  I don't
  see
  that as being the case here.  Should everyone using oslo.messaging (or
 its
  predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't
 know,
  but that's the question at hand.  Ideally this would have come up with a
  more
  broad audience sooner.  If it did, I'm sorry I missed it.

 I understand the concern here and I'm happy to have Barbican look at using
 oslo.messaging during the Icehouse cycle.

 I am a bit surprised at the somewhat strong reactions to our choice. When
 we
 created Barbican, we looked at the messaging frameworks out there for use.
 At
 the time, oslo.messaging was not packaged, not documented, not tested, had
 no
 track record and an unknown level of community support.


The API and developer documentation is at
http://docs.openstack.org/developer/oslo.messaging/

Doug



 Celery is a battle-tested library that is widely deployed with a good track
 record, strong community and decent documentation. We made our choice
 based on
 those factors, just as the same as we would for any library inclusion.

 As celery has met our needs up to this point, we saw no reason to revisit
 the
 decision until now. In that time oslo.messaging  has moved to a separate
 repo.
 It still has little to no documentation, but the packaging and maintenance
 issues seem to be on the way to being sorted.

 So in short, in celery we get a reliable library with good docs that is
 battle
 tested, but is limited to the transports supported by Kombu. Both celery
 and
 Kombu are extendable and have many backends including AMQP, Redis,
 Beanstalk,
 Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.

 Oslo.messaging seems to have good support in OpenStack, but still lacks
 documentation and packaging (though some of that is being sorted out now).
 It
 offers support for qpid which celery seems to lack. It also offers a common
 place for message signing and some other nice to have features for
 OpenStack.

 Based on the commonality in OpenStack (and the lack of anyone else using
 Celery), I think looking to move to oslo.messaging is a good goal. This
 will
 take some time, but I think doing it by Icehouse seems reasonable. I think
 that is what you and Monty are asking for?

 I have added the task to our list on
 https://wiki.openstack.org/wiki/Barbican/Incubation.


 Thanks again for all the eyeballs our on application.


 Jarret


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Jarret Raim

I think there's something else you should take under consideration.
Oslo messaging is not just an OpenStack library. It's the RPC library
that all projects are relying on and one of the strong goals we have
in OpenStack is to reduce code and efforts duplications. We'd love to
have more people testing and contributing to oslo.messaging in order
to make it as battle tested as celery is.

Please, don't get me wrong. I don't mean to say you didn't considered
it, I just want to add another reason why we should always try to
re-use the libraries that other projects are using - unless there's a
strong technical reason ;).

As I¹ve said, we are willing to look at the library for Icehouse. As lots
of projects have implemented it, I hope that the switchover will be
reasonably easy. 

I think this conversation has gotten away from our incubation request and
into an argument about what makes a good library and when and how projects
should choose between oslo and other options. I¹m happy to have the second
one in another thread, but that seems like a longer conversation that is
separate from our request.

It seems like the comments are slowing down now. Does everyone feel our
list (https://wiki.openstack.org/wiki/Barbican/Incubation) accurately
captures the comments that have been brought up?

I filled out the Scope section of our request and I think we¹ve cleared up
the PTL election issue. Is there anything else I missed or have we covered
most of the issues?



Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Jarret Raim

 The API and developer documentation is at
http://docs.openstack.org/developer/oslo.messaging/

This is great, thanks for the link. Would there be any objections to
adding this to the github repo and the openstack wiki pages? I spent a
bunch of time looking and wasn¹t able to turn this up.


Additionally, where is this documentation generated from? I looked at the
doc/ dir in the repo [1] and most of the files in there were empty.


[1] https://github.com/openstack/oslo.messaging/tree/master/doc/source




Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Jeremy Stanley
On 2013-12-03 16:43:32 + (+), Jarret Raim wrote:
 This is great, thanks for the link. Would there be any objections to
 adding this to the github repo

I think you meant the git repo. What's a gi-thub?

URL: http://git.openstack.org/cgit/openstack/oslo.messaging/tree/doc/source/ 

 and the openstack wiki pages? I spent a bunch of time looking and
 wasn¹t able to turn this up.

If you haven't worked on official OpenStack projects before, I can
see how it might be easy to overlook that source for the
Sphinx-built developer documentation lives in a standardized
location within each git repository (for convenience). Possibly
another pattern we should suggest following as part of any
application for incubation.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Russell Bryant
On 12/03/2013 11:40 AM, Jarret Raim wrote:
 
 I think there's something else you should take under consideration.
 Oslo messaging is not just an OpenStack library. It's the RPC library
 that all projects are relying on and one of the strong goals we have
 in OpenStack is to reduce code and efforts duplications. We'd love to
 have more people testing and contributing to oslo.messaging in order
 to make it as battle tested as celery is.

 Please, don't get me wrong. I don't mean to say you didn't considered
 it, I just want to add another reason why we should always try to
 re-use the libraries that other projects are using - unless there's a
 strong technical reason ;).
 
 As I¹ve said, we are willing to look at the library for Icehouse. As lots
 of projects have implemented it, I hope that the switchover will be
 reasonably easy. 
 
 I think this conversation has gotten away from our incubation request and
 into an argument about what makes a good library and when and how projects
 should choose between oslo and other options. I¹m happy to have the second
 one in another thread, but that seems like a longer conversation that is
 separate from our request.

It's absolutely about your incubation request.  Part of this process is
looking at the technical fit with the rest of OpenStack, and this is
well within the scope of that discussion.

 It seems like the comments are slowing down now. Does everyone feel our
 list (https://wiki.openstack.org/wiki/Barbican/Incubation) accurately
 captures the comments that have been brought up?
 
 I filled out the Scope section of our request and I think we¹ve cleared up
 the PTL election issue. Is there anything else I missed or have we covered
 most of the issues?

I need to make another pass over the added info and go deeper into the
technical bits, so I suspect there will be more feedback still.  It's
only been a day.  :-)

Also, note that most of the things brought up so far are based on the
proposed requirements for becoming incubated, not for things to be
worked on during incubation.  Unless the requirements change, so far it
looks like this request should be deferred a bit longer.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-03 Thread Russell Bryant
On 12/03/2013 01:26 PM, Russell Bryant wrote:
 Unless the requirements change, so far it
 looks like this request should be deferred a bit longer.

And note that this is just my opinion, and note a statement of position
on behalf of the entire TC.  We can still officially consider the
request at an upcoming TC meeting.  I was just giving an indication of
where I stand right now.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Monty Taylor


On 12/02/2013 11:53 AM, Russell Bryant wrote:
 On 12/02/2013 10:19 AM, Jarret Raim wrote:
 All,

 Barbican is the OpenStack key management service and we’d like to
 request incubation for the Icehouse release. A Rackspace sponsored team
 has been working for about 9 months now, including following the Havana
 release cycle for our first release.

 Our incubation request is here:
 https://wiki.openstack.org/wiki/Barbican

 Our documentation is mostly hosted at GitHub for the moment, though we
 are in the process of converting much of it to DocBook.
 https://github.com/cloudkeep/barbican
 https://github.com/cloudkeep/barbican/wiki


 The Barbican team will be on IRC today at #openstack-barbican and you
 can contact us using the barbi...@lists.rackspace.com mailing list if
 desired.
 
 The TC is currently working on formalizing requirements for new programs
 and projects [3].  I figured I would give them a try against this
 application.
 
 First, I'm assuming that the application is for a new program that
 contains the new project.  The application doesn't make that bit clear,
 though.
 
 Teams in OpenStack can be created as-needed and grow organically. As the team
 work matures, some technical efforts will be recognized as essential to the
 completion of the OpenStack project mission. By becoming an official Program,
 they place themselves under the authority of the OpenStack Technical
 Committee. In return, their contributors get to vote in the Technical
 Committee election, and they get some space and time to discuss future
 development at our Design Summits. When considering new programs, the TC will
 look into a number of criteria, including (but not limited to):
 
 * Scope
 ** Team must have a specific scope, separated from others teams scope
 
 I would like to see a statement of scope for Barbican on the
 application.  It should specifically cover how the scope differs from
 other programs, in particular the Identity program.
 
 ** Team must have a mission statement
 
 This is missing.
   
 * Maturity
 ** Team must already exist, have regular meetings and produce some work
 
 This seems covered.
 
 ** Team should have a lead, elected by the team contributors
 
 Was the PTL elected?  I can't seem to find record of that.  If not, I
 would like to see an election held for the PTL.
 
 ** Team should have a clear way to grant ATC (voting) status to its
significant contributors
 
 Related to the above
 
 * Deliverables
 ** Team should have a number of clear deliverables
 
 barbican and python-barbicanclient, I presume.  It would be nice to have
 this clearly defined on the application.
 
 
 Now, for the project specific requirements:
 
  Projects wishing to be included in the integrated release of OpenStack must
  first apply for incubation status. During their incubation period, they will
  be able to access new resources and tap into other OpenStack programs (in
  particular the Documentation, QA, Infrastructure and Release management 
 teams)
  to learn about the OpenStack processes and get assistance in their 
 integration
  efforts.
  
  The TC will evaluate the project scope and its complementarity with existing
  integrated projects and other official programs, look into the project
  technical choices, and check a number of requirements, including (but not
  limited to):
  
  * Scope
  ** Project must have a clear and defined scope
 
 This is missing
 
  ** Project should not inadvertently duplicate functionality present in other
 OpenStack projects. If they do, they should have a clear plan and 
 timeframe
 to prevent long-term scope duplication.
  ** Project should leverage existing functionality in other OpenStack 
 projects
 as much as possible
 
 I'm going to hold off on diving into this too far until the scope is
 clarified.

I'm not.

*snip*

  
  * Process
  ** Project must be hosted under stackforge (and therefore use git as its 
 VCS)
 
 I see that barbican is now on stackforge,  but python-barbicanclient is
 still on github.  Is that being moved soon?
 
  ** Project must obey OpenStack coordinated project interface (such as tox,
 pbr, global-requirements...)
 
 Uses tox, but not pbr or global requirements

It's also pretty easy for a stackforge project to opt-in to the global
requirements sync job now too.

  ** Project should use oslo libraries or oslo-incubator where appropriate
 
 The list looks reasonable right now.  Barbican should put migrating to
 oslo.messaging on the Icehouse roadmap though.

*snip*

 
 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
 
 It looks like the only item here not in the global requirements is
 Celery, which is licensed under a 3-clause BSD license.

I'd like to address the use of Celery.

WTF

Barbican has been around for 9 months, which means that it does not
predate the work that has become oslo.messaging. It doesn't even try. It
uses a completely different thing.

The use of celery needs to be 

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Russell Bryant
On 12/02/2013 12:46 PM, Monty Taylor wrote:
 On 12/02/2013 11:53 AM, Russell Bryant wrote:
  * Scope
  ** Project must have a clear and defined scope

 This is missing

  ** Project should not inadvertently duplicate functionality present in 
 other
 OpenStack projects. If they do, they should have a clear plan and 
 timeframe
 to prevent long-term scope duplication.
  ** Project should leverage existing functionality in other OpenStack 
 projects
 as much as possible

 I'm going to hold off on diving into this too far until the scope is
 clarified.
 
 I'm not.
 
 *snip*
 

Ok, I can't help it now.


 The list looks reasonable right now.  Barbican should put migrating to
 oslo.messaging on the Icehouse roadmap though.
 
 *snip*

Yeahhh ... I looked and even though rpc and notifier are imported, they
do not appear to be used at all.


 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires

 It looks like the only item here not in the global requirements is
 Celery, which is licensed under a 3-clause BSD license.
 
 I'd like to address the use of Celery.
 
 WTF
 
 Barbican has been around for 9 months, which means that it does not
 predate the work that has become oslo.messaging. It doesn't even try. It
 uses a completely different thing.
 
 The use of celery needs to be replaced with oslo. Full stop. I do not
 believe it makes any sense to spend further time considering a project
 that's divergent on such a core piece. Which is a shame - because I
 think that Barbican is important and fills an important need and I want
 it to be in. BUT - We don't get to end-run around OpenStack project
 choices by making a new project on the side and then submitting it for
 incubation. It's going to be a pile of suck to fix this I'm sure, and
 I'm sure that it's going to delay getting actually important stuff done
 - but we deal with too much crazy as it is to pull in a non-oslo
 messaging and event substrata.
 

Yeah, I'm afraid I agree with Monty here.  I didn't really address this
because I was trying to just do a first pass and not go too far into the
tech bits.

I think such a big divergence is going to be a hard sell for a number of
reasons.  It's a significant dependency that I don't think is justified.
 Further, it won't work in all of the same environments that OpenStack
works in today.  You can't use Celery with all of the same messaging
transports as oslo.messaging (or the older rpc lib).  One example is Qpid.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Jarret Raim

  * Process
  ** Project must be hosted under stackforge (and therefore use git as
its VCS)
 
 I see that barbican is now on stackforge,  but python-barbicanclient is
 still on github.  Is that being moved soon?
 
  ** Project must obey OpenStack coordinated project interface (such as
tox,
 pbr, global-requirements...)
 
 Uses tox, but not pbr or global requirements

It's also pretty easy for a stackforge project to opt-in to the global
requirements sync job now too.

Are there some docs on how to do this somewhere? I added a task for us to
complete the work as part of the incubation request here:
https://wiki.openstack.org/wiki/Barbican/Incubation


  ** Project should use oslo libraries or oslo-incubator where
appropriate
 
 The list looks reasonable right now.  Barbican should put migrating to
 oslo.messaging on the Icehouse roadmap though.

*snip*

 
 
http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
 
 It looks like the only item here not in the global requirements is
 Celery, which is licensed under a 3-clause BSD license.

I'd like to address the use of Celery.

WTF

Barbican has been around for 9 months, which means that it does not
predate the work that has become oslo.messaging. It doesn't even try. It
uses a completely different thing.

The use of celery needs to be replaced with oslo. Full stop. I do not
believe it makes any sense to spend further time considering a project
that's divergent on such a core piece. Which is a shame - because I
think that Barbican is important and fills an important need and I want
it to be in. BUT - We don't get to end-run around OpenStack project
choices by making a new project on the side and then submitting it for
incubation. It's going to be a pile of suck to fix this I'm sure, and
I'm sure that it's going to delay getting actually important stuff done
- but we deal with too much crazy as it is to pull in a non-oslo
messaging and event substrata.


Is the challenge here that celery has some weird license requirements? Or
that it is a new library?

When we started the Barbican project in February of this year,
oslo.messaging did not exist. If I remember correctly, at the time we were
doing architecture set up, the messaging piece was not available as a
standalone library, was not available on PyPi and had no documentation.

It looks like the project was moved to its own repo in April. However, I
can¹t seem to find the docs anywhere? The only thing I see is a design doc
here [1]. Are there plans for it to be packaged and put into Pypi?

We are probably overdue to look at oslo.messaging again, but I don¹t think
it should be a blocker for our incubation. I'm happy to take a look to see
what we can do during the Icehouse release cycle. Would that be
sufficient? 


[1] https://wiki.openstack.org/wiki/Oslo/Messaging




Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Monty Taylor


On 12/02/2013 05:09 PM, Jarret Raim wrote:
 
  * Process
  ** Project must be hosted under stackforge (and therefore use git as
 its VCS)

 I see that barbican is now on stackforge,  but python-barbicanclient is
 still on github.  Is that being moved soon?

  ** Project must obey OpenStack coordinated project interface (such as
 tox,
 pbr, global-requirements...)

 Uses tox, but not pbr or global requirements

 It's also pretty easy for a stackforge project to opt-in to the global
 requirements sync job now too.
 
 Are there some docs on how to do this somewhere? I added a task for us to
 complete the work as part of the incubation request here:
 https://wiki.openstack.org/wiki/Barbican/Incubation

I'm not sure there are docs per-se - you can essentially just add your
project name to projects.txt in openstack/requirements. Beware - really
soon this will tie you to the openstack pypi mirror (like, possibly
today) so if you have non-aligned requirements (see below) it may block
you until you either align or get requirements changed.

  ** Project should use oslo libraries or oslo-incubator where
 appropriate

 The list looks reasonable right now.  Barbican should put migrating to
 oslo.messaging on the Icehouse roadmap though.

 *snip*



 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires

 It looks like the only item here not in the global requirements is
 Celery, which is licensed under a 3-clause BSD license.

 I'd like to address the use of Celery.

 WTF

 Barbican has been around for 9 months, which means that it does not
 predate the work that has become oslo.messaging. It doesn't even try. It
 uses a completely different thing.

 The use of celery needs to be replaced with oslo. Full stop. I do not
 believe it makes any sense to spend further time considering a project
 that's divergent on such a core piece. Which is a shame - because I
 think that Barbican is important and fills an important need and I want
 it to be in. BUT - We don't get to end-run around OpenStack project
 choices by making a new project on the side and then submitting it for
 incubation. It's going to be a pile of suck to fix this I'm sure, and
 I'm sure that it's going to delay getting actually important stuff done
 - but we deal with too much crazy as it is to pull in a non-oslo
 messaging and event substrata.
 
 
 Is the challenge here that celery has some weird license requirements? Or
 that it is a new library?

The second thing - as a thing to address functionality that has been in
OpenStack since the beginning.

 When we started the Barbican project in February of this year,
 oslo.messaging did not exist. If I remember correctly, at the time we were
 doing architecture set up, the messaging piece was not available as a
 standalone library, was not available on PyPi and had no documentation.

That is true. However, the equivalent oslo-incubator pieces were there.
You can also get the latest pre-release library here:

-f
http://tarballs.openstack.org/oslo.messaging/oslo.messaging-1.2.0a11.tar.gz#egg=oslo.messaging-1.2.0a11
oslo.messaging=1.2.0a11

As soon as we get the wheel publication pre-release code working, those
will go to pypi.

 It looks like the project was moved to its own repo in April. However, I
 can¹t seem to find the docs anywhere? The only thing I see is a design doc
 here [1]. Are there plans for it to be packaged and put into Pypi?
 
 We are probably overdue to look at oslo.messaging again, but I don¹t think
 it should be a blocker for our incubation. I'm happy to take a look to see
 what we can do during the Icehouse release cycle. Would that be
 sufficient? 

I think it would go a long way if you had a roadmap for how you're going
to move to oslo.messaging from celery. The worrying part for me, to be
honest, is that you didn't sync up with the state of the art in that
subject across openstack but instead went to a library that clearly
no-one else was using. A couple of conversations with markmc or
dhellman, or anyone from oslo, or anyone from any of the projects that
are currently also using Rabbit (or zeromq or qpid) for the reasons
you're using it would have gotten you aligned pretty quickly with how
everyone else is doing this ... OR, and argument could have been made as
to why all of that was a terrible idea and that the whole project should
move to celery.

It's not that oslo-incubator was unknown - you're using other things.

When we look at incubating things, part of it is about the tech, but
part of it is the question about whether or not the project is going to
add to the whole's ability to be greater than the sum of its parts, of
if we're going to inherit another group of folks who will be balkanized
in the corner somewhere.

I don't want to sound punitive, that's not my intent - and I want you
guys to succeed and to be a part of things. I just want to be careful
that we don't add more complexity and confusion into an already
difficult to fully digest landscape. I 

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Sylvain Bauza
Hi Jarret,

2013/12/2 Jarret Raim jarret.r...@rackspace.com



 It's also pretty easy for a stackforge project to opt-in to the global
 requirements sync job now too.

 Are there some docs on how to do this somewhere? I added a task for us to
 complete the work as part of the incubation request here:
 https://wiki.openstack.org/wiki/Barbican/Incubation


There is a Stackforge how-to on ci.openstack.org that you can read [1].
Alternatively, you can also find our review [2] where we (Climate) sync'd
our requirements.txt with Oslo.

Hope it can help,
-Sylvain

[1] : http://ci.openstack.org/stackforge.html
[2] : https://review.openstack.org/#/c/56578/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Dolph Mathews
On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant rbry...@redhat.com wrote:

 On 12/02/2013 12:46 PM, Monty Taylor wrote:
  On 12/02/2013 11:53 AM, Russell Bryant wrote:
   * Scope
   ** Project must have a clear and defined scope
 
  This is missing
 
   ** Project should not inadvertently duplicate functionality present
 in other
  OpenStack projects. If they do, they should have a clear plan and
 timeframe
  to prevent long-term scope duplication.
   ** Project should leverage existing functionality in other OpenStack
 projects
  as much as possible
 
  I'm going to hold off on diving into this too far until the scope is
  clarified.
 
  I'm not.
 
  *snip*
 

 Ok, I can't help it now.

 
  The list looks reasonable right now.  Barbican should put migrating to
  oslo.messaging on the Icehouse roadmap though.
 
  *snip*

 Yeahhh ... I looked and even though rpc and notifier are imported, they
 do not appear to be used at all.

 
 
 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
 
  It looks like the only item here not in the global requirements is
  Celery, which is licensed under a 3-clause BSD license.
 
  I'd like to address the use of Celery.
 
  WTF
 
  Barbican has been around for 9 months, which means that it does not
  predate the work that has become oslo.messaging. It doesn't even try. It
  uses a completely different thing.
 
  The use of celery needs to be replaced with oslo. Full stop. I do not
  believe it makes any sense to spend further time considering a project
  that's divergent on such a core piece. Which is a shame - because I
  think that Barbican is important and fills an important need and I want
  it to be in. BUT - We don't get to end-run around OpenStack project
  choices by making a new project on the side and then submitting it for
  incubation. It's going to be a pile of suck to fix this I'm sure, and
  I'm sure that it's going to delay getting actually important stuff done
  - but we deal with too much crazy as it is to pull in a non-oslo
  messaging and event substrata.
 

Yeah, I'm afraid I agree with Monty here.  I didn't really address this
 because I was trying to just do a first pass and not go too far into the
 tech bits.

 I think such a big divergence is going to be a hard sell for a number of
 reasons.  It's a significant dependency that I don't think is justified.
  Further, it won't work in all of the same environments that OpenStack
 works in today.  You can't use Celery with all of the same messaging
 transports as oslo.messaging (or the older rpc lib).  One example is Qpid.


I feel like I'm trying to read past the rant :) so I'd like to stop an ask
for clarification on the exact argument being made. Is the *only* reason
that celery should not be used in openstack because it is incapable of
being deployed against amqp 1.0 brokers (i.e. qpid).

I'm really trying to understand if the actual objection is over the use (or
not) of oslo (which seems like something the TC should express an opinion
on, if that's the case), but rather about limiting OpenStack's deployment
options.



 --
 Russell Bryant

 ___
 OpenStack-TC mailing list
 openstack...@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc


-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Fox, Kevin M
I've been anxious to try out Barbican, but haven't had quite enough time to try 
it yet. But finding out it won't work with Qpid makes it unworkable for us at 
the moment. I think a large swath of the OpenStack community won't be able to 
use it in this form too.

Thanks,
Kevin


From: Dolph Mathews [dolph.math...@gmail.com]
Sent: Monday, December 02, 2013 3:46 PM
To: Russell Bryant
Cc: openstack...@lists.openstack.org; OpenStack Development Mailing List (not 
for usage questions); cloudkeep@googlegroups. com; barbi...@lists.rackspace.com
Subject: Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant 
rbry...@redhat.commailto:rbry...@redhat.com wrote:
On 12/02/2013 12:46 PM, Monty Taylor wrote:
 On 12/02/2013 11:53 AM, Russell Bryant wrote:
  * Scope
  ** Project must have a clear and defined scope

 This is missing

  ** Project should not inadvertently duplicate functionality present in 
 other
 OpenStack projects. If they do, they should have a clear plan and 
 timeframe
 to prevent long-term scope duplication.
  ** Project should leverage existing functionality in other OpenStack 
 projects
 as much as possible

 I'm going to hold off on diving into this too far until the scope is
 clarified.

 I'm not.

 *snip*


Ok, I can't help it now.


 The list looks reasonable right now.  Barbican should put migrating to
 oslo.messaging on the Icehouse roadmap though.

 *snip*

Yeahhh ... I looked and even though rpc and notifier are imported, they
do not appear to be used at all.


 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires

 It looks like the only item here not in the global requirements is
 Celery, which is licensed under a 3-clause BSD license.

 I'd like to address the use of Celery.

 WTF

 Barbican has been around for 9 months, which means that it does not
 predate the work that has become oslo.messaging. It doesn't even try. It
 uses a completely different thing.

 The use of celery needs to be replaced with oslo. Full stop. I do not
 believe it makes any sense to spend further time considering a project
 that's divergent on such a core piece. Which is a shame - because I
 think that Barbican is important and fills an important need and I want
 it to be in. BUT - We don't get to end-run around OpenStack project
 choices by making a new project on the side and then submitting it for
 incubation. It's going to be a pile of suck to fix this I'm sure, and
 I'm sure that it's going to delay getting actually important stuff done
 - but we deal with too much crazy as it is to pull in a non-oslo
 messaging and event substrata.

Yeah, I'm afraid I agree with Monty here.  I didn't really address this
because I was trying to just do a first pass and not go too far into the
tech bits.

I think such a big divergence is going to be a hard sell for a number of
reasons.  It's a significant dependency that I don't think is justified.
 Further, it won't work in all of the same environments that OpenStack
works in today.  You can't use Celery with all of the same messaging
transports as oslo.messaging (or the older rpc lib).  One example is Qpid.

I feel like I'm trying to read past the rant :) so I'd like to stop an ask for 
clarification on the exact argument being made. Is the *only* reason that 
celery should not be used in openstack because it is incapable of being 
deployed against amqp 1.0 brokers (i.e. qpid).

I'm really trying to understand if the actual objection is over the use (or 
not) of oslo (which seems like something the TC should express an opinion on, 
if that's the case), but rather about limiting OpenStack's deployment options.


--
Russell Bryant

___
OpenStack-TC mailing list
openstack...@lists.openstack.orgmailto:openstack...@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc

--

-Dolph

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Russell Bryant
On 12/02/2013 06:46 PM, Dolph Mathews wrote:
 
 
 
 On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant rbry...@redhat.com
 mailto:rbry...@redhat.com wrote:
 
 On 12/02/2013 12:46 PM, Monty Taylor wrote:
  On 12/02/2013 11:53 AM, Russell Bryant wrote:
   * Scope
   ** Project must have a clear and defined scope
 
  This is missing
 
   ** Project should not inadvertently duplicate functionality
 present in other
  OpenStack projects. If they do, they should have a clear
 plan and timeframe
  to prevent long-term scope duplication.
   ** Project should leverage existing functionality in other
 OpenStack projects
  as much as possible
 
  I'm going to hold off on diving into this too far until the scope is
  clarified.
 
  I'm not.
 
  *snip*
 
 
 Ok, I can't help it now.
 
 
  The list looks reasonable right now.  Barbican should put
 migrating to
  oslo.messaging on the Icehouse roadmap though.
 
  *snip*
 
 Yeahhh ... I looked and even though rpc and notifier are imported, they
 do not appear to be used at all.
 
 
 
 http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
 
  It looks like the only item here not in the global requirements is
  Celery, which is licensed under a 3-clause BSD license.
 
  I'd like to address the use of Celery.
 
  WTF
 
  Barbican has been around for 9 months, which means that it does not
  predate the work that has become oslo.messaging. It doesn't even
 try. It
  uses a completely different thing.
 
  The use of celery needs to be replaced with oslo. Full stop. I do not
  believe it makes any sense to spend further time considering a project
  that's divergent on such a core piece. Which is a shame - because I
  think that Barbican is important and fills an important need and I
 want
  it to be in. BUT - We don't get to end-run around OpenStack project
  choices by making a new project on the side and then submitting it for
  incubation. It's going to be a pile of suck to fix this I'm sure, and
  I'm sure that it's going to delay getting actually important stuff
 done
  - but we deal with too much crazy as it is to pull in a non-oslo
  messaging and event substrata.
  
 
 Yeah, I'm afraid I agree with Monty here.  I didn't really address this
 because I was trying to just do a first pass and not go too far into the
 tech bits.
 
 I think such a big divergence is going to be a hard sell for a number of
 reasons.  It's a significant dependency that I don't think is justified.
  Further, it won't work in all of the same environments that OpenStack
 works in today.  You can't use Celery with all of the same messaging
 transports as oslo.messaging (or the older rpc lib).  One example is
 Qpid.
 
 
 I feel like I'm trying to read past the rant :) so I'd like to stop an
 ask for clarification on the exact argument being made. Is the *only*
 reason that celery should not be used in openstack because it is
 incapable of being deployed against amqp 1.0 brokers (i.e. qpid).
 
 I'm really trying to understand if the actual objection is over the use
 (or not) of oslo (which seems like something the TC should express an
 opinion on, if that's the case), but rather about limiting OpenStack's
 deployment options.

There are two big parts to this, I think.  One is techincal - a
significant portion of OpenStack deployments will not work with this
because Celery does not work with their deployed messaging architecture.
 See another reply in this thread for an example of someone that sees
the inability to use Qpid as a roadblock for an example.  This is
solvable, but not quickly.

The other is somewhat technical, but also a community issue.  Monty
articulated this well in another reply.  Barbican has made a conflicting
library choice with what every other project using messaging is using.
With the number of projects we have, it is in our best interest to
strive for consistency where we can.  Differences should not be
arbitrary.  The differences should only be where an exception is well
justified.  I don't see that as being the case here.  Should everyone
using oslo.messaging (or its predecessor rpc in oslo-incubator) be using
Celery?  Maybe.  I don't know, but that's the question at hand.  Ideally
this would have come up with a more broad audience sooner.  If it did,
I'm sorry I missed it.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Jarret Raim
 There are two big parts to this, I think.  One is techincal - a significant 
 portion
 of OpenStack deployments will not work with this because Celery does not
 work with their deployed messaging architecture.
  See another reply in this thread for an example of someone that sees the
 inability to use Qpid as a roadblock for an example.  This is solvable, but 
 not
 quickly.

 The other is somewhat technical, but also a community issue.  Monty
 articulated this well in another reply.  Barbican has made a conflicting 
 library
 choice with what every other project using messaging is using.
 With the number of projects we have, it is in our best interest to strive 
 for
 consistency where we can.  Differences should not be arbitrary.  The
 differences should only be where an exception is well justified.  I don't 
 see
 that as being the case here.  Should everyone using oslo.messaging (or its
 predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't know,
 but that's the question at hand.  Ideally this would have come up with a 
 more
 broad audience sooner.  If it did, I'm sorry I missed it.

I understand the concern here and I'm happy to have Barbican look at using 
oslo.messaging during the Icehouse cycle.

I am a bit surprised at the somewhat strong reactions to our choice. When we 
created Barbican, we looked at the messaging frameworks out there for use. At 
the time, oslo.messaging was not packaged, not documented, not tested, had no 
track record and an unknown level of community support.

Celery is a battle-tested library that is widely deployed with a good track 
record, strong community and decent documentation. We made our choice based on 
those factors, just as the same as we would for any library inclusion.

As celery has met our needs up to this point, we saw no reason to revisit the 
decision until now. In that time oslo.messaging  has moved to a separate repo. 
It still has little to no documentation, but the packaging and maintenance 
issues seem to be on the way to being sorted.

So in short, in celery we get a reliable library with good docs that is battle 
tested, but is limited to the transports supported by Kombu. Both celery and 
Kombu are extendable and have many backends including AMQP, Redis, Beanstalk, 
Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.

Oslo.messaging seems to have good support in OpenStack, but still lacks 
documentation and packaging (though some of that is being sorted out now). It 
offers support for qpid which celery seems to lack. It also offers a common 
place for message signing and some other nice to have features for OpenStack.

Based on the commonality in OpenStack (and the lack of anyone else using 
Celery), I think looking to move to oslo.messaging is a good goal. This will 
take some time, but I think doing it by Icehouse seems reasonable. I think 
that is what you and Monty are asking for?

I have added the task to our list on 
https://wiki.openstack.org/wiki/Barbican/Incubation.


Thanks again for all the eyeballs our on application.


Jarret



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread Jarret Raim
 I've been anxious to try out Barbican, but haven't had quite enough time
to
 try it yet. But finding out it won't work with Qpid makes it unworkable
for us
 at the moment. I think a large swath of the OpenStack community won't be
 able to use it in this form too.

As mentioned in the other thread, Barbican will be looking at oslo messaging
for Icehouse.

In the meantime, you can configure celery to use a pass-through (e.g. no
queue needed). This is the configuration we use for dev / testing. The
documentation for that can be found in the celery docs here:
http://docs.celeryproject.org/en/latest/userguide/


Jarret


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

2013-12-02 Thread John Wood
Hello folks,

I've created a blueprint to move over to oslo.messaging per Jarret's comments 
below: https://blueprints.launchpad.net/barbican

I'd also note that out of the box barbican does not use celery in order to 
simplify demo/standalone deployments...asynchronous calls simply invoke their 
worker tasks directly. 

Thanks,
John

From: cloudk...@googlegroups.com [cloudk...@googlegroups.com] on behalf of 
Jarret Raim [jarret.r...@rackspace.com]
Sent: Monday, December 02, 2013 9:16 PM
To: Fox, Kevin M; OpenStack Development Mailing List (not for usage questions); 
Russell Bryant
Cc: openstack...@lists.openstack.org; cloudkeep@googlegroups. com; 
barbi...@lists.rackspace.com
Subject: RE: [openstack-dev] [openstack-tc] Incubation Request for Barbican

 I've been anxious to try out Barbican, but haven't had quite enough time
to
 try it yet. But finding out it won't work with Qpid makes it unworkable
for us
 at the moment. I think a large swath of the OpenStack community won't be
 able to use it in this form too.

As mentioned in the other thread, Barbican will be looking at oslo messaging
for Icehouse.

In the meantime, you can configure celery to use a pass-through (e.g. no
queue needed). This is the configuration we use for dev / testing. The
documentation for that can be found in the celery docs here:
http://docs.celeryproject.org/en/latest/userguide/


Jarret

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev