Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[John Wood]

>> Just an FYI that I've submitted a pull request [1] to replace Celery
>> with oslo.messaging.
>
> wow. That was quick!
>
> /me is impressed

Ha! Just trying to put the polish on! Thanks for the feedback and hope it is a 
step in the right direction. I'll look to add testing for it on Monday or 
before to take it out of 'in progress'.


> Since you jumped on that - I went ahead and jumped on a pbr-ification
> patch for you. ...

Thanks for the assist! I think we have some competing CRs out there but we'll 
sort those out once the oslo.messaging CR goes in...then we'll be able to 
remove Celery from the requirements once and for all!

Thanks again,
John



From: Monty Taylor [mord...@inaugust.com]
Sent: Friday, December 06, 2013 10:11 AM
To: John Wood; Mark McLoughlin; Douglas Mendizabal
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack...@lists.openstack.org; barbi...@lists.rackspace.com
Subject: Re: [openstack-tc] [openstack-dev] Incubation Request for Barbican

On 12/06/2013 08:35 AM, John Wood wrote:
> Hello folks,
>
> Just an FYI that I've submitted a pull request [1] to replace Celery
> with oslo.messaging.

wow. That was quick!

/me is impressed

Since you jumped on that - I went ahead and jumped on a pbr-ification
patch for you. It may not work yet - I'm on weird network and having
trouble install python things into virtualenvs:

  https://review.openstack.org/60551
  https://review.openstack.org/60552


> I've tagged it as a work in progress per this note:
>
> "Please review this CR, which replaces Celery with oslo.messaging
> components. I've verified that this works in my local environment,
> but I still need to add unit testing. I also need to verify that it
> works correctly with an HA Rabbit MQ cluster, as that is a hard
> requirement for Barbican."
>
> Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me
> to very useful links here [2] and here [3] respectively.
>
> [1] https://review.openstack.org/#/c/60427/ [2]
> https://review.openstack.org/#/c/39929 [3]
> https://review.openstack.org/#/c/57880
>
> Thanks, John
>
>  From: Monty Taylor
> [mord...@inaugust.com] Sent: Thursday, December 05, 2013 8:35 PM To:
> Mark McLoughlin; Douglas Mendizabal Cc: OpenStack Development Mailing
> List (not for usage questions); openstack...@lists.openstack.org;
> barbi...@lists.rackspace.com Subject: Re: [openstack-tc]
> [openstack-dev] Incubation Request for Barbican
>
> On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
>> On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:

 I agree that this is concerning. And that what's concerning
 isn't so much that the project did something different, but
 rather that choice was apparently made because the project
 thought it was perfectly fine for them to ignore what other
 OpenStack projects do and go off and do its own thing.

 We can't make this growth in the number of OpenStack projects
 work if each project goes off randomly and does its own thing
 without any concern for the difficulties that creates.

 Mark.
>>>
>>> Hi Mark,
>>>
>>> You may have missed it, but barbican has added a blueprint to
>>> change our queue to use oslo.messaging [1]
>>>
>>> I just wanted to clarify that we didn’t choose Celery because we
>>> thought that “it was perfectly fine to ignore what other
>>> OpenStack projects do”. Incubation has been one of our goals
>>> since the project began.  If you’ve taken the time to look at our
>>> code, you’ve seen that we have been using oslo.config this whole
>>> time.  We chose Celery because it was
>>>
>>> a) Properly packaged like any other python library, so we could
>>> just pip-install it. b) Well documented c) Well tested in
>>> production environments
>>>
>>> At the time none of those were true for oslo.messaging.  In
>>> fact, oslo.messgaging still cannot be pip-installed as of today.
>>> Obviously, had we know that using oslo.messaging is hard
>>> requirement in advance, we would have chosen it despite its poor
>>> distribution story.
>>
>> I do sympathise, but it's also true is that all other projects
>> were using the oslo-incubator RPC code at the time you chose
>> Celery.
>>
>> I think all the verbiage in this thread about celery is just to
>> reinforce that we need to be very sure that new projects feel a
>> responsibility to fit closely in with the rest of OpenStack. It's
>> not about technical requirements so much as social responsibility.
>>
>> But look - I think you've reacted well to the concern and hopefully
>> if it feels like there was an overreaction that you can understand
>> the broader thing we're trying to get at here.
>
> I agree. I think you've done an excellent job in responding to it -
> and I appreciate that. We're trying to be clearer about expectations
> moving forward, which I hope this thread in some part helps with.
>
> Monty
>

_

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Monty Taylor]

On 12/06/2013 08:35 AM, John Wood wrote:
> Hello folks,
> 
> Just an FYI that I've submitted a pull request [1] to replace Celery
> with oslo.messaging.

wow. That was quick!

/me is impressed

Since you jumped on that - I went ahead and jumped on a pbr-ification
patch for you. It may not work yet - I'm on weird network and having
trouble install python things into virtualenvs:

  https://review.openstack.org/60551
  https://review.openstack.org/60552


> I've tagged it as a work in progress per this note:
> 
> "Please review this CR, which replaces Celery with oslo.messaging
> components. I've verified that this works in my local environment,
> but I still need to add unit testing. I also need to verify that it
> works correctly with an HA Rabbit MQ cluster, as that is a hard
> requirement for Barbican."
> 
> Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me
> to very useful links here [2] and here [3] respectively.
> 
> [1] https://review.openstack.org/#/c/60427/ [2]
> https://review.openstack.org/#/c/39929 [3]
> https://review.openstack.org/#/c/57880
> 
> Thanks, John
> 
>  From: Monty Taylor
> [mord...@inaugust.com] Sent: Thursday, December 05, 2013 8:35 PM To:
> Mark McLoughlin; Douglas Mendizabal Cc: OpenStack Development Mailing
> List (not for usage questions); openstack...@lists.openstack.org;
> barbi...@lists.rackspace.com Subject: Re: [openstack-tc]
> [openstack-dev] Incubation Request for Barbican
> 
> On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
>> On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
 
 I agree that this is concerning. And that what's concerning
 isn't so much that the project did something different, but
 rather that choice was apparently made because the project
 thought it was perfectly fine for them to ignore what other
 OpenStack projects do and go off and do its own thing.
 
 We can't make this growth in the number of OpenStack projects
 work if each project goes off randomly and does its own thing
 without any concern for the difficulties that creates.
 
 Mark.
>>> 
>>> Hi Mark,
>>> 
>>> You may have missed it, but barbican has added a blueprint to
>>> change our queue to use oslo.messaging [1]
>>> 
>>> I just wanted to clarify that we didn’t choose Celery because we
>>> thought that “it was perfectly fine to ignore what other
>>> OpenStack projects do”. Incubation has been one of our goals
>>> since the project began.  If you’ve taken the time to look at our
>>> code, you’ve seen that we have been using oslo.config this whole
>>> time.  We chose Celery because it was
>>> 
>>> a) Properly packaged like any other python library, so we could
>>> just pip-install it. b) Well documented c) Well tested in
>>> production environments
>>> 
>>> At the time none of those were true for oslo.messaging.  In
>>> fact, oslo.messgaging still cannot be pip-installed as of today.
>>> Obviously, had we know that using oslo.messaging is hard
>>> requirement in advance, we would have chosen it despite its poor
>>> distribution story.
>> 
>> I do sympathise, but it's also true is that all other projects
>> were using the oslo-incubator RPC code at the time you chose
>> Celery.
>> 
>> I think all the verbiage in this thread about celery is just to 
>> reinforce that we need to be very sure that new projects feel a 
>> responsibility to fit closely in with the rest of OpenStack. It's
>> not about technical requirements so much as social responsibility.
>> 
>> But look - I think you've reacted well to the concern and hopefully
>> if it feels like there was an overreaction that you can understand
>> the broader thing we're trying to get at here.
> 
> I agree. I think you've done an excellent job in responding to it -
> and I appreciate that. We're trying to be clearer about expectations
> moving forward, which I hope this thread in some part helps with.
> 
> Monty
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[John Wood]

Hello folks,

Just an FYI that I've submitted a pull request [1] to replace Celery with 
oslo.messaging.

I've tagged it as a work in progress per this note:

"Please review this CR, which replaces Celery with oslo.messaging components. 
I've verified that this works in my local environment, but I still need to add 
unit testing. I also need to verify that it works correctly with an HA Rabbit 
MQ cluster, as that is a hard requirement for Barbican."

Special thanks to Mark McLoughlin and Sylvain Bauza for pointing me to very 
useful links here [2] and here [3] respectively.

[1] https://review.openstack.org/#/c/60427/
[2] https://review.openstack.org/#/c/39929
[3] https://review.openstack.org/#/c/57880

Thanks,
John


From: Monty Taylor [mord...@inaugust.com]
Sent: Thursday, December 05, 2013 8:35 PM
To: Mark McLoughlin; Douglas Mendizabal
Cc: OpenStack Development Mailing List (not for usage questions); 
openstack...@lists.openstack.org; barbi...@lists.rackspace.com
Subject: Re: [openstack-tc] [openstack-dev] Incubation Request for Barbican

On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
> On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
>>>
>>> I agree that this is concerning. And that what's concerning isn't so
>>> much that the project did something different, but rather that choice
>>> was apparently made because the project thought it was perfectly fine
>>> for them to ignore what other OpenStack projects do and go off and do
>>> its own thing.
>>>
>>> We can't make this growth in the number of OpenStack projects work if
>>> each project goes off randomly and does its own thing without any
>>> concern for the difficulties that creates.
>>>
>>> Mark.
>>
>> Hi Mark,
>>
>> You may have missed it, but barbican has added a blueprint to change our
>> queue to use oslo.messaging [1]
>>
>> I just wanted to clarify that we didn’t choose Celery because we thought
>> that “it was perfectly fine to ignore what other OpenStack projects do”.
>> Incubation has been one of our goals since the project began.  If you’ve
>> taken the time to look at our code, you’ve seen that we have been using
>> oslo.config this whole time.  We chose Celery because it was
>>
>> a) Properly packaged like any other python library, so we could just
>> pip-install it.
>> b) Well documented
>> c) Well tested in production environments
>>
>> At the time none of those were true for oslo.messaging.  In fact,
>> oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
>> we know that using oslo.messaging is hard requirement in advance, we would
>> have chosen it despite its poor distribution story.
>
> I do sympathise, but it's also true is that all other projects were
> using the oslo-incubator RPC code at the time you chose Celery.
>
> I think all the verbiage in this thread about celery is just to
> reinforce that we need to be very sure that new projects feel a
> responsibility to fit closely in with the rest of OpenStack. It's not
> about technical requirements so much as social responsibility.
>
> But look - I think you've reacted well to the concern and hopefully if
> it feels like there was an overreaction that you can understand the
> broader thing we're trying to get at here.

I agree. I think you've done an excellent job in responding to it - and
I appreciate that. We're trying to be clearer about expectations moving
forward, which I hope this thread in some part helps with.

Monty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Monty Taylor]

On 12/06/2013 01:53 AM, Mark McLoughlin wrote:
> On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
>>>
>>> I agree that this is concerning. And that what's concerning isn't so
>>> much that the project did something different, but rather that choice
>>> was apparently made because the project thought it was perfectly fine
>>> for them to ignore what other OpenStack projects do and go off and do
>>> its own thing.
>>>
>>> We can't make this growth in the number of OpenStack projects work if
>>> each project goes off randomly and does its own thing without any
>>> concern for the difficulties that creates.
>>>
>>> Mark.
>>
>> Hi Mark,
>>
>> You may have missed it, but barbican has added a blueprint to change our
>> queue to use oslo.messaging [1]
>>
>> I just wanted to clarify that we didn’t choose Celery because we thought
>> that “it was perfectly fine to ignore what other OpenStack projects do”.
>> Incubation has been one of our goals since the project began.  If you’ve
>> taken the time to look at our code, you’ve seen that we have been using
>> oslo.config this whole time.  We chose Celery because it was
>>
>> a) Properly packaged like any other python library, so we could just
>> pip-install it.
>> b) Well documented
>> c) Well tested in production environments
>>
>> At the time none of those were true for oslo.messaging.  In fact,
>> oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
>> we know that using oslo.messaging is hard requirement in advance, we would
>> have chosen it despite its poor distribution story.
> 
> I do sympathise, but it's also true is that all other projects were
> using the oslo-incubator RPC code at the time you chose Celery.
> 
> I think all the verbiage in this thread about celery is just to
> reinforce that we need to be very sure that new projects feel a
> responsibility to fit closely in with the rest of OpenStack. It's not
> about technical requirements so much as social responsibility.
> 
> But look - I think you've reacted well to the concern and hopefully if
> it feels like there was an overreaction that you can understand the
> broader thing we're trying to get at here.

I agree. I think you've done an excellent job in responding to it - and
I appreciate that. We're trying to be clearer about expectations moving
forward, which I hope this thread in some part helps with.

Monty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Mark McLoughlin]

On Thu, 2013-12-05 at 23:37 +, Douglas Mendizabal wrote:
> >
> >I agree that this is concerning. And that what's concerning isn't so
> >much that the project did something different, but rather that choice
> >was apparently made because the project thought it was perfectly fine
> >for them to ignore what other OpenStack projects do and go off and do
> >its own thing.
> >
> >We can't make this growth in the number of OpenStack projects work if
> >each project goes off randomly and does its own thing without any
> >concern for the difficulties that creates.
> >
> >Mark.
> 
> Hi Mark,
> 
> You may have missed it, but barbican has added a blueprint to change our
> queue to use oslo.messaging [1]
> 
> I just wanted to clarify that we didn’t choose Celery because we thought
> that “it was perfectly fine to ignore what other OpenStack projects do”.
> Incubation has been one of our goals since the project began.  If you’ve
> taken the time to look at our code, you’ve seen that we have been using
> oslo.config this whole time.  We chose Celery because it was
> 
> a) Properly packaged like any other python library, so we could just
> pip-install it.
> b) Well documented
> c) Well tested in production environments
> 
> At the time none of those were true for oslo.messaging.  In fact,
> oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
> we know that using oslo.messaging is hard requirement in advance, we would
> have chosen it despite its poor distribution story.

I do sympathise, but it's also true is that all other projects were
using the oslo-incubator RPC code at the time you chose Celery.

I think all the verbiage in this thread about celery is just to
reinforce that we need to be very sure that new projects feel a
responsibility to fit closely in with the rest of OpenStack. It's not
about technical requirements so much as social responsibility.

But look - I think you've reacted well to the concern and hopefully if
it feels like there was an overreaction that you can understand the
broader thing we're trying to get at here.

Thanks,
Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Douglas Mendizabal]

>
>I agree that this is concerning. And that what's concerning isn't so
>much that the project did something different, but rather that choice
>was apparently made because the project thought it was perfectly fine
>for them to ignore what other OpenStack projects do and go off and do
>its own thing.
>
>We can't make this growth in the number of OpenStack projects work if
>each project goes off randomly and does its own thing without any
>concern for the difficulties that creates.
>
>Mark.

Hi Mark,

You may have missed it, but barbican has added a blueprint to change our
queue to use oslo.messaging [1]

I just wanted to clarify that we didn’t choose Celery because we thought
that “it was perfectly fine to ignore what other OpenStack projects do”.
Incubation has been one of our goals since the project began.  If you’ve
taken the time to look at our code, you’ve seen that we have been using
oslo.config this whole time.  We chose Celery because it was

a) Properly packaged like any other python library, so we could just
pip-install it.
b) Well documented
c) Well tested in production environments

At the time none of those were true for oslo.messaging.  In fact,
oslo.messgaging still cannot be pip-installed as of today.  Obviously, had
we know that using oslo.messaging is hard requirement in advance, we would
have chosen it despite its poor distribution story.

- Doug Mendizabal

[1] 
https://blueprints.launchpad.net/barbican/+spec/queue-use-oslo-messaging

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Mark McLoughlin]

On Tue, 2013-12-03 at 16:43 +, Jarret Raim wrote:
> > The API and developer documentation is at
> >http://docs.openstack.org/developer/oslo.messaging/
> 
> This is great, thanks for the link. Would there be any objections to
> adding this to the github repo and the openstack wiki pages? I spent a
> bunch of time looking and wasn¹t able to turn this up.
> 
> 
> Additionally, where is this documentation generated from? I looked at the
> doc/ dir in the repo [1] and most of the files in there were empty.
> 
> 
> [1] https://github.com/openstack/oslo.messaging/tree/master/doc/source

It's generated from inline docstrings in the code.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Mark McLoughlin]

On Mon, 2013-12-02 at 17:33 -0500, Monty Taylor wrote:
> 
> On 12/02/2013 05:09 PM, Jarret Raim wrote:
...
> >> I'd like to address the use of Celery.
> >>
> >> WTF
> >>
> >> Barbican has been around for 9 months, which means that it does not
> >> predate the work that has become oslo.messaging. It doesn't even try. It
> >> uses a completely different thing.
> >>
> >> The use of celery needs to be replaced with oslo. Full stop. I do not
> >> believe it makes any sense to spend further time considering a project
> >> that's divergent on such a core piece. Which is a shame - because I
> >> think that Barbican is important and fills an important need and I want
> >> it to be in. BUT - We don't get to end-run around OpenStack project
> >> choices by making a new project on the side and then submitting it for
> >> incubation. It's going to be a pile of suck to fix this I'm sure, and
> >> I'm sure that it's going to delay getting actually important stuff done
> >> - but we deal with too much crazy as it is to pull in a non-oslo
> >> messaging and event substrata.
> > 
> > 
> > Is the challenge here that celery has some weird license requirements? Or
> > that it is a new library?
> 
> The second thing - as a thing to address functionality that has been in
> OpenStack since the beginning.
> 
> > When we started the Barbican project in February of this year,
> > oslo.messaging did not exist. If I remember correctly, at the time we were
> > doing architecture set up, the messaging piece was not available as a
> > standalone library, was not available on PyPi and had no documentation.
> 
> That is true. However, the equivalent oslo-incubator pieces were there.
> You can also get the latest pre-release library here:
> 
> -f
> http://tarballs.openstack.org/oslo.messaging/oslo.messaging-1.2.0a11.tar.gz#egg=oslo.messaging-1.2.0a11
> oslo.messaging>=1.2.0a11
> 
> As soon as we get the wheel publication pre-release code working, those
> will go to pypi.
> 
> > It looks like the project was moved to its own repo in April. However, I
> > can¹t seem to find the docs anywhere? The only thing I see is a design doc
> > here [1]. Are there plans for it to be packaged and put into Pypi?
> > 
> > We are probably overdue to look at oslo.messaging again, but I don¹t think
> > it should be a blocker for our incubation. I'm happy to take a look to see
> > what we can do during the Icehouse release cycle. Would that be
> > sufficient? 
> 
> I think it would go a long way if you had a roadmap for how you're going
> to move to oslo.messaging from celery. The worrying part for me, to be
> honest, is that you didn't sync up with the state of the art in that
> subject across openstack but instead went to a library that clearly
> no-one else was using. A couple of conversations with markmc or
> dhellman, or anyone from oslo, or anyone from any of the projects that
> are currently also using Rabbit (or zeromq or qpid) for the reasons
> you're using it would have gotten you aligned pretty quickly with how
> everyone else is doing this ... OR, and argument could have been made as
> to why all of that was a terrible idea and that the whole project should
> move to celery.
> 
> It's not that oslo-incubator was unknown - you're using other things.
> 
> When we look at incubating things, part of it is about the tech, but
> part of it is the question about whether or not the project is going to
> add to the whole's ability to be greater than the sum of its parts, of
> if we're going to inherit another group of folks who will be balkanized
> in the corner somewhere.
> 
> I don't want to sound punitive, that's not my intent - and I want you
> guys to succeed and to be a part of things. I just want to be careful
> that we don't add more complexity and confusion into an already
> difficult to fully digest landscape. I want to make sure that when we
> release an integrated release of OpenStack that it's an actual
> integrated stack that works together, and I want to make sure that we
> have an integrated developer community.
> 
> SO - without drawing a hard line in the sand or anything, I think that a
> discussion of what the real concrete plans are around alignment with the
> messaging strata in OpenStack is will probably be one of the more
> important questions we look at.

I agree that this is concerning. And that what's concerning isn't so
much that the project did something different, but rather that choice
was apparently made because the project thought it was perfectly fine
for them to ignore what other OpenStack projects do and go off and do
its own thing.

We can't make this growth in the number of OpenStack projects work if
each project goes off randomly and does its own thing without any
concern for the difficulties that creates.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Mark McLoughlin]

On Mon, 2013-12-02 at 22:09 +, Jarret Raim wrote:

> It looks like the project was moved to its own repo in April. However, I
> can¹t seem to find the docs anywhere? The only thing I see is a design doc
> here [1]. Are there plans for it to be packaged and put into Pypi?
> 
...
> 
> [1] https://wiki.openstack.org/wiki/Oslo/Messaging

Sorry for the slow response to this and that all you could find is that
design document. I've added a link to the full API docs from the design
doc:

  http://docs.openstack.org/developer/oslo.messaging/

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

>> 
>> I think this conversation has gotten away from our incubation request
>>and
>> into an argument about what makes a good library and when and how
>>projects
>> should choose between oslo and other options. I¹m happy to have the
>>second
>> one in another thread, but that seems like a longer conversation that is
>> separate from our request.
>
>It's absolutely about your incubation request.  Part of this process is
>looking at the technical fit with the rest of OpenStack, and this is
>well within the scope of that discussion.

Oh understood, I’m just saying that the ask was to move away from Celery
to oslo.messaging. We’ve agreed to work on that and I don’t think there is
a problem in getting that done for Icehouse.

It is the larger discussion about whether oslo.messaging is really the
best approach that seems off topic. We’re going to use it, so the argument
is moot for now. If we want to take up that discussion, we can certainly
do that, but it more like a conversation that happens within the oslo
group rather than part of Barbican's request.

>I need to make another pass over the added info and go deeper into the
>technical bits, so I suspect there will be more feedback still.  It's
>only been a day.  :-)

Great. Always happy to have another set of eyes on the code.

>Also, note that most of the things brought up so far are based on the
>proposed requirements for becoming incubated, not for things to be
>worked on during incubation.  Unless the requirements change, so far it
>looks like this request should be deferred a bit longer.

We are working on the issues that have been brought up so far. Today, we
submitted a PR to add in pbr and the global requirements [1]. Once that
has been reviewed, we'll land it after the Icehouse-1 release. Note that
celery is still in the list until we complete the oslo.messengaging work.


Jarret




[1] https://review.openstack.org/#/c/59817/1


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Russell Bryant]

On 12/03/2013 01:26 PM, Russell Bryant wrote:
> Unless the requirements change, so far it
> looks like this request should be deferred a bit longer.

And note that this is just my opinion, and note a statement of position
on behalf of the entire TC.  We can still officially consider the
request at an upcoming TC meeting.  I was just giving an indication of
where I stand right now.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Russell Bryant]

On 12/03/2013 11:40 AM, Jarret Raim wrote:
> 
>> I think there's something else you should take under consideration.
>> Oslo messaging is not just an OpenStack library. It's the RPC library
>> that all projects are relying on and one of the strong goals we have
>> in OpenStack is to reduce code and efforts duplications. We'd love to
>> have more people testing and contributing to oslo.messaging in order
>> to make it as battle tested as celery is.
>>
>> Please, don't get me wrong. I don't mean to say you didn't considered
>> it, I just want to add another reason why we should always try to
>> re-use the libraries that other projects are using - unless there's a
>> strong technical reason ;).
> 
> As I¹ve said, we are willing to look at the library for Icehouse. As lots
> of projects have implemented it, I hope that the switchover will be
> reasonably easy. 
> 
> I think this conversation has gotten away from our incubation request and
> into an argument about what makes a good library and when and how projects
> should choose between oslo and other options. I¹m happy to have the second
> one in another thread, but that seems like a longer conversation that is
> separate from our request.

It's absolutely about your incubation request.  Part of this process is
looking at the technical fit with the rest of OpenStack, and this is
well within the scope of that discussion.

> It seems like the comments are slowing down now. Does everyone feel our
> list (https://wiki.openstack.org/wiki/Barbican/Incubation) accurately
> captures the comments that have been brought up?
> 
> I filled out the Scope section of our request and I think we¹ve cleared up
> the PTL election issue. Is there anything else I missed or have we covered
> most of the issues?

I need to make another pass over the added info and go deeper into the
technical bits, so I suspect there will be more feedback still.  It's
only been a day.  :-)

Also, note that most of the things brought up so far are based on the
proposed requirements for becoming incubated, not for things to be
worked on during incubation.  Unless the requirements change, so far it
looks like this request should be deferred a bit longer.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jeremy Stanley]

On 2013-12-03 16:43:32 + (+), Jarret Raim wrote:
> This is great, thanks for the link. Would there be any objections to
> adding this to the github repo

I think you meant the "git" repo. What's a gi-thub?

http://git.openstack.org/cgit/openstack/oslo.messaging/tree/doc/source/ >

> and the openstack wiki pages? I spent a bunch of time looking and
> wasn¹t able to turn this up.

If you haven't worked on official OpenStack projects before, I can
see how it might be easy to overlook that source for the
Sphinx-built developer documentation lives in a standardized
location within each git repository (for convenience). Possibly
another pattern we should suggest following as part of any
application for incubation.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

> The API and developer documentation is at
>http://docs.openstack.org/developer/oslo.messaging/

This is great, thanks for the link. Would there be any objections to
adding this to the github repo and the openstack wiki pages? I spent a
bunch of time looking and wasn¹t able to turn this up.


Additionally, where is this documentation generated from? I looked at the
doc/ dir in the repo [1] and most of the files in there were empty.


[1] https://github.com/openstack/oslo.messaging/tree/master/doc/source




Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

>I think there's something else you should take under consideration.
>Oslo messaging is not just an OpenStack library. It's the RPC library
>that all projects are relying on and one of the strong goals we have
>in OpenStack is to reduce code and efforts duplications. We'd love to
>have more people testing and contributing to oslo.messaging in order
>to make it as battle tested as celery is.
>
>Please, don't get me wrong. I don't mean to say you didn't considered
>it, I just want to add another reason why we should always try to
>re-use the libraries that other projects are using - unless there's a
>strong technical reason ;).

As I¹ve said, we are willing to look at the library for Icehouse. As lots
of projects have implemented it, I hope that the switchover will be
reasonably easy. 

I think this conversation has gotten away from our incubation request and
into an argument about what makes a good library and when and how projects
should choose between oslo and other options. I¹m happy to have the second
one in another thread, but that seems like a longer conversation that is
separate from our request.

It seems like the comments are slowing down now. Does everyone feel our
list (https://wiki.openstack.org/wiki/Barbican/Incubation) accurately
captures the comments that have been brought up?

I filled out the Scope section of our request and I think we¹ve cleared up
the PTL election issue. Is there anything else I missed or have we covered
most of the issues?



Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Doug Hellmann]

On Mon, Dec 2, 2013 at 10:12 PM, Jarret Raim wrote:

> > There are two big parts to this, I think.  One is techincal - a
> significant
> > portion
> > of OpenStack deployments will not work with this because Celery does not
> > work with their deployed messaging architecture.
> >  See another reply in this thread for an example of someone that sees the
> > inability to use Qpid as a roadblock for an example.  This is solvable,
> but
> > not
> > quickly.
> >
> > The other is somewhat technical, but also a community issue.  Monty
> > articulated this well in another reply.  Barbican has made a conflicting
> > library
> > choice with what every other project using messaging is using.
> > With the number of projects we have, it is in our best interest to strive
> > for
> > consistency where we can.  Differences should not be arbitrary.  The
> > differences should only be where an exception is well justified.  I don't
> > see
> > that as being the case here.  Should everyone using oslo.messaging (or
> its
> > predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't
> know,
> > but that's the question at hand.  Ideally this would have come up with a
> > more
> > broad audience sooner.  If it did, I'm sorry I missed it.
>
> I understand the concern here and I'm happy to have Barbican look at using
> oslo.messaging during the Icehouse cycle.
>
> I am a bit surprised at the somewhat strong reactions to our choice. When
> we
> created Barbican, we looked at the messaging frameworks out there for use.
> At
> the time, oslo.messaging was not packaged, not documented, not tested, had
> no
> track record and an unknown level of community support.
>

The API and developer documentation is at
http://docs.openstack.org/developer/oslo.messaging/

Doug



> Celery is a battle-tested library that is widely deployed with a good track
> record, strong community and decent documentation. We made our choice
> based on
> those factors, just as the same as we would for any library inclusion.
>
> As celery has met our needs up to this point, we saw no reason to revisit
> the
> decision until now. In that time oslo.messaging  has moved to a separate
> repo.
> It still has little to no documentation, but the packaging and maintenance
> issues seem to be on the way to being sorted.
>
> So in short, in celery we get a reliable library with good docs that is
> battle
> tested, but is limited to the transports supported by Kombu. Both celery
> and
> Kombu are extendable and have many backends including AMQP, Redis,
> Beanstalk,
> Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.
>
> Oslo.messaging seems to have good support in OpenStack, but still lacks
> documentation and packaging (though some of that is being sorted out now).
> It
> offers support for qpid which celery seems to lack. It also offers a common
> place for message signing and some other nice to have features for
> OpenStack.
>
> Based on the commonality in OpenStack (and the lack of anyone else using
> Celery), I think looking to move to oslo.messaging is a good goal. This
> will
> take some time, but I think doing it by Icehouse seems reasonable. I think
> that is what you and Monty are asking for?
>
> I have added the task to our list on
> https://wiki.openstack.org/wiki/Barbican/Incubation.
>
>
> Thanks again for all the eyeballs our on application.
>
>
> Jarret
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Flavio Percoco]

On 03/12/13 03:12 +, Jarret Raim wrote:

There are two big parts to this, I think.  One is techincal - a significant
portion
of OpenStack deployments will not work with this because Celery does not
work with their deployed messaging architecture.
 See another reply in this thread for an example of someone that sees the
inability to use Qpid as a roadblock for an example.  This is solvable, but
not
quickly.

The other is somewhat technical, but also a community issue.  Monty
articulated this well in another reply.  Barbican has made a conflicting
library
choice with what every other project using messaging is using.
With the number of projects we have, it is in our best interest to strive
for
consistency where we can.  Differences should not be arbitrary.  The
differences should only be where an exception is well justified.  I don't
see
that as being the case here.  Should everyone using oslo.messaging (or its
predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't know,
but that's the question at hand.  Ideally this would have come up with a
more
broad audience sooner.  If it did, I'm sorry I missed it.


I understand the concern here and I'm happy to have Barbican look at using
oslo.messaging during the Icehouse cycle.

I am a bit surprised at the somewhat strong reactions to our choice. When we
created Barbican, we looked at the messaging frameworks out there for use. At
the time, oslo.messaging was not packaged, not documented, not tested, had no
track record and an unknown level of community support.


But there was oslo-incubator/rpc which all projects where already
using.


Celery is a battle-tested library that is widely deployed with a good track
record, strong community and decent documentation. We made our choice based on
those factors, just as the same as we would for any library inclusion.

As celery has met our needs up to this point, we saw no reason to revisit the
decision until now. In that time oslo.messaging  has moved to a separate repo.
It still has little to no documentation, but the packaging and maintenance
issues seem to be on the way to being sorted.

So in short, in celery we get a reliable library with good docs that is battle
tested, but is limited to the transports supported by Kombu. Both celery and
Kombu are extendable and have many backends including AMQP, Redis, Beanstalk,
Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.

Oslo.messaging seems to have good support in OpenStack, but still lacks
documentation and packaging (though some of that is being sorted out now). It
offers support for qpid which celery seems to lack. It also offers a common
place for message signing and some other nice to have features for OpenStack.


I think there's something else you should take under consideration.
Oslo messaging is not just an OpenStack library. It's the RPC library
that all projects are relying on and one of the strong goals we have
in OpenStack is to reduce code and efforts duplications. We'd love to
have more people testing and contributing to oslo.messaging in order
to make it as battle tested as celery is.

Please, don't get me wrong. I don't mean to say you didn't considered
it, I just want to add another reason why we should always try to
re-use the libraries that other projects are using - unless there's a
strong technical reason ;).


Based on the commonality in OpenStack (and the lack of anyone else using
Celery), I think looking to move to oslo.messaging is a good goal. This will
take some time, but I think doing it by Icehouse seems reasonable. I think
that is what you and Monty are asking for?




I have added the task to our list on
https://wiki.openstack.org/wiki/Barbican/Incubation.



Thanks a lot for this, really!

Cheers,
FF

--
@flaper87
Flavio Percoco


pgpznZ7d3tIAe.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[John Wood]

Hello folks,

I've created a blueprint to move over to oslo.messaging per Jarret's comments 
below: https://blueprints.launchpad.net/barbican

I'd also note that out of the box barbican does not use celery in order to 
simplify demo/standalone deployments...asynchronous calls simply invoke their 
worker tasks directly. 

Thanks,
John

From: cloudk...@googlegroups.com [cloudk...@googlegroups.com] on behalf of 
Jarret Raim [jarret.r...@rackspace.com]
Sent: Monday, December 02, 2013 9:16 PM
To: Fox, Kevin M; OpenStack Development Mailing List (not for usage questions); 
Russell Bryant
Cc: openstack...@lists.openstack.org; cloudkeep@googlegroups. com; 
barbi...@lists.rackspace.com
Subject: RE: [openstack-dev] [openstack-tc] Incubation Request for Barbican

> I've been anxious to try out Barbican, but haven't had quite enough time
to
> try it yet. But finding out it won't work with Qpid makes it unworkable
for us
> at the moment. I think a large swath of the OpenStack community won't be
> able to use it in this form too.

As mentioned in the other thread, Barbican will be looking at oslo messaging
for Icehouse.

In the meantime, you can configure celery to use a pass-through (e.g. no
queue needed). This is the configuration we use for dev / testing. The
documentation for that can be found in the celery docs here:
http://docs.celeryproject.org/en/latest/userguide/


Jarret

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

> I've been anxious to try out Barbican, but haven't had quite enough time
to
> try it yet. But finding out it won't work with Qpid makes it unworkable
for us
> at the moment. I think a large swath of the OpenStack community won't be
> able to use it in this form too.

As mentioned in the other thread, Barbican will be looking at oslo messaging
for Icehouse.

In the meantime, you can configure celery to use a pass-through (e.g. no
queue needed). This is the configuration we use for dev / testing. The
documentation for that can be found in the celery docs here:
http://docs.celeryproject.org/en/latest/userguide/


Jarret


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

> There are two big parts to this, I think.  One is techincal - a significant 
> portion
> of OpenStack deployments will not work with this because Celery does not
> work with their deployed messaging architecture.
>  See another reply in this thread for an example of someone that sees the
> inability to use Qpid as a roadblock for an example.  This is solvable, but 
> not
> quickly.
>
> The other is somewhat technical, but also a community issue.  Monty
> articulated this well in another reply.  Barbican has made a conflicting 
> library
> choice with what every other project using messaging is using.
> With the number of projects we have, it is in our best interest to strive 
> for
> consistency where we can.  Differences should not be arbitrary.  The
> differences should only be where an exception is well justified.  I don't 
> see
> that as being the case here.  Should everyone using oslo.messaging (or its
> predecessor rpc in oslo-incubator) be using Celery?  Maybe.  I don't know,
> but that's the question at hand.  Ideally this would have come up with a 
> more
> broad audience sooner.  If it did, I'm sorry I missed it.

I understand the concern here and I'm happy to have Barbican look at using 
oslo.messaging during the Icehouse cycle.

I am a bit surprised at the somewhat strong reactions to our choice. When we 
created Barbican, we looked at the messaging frameworks out there for use. At 
the time, oslo.messaging was not packaged, not documented, not tested, had no 
track record and an unknown level of community support.

Celery is a battle-tested library that is widely deployed with a good track 
record, strong community and decent documentation. We made our choice based on 
those factors, just as the same as we would for any library inclusion.

As celery has met our needs up to this point, we saw no reason to revisit the 
decision until now. In that time oslo.messaging  has moved to a separate repo. 
It still has little to no documentation, but the packaging and maintenance 
issues seem to be on the way to being sorted.

So in short, in celery we get a reliable library with good docs that is battle 
tested, but is limited to the transports supported by Kombu. Both celery and 
Kombu are extendable and have many backends including AMQP, Redis, Beanstalk, 
Amazon SQS, CouchDB, MongoDB, ZeroMQ, ZooKeeper, SoftLayer MQ and Pyro.

Oslo.messaging seems to have good support in OpenStack, but still lacks 
documentation and packaging (though some of that is being sorted out now). It 
offers support for qpid which celery seems to lack. It also offers a common 
place for message signing and some other nice to have features for OpenStack.

Based on the commonality in OpenStack (and the lack of anyone else using 
Celery), I think looking to move to oslo.messaging is a good goal. This will 
take some time, but I think doing it by Icehouse seems reasonable. I think 
that is what you and Monty are asking for?

I have added the task to our list on 
https://wiki.openstack.org/wiki/Barbican/Incubation.


Thanks again for all the eyeballs our on application.


Jarret



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Russell Bryant]

On 12/02/2013 06:46 PM, Dolph Mathews wrote:
> 
> 
> 
> On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant  > wrote:
> 
> On 12/02/2013 12:46 PM, Monty Taylor wrote:
> > On 12/02/2013 11:53 AM, Russell Bryant wrote:
> >>>  * Scope
> >>>  ** Project must have a clear and defined scope
> >>
> >> This is missing
> >>
> >>>  ** Project should not inadvertently duplicate functionality
> present in other
> >>> OpenStack projects. If they do, they should have a clear
> plan and timeframe
> >>> to prevent long-term scope duplication.
> >>>  ** Project should leverage existing functionality in other
> OpenStack projects
> >>> as much as possible
> >>
> >> I'm going to hold off on diving into this too far until the scope is
> >> clarified.
> >
> > I'm not.
> >
> > *snip*
> >
> 
> Ok, I can't help it now.
> 
> >>
> >> The list looks reasonable right now.  Barbican should put
> migrating to
> >> oslo.messaging on the Icehouse roadmap though.
> >
> > *snip*
> 
> Yeahhh ... I looked and even though rpc and notifier are imported, they
> do not appear to be used at all.
> 
> >>
> >>
> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
> >>
> >> It looks like the only item here not in the global requirements is
> >> Celery, which is licensed under a 3-clause BSD license.
> >
> > I'd like to address the use of Celery.
> >
> > WTF
> >
> > Barbican has been around for 9 months, which means that it does not
> > predate the work that has become oslo.messaging. It doesn't even
> try. It
> > uses a completely different thing.
> >
> > The use of celery needs to be replaced with oslo. Full stop. I do not
> > believe it makes any sense to spend further time considering a project
> > that's divergent on such a core piece. Which is a shame - because I
> > think that Barbican is important and fills an important need and I
> want
> > it to be in. BUT - We don't get to end-run around OpenStack project
> > choices by making a new project on the side and then submitting it for
> > incubation. It's going to be a pile of suck to fix this I'm sure, and
> > I'm sure that it's going to delay getting actually important stuff
> done
> > - but we deal with too much crazy as it is to pull in a non-oslo
> > messaging and event substrata.
> > 
> 
> Yeah, I'm afraid I agree with Monty here.  I didn't really address this
> because I was trying to just do a first pass and not go too far into the
> tech bits.
> 
> I think such a big divergence is going to be a hard sell for a number of
> reasons.  It's a significant dependency that I don't think is justified.
>  Further, it won't work in all of the same environments that OpenStack
> works in today.  You can't use Celery with all of the same messaging
> transports as oslo.messaging (or the older rpc lib).  One example is
> Qpid.
> 
> 
> I feel like I'm trying to read past the rant :) so I'd like to stop an
> ask for clarification on the exact argument being made. Is the *only*
> reason that celery should not be used in openstack because it is
> incapable of being deployed against amqp 1.0 brokers (i.e. qpid).
> 
> I'm really trying to understand if the actual objection is over the use
> (or not) of oslo (which seems like something the TC should express an
> opinion on, if that's the case), but rather about limiting OpenStack's
> deployment options.

There are two big parts to this, I think.  One is techincal - a
significant portion of OpenStack deployments will not work with this
because Celery does not work with their deployed messaging architecture.
 See another reply in this thread for an example of someone that sees
the inability to use Qpid as a roadblock for an example.  This is
solvable, but not quickly.

The other is somewhat technical, but also a community issue.  Monty
articulated this well in another reply.  Barbican has made a conflicting
library choice with what every other project using messaging is using.
With the number of projects we have, it is in our best interest to
strive for consistency where we can.  Differences should not be
arbitrary.  The differences should only be where an exception is well
justified.  I don't see that as being the case here.  Should everyone
using oslo.messaging (or its predecessor rpc in oslo-incubator) be using
Celery?  Maybe.  I don't know, but that's the question at hand.  Ideally
this would have come up with a more broad audience sooner.  If it did,
I'm sorry I missed it.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Fox, Kevin M]

I've been anxious to try out Barbican, but haven't had quite enough time to try 
it yet. But finding out it won't work with Qpid makes it unworkable for us at 
the moment. I think a large swath of the OpenStack community won't be able to 
use it in this form too.

Thanks,
Kevin


From: Dolph Mathews [dolph.math...@gmail.com]
Sent: Monday, December 02, 2013 3:46 PM
To: Russell Bryant
Cc: openstack...@lists.openstack.org; OpenStack Development Mailing List (not 
for usage questions); cloudkeep@googlegroups. com; barbi...@lists.rackspace.com
Subject: Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant 
mailto:rbry...@redhat.com>> wrote:
On 12/02/2013 12:46 PM, Monty Taylor wrote:
> On 12/02/2013 11:53 AM, Russell Bryant wrote:
>>>  * Scope
>>>  ** Project must have a clear and defined scope
>>
>> This is missing
>>
>>>  ** Project should not inadvertently duplicate functionality present in 
>>> other
>>> OpenStack projects. If they do, they should have a clear plan and 
>>> timeframe
>>> to prevent long-term scope duplication.
>>>  ** Project should leverage existing functionality in other OpenStack 
>>> projects
>>> as much as possible
>>
>> I'm going to hold off on diving into this too far until the scope is
>> clarified.
>
> I'm not.
>
> *snip*
>

Ok, I can't help it now.

>>
>> The list looks reasonable right now.  Barbican should put migrating to
>> oslo.messaging on the Icehouse roadmap though.
>
> *snip*

Yeahhh ... I looked and even though rpc and notifier are imported, they
do not appear to be used at all.

>>
>> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
>>
>> It looks like the only item here not in the global requirements is
>> Celery, which is licensed under a 3-clause BSD license.
>
> I'd like to address the use of Celery.
>
> WTF
>
> Barbican has been around for 9 months, which means that it does not
> predate the work that has become oslo.messaging. It doesn't even try. It
> uses a completely different thing.
>
> The use of celery needs to be replaced with oslo. Full stop. I do not
> believe it makes any sense to spend further time considering a project
> that's divergent on such a core piece. Which is a shame - because I
> think that Barbican is important and fills an important need and I want
> it to be in. BUT - We don't get to end-run around OpenStack project
> choices by making a new project on the side and then submitting it for
> incubation. It's going to be a pile of suck to fix this I'm sure, and
> I'm sure that it's going to delay getting actually important stuff done
> - but we deal with too much crazy as it is to pull in a non-oslo
> messaging and event substrata.
>
Yeah, I'm afraid I agree with Monty here.  I didn't really address this
because I was trying to just do a first pass and not go too far into the
tech bits.

I think such a big divergence is going to be a hard sell for a number of
reasons.  It's a significant dependency that I don't think is justified.
 Further, it won't work in all of the same environments that OpenStack
works in today.  You can't use Celery with all of the same messaging
transports as oslo.messaging (or the older rpc lib).  One example is Qpid.

I feel like I'm trying to read past the rant :) so I'd like to stop an ask for 
clarification on the exact argument being made. Is the *only* reason that 
celery should not be used in openstack because it is incapable of being 
deployed against amqp 1.0 brokers (i.e. qpid).

I'm really trying to understand if the actual objection is over the use (or 
not) of oslo (which seems like something the TC should express an opinion on, 
if that's the case), but rather about limiting OpenStack's deployment options.


--
Russell Bryant

___
OpenStack-TC mailing list
openstack...@lists.openstack.org<mailto:openstack...@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc

--

-Dolph

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Dolph Mathews]

On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant  wrote:

> On 12/02/2013 12:46 PM, Monty Taylor wrote:
> > On 12/02/2013 11:53 AM, Russell Bryant wrote:
> >>>  * Scope
> >>>  ** Project must have a clear and defined scope
> >>
> >> This is missing
> >>
> >>>  ** Project should not inadvertently duplicate functionality present
> in other
> >>> OpenStack projects. If they do, they should have a clear plan and
> timeframe
> >>> to prevent long-term scope duplication.
> >>>  ** Project should leverage existing functionality in other OpenStack
> projects
> >>> as much as possible
> >>
> >> I'm going to hold off on diving into this too far until the scope is
> >> clarified.
> >
> > I'm not.
> >
> > *snip*
> >
>
> Ok, I can't help it now.
>
> >>
> >> The list looks reasonable right now.  Barbican should put migrating to
> >> oslo.messaging on the Icehouse roadmap though.
> >
> > *snip*
>
> Yeahhh ... I looked and even though rpc and notifier are imported, they
> do not appear to be used at all.
>
> >>
> >>
> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
> >>
> >> It looks like the only item here not in the global requirements is
> >> Celery, which is licensed under a 3-clause BSD license.
> >
> > I'd like to address the use of Celery.
> >
> > WTF
> >
> > Barbican has been around for 9 months, which means that it does not
> > predate the work that has become oslo.messaging. It doesn't even try. It
> > uses a completely different thing.
> >
> > The use of celery needs to be replaced with oslo. Full stop. I do not
> > believe it makes any sense to spend further time considering a project
> > that's divergent on such a core piece. Which is a shame - because I
> > think that Barbican is important and fills an important need and I want
> > it to be in. BUT - We don't get to end-run around OpenStack project
> > choices by making a new project on the side and then submitting it for
> > incubation. It's going to be a pile of suck to fix this I'm sure, and
> > I'm sure that it's going to delay getting actually important stuff done
> > - but we deal with too much crazy as it is to pull in a non-oslo
> > messaging and event substrata.
> >
>
Yeah, I'm afraid I agree with Monty here.  I didn't really address this
> because I was trying to just do a first pass and not go too far into the
> tech bits.
>
> I think such a big divergence is going to be a hard sell for a number of
> reasons.  It's a significant dependency that I don't think is justified.
>  Further, it won't work in all of the same environments that OpenStack
> works in today.  You can't use Celery with all of the same messaging
> transports as oslo.messaging (or the older rpc lib).  One example is Qpid.


I feel like I'm trying to read past the rant :) so I'd like to stop an ask
for clarification on the exact argument being made. Is the *only* reason
that celery should not be used in openstack because it is incapable of
being deployed against amqp 1.0 brokers (i.e. qpid).

I'm really trying to understand if the actual objection is over the use (or
not) of oslo (which seems like something the TC should express an opinion
on, if that's the case), but rather about limiting OpenStack's deployment
options.


>
> --
> Russell Bryant
>
> ___
> OpenStack-TC mailing list
> openstack...@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc
>

-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Sylvain Bauza]

Hi Jarret,

2013/12/2 Jarret Raim 

>
>
> >It's also pretty easy for a stackforge project to opt-in to the global
> >requirements sync job now too.
>
> Are there some docs on how to do this somewhere? I added a task for us to
> complete the work as part of the incubation request here:
> https://wiki.openstack.org/wiki/Barbican/Incubation
>
>
There is a Stackforge how-to on ci.openstack.org that you can read [1].
Alternatively, you can also find our review [2] where we (Climate) sync'd
our requirements.txt with Oslo.

Hope it can help,
-Sylvain

[1] : http://ci.openstack.org/stackforge.html
[2] : https://review.openstack.org/#/c/56578/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Monty Taylor]

On 12/02/2013 05:09 PM, Jarret Raim wrote:
> 
  * Process
  ** Project must be hosted under stackforge (and therefore use git as
 its VCS)
>>>
>>> I see that barbican is now on stackforge,  but python-barbicanclient is
>>> still on github.  Is that being moved soon?
>>>
  ** Project must obey OpenStack coordinated project interface (such as
 tox,
 pbr, global-requirements...)
>>>
>>> Uses tox, but not pbr or global requirements
>>
>> It's also pretty easy for a stackforge project to opt-in to the global
>> requirements sync job now too.
> 
> Are there some docs on how to do this somewhere? I added a task for us to
> complete the work as part of the incubation request here:
> https://wiki.openstack.org/wiki/Barbican/Incubation

I'm not sure there are docs per-se - you can essentially just add your
project name to projects.txt in openstack/requirements. Beware - really
soon this will tie you to the openstack pypi mirror (like, possibly
today) so if you have non-aligned requirements (see below) it may block
you until you either align or get requirements changed.

  ** Project should use oslo libraries or oslo-incubator where
 appropriate
>>>
>>> The list looks reasonable right now.  Barbican should put migrating to
>>> oslo.messaging on the Icehouse roadmap though.
>>
>> *snip*
>>
>>>
>>>
>>> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
>>>
>>> It looks like the only item here not in the global requirements is
>>> Celery, which is licensed under a 3-clause BSD license.
>>
>> I'd like to address the use of Celery.
>>
>> WTF
>>
>> Barbican has been around for 9 months, which means that it does not
>> predate the work that has become oslo.messaging. It doesn't even try. It
>> uses a completely different thing.
>>
>> The use of celery needs to be replaced with oslo. Full stop. I do not
>> believe it makes any sense to spend further time considering a project
>> that's divergent on such a core piece. Which is a shame - because I
>> think that Barbican is important and fills an important need and I want
>> it to be in. BUT - We don't get to end-run around OpenStack project
>> choices by making a new project on the side and then submitting it for
>> incubation. It's going to be a pile of suck to fix this I'm sure, and
>> I'm sure that it's going to delay getting actually important stuff done
>> - but we deal with too much crazy as it is to pull in a non-oslo
>> messaging and event substrata.
> 
> 
> Is the challenge here that celery has some weird license requirements? Or
> that it is a new library?

The second thing - as a thing to address functionality that has been in
OpenStack since the beginning.

> When we started the Barbican project in February of this year,
> oslo.messaging did not exist. If I remember correctly, at the time we were
> doing architecture set up, the messaging piece was not available as a
> standalone library, was not available on PyPi and had no documentation.

That is true. However, the equivalent oslo-incubator pieces were there.
You can also get the latest pre-release library here:

-f
http://tarballs.openstack.org/oslo.messaging/oslo.messaging-1.2.0a11.tar.gz#egg=oslo.messaging-1.2.0a11
oslo.messaging>=1.2.0a11

As soon as we get the wheel publication pre-release code working, those
will go to pypi.

> It looks like the project was moved to its own repo in April. However, I
> can¹t seem to find the docs anywhere? The only thing I see is a design doc
> here [1]. Are there plans for it to be packaged and put into Pypi?
> 
> We are probably overdue to look at oslo.messaging again, but I don¹t think
> it should be a blocker for our incubation. I'm happy to take a look to see
> what we can do during the Icehouse release cycle. Would that be
> sufficient? 

I think it would go a long way if you had a roadmap for how you're going
to move to oslo.messaging from celery. The worrying part for me, to be
honest, is that you didn't sync up with the state of the art in that
subject across openstack but instead went to a library that clearly
no-one else was using. A couple of conversations with markmc or
dhellman, or anyone from oslo, or anyone from any of the projects that
are currently also using Rabbit (or zeromq or qpid) for the reasons
you're using it would have gotten you aligned pretty quickly with how
everyone else is doing this ... OR, and argument could have been made as
to why all of that was a terrible idea and that the whole project should
move to celery.

It's not that oslo-incubator was unknown - you're using other things.

When we look at incubating things, part of it is about the tech, but
part of it is the question about whether or not the project is going to
add to the whole's ability to be greater than the sum of its parts, of
if we're going to inherit another group of folks who will be balkanized
in the corner somewhere.

I don't want to sound punitive, that's not my intent - and I want you
guys to succeed and to be

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Jarret Raim]

>>>  * Process
>>>  ** Project must be hosted under stackforge (and therefore use git as
>>>its VCS)
>> 
>> I see that barbican is now on stackforge,  but python-barbicanclient is
>> still on github.  Is that being moved soon?
>> 
>>>  ** Project must obey OpenStack coordinated project interface (such as
>>>tox,
>>> pbr, global-requirements...)
>> 
>> Uses tox, but not pbr or global requirements
>
>It's also pretty easy for a stackforge project to opt-in to the global
>requirements sync job now too.

Are there some docs on how to do this somewhere? I added a task for us to
complete the work as part of the incubation request here:
https://wiki.openstack.org/wiki/Barbican/Incubation


>>>  ** Project should use oslo libraries or oslo-incubator where
>>>appropriate
>> 
>> The list looks reasonable right now.  Barbican should put migrating to
>> oslo.messaging on the Icehouse roadmap though.
>
>*snip*
>
>> 
>> 
>>http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
>> 
>> It looks like the only item here not in the global requirements is
>> Celery, which is licensed under a 3-clause BSD license.
>
>I'd like to address the use of Celery.
>
>WTF
>
>Barbican has been around for 9 months, which means that it does not
>predate the work that has become oslo.messaging. It doesn't even try. It
>uses a completely different thing.
>
>The use of celery needs to be replaced with oslo. Full stop. I do not
>believe it makes any sense to spend further time considering a project
>that's divergent on such a core piece. Which is a shame - because I
>think that Barbican is important and fills an important need and I want
>it to be in. BUT - We don't get to end-run around OpenStack project
>choices by making a new project on the side and then submitting it for
>incubation. It's going to be a pile of suck to fix this I'm sure, and
>I'm sure that it's going to delay getting actually important stuff done
>- but we deal with too much crazy as it is to pull in a non-oslo
>messaging and event substrata.


Is the challenge here that celery has some weird license requirements? Or
that it is a new library?

When we started the Barbican project in February of this year,
oslo.messaging did not exist. If I remember correctly, at the time we were
doing architecture set up, the messaging piece was not available as a
standalone library, was not available on PyPi and had no documentation.

It looks like the project was moved to its own repo in April. However, I
can¹t seem to find the docs anywhere? The only thing I see is a design doc
here [1]. Are there plans for it to be packaged and put into Pypi?

We are probably overdue to look at oslo.messaging again, but I don¹t think
it should be a blocker for our incubation. I'm happy to take a look to see
what we can do during the Icehouse release cycle. Would that be
sufficient? 


[1] https://wiki.openstack.org/wiki/Oslo/Messaging




Jarret


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Russell Bryant]

On 12/02/2013 12:46 PM, Monty Taylor wrote:
> On 12/02/2013 11:53 AM, Russell Bryant wrote:
>>>  * Scope
>>>  ** Project must have a clear and defined scope
>>
>> This is missing
>>
>>>  ** Project should not inadvertently duplicate functionality present in 
>>> other
>>> OpenStack projects. If they do, they should have a clear plan and 
>>> timeframe
>>> to prevent long-term scope duplication.
>>>  ** Project should leverage existing functionality in other OpenStack 
>>> projects
>>> as much as possible
>>
>> I'm going to hold off on diving into this too far until the scope is
>> clarified.
> 
> I'm not.
> 
> *snip*
> 

Ok, I can't help it now.

>>
>> The list looks reasonable right now.  Barbican should put migrating to
>> oslo.messaging on the Icehouse roadmap though.
> 
> *snip*

Yeahhh ... I looked and even though rpc and notifier are imported, they
do not appear to be used at all.

>>
>> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
>>
>> It looks like the only item here not in the global requirements is
>> Celery, which is licensed under a 3-clause BSD license.
> 
> I'd like to address the use of Celery.
> 
> WTF
> 
> Barbican has been around for 9 months, which means that it does not
> predate the work that has become oslo.messaging. It doesn't even try. It
> uses a completely different thing.
> 
> The use of celery needs to be replaced with oslo. Full stop. I do not
> believe it makes any sense to spend further time considering a project
> that's divergent on such a core piece. Which is a shame - because I
> think that Barbican is important and fills an important need and I want
> it to be in. BUT - We don't get to end-run around OpenStack project
> choices by making a new project on the side and then submitting it for
> incubation. It's going to be a pile of suck to fix this I'm sure, and
> I'm sure that it's going to delay getting actually important stuff done
> - but we deal with too much crazy as it is to pull in a non-oslo
> messaging and event substrata.
> 

Yeah, I'm afraid I agree with Monty here.  I didn't really address this
because I was trying to just do a first pass and not go too far into the
tech bits.

I think such a big divergence is going to be a hard sell for a number of
reasons.  It's a significant dependency that I don't think is justified.
 Further, it won't work in all of the same environments that OpenStack
works in today.  You can't use Celery with all of the same messaging
transports as oslo.messaging (or the older rpc lib).  One example is Qpid.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican

[Monty Taylor]

On 12/02/2013 11:53 AM, Russell Bryant wrote:
> On 12/02/2013 10:19 AM, Jarret Raim wrote:
>> All,
>>
>> Barbican is the OpenStack key management service and we’d like to
>> request incubation for the Icehouse release. A Rackspace sponsored team
>> has been working for about 9 months now, including following the Havana
>> release cycle for our first release.
>>
>> Our incubation request is here:
>> https://wiki.openstack.org/wiki/Barbican
>>
>> Our documentation is mostly hosted at GitHub for the moment, though we
>> are in the process of converting much of it to DocBook.
>> https://github.com/cloudkeep/barbican
>> https://github.com/cloudkeep/barbican/wiki
>>
>>
>> The Barbican team will be on IRC today at #openstack-barbican and you
>> can contact us using the barbi...@lists.rackspace.com mailing list if
>> desired.
> 
> The TC is currently working on formalizing requirements for new programs
> and projects [3].  I figured I would give them a try against this
> application.
> 
> First, I'm assuming that the application is for a new program that
> contains the new project.  The application doesn't make that bit clear,
> though.
> 
>> Teams in OpenStack can be created as-needed and grow organically. As the team
>> work matures, some technical efforts will be recognized as essential to the
>> completion of the OpenStack project mission. By becoming an official Program,
>> they place themselves under the authority of the OpenStack Technical
>> Committee. In return, their contributors get to vote in the Technical
>> Committee election, and they get some space and time to discuss future
>> development at our Design Summits. When considering new programs, the TC will
>> look into a number of criteria, including (but not limited to):
> 
>> * Scope
>> ** Team must have a specific scope, separated from others teams scope
> 
> I would like to see a statement of scope for Barbican on the
> application.  It should specifically cover how the scope differs from
> other programs, in particular the Identity program.
> 
>> ** Team must have a mission statement
> 
> This is missing.
>   
>> * Maturity
>> ** Team must already exist, have regular meetings and produce some work
> 
> This seems covered.
> 
>> ** Team should have a lead, elected by the team contributors
> 
> Was the PTL elected?  I can't seem to find record of that.  If not, I
> would like to see an election held for the PTL.
> 
>> ** Team should have a clear way to grant ATC (voting) status to its
>>significant contributors
> 
> Related to the above
> 
>> * Deliverables
>> ** Team should have a number of clear deliverables
> 
> barbican and python-barbicanclient, I presume.  It would be nice to have
> this clearly defined on the application.
> 
> 
> Now, for the project specific requirements:
> 
>>  Projects wishing to be included in the integrated release of OpenStack must
>>  first apply for incubation status. During their incubation period, they will
>>  be able to access new resources and tap into other OpenStack programs (in
>>  particular the Documentation, QA, Infrastructure and Release management 
>> teams)
>>  to learn about the OpenStack processes and get assistance in their 
>> integration
>>  efforts.
>>  
>>  The TC will evaluate the project scope and its complementarity with existing
>>  integrated projects and other official programs, look into the project
>>  technical choices, and check a number of requirements, including (but not
>>  limited to):
>>  
>>  * Scope
>>  ** Project must have a clear and defined scope
> 
> This is missing
> 
>>  ** Project should not inadvertently duplicate functionality present in other
>> OpenStack projects. If they do, they should have a clear plan and 
>> timeframe
>> to prevent long-term scope duplication.
>>  ** Project should leverage existing functionality in other OpenStack 
>> projects
>> as much as possible
> 
> I'm going to hold off on diving into this too far until the scope is
> clarified.

I'm not.

*snip*

>>  
>>  * Process
>>  ** Project must be hosted under stackforge (and therefore use git as its 
>> VCS)
> 
> I see that barbican is now on stackforge,  but python-barbicanclient is
> still on github.  Is that being moved soon?
> 
>>  ** Project must obey OpenStack coordinated project interface (such as tox,
>> pbr, global-requirements...)
> 
> Uses tox, but not pbr or global requirements

It's also pretty easy for a stackforge project to opt-in to the global
requirements sync job now too.

>>  ** Project should use oslo libraries or oslo-incubator where appropriate
> 
> The list looks reasonable right now.  Barbican should put migrating to
> oslo.messaging on the Icehouse roadmap though.

*snip*

> 
> http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
> 
> It looks like the only item here not in the global requirements is
> Celery, which is licensed under a 3-clause BSD license.

I'd like to address the use of Celery.

WTF

Barbican has been aroun