Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Thierry Carrez]

Ed Leafe wrote:

On May 25, 2016, at 7:25 AM, Denis Makogon  wrote:


Correct me if i'm wrong, none of the messages above were stating about support 
Go-extensions for Python (C extensions were mentioned couple times). Starting 
Go v1.5 it is possible to develop extension for Python [1] (lib that helps to 
develop extensions [2])


No, you’re not wrong at all.

This is much more in the original spirit for dealing with the inevitable issues 
where Python just doesn’t cut it performance-wise. The idea was to do 
everything in Python, and where there was a bottleneck, write a C module for 
that function and integrate it using ctypes.

So could someone from the Designate team do the following: isolate the part(s) 
of the process where Go kicks Python’s butt, create small Go packages to handle 
them, and then use gopy to integrate it? I think there would be little or no 
controversy with this approach, as it’s much less disruptive to the overall 
community.


Yes, this is a variant on the "external dependency" approach that would 
address most of the community fragmentation concerns by keeping the 
optimized parts small and Python-driven.


I could see that working for Designate's MiniDNS (and other partial 
optimizations), but I'm not sure that would work in the Hummingbird 
case, where all the node is rewritten in Go. If we mandated that 
approach, that would probably mean a lot of rework there...


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ed Leafe]

On May 25, 2016, at 7:25 AM, Denis Makogon  wrote:

> Correct me if i'm wrong, none of the messages above were stating about 
> support Go-extensions for Python (C extensions were mentioned couple times). 
> Starting Go v1.5 it is possible to develop extension for Python [1] (lib that 
> helps to develop extensions [2])

No, you’re not wrong at all.

This is much more in the original spirit for dealing with the inevitable issues 
where Python just doesn’t cut it performance-wise. The idea was to do 
everything in Python, and where there was a bottleneck, write a C module for 
that function and integrate it using ctypes. 

So could someone from the Designate team do the following: isolate the part(s) 
of the process where Go kicks Python’s butt, create small Go packages to handle 
them, and then use gopy to integrate it? I think there would be little or no 
controversy with this approach, as it’s much less disruptive to the overall 
community.


-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ed Leafe]

On May 25, 2016, at 7:09 PM, Adrian Otto  wrote:

> In order to evolve, OpenStack must allow alternatives.

A tautology, for sure, and it certainly doesn’t require changing languages in 
order to evolve. All sorts of alternative approaches can bring about evolution.

Evolution is also not without pain; without such stress evolution simply 
doesn’t happen. The discussion is about one such pain point, and how to best 
address it so as to eliminate the pain with the minimal amount of impact on the 
community.

-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Adrian Otto]

> On May 25, 2016, at 12:43 PM, Ben Swartzlander  wrote:
> 
> On 05/25/2016 06:48 AM, Sean Dague wrote:
>> I've been watching the threads, trying to digest, and find the way's
>> this is getting sliced doesn't quite slice the way I've been thinking
>> about it. (which might just means I've been thinking about it wrong).
>> However, here is my current set of thoughts on things.
>> 
>> 1. Should OpenStack be open to more languages?
>> 
>> I've long thought the answer should be yes. Especially if it means we
>> end up with keystonemiddleware, keystoneauth, oslo.config in other
>> languages that let us share elements of infrastructure pretty
>> seamlessly. The OpenStack model of building services that register in a
>> service catalog and use common tokens for permissions through a bunch of
>> services is quite valuable. There are definitely people that have Java
>> applications that fit into the OpenStack model, but have no place to
>> collaborate on them.
>> 
>> (Note: nothing about the current proposal goes anywhere near this)
>> 
>> 2. Is Go a "good" language to add to the community?
>> 
>> Here I am far more mixed. In programming language time, Go is super new.
>> It is roughly the same age as the OpenStack project. The idea that Go and
>> Python programmers overlap seems to be because some shops that used
>> to do a lot in Python, now do some things in Go.
>> 
>> But when compared to other languages in our bag, Javascript, Bash. These
>> are things that go back 2 decades. Unless you have avoided Linux or the
>> Web successfully for 2 decades, you've done these in some form. Maybe
>> not being an expert, but there is vestigial bits of knowledge there. So
>> they *are* different. In the same way that C or Java are different, for
>> having age. The likelihood of finding community members than know Python
>> + one of these is actually *way* higher than Python + Go, just based on
>> duration of existence. In a decade that probably won't be true.
> 
> Thank you for bringing up this point. My major concern boils down to the 
> likelihood that Go will never be well understood by more than a small subset 
> of the community. (When I say "well understood" I mean years of experiences 
> with thousands of lines of code -- not "I can write hello world").
> 
> You expect this problem to get better in the future -- I expect this problem 
> to get worse. Not all programming languages survive. Google for "dead 
> programming languages" some time and you'll find many examples. The problem 
> is that it's never obvious when the languages are young that something more 
> popular will come along and kill a language.
> 
> I don't want to imply that Golang is especially likely to die any time soon. 
> But every time you add a new language to a community, you increase the *risk* 
> that one of the programming languages used by the community will eventually 
> fall out of popularity, and it will become hard or impossible to find people 
> to maintain parts of the code.
> 
> I tend to take a long view of software lifecycles, having witnessed the death 
> of projects due to bad decisions before. Does anyone expect OpenStack to 
> still be around in 10 years? 20 years? What is the likelihood that both 
> Python and Golang are both still popular languages then? I guarantee [1] that 
> it's lower than the likelihood that only Python is still a popular language.
> 
> Adding a new language adds risk that new contributors won't understand some 
> parts of the code. Period. It doesn't matter what the language is.
> 
> My proposed solution is to draw the community line at the language barrier 
> line. People in this community are expected to understand Python. Anyone can 
> start other communities, and they can overlap with ours, but let's make it 
> clear that they're not the same.

Take all the names of the programming languages out for a moment here. The 
point is not that one is any more appropriate than another. In order to evolve, 
OpenStack must allow alternatives. It sets us up for long term success. 
Evolution is gradual change. Will we ever need to refactor things from one 
language to another, or have the same API implemented in two languages? Sure. 
That’s fine. Optimize for a long term outcome, not short term efficiencies. 
Twenty years from now if OpenStack still has a  “Python only” attitude, I’m 
sure it will be totally and utterly irrelevant. We will have all moved on by 
then. Let’s get this right, and offer individual projects freedom to do what 
they feel is best. Have a selection of designated languages, and rationale for 
why to stick with whichever one is preferred at a point in time.

Adrian

> 
> -Ben Swartzlander
> 
> [1] For all X, Y in (0, 1): X * Y < X
> 
>> 3. Are there performance problems where python really can't get there?
>> 
>> This seems like a pretty clear "yes". It shouldn't be surprising. Python
>> has no jit (yes there is pypy, but it's compat story isn't here). There
>> is 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[John Dickinson]

My responses are inline and to question 5, which, like you, I think is the key.

On 25 May 2016, at 3:48, Sean Dague wrote:

> I've been watching the threads, trying to digest, and find the way's
> this is getting sliced doesn't quite slice the way I've been thinking
> about it. (which might just means I've been thinking about it wrong).
> However, here is my current set of thoughts on things.
>
> 1. Should OpenStack be open to more languages?
>
> I've long thought the answer should be yes. Especially if it means we
> end up with keystonemiddleware, keystoneauth, oslo.config in other
> languages that let us share elements of infrastructure pretty
> seamlessly. The OpenStack model of building services that register in a
> service catalog and use common tokens for permissions through a bunch of
> services is quite valuable. There are definitely people that have Java
> applications that fit into the OpenStack model, but have no place to
> collaborate on them.
>
> (Note: nothing about the current proposal goes anywhere near this)
>
> 2. Is Go a "good" language to add to the community?
>
> Here I am far more mixed. In programming language time, Go is super new.
> It is roughly the same age as the OpenStack project. The idea that Go and
> Python programmers overlap seems to be because some shops that used
> to do a lot in Python, now do some things in Go.
>
> But when compared to other languages in our bag, Javascript, Bash. These
> are things that go back 2 decades. Unless you have avoided Linux or the
> Web successfully for 2 decades, you've done these in some form. Maybe
> not being an expert, but there is vestigial bits of knowledge there. So
> they *are* different. In the same way that C or Java are different, for
> having age. The likelihood of finding community members than know Python
> + one of these is actually *way* higher than Python + Go, just based on
> duration of existence. In a decade that probably won't be true.
>
> 3. Are there performance problems where python really can't get there?
>
> This seems like a pretty clear "yes". It shouldn't be surprising. Python
> has no jit (yes there is pypy, but it's compat story isn't here). There
> is a reason a bunch of python libs have native components for speed -
> numpy, lxml, cryptography, even yaml throws a warning that you should
> really compile the native version for performance when there is full
> python fallback.
>
> The Swift team did a very good job demonstrating where these issues are
> with trying to get raw disk IO. It was a great analysis, and kudos to
> that team for looking at so many angles here.
>
> 4. Do we want to be in the business of building data plane services that
> will all run into python limitations, and will all need to be rewritten
> in another language?
>
> This is a slightly different spin on the question Thierry is asking.
>
> Control Plane services are very unlikely to ever hit a scaling concern
> where rewriting the service in another language is needed for
> performance issues. These are orchestrators, and the time spent in them
> is vastly less than the operations they trigger (start a vm, configure a
> switch, boot a database server). There was a whole lot of talk in the
> threads of "well that's not innovative, no one will want to do just
> that", which seems weird, because that's most of OpenStack. And it's
> pretty much where all the effort in the containers space is right now,
> with a new container fleet manager every couple of weeks. So thinking
> that this is a boring problem no one wants to solve, doesn't hold water
> with me.
>
> Data Plane services seem like they will all end up in the boat of
> "python is not fast enough". Be it serving data from disk, mass DNS
> transfers, time series database, message queues. They will all
> eventually hit the python wall. Swift hit it first because of the
> maturity of the project and they are now focused on this kind of
> optimization, as that's what their user base demands. However I think
> all other data plane services will hit this as well.
>
> Glance (which is partially a data plane service) did hit this limit, and
> the way it is largely mitigated by folks is by using Ceph and exposing that
> directly to Nova so now Glance is only in the location game and metadata
> game, and Ceph is in the data plane game.
>
> When it comes to doing data plan services in OpenStack, I'm quite mixed.
> The technology concerns for data plane
> services are quite different. All the control plane services kind of
> look and feel the same. An API + worker model, a DB for state, message
> passing / rpc to put work to the workers. This is a common pattern and
> is something which even for all the project differences, does end up
> kind of common between parts. Projects that follow this model are
> debuggable as a group not too badly.
>
> 5. Where does Swift fit?
>
> This I think has always been a tension point in the community (at least
> since I joined in 2012). Swift is an original 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ben Swartzlander]

On 05/25/2016 06:48 AM, Sean Dague wrote:

I've been watching the threads, trying to digest, and find the way's
this is getting sliced doesn't quite slice the way I've been thinking
about it. (which might just means I've been thinking about it wrong).
However, here is my current set of thoughts on things.

1. Should OpenStack be open to more languages?

I've long thought the answer should be yes. Especially if it means we
end up with keystonemiddleware, keystoneauth, oslo.config in other
languages that let us share elements of infrastructure pretty
seamlessly. The OpenStack model of building services that register in a
service catalog and use common tokens for permissions through a bunch of
services is quite valuable. There are definitely people that have Java
applications that fit into the OpenStack model, but have no place to
collaborate on them.

(Note: nothing about the current proposal goes anywhere near this)

2. Is Go a "good" language to add to the community?

Here I am far more mixed. In programming language time, Go is super new.
It is roughly the same age as the OpenStack project. The idea that Go and
Python programmers overlap seems to be because some shops that used
to do a lot in Python, now do some things in Go.

But when compared to other languages in our bag, Javascript, Bash. These
are things that go back 2 decades. Unless you have avoided Linux or the
Web successfully for 2 decades, you've done these in some form. Maybe
not being an expert, but there is vestigial bits of knowledge there. So
they *are* different. In the same way that C or Java are different, for
having age. The likelihood of finding community members than know Python
+ one of these is actually *way* higher than Python + Go, just based on
duration of existence. In a decade that probably won't be true.


Thank you for bringing up this point. My major concern boils down to the 
likelihood that Go will never be well understood by more than a small 
subset of the community. (When I say "well understood" I mean years of 
experiences with thousands of lines of code -- not "I can write hello 
world").


You expect this problem to get better in the future -- I expect this 
problem to get worse. Not all programming languages survive. Google for 
"dead programming languages" some time and you'll find many examples. 
The problem is that it's never obvious when the languages are young that 
something more popular will come along and kill a language.


I don't want to imply that Golang is especially likely to die any time 
soon. But every time you add a new language to a community, you increase 
the *risk* that one of the programming languages used by the community 
will eventually fall out of popularity, and it will become hard or 
impossible to find people to maintain parts of the code.


I tend to take a long view of software lifecycles, having witnessed the 
death of projects due to bad decisions before. Does anyone expect 
OpenStack to still be around in 10 years? 20 years? What is the 
likelihood that both Python and Golang are both still popular languages 
then? I guarantee [1] that it's lower than the likelihood that only 
Python is still a popular language.


Adding a new language adds risk that new contributors won't understand 
some parts of the code. Period. It doesn't matter what the language is.


My proposed solution is to draw the community line at the language 
barrier line. People in this community are expected to understand 
Python. Anyone can start other communities, and they can overlap with 
ours, but let's make it clear that they're not the same.


-Ben Swartzlander

[1] For all X, Y in (0, 1): X * Y < X


3. Are there performance problems where python really can't get there?

This seems like a pretty clear "yes". It shouldn't be surprising. Python
has no jit (yes there is pypy, but it's compat story isn't here). There
is a reason a bunch of python libs have native components for speed -
numpy, lxml, cryptography, even yaml throws a warning that you should
really compile the native version for performance when there is full
python fallback.

The Swift team did a very good job demonstrating where these issues are
with trying to get raw disk IO. It was a great analysis, and kudos to
that team for looking at so many angles here.

4. Do we want to be in the business of building data plane services that
will all run into python limitations, and will all need to be rewritten
in another language?

This is a slightly different spin on the question Thierry is asking.

Control Plane services are very unlikely to ever hit a scaling concern
where rewriting the service in another language is needed for
performance issues. These are orchestrators, and the time spent in them
is vastly less than the operations they trigger (start a vm, configure a
switch, boot a database server). There was a whole lot of talk in the
threads of "well that's not innovative, no one will want to do just
that", which seems weird, because that's 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Fox, Kevin M]

+1. very good discussion.

From: Sean Dague [s...@dague.net]
Sent: Wednesday, May 25, 2016 3:48 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

I've been watching the threads, trying to digest, and find the way's
this is getting sliced doesn't quite slice the way I've been thinking
about it. (which might just means I've been thinking about it wrong).
However, here is my current set of thoughts on things.

1. Should OpenStack be open to more languages?

I've long thought the answer should be yes. Especially if it means we
end up with keystonemiddleware, keystoneauth, oslo.config in other
languages that let us share elements of infrastructure pretty
seamlessly. The OpenStack model of building services that register in a
service catalog and use common tokens for permissions through a bunch of
services is quite valuable. There are definitely people that have Java
applications that fit into the OpenStack model, but have no place to
collaborate on them.

(Note: nothing about the current proposal goes anywhere near this)

2. Is Go a "good" language to add to the community?

Here I am far more mixed. In programming language time, Go is super new.
It is roughly the same age as the OpenStack project. The idea that Go and
Python programmers overlap seems to be because some shops that used
to do a lot in Python, now do some things in Go.

But when compared to other languages in our bag, Javascript, Bash. These
are things that go back 2 decades. Unless you have avoided Linux or the
Web successfully for 2 decades, you've done these in some form. Maybe
not being an expert, but there is vestigial bits of knowledge there. So
they *are* different. In the same way that C or Java are different, for
having age. The likelihood of finding community members than know Python
+ one of these is actually *way* higher than Python + Go, just based on
duration of existence. In a decade that probably won't be true.

3. Are there performance problems where python really can't get there?

This seems like a pretty clear "yes". It shouldn't be surprising. Python
has no jit (yes there is pypy, but it's compat story isn't here). There
is a reason a bunch of python libs have native components for speed -
numpy, lxml, cryptography, even yaml throws a warning that you should
really compile the native version for performance when there is full
python fallback.

The Swift team did a very good job demonstrating where these issues are
with trying to get raw disk IO. It was a great analysis, and kudos to
that team for looking at so many angles here.

4. Do we want to be in the business of building data plane services that
will all run into python limitations, and will all need to be rewritten
in another language?

This is a slightly different spin on the question Thierry is asking.

Control Plane services are very unlikely to ever hit a scaling concern
where rewriting the service in another language is needed for
performance issues. These are orchestrators, and the time spent in them
is vastly less than the operations they trigger (start a vm, configure a
switch, boot a database server). There was a whole lot of talk in the
threads of "well that's not innovative, no one will want to do just
that", which seems weird, because that's most of OpenStack. And it's
pretty much where all the effort in the containers space is right now,
with a new container fleet manager every couple of weeks. So thinking
that this is a boring problem no one wants to solve, doesn't hold water
with me.

Data Plane services seem like they will all end up in the boat of
"python is not fast enough". Be it serving data from disk, mass DNS
transfers, time series database, message queues. They will all
eventually hit the python wall. Swift hit it first because of the
maturity of the project and they are now focused on this kind of
optimization, as that's what their user base demands. However I think
all other data plane services will hit this as well.

Glance (which is partially a data plane service) did hit this limit, and
the way it is largely mitigated by folks is by using Ceph and exposing that
directly to Nova so now Glance is only in the location game and metadata
game, and Ceph is in the data plane game.

When it comes to doing data plan services in OpenStack, I'm quite mixed.
The technology concerns for data plane
services are quite different. All the control plane services kind of
look and feel the same. An API + worker model, a DB for state, message
passing / rpc to put work to the workers. This is a common pattern and
is something which even for all the project differences, does end up
kind of common between parts. Projects that follow this model are
debuggable as a group not too badly.

5. Where does Swift fit?

This I think has always been a tension point in the community (at least
since I joined in 2012). Swift 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Doug Hellmann]

Excerpts from Flavio Percoco's message of 2016-05-25 13:21:49 +0200:
> On 25/05/16 06:48 -0400, Sean Dague wrote:
> 
> [snip]
> 
> >4. Do we want to be in the business of building data plane services that
> >will all run into python limitations, and will all need to be rewritten
> >in another language?
> >
> >This is a slightly different spin on the question Thierry is asking.
> >
> >Control Plane services are very unlikely to ever hit a scaling concern
> >where rewriting the service in another language is needed for
> >performance issues. These are orchestrators, and the time spent in them
> >is vastly less than the operations they trigger (start a vm, configure a
> >switch, boot a database server). There was a whole lot of talk in the
> >threads of "well that's not innovative, no one will want to do just
> >that", which seems weird, because that's most of OpenStack. And it's
> >pretty much where all the effort in the containers space is right now,
> >with a new container fleet manager every couple of weeks. So thinking
> >that this is a boring problem no one wants to solve, doesn't hold water
> >with me.
> >
> >Data Plane services seem like they will all end up in the boat of
> >"python is not fast enough". Be it serving data from disk, mass DNS
> >transfers, time series database, message queues. They will all
> >eventually hit the python wall. Swift hit it first because of the
> >maturity of the project and they are now focused on this kind of
> >optimization, as that's what their user base demands. However I think
> >all other data plane services will hit this as well.
> >
> >Glance (which is partially a data plane service) did hit this limit, and
> >the way it is largely mitigated by folks is by using Ceph and exposing that
> >directly to Nova so now Glance is only in the location game and metadata
> >game, and Ceph is in the data plane game.
> 
> Sorry for nitpicking here but Glance's API keeps being a data API. Sure it
> stores locations and sure you can do fancy things with those locations but, as
> far as end users go, it's still a data API. It is not be used as intensively 
> as
> Swift's, though. Ceph's driver allows for fancier things to be done but there
> are deployments which don't use Ceph.

FWIW, that's part of why my original suggestion for the new import
API only had the equivalent of the copy-from feature. Monty made a
good point that doing it that way required users to have some other
service from which they could import the data, so we still have a data
API in glance. Maybe we should revisit that decision after this issue is
resolved.

> I believe it'd be better to separate data services that *own* the data from
> those that integrate other backends. Swift owns the data. You upload it to
> swift, it stores the data using its own strategies and it serves it. Glance 
> gets
> the data, puts it in some other store and then you can either access it (not
> always) directly from the store or have Glance serving it back.

> 
> >When it comes to doing data plan services in OpenStack, I'm quite mixed.
> >The technology concerns for data plane
> >services are quite different. All the control plane services kind of
> >look and feel the same. An API + worker model, a DB for state, message
> >passing / rpc to put work to the workers. This is a common pattern and
> >is something which even for all the project differences, does end up
> >kind of common between parts. Projects that follow this model are
> >debuggable as a group not too badly.
> >
> >5. Where does Swift fit?
> >
> >This I think has always been a tension point in the community (at least
> >since I joined in 2012). Swift is an original service of OpenStack, as
> >it started as Swift and Nova. But they were very different things. Swift
> >is a data service, Nova was a control plane. Much of what is now
> >OpenStack is Nova derivative in some way (some times direct extractions
> >(Glance, Cinder, Ironic), some times convergent paths (Neutron). And
> >then with that many examples, lots of other things built in similar ways.
> >
> >Swift doesn't use common oslo components. That actually makes debugging
> >it quite different compared to the rest of OpenStack. The lack of
> >oslo.log means structured JSON log messages to Elastic Search, are not
> >a thing. Swift has a very different model in it's service split.
> >Swift doesn't use global requirements. Swift ensures it can run without
> >Keystone, because their goal is Swift everywhere, whether or not it's
> >part of the rest of OpenStack.
> >
> >These are all fine goals, but they definitely have led to tensions on
> >all sides.
> >
> >And I think part of the question is "are these tensions that need to be
> >solved" or "is this data that this thing is different". Which isn't to
> >say that Swift is bad, it's just definitively different than much of the
> >ecosystem. Maybe Swift should be graduated beyond OpenStack, because
> >it's scope cross cuts much differently. Ceph isn't part of 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Doug Hellmann]

Excerpts from Fox, Kevin M's message of 2016-05-24 19:38:06 +:
> Frankly, this is one of the major negatives we've felt from the Big Tent 
> idea...
> 
> OpenStack use to be more of a product then it is now. When there were common 
> problems to be solved, there was pressure applied to solve them in a way 
> everyone (OpenStack Project, OpenStack Users, and Openstack Operators) would 
> benefit from.
> 
> For recent example, there has been a lot of talk about reimplementing 
> features from Barbican in Magnum, Keystone, etc, and not wanting to depend on 
> Barbican. In the pre-tent days, we'd just fix Barbican to do the things we 
> all need it to, and then start depending on it. Then, everyone could start 
> depending on a solid secret store being there since everyone would deploy it 
> because they would want at least one thing that depends on it. Say, Lbaas, or 
> COE orchestration, and then adding more projects that depend on it would be 
> easier for the operator. Instead I see a lot of of trying to implement a hack 
> in each project to try and not depend on it, solving the problem for one 
> project but for no one else.

That's unfortunate. I remember talking about encouraging projects
to build on the work of others when we were discussing the big tent
changes.

  "Where it makes sense, the project cooperates with existing projects
  rather than gratuitously competing or reinventing the wheel." [1]

[1] http://governance.openstack.org/reference/new-projects-requirements.html

> Its a vicious chicken and the egg cycle we have created. Projects don't want 
> to depend on things if its not commonly deployed. Operators don't want to 
> deploy it if there's not a direct reason to, or something they do care about 
> depending on it. So our projects are encouraged to do bad things now and I 
> think we're all suffering for it.

Project teams seem to have misinterpreted or misunderstood, but that's
not the guidance we've given them.

> Cross project work became much harder after the big tent because there was 
> less reason to play nice with each other. OpenStack projects are already 
> fairly insular and this has made it worse. Opening up additional languages 
> makes it yet harder to work on the common stuff. I'm not against picking 1 
> additional language for performance critical stuff, but it should be 
> carefully considered, and should have to be carefully reasoned about the need 
> for.
> 
> Thanks,
> Kevin
> 
> 
> From: Ian Cordasco [sigmaviru...@gmail.com]
> Sent: Tuesday, May 24, 2016 12:12 PM
> To: Jay Pipes; OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"
> 
> -Original Message-
> From: Jay Pipes <jaypi...@gmail.com>
> Reply: OpenStack Development Mailing List (not for usage questions) 
> <openstack-dev@lists.openstack.org>
> Date: May 24, 2016 at 11:35:42
> To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
> Subject:  Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"
> 
> > On 05/24/2016 06:19 AM, Thierry Carrez wrote:
> > > Chris Dent wrote:
> > >> [...]
> > >> I don't really know. I'm firmly in the camp that OpenStack needs to
> > >> be smaller and more tightly focused if a unitary thing called OpenStack
> > >> expects to be any good. So I'm curious about and interested in
> > >> strategies for figuring out where the boundaries are.
> > >>
> > >> So that, of course, leads back to the original question: Is OpenStack
> > >> supposed to be a unitary.
> > >
> > > As a data point, since I heard that question rhetorically asked quite a
> > > few times over the past year... There is an old answer to that, since a
> > > vote of the PPB (the ancestor of our TC) from June, 2011 which was never
> > > overruled or changed afterwards:
> > >
> > > "OpenStack is a single product made of a lot of independent, but
> > > cooperating, components."
> > >
> > > The log is an interesting read:
> > > http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html
> >
> > Hmm, blast from the past. I'm sad I didn't make it to that meeting.
> >
> > I would (now at least) have voted for #2: OpenStack is "a collection of
> > independent projects that work together for some level of integration
> > and releases".
> >
> > This is how I believe OpenStack should be seen, as I wrote on Twitter
> > relatively recently:
> >
>

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Denis Makogon]

Hello to All.

This message is not about arguing weather OpenStack needs Go and other
language.

This is a good discussion. So, the main question here is "Go along with
Python for OpenStack" and the problem is to support Go code starting for
necessity of skilled Go developers to infrastructure for CI/CD, etc.

Correct me if i'm wrong, none of the messages above were stating about
support Go-extensions for Python (C extensions were mentioned couple
times). Starting Go v1.5 it is possible to develop extension for Python [1]
(lib that helps to develop extensions [2])
An idea is in:
  - "If think that your project is an exceptional one (swift, designate,
etc.) and you really think that Golang is what you need,
 then why can't you develop your own Go-extensions  and write Python
libs that are utilizing that code
 and then add that new python dependency to your project?"
  - distribute your go-extensions (*.so files) and DEB/RPM for further
consumption in DevStack for example (like we do for multiple components -
Kafka, Cassandra, MySQL, RMQ, etc.)

As i can see such behaviour would allow Python to be main lang for
OpenStack development, we wouldn't have an overhead for building new
infrastructure for Go and would allow projects to use Go for developing
their extensions out of OpenStack Big Tent.

[1] https://blog.filippo.io/building-python-modules-with-go-1-5/
[2] https://github.com/sbinet/go-python

Kind regards,
Denys Makogon


2016-05-25 14:21 GMT+03:00 Flavio Percoco :

> On 25/05/16 06:48 -0400, Sean Dague wrote:
>
> [snip]
>
>
> 4. Do we want to be in the business of building data plane services that
>> will all run into python limitations, and will all need to be rewritten
>> in another language?
>>
>> This is a slightly different spin on the question Thierry is asking.
>>
>> Control Plane services are very unlikely to ever hit a scaling concern
>> where rewriting the service in another language is needed for
>> performance issues. These are orchestrators, and the time spent in them
>> is vastly less than the operations they trigger (start a vm, configure a
>> switch, boot a database server). There was a whole lot of talk in the
>> threads of "well that's not innovative, no one will want to do just
>> that", which seems weird, because that's most of OpenStack. And it's
>> pretty much where all the effort in the containers space is right now,
>> with a new container fleet manager every couple of weeks. So thinking
>> that this is a boring problem no one wants to solve, doesn't hold water
>> with me.
>>
>> Data Plane services seem like they will all end up in the boat of
>> "python is not fast enough". Be it serving data from disk, mass DNS
>> transfers, time series database, message queues. They will all
>> eventually hit the python wall. Swift hit it first because of the
>> maturity of the project and they are now focused on this kind of
>> optimization, as that's what their user base demands. However I think
>> all other data plane services will hit this as well.
>>
>> Glance (which is partially a data plane service) did hit this limit, and
>> the way it is largely mitigated by folks is by using Ceph and exposing
>> that
>> directly to Nova so now Glance is only in the location game and metadata
>> game, and Ceph is in the data plane game.
>>
>
> Sorry for nitpicking here but Glance's API keeps being a data API. Sure it
> stores locations and sure you can do fancy things with those locations
> but, as
> far as end users go, it's still a data API. It is not be used as
> intensively as
> Swift's, though. Ceph's driver allows for fancier things to be done but
> there
> are deployments which don't use Ceph.
>
> I believe it'd be better to separate data services that *own* the data from
> those that integrate other backends. Swift owns the data. You upload it to
> swift, it stores the data using its own strategies and it serves it.
> Glance gets
> the data, puts it in some other store and then you can either access it
> (not
> always) directly from the store or have Glance serving it back.
>
>
> When it comes to doing data plan services in OpenStack, I'm quite mixed.
>> The technology concerns for data plane
>> services are quite different. All the control plane services kind of
>> look and feel the same. An API + worker model, a DB for state, message
>> passing / rpc to put work to the workers. This is a common pattern and
>> is something which even for all the project differences, does end up
>> kind of common between parts. Projects that follow this model are
>> debuggable as a group not too badly.
>>
>> 5. Where does Swift fit?
>>
>> This I think has always been a tension point in the community (at least
>> since I joined in 2012). Swift is an original service of OpenStack, as
>> it started as Swift and Nova. But they were very different things. Swift
>> is a data service, Nova was a control plane. Much of what is now
>> OpenStack is Nova derivative in some way (some times 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Flavio Percoco]

On 25/05/16 06:48 -0400, Sean Dague wrote:

[snip]


4. Do we want to be in the business of building data plane services that
will all run into python limitations, and will all need to be rewritten
in another language?

This is a slightly different spin on the question Thierry is asking.

Control Plane services are very unlikely to ever hit a scaling concern
where rewriting the service in another language is needed for
performance issues. These are orchestrators, and the time spent in them
is vastly less than the operations they trigger (start a vm, configure a
switch, boot a database server). There was a whole lot of talk in the
threads of "well that's not innovative, no one will want to do just
that", which seems weird, because that's most of OpenStack. And it's
pretty much where all the effort in the containers space is right now,
with a new container fleet manager every couple of weeks. So thinking
that this is a boring problem no one wants to solve, doesn't hold water
with me.

Data Plane services seem like they will all end up in the boat of
"python is not fast enough". Be it serving data from disk, mass DNS
transfers, time series database, message queues. They will all
eventually hit the python wall. Swift hit it first because of the
maturity of the project and they are now focused on this kind of
optimization, as that's what their user base demands. However I think
all other data plane services will hit this as well.

Glance (which is partially a data plane service) did hit this limit, and
the way it is largely mitigated by folks is by using Ceph and exposing that
directly to Nova so now Glance is only in the location game and metadata
game, and Ceph is in the data plane game.


Sorry for nitpicking here but Glance's API keeps being a data API. Sure it
stores locations and sure you can do fancy things with those locations but, as
far as end users go, it's still a data API. It is not be used as intensively as
Swift's, though. Ceph's driver allows for fancier things to be done but there
are deployments which don't use Ceph.

I believe it'd be better to separate data services that *own* the data from
those that integrate other backends. Swift owns the data. You upload it to
swift, it stores the data using its own strategies and it serves it. Glance gets
the data, puts it in some other store and then you can either access it (not
always) directly from the store or have Glance serving it back.


When it comes to doing data plan services in OpenStack, I'm quite mixed.
The technology concerns for data plane
services are quite different. All the control plane services kind of
look and feel the same. An API + worker model, a DB for state, message
passing / rpc to put work to the workers. This is a common pattern and
is something which even for all the project differences, does end up
kind of common between parts. Projects that follow this model are
debuggable as a group not too badly.

5. Where does Swift fit?

This I think has always been a tension point in the community (at least
since I joined in 2012). Swift is an original service of OpenStack, as
it started as Swift and Nova. But they were very different things. Swift
is a data service, Nova was a control plane. Much of what is now
OpenStack is Nova derivative in some way (some times direct extractions
(Glance, Cinder, Ironic), some times convergent paths (Neutron). And
then with that many examples, lots of other things built in similar ways.

Swift doesn't use common oslo components. That actually makes debugging
it quite different compared to the rest of OpenStack. The lack of
oslo.log means structured JSON log messages to Elastic Search, are not
a thing. Swift has a very different model in it's service split.
Swift doesn't use global requirements. Swift ensures it can run without
Keystone, because their goal is Swift everywhere, whether or not it's
part of the rest of OpenStack.

These are all fine goals, but they definitely have led to tensions on
all sides.

And I think part of the question is "are these tensions that need to be
solved" or "is this data that this thing is different". Which isn't to
say that Swift is bad, it's just definitively different than much of the
ecosystem. Maybe Swift should be graduated beyond OpenStack, because
it's scope cross cuts much differently. Ceph isn't part of OpenStack,
but it's in 50% of installs. libvirt isn't part of OpenStack, but it's
in 90% of installs. And in both of those cases OpenStack is one of the
biggest drivers of their use.

Which, gets contentious because people feel like this is kicking
something out. And that I can understand. There is a lot of emotion
wrapped up in labels and who gets to be on the the OpenStack home page.
I wish there wasn't. Good software should get deployed because it is
good and solves a need, not because of labels. I'm not sure Swift users
really care that Swift is OpenStack. They care that Swift is Swift. And
Swift being Swift, but not being OpenStack would open up 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Sean Dague]

I've been watching the threads, trying to digest, and find the way's
this is getting sliced doesn't quite slice the way I've been thinking
about it. (which might just means I've been thinking about it wrong).
However, here is my current set of thoughts on things.

1. Should OpenStack be open to more languages?

I've long thought the answer should be yes. Especially if it means we
end up with keystonemiddleware, keystoneauth, oslo.config in other
languages that let us share elements of infrastructure pretty
seamlessly. The OpenStack model of building services that register in a
service catalog and use common tokens for permissions through a bunch of
services is quite valuable. There are definitely people that have Java
applications that fit into the OpenStack model, but have no place to
collaborate on them.

(Note: nothing about the current proposal goes anywhere near this)

2. Is Go a "good" language to add to the community?

Here I am far more mixed. In programming language time, Go is super new.
It is roughly the same age as the OpenStack project. The idea that Go and
Python programmers overlap seems to be because some shops that used
to do a lot in Python, now do some things in Go.

But when compared to other languages in our bag, Javascript, Bash. These
are things that go back 2 decades. Unless you have avoided Linux or the
Web successfully for 2 decades, you've done these in some form. Maybe
not being an expert, but there is vestigial bits of knowledge there. So
they *are* different. In the same way that C or Java are different, for
having age. The likelihood of finding community members than know Python
+ one of these is actually *way* higher than Python + Go, just based on
duration of existence. In a decade that probably won't be true.

3. Are there performance problems where python really can't get there?

This seems like a pretty clear "yes". It shouldn't be surprising. Python
has no jit (yes there is pypy, but it's compat story isn't here). There
is a reason a bunch of python libs have native components for speed -
numpy, lxml, cryptography, even yaml throws a warning that you should
really compile the native version for performance when there is full
python fallback.

The Swift team did a very good job demonstrating where these issues are
with trying to get raw disk IO. It was a great analysis, and kudos to
that team for looking at so many angles here.

4. Do we want to be in the business of building data plane services that
will all run into python limitations, and will all need to be rewritten
in another language?

This is a slightly different spin on the question Thierry is asking.

Control Plane services are very unlikely to ever hit a scaling concern
where rewriting the service in another language is needed for
performance issues. These are orchestrators, and the time spent in them
is vastly less than the operations they trigger (start a vm, configure a
switch, boot a database server). There was a whole lot of talk in the
threads of "well that's not innovative, no one will want to do just
that", which seems weird, because that's most of OpenStack. And it's
pretty much where all the effort in the containers space is right now,
with a new container fleet manager every couple of weeks. So thinking
that this is a boring problem no one wants to solve, doesn't hold water
with me.

Data Plane services seem like they will all end up in the boat of
"python is not fast enough". Be it serving data from disk, mass DNS
transfers, time series database, message queues. They will all
eventually hit the python wall. Swift hit it first because of the
maturity of the project and they are now focused on this kind of
optimization, as that's what their user base demands. However I think
all other data plane services will hit this as well.

Glance (which is partially a data plane service) did hit this limit, and
the way it is largely mitigated by folks is by using Ceph and exposing that
directly to Nova so now Glance is only in the location game and metadata
game, and Ceph is in the data plane game.

When it comes to doing data plan services in OpenStack, I'm quite mixed.
The technology concerns for data plane
services are quite different. All the control plane services kind of
look and feel the same. An API + worker model, a DB for state, message
passing / rpc to put work to the workers. This is a common pattern and
is something which even for all the project differences, does end up
kind of common between parts. Projects that follow this model are
debuggable as a group not too badly.

5. Where does Swift fit?

This I think has always been a tension point in the community (at least
since I joined in 2012). Swift is an original service of OpenStack, as
it started as Swift and Nova. But they were very different things. Swift
is a data service, Nova was a control plane. Much of what is now
OpenStack is Nova derivative in some way (some times direct extractions
(Glance, Cinder, Ironic), some times convergent paths 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Julien Danjou]

On Tue, May 24 2016, Fox, Kevin M wrote:

> For recent example, there has been a lot of talk about reimplementing features
> from Barbican in Magnum, Keystone, etc, and not wanting to depend on Barbican.
> In the pre-tent days, we'd just fix Barbican to do the things we all need it
> to, and then start depending on it. Then, everyone could start depending on a
> solid secret store being there since everyone would deploy it because they
> would want at least one thing that depends on it.

Sorry, but before Big Tent, it was impossible to have an integrated
project depending of a non-integrated project.

So what you're imagining was even less likely to happen.

-- 
Julien Danjou
-- Free Software hacker
-- https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Adrian Otto]

> On May 24, 2016, at 12:09 PM, Mike Perez  wrote:
> 
> On 12:24 May 24, Thierry Carrez wrote:
>> Morgan Fainberg wrote:
>>> [...]  If we are accepting golang, I want it to be clearly
>>> documented that the expectation is it is used exclusively where there is
>>> a demonstrable case (such as with swift) and not a carte blanche to use
>>> it wherever-you-please.
>>> 
>>> I want this to be a social contract looked at and enforced by the
>>> community, not special permissions that are granted by the TC (I don't
>>> want the TC to need to step in an approve every single use case of
>>> golang, or javascript ...). It's bottlenecking back to the TC for
>>> special permissions or inclusion (see reasons for the dissolution of the
>>> "integrated release").
>>> 
>>> This isn't strictly an all or nothing case, this is a "how would we
>>> enforce this?" type deal. Lean on infra to enforce that only projects
>>> with the golang-is-ok-here tag are allowed to use it? I don't want
>>> people to write their APIs in javascript (and node.js) nor in golang. I
>>> would like to see most of the work continue with python as the primary
>>> language. I just think it's unreasonable to lock tools behind a gate
>>> that is stronger than the social / community contract (and outlined in
>>> the resolution including X language).
>> 
>> +1
>> 
>> I'd prefer if we didn't have to special-case anyone, and we could come up
>> with general rules that every OpenStack project follows. Any other solution
>> is an administrative nightmare and a source of tension between projects (why
>> are they special and not me).
> 
> I'm in agreement that I don't want to see the TC enforcing this. In fact as
> Thierry has said, lets not special case anyone.
> 
> As soon as a special case is accepted, as nortoriously happens people are 
> going
> to go in a corner and rewrite things in Go. They will be upset later for not
> communicating well on their intentions upfront, and the TC or a few strongly
> opinionated folks in the community are going to be made the bad people just
> about every time.
> 
> Community enforcing or not, I predict this to get out of hand and it's going 
> to
> create more community divide regardless.

I remember in 2010, our founding intent was to converge on two languages for 
OpenStack Development: Python and C. We would prefer Python for things like 
control plane API services, and when needed for performance or other reasons, 
we would use C as an alternative. To my knowledge, since then nothing was ever 
written in C. We have a clear trend of high performance alternative solutions 
showing up in Golang. So, I suggest we go back to the original intent that we 
build things in Python as our preference, and allow teams to select a 
designated alternative when they have their own reasons to do that. I see no 
reason why that designated alternative can not be Golang[1].

Programming syles and languages evolve over time. Otherwise we’d all still be 
using FORTRAN from 1954. OpenStack, as a community needs to have a deliberate 
plan for how to track such evolution. Digging our heels in with a Python only 
attitude is not progressive enough. Giving choice of any option under the sun 
is not practical. We will strike a balance. Recognize that evolution requires 
duplication. There must be overlap (wasted effort maintaining common code in 
multiple languages) in order to allow evolution. This overlap is healthy. We 
don’t need our TC to decide when a project should be allowed to use a 
designated alternative. Set rules that allow for innovation, and then let 
projects decide on their own within such guidelines.

My proposal:

"Openstack projects shall use Python as the preferred programming language. 
Golang may be used as an alternative if the project leadership decides it is 
justified."

Additional (non-regulatory) guidance can also be offered by the OpenStack 
community to indicate when individual projects should decide to use an 
alternative language. In the future as we notice evolution around us, we may 
add other alternatives to that list.

Thanks,

Adrian

[1] I categorically reject the previous rhetoric that casts Golang as a 
premature language that we can’t rely on. That’s FUD, plain and simple. Stop it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Fox, Kevin M]

Frankly, this is one of the major negatives we've felt from the Big Tent idea...

OpenStack use to be more of a product then it is now. When there were common 
problems to be solved, there was pressure applied to solve them in a way 
everyone (OpenStack Project, OpenStack Users, and Openstack Operators) would 
benefit from.

For recent example, there has been a lot of talk about reimplementing features 
from Barbican in Magnum, Keystone, etc, and not wanting to depend on Barbican. 
In the pre-tent days, we'd just fix Barbican to do the things we all need it 
to, and then start depending on it. Then, everyone could start depending on a 
solid secret store being there since everyone would deploy it because they 
would want at least one thing that depends on it. Say, Lbaas, or COE 
orchestration, and then adding more projects that depend on it would be easier 
for the operator. Instead I see a lot of of trying to implement a hack in each 
project to try and not depend on it, solving the problem for one project but 
for no one else.

Its a vicious chicken and the egg cycle we have created. Projects don't want to 
depend on things if its not commonly deployed. Operators don't want to deploy 
it if there's not a direct reason to, or something they do care about depending 
on it. So our projects are encouraged to do bad things now and I think we're 
all suffering for it.

Cross project work became much harder after the big tent because there was less 
reason to play nice with each other. OpenStack projects are already fairly 
insular and this has made it worse. Opening up additional languages makes it 
yet harder to work on the common stuff. I'm not against picking 1 additional 
language for performance critical stuff, but it should be carefully considered, 
and should have to be carefully reasoned about the need for.

Thanks,
Kevin


From: Ian Cordasco [sigmaviru...@gmail.com]
Sent: Tuesday, May 24, 2016 12:12 PM
To: Jay Pipes; OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

-Original Message-
From: Jay Pipes <jaypi...@gmail.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 24, 2016 at 11:35:42
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

> On 05/24/2016 06:19 AM, Thierry Carrez wrote:
> > Chris Dent wrote:
> >> [...]
> >> I don't really know. I'm firmly in the camp that OpenStack needs to
> >> be smaller and more tightly focused if a unitary thing called OpenStack
> >> expects to be any good. So I'm curious about and interested in
> >> strategies for figuring out where the boundaries are.
> >>
> >> So that, of course, leads back to the original question: Is OpenStack
> >> supposed to be a unitary.
> >
> > As a data point, since I heard that question rhetorically asked quite a
> > few times over the past year... There is an old answer to that, since a
> > vote of the PPB (the ancestor of our TC) from June, 2011 which was never
> > overruled or changed afterwards:
> >
> > "OpenStack is a single product made of a lot of independent, but
> > cooperating, components."
> >
> > The log is an interesting read:
> > http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html
>
> Hmm, blast from the past. I'm sad I didn't make it to that meeting.
>
> I would (now at least) have voted for #2: OpenStack is "a collection of
> independent projects that work together for some level of integration
> and releases".
>
> This is how I believe OpenStack should be seen, as I wrote on Twitter
> relatively recently:
>
> https://twitter.com/jaypipes/status/705794815338741761
> https://twitter.com/jaypipes/status/705795095262441472

I'm honestly in the same boat as Chris. And I've constantly heard both. I also 
frankly am not sure I agree with the idea that OpenStack is one product. I 
think more along the lines of the way DefCore specifies OpenStack Compute as a 
Product, etc. I feel like if every project contributed to the OpenStack 
product, we might have a better adoption rate and a better knowledge base for 
how to make new services scale from day 1. Instead, we are definitely a loose 
collection of projects that integrate on some levels and produce what various 
people might combine to create a cloud.

I'm also not entirely that the answer remains true with the different defcore 
programs. It seems like DefCore makes us define a minimum viable OpenStack 
{Compute,Object Storage} and then you can add to that. But those two things are 
"

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ian Cordasco]

-Original Message-
From: Jay Pipes <jaypi...@gmail.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 24, 2016 at 11:35:42
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

> On 05/24/2016 06:19 AM, Thierry Carrez wrote:
> > Chris Dent wrote:
> >> [...]
> >> I don't really know. I'm firmly in the camp that OpenStack needs to
> >> be smaller and more tightly focused if a unitary thing called OpenStack
> >> expects to be any good. So I'm curious about and interested in
> >> strategies for figuring out where the boundaries are.
> >>
> >> So that, of course, leads back to the original question: Is OpenStack
> >> supposed to be a unitary.
> >
> > As a data point, since I heard that question rhetorically asked quite a
> > few times over the past year... There is an old answer to that, since a
> > vote of the PPB (the ancestor of our TC) from June, 2011 which was never
> > overruled or changed afterwards:
> >
> > "OpenStack is a single product made of a lot of independent, but
> > cooperating, components."
> >
> > The log is an interesting read:
> > http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html
> >   
>  
> Hmm, blast from the past. I'm sad I didn't make it to that meeting.
>  
> I would (now at least) have voted for #2: OpenStack is "a collection of
> independent projects that work together for some level of integration
> and releases".
>  
> This is how I believe OpenStack should be seen, as I wrote on Twitter
> relatively recently:
>  
> https://twitter.com/jaypipes/status/705794815338741761
> https://twitter.com/jaypipes/status/705795095262441472

I'm honestly in the same boat as Chris. And I've constantly heard both. I also 
frankly am not sure I agree with the idea that OpenStack is one product. I 
think more along the lines of the way DefCore specifies OpenStack Compute as a 
Product, etc. I feel like if every project contributed to the OpenStack 
product, we might have a better adoption rate and a better knowledge base for 
how to make new services scale from day 1. Instead, we are definitely a loose 
collection of projects that integrate on some levels and produce what various 
people might combine to create a cloud.

I'm also not entirely that the answer remains true with the different defcore 
programs. It seems like DefCore makes us define a minimum viable OpenStack 
{Compute,Object Storage} and then you can add to that. But those two things are 
"OpenStack" and everything else is a nice additional feature. There's nothing 
that makes Barbican or Magnum or Ceilometer a core part of OpenStack. Yet 
they're projects of varying popularity that different people choose whether or 
not to deploy. If OpenStack were a product, I'd think that not deploying 
Ceilometer would be the exception.

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Mike Perez]

On 12:24 May 24, Thierry Carrez wrote:
> Morgan Fainberg wrote:
> >[...]  If we are accepting golang, I want it to be clearly
> >documented that the expectation is it is used exclusively where there is
> >a demonstrable case (such as with swift) and not a carte blanche to use
> >it wherever-you-please.
> >
> >I want this to be a social contract looked at and enforced by the
> >community, not special permissions that are granted by the TC (I don't
> >want the TC to need to step in an approve every single use case of
> >golang, or javascript ...). It's bottlenecking back to the TC for
> >special permissions or inclusion (see reasons for the dissolution of the
> >"integrated release").
> >
> >This isn't strictly an all or nothing case, this is a "how would we
> >enforce this?" type deal. Lean on infra to enforce that only projects
> >with the golang-is-ok-here tag are allowed to use it? I don't want
> >people to write their APIs in javascript (and node.js) nor in golang. I
> >would like to see most of the work continue with python as the primary
> >language. I just think it's unreasonable to lock tools behind a gate
> >that is stronger than the social / community contract (and outlined in
> >the resolution including X language).
> 
> +1
> 
> I'd prefer if we didn't have to special-case anyone, and we could come up
> with general rules that every OpenStack project follows. Any other solution
> is an administrative nightmare and a source of tension between projects (why
> are they special and not me).

I'm in agreement that I don't want to see the TC enforcing this. In fact as
Thierry has said, lets not special case anyone.

As soon as a special case is accepted, as nortoriously happens people are going
to go in a corner and rewrite things in Go. They will be upset later for not
communicating well on their intentions upfront, and the TC or a few strongly
opinionated folks in the community are going to be made the bad people just
about every time.

Community enforcing or not, I predict this to get out of hand and it's going to
create more community divide regardless.

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Joshua Harlow]

Ok, probably should be chopped out of the oslo lib list then ;)

I'll do that, since it seems to be really not the right name for what it 
is :)


Ian Cordasco wrote:

-Original Message-
From: Joshua Harlow<harlo...@fastmail.com>
Reply: OpenStack Development Mailing List (not for usage 
questions)<openstack-dev@lists.openstack.org>
Date: May 23, 2016 at 15:23:32
To: OpenStack Development Mailing List (not for usage 
questions)<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"


Sean Dague wrote:

On 05/23/2016 03:34 PM, Gregory Haynes wrote:

On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:

On Mon, 23 May 2016, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.

Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.

Only if you are willing to pay the complexity and debt cost of having
optional backends all over the place.


We seem to do this quite well even without building tools/projects that
work outside of openstack ;)

Or are you saying that using such library/project(s) u have to accept
that there will be many optional backends that you will likely not/never
use that exist in said library/project (and thus are akin to dead weight)?


For instance, I think we're well beyond that point that Keystone being
optional should be a thing anywhere (and it is a thing in a number of
places). Keystone should be our auth system, all projects 100% depend on
it, and if you have different site needs, put that into a Keystone backend.

Most of the oslo libraries require other oslo libraries, which is fine.
They aren't trying to solve the general purpose case of logging or
configuration or db access. They are trying to solve a specific set of
patterns that are applicable to OpenStack projects.


I just took a quick stab at annotating which ones (I think are) useable
outside of openstack (without say bringing in the configuration
ideology/pattern that oslo.config adds) and made the following:

automaton (useable)
cliff (useable)
cookiecutter (useable)


I'm catching up on this thread, but cookiecutter most certainly is *NOT* an 
OpenStack project: https://pypi.io/project/cookiecutter/

It was createdy by Audrey Roy Greenfeld.


debtcollector (useable)
futurist (useable)
osprofiler (useable?)
oslo.cache
oslo.concurrency
oslo.context
oslo.config
oslo-cookiecutter
oslo.db
oslo.i18n
oslo.log
oslo.messaging
oslo.middleware
oslo.policy (useable?)
oslo.privsep (useable?)
oslo.reports
oslo.rootwrap
oslo.serialization (useable)
oslo.service
oslosphinx
oslotest (useable)
oslo.utils (useable)
oslo.versionedobjects
oslo.vmware
hacking (useable)
pbr (useable)
pyCADF (useable?)
stevedore (useable)
taskflow (useable)
tooz (useable)

So out of 33 that's about half (~17) that I think are useable outside
without to many patterns/ideologies being forced on non-openstack folks
(if your external project accepts the pattern of oslo.config then the
number increases).


-Sean



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Ian Cordasco



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ian Cordasco]

-Original Message-
From: Joshua Harlow <harlo...@fastmail.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 23, 2016 at 15:23:32
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

> Sean Dague wrote:
> > On 05/23/2016 03:34 PM, Gregory Haynes wrote:
> >> On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
> >>> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> >>>> On Mon, 23 May 2016, Doug Hellmann wrote:
> >>>>> Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> >>>>>> I don't think language does (or should) have anything to do with it.
> >>>>>>
> >>>>>> The question is whether or not the tool (whether service or
> >>>>>> dependent library) is useful to and usable outside the openstack-stack.
> >>>>>> For example gnocchi is useful to openstack but you can use it with 
> >>>>>> other
> >>>>>> stuff, therefore _not_ openstack. More controversially: swift can be
> >>>>>> usefully used all by its lonesome: _not_ openstack.
> >> Making a tool which is useful outside of the OpenStack context just
> >> seems like good software engineering - it seems odd that we would try
> >> and ensure our tools do not fit this description. Fortunately, many (or
> >> even most) of the tools we create *are* useful outside of the OpenStack
> >> world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
> >> libraries. This is really a question of defining useful interfaces more
> >> than anything else, not a statement of whether a tool is part of our
> >> community.
> >
> > Only if you are willing to pay the complexity and debt cost of having
> > optional backends all over the place.
>  
> We seem to do this quite well even without building tools/projects that
> work outside of openstack ;)
>  
> Or are you saying that using such library/project(s) u have to accept
> that there will be many optional backends that you will likely not/never
> use that exist in said library/project (and thus are akin to dead weight)?
>  
> >
> > For instance, I think we're well beyond that point that Keystone being
> > optional should be a thing anywhere (and it is a thing in a number of
> > places). Keystone should be our auth system, all projects 100% depend on
> > it, and if you have different site needs, put that into a Keystone backend.
> >
> > Most of the oslo libraries require other oslo libraries, which is fine.
> > They aren't trying to solve the general purpose case of logging or
> > configuration or db access. They are trying to solve a specific set of
> > patterns that are applicable to OpenStack projects.
>  
> I just took a quick stab at annotating which ones (I think are) useable
> outside of openstack (without say bringing in the configuration
> ideology/pattern that oslo.config adds) and made the following:
>  
> automaton (useable)
> cliff (useable)
> cookiecutter (useable)

I'm catching up on this thread, but cookiecutter most certainly is *NOT* an 
OpenStack project: https://pypi.io/project/cookiecutter/

It was createdy by Audrey Roy Greenfeld. 

> debtcollector (useable)
> futurist (useable)
> osprofiler (useable?)
> oslo.cache
> oslo.concurrency
> oslo.context
> oslo.config
> oslo-cookiecutter
> oslo.db
> oslo.i18n
> oslo.log
> oslo.messaging
> oslo.middleware
> oslo.policy (useable?)
> oslo.privsep (useable?)
> oslo.reports
> oslo.rootwrap
> oslo.serialization (useable)
> oslo.service
> oslosphinx
> oslotest (useable)
> oslo.utils (useable)
> oslo.versionedobjects
> oslo.vmware
> hacking (useable)
> pbr (useable)
> pyCADF (useable?)
> stevedore (useable)
> taskflow (useable)
> tooz (useable)
>  
> So out of 33 that's about half (~17) that I think are useable outside
> without to many patterns/ideologies being forced on non-openstack folks
> (if your external project accepts the pattern of oslo.config then the
> number increases).
>  
> >
> > -Sean
> >
>  
> __  
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>  

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Jay Pipes]

On 05/24/2016 06:19 AM, Thierry Carrez wrote:

Chris Dent wrote:

[...]
I don't really know. I'm firmly in the camp that OpenStack needs to
be smaller and more tightly focused if a unitary thing called OpenStack
expects to be any good. So I'm curious about and interested in
strategies for figuring out where the boundaries are.

So that, of course, leads back to the original question: Is OpenStack
supposed to be a unitary.


As a data point, since I heard that question rhetorically asked quite a
few times over the past year... There is an old answer to that, since a
vote of the PPB (the ancestor of our TC) from June, 2011 which was never
overruled or changed afterwards:

"OpenStack is a single product made of a lot of independent, but
cooperating, components."

The log is an interesting read:
http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html


Hmm, blast from the past. I'm sad I didn't make it to that meeting.

I would (now at least) have voted for #2: OpenStack is "a collection of 
independent projects that work together for some level of integration 
and releases".


This is how I believe OpenStack should be seen, as I wrote on Twitter 
relatively recently:


https://twitter.com/jaypipes/status/705794815338741761
https://twitter.com/jaypipes/status/705795095262441472

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Fox, Kevin M]

OpenStack is more then the sum of its various pieces. Often features need to 
cross more then one project. Cross project work is already extremely hard 
without having to change languages between. Language change should be done very 
carefully and deliberately.

Thanks,
Kevin


From: Geoff O'Callaghan
Sent: Monday, May 23, 2016 5:59:13 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

Surely openstack is defined by it’s capabilities and interfaces rather than 
it’s internals.  Given the simplistic view that openstack is a collection of 
micro services connected by well defined api’s does it really matter what code 
is used inside that micro service (or macro service )?   If there is a 
community willing to bring and support code in language ‘x’,  isn’t that better 
than worrying about the off chance of existing developers wishing to move 
projects and not knowing the target language?Is there a fear that we’ll end 
up with a fork of nova (or others) written in rust ?
If we’re not open to evolving then we’ll die.

Just throwing in a different perspective.

Sorry about the top-post but it applies in general to the below discussion
Tks
Geoff

On 24 May 2016, at 9:28 AM, Gregory Haynes 
<g...@greghaynes.net<mailto:g...@greghaynes.net>> wrote:

On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:


On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes 
<g...@greghaynes.net<mailto:g...@greghaynes.net>> wrote:
On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> John Dickinson wrote:
> > [...]
> >> So the real question we need to answer is... where does OpenStack
> >> stop, and where does the wider open source community start ? If
> >> OpenStack is purely an "integration engine", glue code for other
> >> lower-level technologies like hypervisors, databases, or distributed
> >> block storage, then the scope is limited, Python should be plenty
> >> enough, and we don't need to fragment our community. If OpenStack is
> >> "whatever it takes to reach our mission", then yes we need to add one
> >> language to cover lower-level/native optimization, because we'll
> >> need that... and we need to accept the community cost as a
> >> consequence of that scope choice. Those are the only two options on
> >> the table.
> >>
> >> I'm actually not sure what is the best answer. But I'm convinced we,
> >> as a community, need to have a clear answer to that. We've been
> >> avoiding that clear answer until now, creating tension between the
> >> advocates of an ASF-like collection of tools and the advocates of a
> >> tighter-integrated "openstack" product. We have created silos and
> >> specialized areas as we got into the business of developing time-
> >> series databases or SDNs. As a result, it's not "one community"
> >> anymore. Should we further encourage that, or should we focus on
> >> what the core of our mission is, what we have in common, this
> >> integration engine that binds all those other open source projects
> >> into one programmable infrastructure solution ?
> >
> > You said the answer in your question. OpenStack isn't defined as an
> > integration engine[3]. The definition of OpenStack is whatever it
> > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > I mean that we've already gone to the effort of defining OpenStack. It's
> > our mission statement. We're all about building a cloud platform upon
> > which people can run their apps ("cloud-native" or otherwise), so we
> > write the software needed to do that.
> >
> > So where does OpenStack stop and the wider community start? OpenStack
> > includes the projects needed to fulfill its mission.
>
> I'd totally agree with you if OpenStack was developed in a vacuum. But
> there is a large number of open source projects and libraries that
> OpenStack needs to fulfill its mission that are not in "OpenStack": they
> are external open source projects we depend on. Python, MySQL, libvirt,
> KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> be included in OpenStack, and we are not NIHing replacements for those
> in OpenStack either.
>
> So it is not as clear-cut as you present it, and you can approach this
> dependency question from two directions.
>
> One is community-centric: "anything produced by our community is
> OpenStack". If we are missing a lower-level piece to achieve our mission
> and are developing it ourselves as a result, then it is OpenStack, even
> i

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ben Meyer]

On 05/24/2016 11:13 AM, Dean Troyer wrote:
> On Tue, May 24, 2016 at 8:20 AM, Flavio Percoco  > wrote:
>
> So, just to make sure I'm making myself clear, I believe we should
> go with
> option #2 in Thierry's comment from May 23 11:3 on this[0] review.
> While I'm not
> entirely opposed to #1 I think #2 is better for us at this point
> in time. Here's
> a quote of Thierry's comment:
>
>   "To summarize my view on this, I think our only two options
> here are (1)
>   approve the addition of golang (with caveats on where it
> should be used
>   to try to minimize useless churn), or (2) precise the line
> between
>   'openstack projects' and 'dependencies of openstack
> projects' in a way
>   that makes it obvious that components requiring such
> optimization as to
>   require golang (or any other such language) should be
> developed as
>   dependencies"
>
> My main motivation is that I still believe option #1 will have a
> negative impact
> on the community and, perhaps more importantly, I don't think
> it'll help
> reaching the goal we've been talking about in this thread. Many
> people have been
> asking for focus and I think #2 will do that, whereas #1 will open
> the doors to
> a different set of problems and complexities that won't help with
> keeping the focus.
>
>
> Option #2 without the followup of actually evaluating and removing
> things that do not fit is really Option #3, do nothing. Which is what
> I am afraid will happen.  No renewed focus, no growth, no goal.
>
> On the language front, since we want focus, the exiting decisions re
> languages should also be part of that re-evaluation for focus.  It
> sure feels like JavaScript is in exactly the same boat as folks fear
> Golang will be here (a special case, domain-specific, division of
> community (ask Horizon devs)).  And Bash, well, that isn't even a
> language.

Just $0.02 - if you want to support a language, then it would seem like
having a full SDK for that language would be a first step so that people
inside and outside the community can use the language in a supported
manner. With an SDK, it seems like everyone will just reinvent the
wheel. That would also seem to further the goal of using the language as
the community intends - whether for services, clients, or UI - since the
SDK would be targeted appropriately. If no SDK, then special casing
would seem to the proper place.

Again, $0.02

Ben
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Dean Troyer]

On Tue, May 24, 2016 at 8:20 AM, Flavio Percoco  wrote:
>
> So, just to make sure I'm making myself clear, I believe we should go with
> option #2 in Thierry's comment from May 23 11:3 on this[0] review. While
> I'm not
> entirely opposed to #1 I think #2 is better for us at this point in time.
> Here's
> a quote of Thierry's comment:
>
>   "To summarize my view on this, I think our only two options here are
> (1)
>   approve the addition of golang (with caveats on where it should be
> used
>   to try to minimize useless churn), or (2) precise the line between
>   'openstack projects' and 'dependencies of openstack projects' in a
> way
>   that makes it obvious that components requiring such optimization as
> to
>   require golang (or any other such language) should be developed as
>   dependencies"
>
> My main motivation is that I still believe option #1 will have a negative
> impact
> on the community and, perhaps more importantly, I don't think it'll help
> reaching the goal we've been talking about in this thread. Many people
> have been
> asking for focus and I think #2 will do that, whereas #1 will open the
> doors to
> a different set of problems and complexities that won't help with keeping
> the focus.
>

Option #2 without the followup of actually evaluating and removing things
that do not fit is really Option #3, do nothing. Which is what I am afraid
will happen.  No renewed focus, no growth, no goal.

On the language front, since we want focus, the exiting decisions re
languages should also be part of that re-evaluation for focus.  It sure
feels like JavaScript is in exactly the same boat as folks fear Golang will
be here (a special case, domain-specific, division of community (ask
Horizon devs)).  And Bash, well, that isn't even a language.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Flavio Percoco]

On 23/05/16 21:57 +0100, Chris Dent wrote:

[snip]


So, yet another way to frame the original question (in a loaded way)
may be: Are we trying to come up with a way of defining the community
that lets us carry on doing what we've been doing, haphazardly, or
are we trying to get the process of defining the community to bring
us to a point where we have some useful constraints that allow us to
more effectively reach goal X?

Begging, of course: What's X?

(To me, an unfettered big tent is a great way to keep riding the
great OpenStack enterprise boondoggle, but I'm not sure it's
resulting in a great experience for humans who aren't on that
train.)


The question I believe we're trying to answer is the second one. It seems clear
to me that what we're doing has some problems we need to fix and coming up with
a way to explain what we're doing without fixing those problems won't help us at
all.

I've expressed several times my concerns related to an unfettered big tent. My
belief is that we should focus on the goal and accept that we might need to set
stronger rules (for lack of a better word in my vocabulary) that will help
preserving the tent.

So, just to make sure I'm making myself clear, I believe we should go with
option #2 in Thierry's comment from May 23 11:3 on this[0] review. While I'm not
entirely opposed to #1 I think #2 is better for us at this point in time. Here's
a quote of Thierry's comment:

  "To summarize my view on this, I think our only two options here are (1)
  approve the addition of golang (with caveats on where it should be used
  to try to minimize useless churn), or (2) precise the line between
  'openstack projects' and 'dependencies of openstack projects' in a way
  that makes it obvious that components requiring such optimization as to
  require golang (or any other such language) should be developed as
  dependencies"

My main motivation is that I still believe option #1 will have a negative impact
on the community and, perhaps more importantly, I don't think it'll help
reaching the goal we've been talking about in this thread. Many people have been
asking for focus and I think #2 will do that, whereas #1 will open the doors to
a different set of problems and complexities that won't help with keeping the 
focus.

This is, of course, just my opinion.

[0] https://review.openstack.org/#/c/312267/

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Clint Byrum]

Excerpts from Geoff O'Callaghan's message of 2016-05-24 15:31:28 +1000:
> 
> > On 24 May 2016, at 3:13 PM, Clint Byrum  wrote:
> > 
> > 
> [snip]
> 
> > those other needs. Grab a python developer, land some code, and your
> > feature is there.
> 
> s/python/whateverlanguage/
> 

But that's just the point. If you have lots of languages, you have to
find developers who know whichever one your feature or bug fix needs to
be written in.

> > 
> >> I also never said, ship the source code and say ‘good luck’.   What I did 
> >> imply  was, due to a relaxing of coding platform requirements we might be 
> >> able to deliver a function at this performance point that  we may not have 
> >> been able to do otherwise.   We should always provide support and the 
> >> code,  but as to what language it’s written it i’m personally not fussed 
> >> and I have to deal with a variety of languages already so maybe that’s why 
> >> I don’t see it as a big problem.
> > 
> > This again assumes that one only buys software and does not ever
> > participate in its development in an ongoing basis. There's nothing
> > wrong with that, but this particular community is highly focused on
> > people who do want to participate and think that the ability to
> > adapt this cloud they've invested in to their changing business needs is
> > more important than any one feature.
> 
> No I didn’t say that at all and I don’t believe it’s assumed.I just said 
> I wasn’t fussed about what language it’s written in and just wanted 
> developers to be able to contribute if they had something to contribute.   
> 

Not being fussed about the language means not being fussed about who can
develop on it, so I took that to mean not being interested in developing
on it. I'm not sure either of us was "wrong" here, but I apologize for
assuming that's what you meant, if that's indeed not what you meant.

> > 
> >> 
> >> I understand there will be integration challenges and I agree with 
> >> cohesiveness being a good thing, but I also believe we must deliver value 
> >> more than cohesiveness.   The value is what operators want,  the 
> >> cohesiveness is what the developers may or may not want.
> >> 
> > 
> > We agree that delivering value to end users and operators is the #1
> > priority. I think we just disagree on the value of an open development
> > model and cohesion in the _community_.
> 
> It’s not open if you restrict developers based on programming language.
> Trust me I get cohesion and it’s value, we’ve reached the stage now where 
> cohesion is being questioned.  The questioning is a good thing and it is a 
> measure of the health of the community.

So, there's a funny principle here, where the word open is _so open_
that one can use it to classify any number of aspects, while ignoring
others, and still be correct.

I qualified the development model with the word open, because the way
we govern it, the way code and change move through the system, are 100%
transparent and available to anyone who wants to participate. But I agree,
it is less available to those who want to participate using languages
we've chosen to avoid. They have to begin at the governance level,
which they have, in fact, done by approaching the TC. But they may be
shout out, and that would make developing on OpenStack closed to them.

However, I don't think the TC takes this lightly, and they understand
that having it open to _more_ contribution is the goal. What I think may
not make sense to all parties, is that closing it to some, will keep it
open to many others. And what I think Thierry did by opening this thread
was try to figure out how many stand on either side of that decision.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Duncan Thomas]

On 24 May 2016 at 02:28, Gregory Haynes  wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
> I really do not want to "special case" swift. It really doesn't go with
> the spirit of inclusion.
>
>
> I am not sure how inclusion is related to special casing. Inclusion here
> implies that some group is not being accepted in to our community. The
> excluded group here would be one writing software not in python, which is
> the same groups being excluded currently. This is an argument against the
> status quo, not against special casing.
>
>

It excludes any other project that might be possible under the more relaxed
rules. It says 'swift is a special snowflake that the rules don't apply to,
but you are not special enough, go away', which is the very definition of
exclusionary.


> I expect we will see a continued drive for whichever additional languages
> are supported once it's in place/allowed.
>
>
> That is the problem. The social and debugging costs of adding another
> language are a function of how much that other language is used. If one
> component of one project is written in another language then these costs
> should be fairly low. I agree that once we allow another language we should
> expect many projects to begin using it, and IMO most if not all of these
> cases except swift will not be warranted.
>

So designate have come up with a use-case detailed in this thread. Gnocchi
have suggested they might have one. Others quite possibly exist that
haven't even been explored yet because the rules were against it.

One alternative to ffor swift to split into two layer, a control plane and
a data plane, and for alternative dataplane implementations (i.e. the go
one, maybe something ceph based, etc) to sit outside of the openstack
umbrella. This is the model nearly every other openstack service has.


-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Davanum Srinivas]

+1 ttx. ("I'd prefer if we didn't have to special-case anyone")

-- Dims

On Tue, May 24, 2016 at 6:24 AM, Thierry Carrez  wrote:
> Morgan Fainberg wrote:
>>
>> [...]  If we are accepting golang, I want it to be clearly
>> documented that the expectation is it is used exclusively where there is
>> a demonstrable case (such as with swift) and not a carte blanche to use
>> it wherever-you-please.
>>
>> I want this to be a social contract looked at and enforced by the
>> community, not special permissions that are granted by the TC (I don't
>> want the TC to need to step in an approve every single use case of
>> golang, or javascript ...). It's bottlenecking back to the TC for
>> special permissions or inclusion (see reasons for the dissolution of the
>> "integrated release").
>>
>> This isn't strictly an all or nothing case, this is a "how would we
>> enforce this?" type deal. Lean on infra to enforce that only projects
>> with the golang-is-ok-here tag are allowed to use it? I don't want
>> people to write their APIs in javascript (and node.js) nor in golang. I
>> would like to see most of the work continue with python as the primary
>> language. I just think it's unreasonable to lock tools behind a gate
>> that is stronger than the social / community contract (and outlined in
>> the resolution including X language).
>
>
> +1
>
> I'd prefer if we didn't have to special-case anyone, and we could come up
> with general rules that every OpenStack project follows. Any other solution
> is an administrative nightmare and a source of tension between projects (why
> are they special and not me).
>
> --
> Thierry Carrez (ttx)
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Chris Dent]

On Tue, 24 May 2016, Thierry Carrez wrote:

Chris Dent wrote:

So that, of course, leads back to the original question: Is OpenStack
supposed to be a unitary.


As a data point, since I heard that question rhetorically asked quite a few 
times over the past year... There is an old answer to that, since a vote of 
the PPB (the ancestor of our TC) from June, 2011 which was never overruled or 
changed afterwards:


"OpenStack is a single product made of a lot of independent, but cooperating, 
components."


The log is an interesting read: 
http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html


That is an interesting read, thank you for pointing that out. It all
feel very familiar.

As someone who has only entered the community in the last two years, and
when I did enter I was completely uninitiated (in the sense that I had
never really even heard of OpenStack so didn't have much in the way of
preconceived notions), I can say that the meaning and message behind
that quote was _not_ drilled into me. If we want it to be true and
active part of our culture, it should have been.

I'm not certain I have a strong preference for how things should be
either way (reading the log I felt a lot of empathy for notmyname's
position and thought "we'd be a much different (maybe better, unclear)
thing now if we'd headed that way") but I can say unequivocally that
it would make decision processes, now, much better if we could commit
to one way.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Thierry Carrez]

Morgan Fainberg wrote:

[...]  If we are accepting golang, I want it to be clearly
documented that the expectation is it is used exclusively where there is
a demonstrable case (such as with swift) and not a carte blanche to use
it wherever-you-please.

I want this to be a social contract looked at and enforced by the
community, not special permissions that are granted by the TC (I don't
want the TC to need to step in an approve every single use case of
golang, or javascript ...). It's bottlenecking back to the TC for
special permissions or inclusion (see reasons for the dissolution of the
"integrated release").

This isn't strictly an all or nothing case, this is a "how would we
enforce this?" type deal. Lean on infra to enforce that only projects
with the golang-is-ok-here tag are allowed to use it? I don't want
people to write their APIs in javascript (and node.js) nor in golang. I
would like to see most of the work continue with python as the primary
language. I just think it's unreasonable to lock tools behind a gate
that is stronger than the social / community contract (and outlined in
the resolution including X language).


+1

I'd prefer if we didn't have to special-case anyone, and we could come 
up with general rules that every OpenStack project follows. Any other 
solution is an administrative nightmare and a source of tension between 
projects (why are they special and not me).


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Thierry Carrez]

Chris Dent wrote:

[...]
I don't really know. I'm firmly in the camp that OpenStack needs to
be smaller and more tightly focused if a unitary thing called OpenStack
expects to be any good. So I'm curious about and interested in
strategies for figuring out where the boundaries are.

So that, of course, leads back to the original question: Is OpenStack
supposed to be a unitary.


As a data point, since I heard that question rhetorically asked quite a 
few times over the past year... There is an old answer to that, since a 
vote of the PPB (the ancestor of our TC) from June, 2011 which was never 
overruled or changed afterwards:


"OpenStack is a single product made of a lot of independent, but 
cooperating, components."


The log is an interesting read: 
http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-06-28-20.06.log.html


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Geoff O'Callaghan]

> On 24 May 2016, at 3:13 PM, Clint Byrum  wrote:
> 
> 
[snip]

> those other needs. Grab a python developer, land some code, and your
> feature is there.

s/python/whateverlanguage/

> 
>> I also never said, ship the source code and say ‘good luck’.   What I did 
>> imply  was, due to a relaxing of coding platform requirements we might be 
>> able to deliver a function at this performance point that  we may not have 
>> been able to do otherwise.   We should always provide support and the code,  
>> but as to what language it’s written it i’m personally not fussed and I have 
>> to deal with a variety of languages already so maybe that’s why I don’t see 
>> it as a big problem.
> 
> This again assumes that one only buys software and does not ever
> participate in its development in an ongoing basis. There's nothing
> wrong with that, but this particular community is highly focused on
> people who do want to participate and think that the ability to
> adapt this cloud they've invested in to their changing business needs is
> more important than any one feature.

No I didn’t say that at all and I don’t believe it’s assumed.I just said I 
wasn’t fussed about what language it’s written in and just wanted developers to 
be able to contribute if they had something to contribute.   

> 
>> 
>> I understand there will be integration challenges and I agree with 
>> cohesiveness being a good thing, but I also believe we must deliver value 
>> more than cohesiveness.   The value is what operators want,  the 
>> cohesiveness is what the developers may or may not want.
>> 
> 
> We agree that delivering value to end users and operators is the #1
> priority. I think we just disagree on the value of an open development
> model and cohesion in the _community_.

It’s not open if you restrict developers based on programming language.
Trust me I get cohesion and it’s value, we’ve reached the stage now where 
cohesion is being questioned.  The questioning is a good thing and it is a 
measure of the health of the community.

> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Clint Byrum]

Excerpts from Geoff O'Callaghan's message of 2016-05-24 14:34:46 +1000:
> 
> > On 24 May 2016, at 2:04 PM, Clint Byrum  wrote:
> > 
> > Excerpts from Geoff O'Callaghan's message of 2016-05-24 10:59:13 +1000:
> >> Surely openstack is defined by it’s capabilities and interfaces rather 
> >> than it’s internals.  Given the simplistic view that openstack is a 
> >> collection of micro services connected by well defined api’s does it 
> >> really matter what code is used inside that micro service (or macro 
> >> service )?   If there is a community willing to bring and support code in 
> >> language ‘x’,  isn’t that better than worrying about the off chance of 
> >> existing developers wishing to move projects and not knowing the target 
> >> language?Is there a fear that we’ll end up with a fork of nova (or 
> >> others) written in rust ?
> >> If we’re not open to evolving then we’ll die.
> >> 
> >> Just throwing in a different perspective.
> > 
> > Thanks Geoff. Your perspective is one that has been considered many
> > times. That is an engineering perspective though, and ignores the people
> > and businesses that are the users of OpenStack. We don't just shove the
> > code out the door and say "good luck!”.
> 
> Hey Clint,  That is exactly what I wasn’t saying.   Businesses and people out 
> there want the platform to have the features they want and work.  They could 
> care less about what it’s written in.   You tend to care when it doesn’t work 
> and / or it doesn’t have the features you want.   So I can understand for 
> operators now they have a vested interest in making sure they can debug what 
> is given to them as we don’t meet Geoff’s rule # 1 - the code must work and 
> it must do what I want.
> 

I completely respect that you may be in a situation where you can
actually define all of the things you want for your IT department today.

My experience is that those things change constantly, and there is no
product that will ever add all of the features that everyone needs. But
if one can hit 80%, and 100% of the startup features, then being open
source, there's no worry that some vendor will simply refuse to respect
those other needs. Grab a python developer, land some code, and your
feature is there.

> I also never said, ship the source code and say ‘good luck’.   What I did 
> imply  was, due to a relaxing of coding platform requirements we might be 
> able to deliver a function at this performance point that  we may not have 
> been able to do otherwise.   We should always provide support and the code,  
> but as to what language it’s written it i’m personally not fussed and I have 
> to deal with a variety of languages already so maybe that’s why I don’t see 
> it as a big problem.

This again assumes that one only buys software and does not ever
participate in its development in an ongoing basis. There's nothing
wrong with that, but this particular community is highly focused on
people who do want to participate and think that the ability to
adapt this cloud they've invested in to their changing business needs is
more important than any one feature.

> 
> I understand there will be integration challenges and I agree with 
> cohesiveness being a good thing, but I also believe we must deliver value 
> more than cohesiveness.   The value is what operators want,  the cohesiveness 
> is what the developers may or may not want.
> 

We agree that delivering value to end users and operators is the #1
priority. I think we just disagree on the value of an open development
model and cohesion in the _community_.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Geoff O'Callaghan]

> On 24 May 2016, at 2:04 PM, Clint Byrum  wrote:
> 
> Excerpts from Geoff O'Callaghan's message of 2016-05-24 10:59:13 +1000:
>> Surely openstack is defined by it’s capabilities and interfaces rather than 
>> it’s internals.  Given the simplistic view that openstack is a collection of 
>> micro services connected by well defined api’s does it really matter what 
>> code is used inside that micro service (or macro service )?   If there is a 
>> community willing to bring and support code in language ‘x’,  isn’t that 
>> better than worrying about the off chance of existing developers wishing to 
>> move projects and not knowing the target language?Is there a fear that 
>> we’ll end up with a fork of nova (or others) written in rust ?
>> If we’re not open to evolving then we’ll die.
>> 
>> Just throwing in a different perspective.
> 
> Thanks Geoff. Your perspective is one that has been considered many
> times. That is an engineering perspective though, and ignores the people
> and businesses that are the users of OpenStack. We don't just shove the
> code out the door and say "good luck!”.

Hey Clint,  That is exactly what I wasn’t saying.   Businesses and people out 
there want the platform to have the features they want and work.  They could 
care less about what it’s written in.   You tend to care when it doesn’t work 
and / or it doesn’t have the features you want.   So I can understand for 
operators now they have a vested interest in making sure they can debug what is 
given to them as we don’t meet Geoff’s rule # 1 - the code must work and it 
must do what I want.

I also never said, ship the source code and say ‘good luck’.   What I did imply 
 was, due to a relaxing of coding platform requirements we might be able to 
deliver a function at this performance point that  we may not have been able to 
do otherwise.   We should always provide support and the code,  but as to what 
language it’s written it i’m personally not fussed and I have to deal with a 
variety of languages already so maybe that’s why I don’t see it as a big 
problem.

I understand there will be integration challenges and I agree with cohesiveness 
being a good thing, but I also believe we must deliver value more than 
cohesiveness.   The value is what operators want,  the cohesiveness is what the 
developers may or may not want.

> 
> So, each new aspect of the internals that the people and business must
> evaluate is another hurdle to adoption at some level. If OpenStack is
> the fastest open source cloud platform ever, but only gets adopted by
> the 5 largest cloud users on the planet, and smaller organizations are
> forced to invest their money in closed source appliance based clouds,
> then we'll all suffer, as neither of those options will stand a chance
> against the large scale efforts of the established players like Amazon,
> Google, and Microsoft.
> 
> Likewise, if OpenStack is adopted by the bulk of small/medium businesses
> needing clouds, but nobody can run it at scale, we will also be crushed
> as users develop elastic workloads that need to be moved onto a large
> scale cloud.
> 
> And if there are simply two, one for big clouds, and one for small,
> they'll never be fully compatible. It will be like POSIX... moderately
> useful for a limited set of applications, but most things will have to
> be optimized for each different implementation, wasting everyone's time
> porting when they should be developing.
> 
> So, while this doesn't mean we should just force everything onto Python,
> it does mean that we should remain as cohesive as we can when making
> choices like this. So the question "What is OpenStack" needs to be
> asked, so we can evaluate what should be kept close together, and what
> might be better off as an independent component.

I thought that had already been asked and answered.  The question becomes :- is 
what openstack is written in a core part of openstack?

Thanks for the reply.
Geoff


> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Clint Byrum]

Excerpts from Geoff O'Callaghan's message of 2016-05-24 10:59:13 +1000:
> Surely openstack is defined by it’s capabilities and interfaces rather than 
> it’s internals.  Given the simplistic view that openstack is a collection of 
> micro services connected by well defined api’s does it really matter what 
> code is used inside that micro service (or macro service )?   If there is a 
> community willing to bring and support code in language ‘x’,  isn’t that 
> better than worrying about the off chance of existing developers wishing to 
> move projects and not knowing the target language?Is there a fear that 
> we’ll end up with a fork of nova (or others) written in rust ?
> If we’re not open to evolving then we’ll die.
> 
> Just throwing in a different perspective.

Thanks Geoff. Your perspective is one that has been considered many
times. That is an engineering perspective though, and ignores the people
and businesses that are the users of OpenStack. We don't just shove the
code out the door and say "good luck!".

So, each new aspect of the internals that the people and business must
evaluate is another hurdle to adoption at some level. If OpenStack is
the fastest open source cloud platform ever, but only gets adopted by
the 5 largest cloud users on the planet, and smaller organizations are
forced to invest their money in closed source appliance based clouds,
then we'll all suffer, as neither of those options will stand a chance
against the large scale efforts of the established players like Amazon,
Google, and Microsoft.

Likewise, if OpenStack is adopted by the bulk of small/medium businesses
needing clouds, but nobody can run it at scale, we will also be crushed
as users develop elastic workloads that need to be moved onto a large
scale cloud.

And if there are simply two, one for big clouds, and one for small,
they'll never be fully compatible. It will be like POSIX... moderately
useful for a limited set of applications, but most things will have to
be optimized for each different implementation, wasting everyone's time
porting when they should be developing.

So, while this doesn't mean we should just force everything onto Python,
it does mean that we should remain as cohesive as we can when making
choices like this. So the question "What is OpenStack" needs to be
asked, so we can evaluate what should be kept close together, and what
might be better off as an independent component.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Morgan Fainberg]

On Mon, May 23, 2016 at 4:28 PM, Gregory Haynes  wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes 
> wrote:
>
> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Geoff O'Callaghan]

Surely openstack is defined by it’s capabilities and interfaces rather than 
it’s internals.  Given the simplistic view that openstack is a collection of 
micro services connected by well defined api’s does it really matter what code 
is used inside that micro service (or macro service )?   If there is a 
community willing to bring and support code in language ‘x’,  isn’t that better 
than worrying about the off chance of existing developers wishing to move 
projects and not knowing the target language?Is there a fear that we’ll end 
up with a fork of nova (or others) written in rust ?
If we’re not open to evolving then we’ll die.

Just throwing in a different perspective.

Sorry about the top-post but it applies in general to the below discussion
Tks
Geoff

> On 24 May 2016, at 9:28 AM, Gregory Haynes  wrote:
> 
> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>>  
>>  
>> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes > > wrote:
>> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
>> > John Dickinson wrote:
>> > > [...]
>> > >> So the real question we need to answer is... where does OpenStack
>> > >> stop, and where does the wider open source community start ? If
>> > >> OpenStack is purely an "integration engine", glue code for other
>> > >> lower-level technologies like hypervisors, databases, or distributed
>> > >> block storage, then the scope is limited, Python should be plenty
>> > >> enough, and we don't need to fragment our community. If OpenStack is
>> > >> "whatever it takes to reach our mission", then yes we need to add one
>> > >> language to cover lower-level/native optimization, because we'll
>> > >> need that... and we need to accept the community cost as a
>> > >> consequence of that scope choice. Those are the only two options on
>> > >> the table.
>> > >>
>> > >> I'm actually not sure what is the best answer. But I'm convinced we,
>> > >> as a community, need to have a clear answer to that. We've been
>> > >> avoiding that clear answer until now, creating tension between the
>> > >> advocates of an ASF-like collection of tools and the advocates of a
>> > >> tighter-integrated "openstack" product. We have created silos and
>> > >> specialized areas as we got into the business of developing time-
>> > >> series databases or SDNs. As a result, it's not "one community"
>> > >> anymore. Should we further encourage that, or should we focus on
>> > >> what the core of our mission is, what we have in common, this
>> > >> integration engine that binds all those other open source projects
>> > >> into one programmable infrastructure solution ?
>> > >
>> > > You said the answer in your question. OpenStack isn't defined as an
>> > > integration engine[3]. The definition of OpenStack is whatever it
>> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
>> > > I mean that we've already gone to the effort of defining OpenStack. It's
>> > > our mission statement. We're all about building a cloud platform upon
>> > > which people can run their apps ("cloud-native" or otherwise), so we
>> > > write the software needed to do that.
>> > >
>> > > So where does OpenStack stop and the wider community start? OpenStack
>> > > includes the projects needed to fulfill its mission.
>> >
>> > I'd totally agree with you if OpenStack was developed in a vacuum. But
>> > there is a large number of open source projects and libraries that
>> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
>> > are external open source projects we depend on. Python, MySQL, libvirt,
>> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
>> > be included in OpenStack, and we are not NIHing replacements for those
>> > in OpenStack either.
>> >
>> > So it is not as clear-cut as you present it, and you can approach this
>> > dependency question from two directions.
>> >
>> > One is community-centric: "anything produced by our community is
>> > OpenStack". If we are missing a lower-level piece to achieve our mission
>> > and are developing it ourselves as a result, then it is OpenStack, even
>> > if it ends up being a message queue or a database.
>> >
>> > The other approach is product-centric: "lower-level pieces are OpenStack
>> > dependencies, rather than OpenStack itself". If we are missing a
>> > lower-level piece to achieve our mission and are developing it as a
>> > result, it could be developed on OpenStack infrastructure by members of
>> > the OpenStack community but it is not "OpenStack the product", it's an
>> > OpenStack *dependency*. It is not governed by the TC, it can use any
>> > language and tool deemed necessary.
>> >
>> > On this second approach, there is the obvious question of where
>> > "lower-level" starts, which as you explained above is not really
>> > clear-cut. A good litmus test for it could be whenever Python is not
>> > enough. If you can't develop it 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Erno Kuvaja]

On Tue, May 24, 2016 at 12:28 AM, Gregory Haynes 
wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes 
> wrote:
>
> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Gregory Haynes]

On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes
>  wrote:
>> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
>> > John Dickinson wrote:
>> > > [...]
>> > >> So the real question we need to answer is... where does
>> > >> OpenStack
>> > >> stop, and where does the wider open source community start ? If
>> > >> OpenStack is purely an "integration engine", glue code for other
>> > >> lower-level technologies like hypervisors, databases, or
>> > >> distributed
>> > >> block storage, then the scope is limited, Python should be
>> > >> plenty
>> > >> enough, and we don't need to fragment our community. If
>> > >> OpenStack is
>> > >> "whatever it takes to reach our mission", then yes we need to
>> > >> add one
>> > >> language to cover lower-level/native optimization, because we'll
>> > >> need that... and we need to accept the community cost as a
>> > >> consequence of that scope choice. Those are the only two options
>> > >> on
>> > >> the table.
>> > >>
>> > >> I'm actually not sure what is the best answer. But I'm convinced
>> > >> we,
>> > >> as a community, need to have a clear answer to that. We've been
>> > >> avoiding that clear answer until now, creating tension between
>> > >> the
>> > >> advocates of an ASF-like collection of tools and the advocates
>> > >> of a
>> > >> tighter-integrated "openstack" product. We have created silos
>> > >> and
>> > >> specialized areas as we got into the business of developing time-
>> > >> series databases or SDNs. As a result, it's not "one community"
>> > >> anymore. Should we further encourage that, or should we focus on
>> > >> what the core of our mission is, what we have in common, this
>> > >> integration engine that binds all those other open source
>> > >> projects
>> > >> into one programmable infrastructure solution ?
>> > >
>> > > You said the answer in your question. OpenStack isn't defined
>> > > as an
>> > > integration engine[3]. The definition of OpenStack is whatever it
>> > > takes to fulfill our mission[4][5]. I don't mean that as a
>> > > tautology.
>> > > I mean that we've already gone to the effort of defining
>> > > OpenStack. It's
>> > > our mission statement. We're all about building a cloud platform
>> > > upon
>> > > which people can run their apps ("cloud-native" or otherwise),
>> > > so we
>> > > write the software needed to do that.
>> > >
>> > > So where does OpenStack stop and the wider community start?
>> > > OpenStack
>> > > includes the projects needed to fulfill its mission.
>> >
>> > I'd totally agree with you if OpenStack was developed in a
>> > vacuum. But
>> > there is a large number of open source projects and libraries that
>> > OpenStack needs to fulfill its mission that are not in
>> > "OpenStack": they
>> > are external open source projects we depend on. Python, MySQL,
>> > libvirt,
>> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those
>> > should
>> > be included in OpenStack, and we are not NIHing replacements for
>> > those
>> > in OpenStack either.
>> >
>> > So it is not as clear-cut as you present it, and you can
>> > approach this
>> > dependency question from two directions.
>> >
>> > One is community-centric: "anything produced by our community is
>> > OpenStack". If we are missing a lower-level piece to achieve our
>> > mission
>> > and are developing it ourselves as a result, then it is
>> > OpenStack, even
>> > if it ends up being a message queue or a database.
>> >
>> > The other approach is product-centric: "lower-level pieces are
>> > OpenStack
>> > dependencies, rather than OpenStack itself". If we are missing a
>> > lower-level piece to achieve our mission and are developing it as a
>> > result, it could be developed on OpenStack infrastructure by
>> > members of
>> > the OpenStack community but it is not "OpenStack the product",
>> > it's an
>> > OpenStack *dependency*. It is not governed by the TC, it can
>> > use any
>> > language and tool deemed necessary.
>> >
>> > On this second approach, there is the obvious question of where
>> > "lower-level" starts, which as you explained above is not really
>> > clear-cut. A good litmus test for it could be whenever Python
>> > is not
>> > enough. If you can't develop it effectively with the language
>> > that is
>> > currently sufficient for the rest of OpenStack, then developing it
>> > as an
>> > OpenStack dependency in whatever language is appropriate might
>> > be the
>> > solution...
>> >
>> > That is what I mean by 'scope': where does "OpenStack" stop, and
>> > where
>> > do "OpenStack dependencies" start ? It is a lot easier and a
>> > lot less
>> > community-costly to allow additional languages in OpenStack
>> > dependencies
>> > (we already have plenty there).
>> >
>>
>> I strongly agree with both of the points about what OpenStack is
>> defined
>> as. We are  a set of projects attempting to fulfill our mission. In
>> doing so, we try to use 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Morgan Fainberg]

On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes  wrote:

> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an outside project doesn't exist. That is
> when things become less clear-cut and we might need to develop software
> not purely/directly related to 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Gregory Haynes]

On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> John Dickinson wrote:
> > [...]
> >> So the real question we need to answer is... where does OpenStack
> >> stop, and where does the wider open source community start ? If
> >> OpenStack is purely an "integration engine", glue code for other
> >> lower-level technologies like hypervisors, databases, or distributed
> >> block storage, then the scope is limited, Python should be plenty
> >> enough, and we don't need to fragment our community. If OpenStack is
> >> "whatever it takes to reach our mission", then yes we need to add one
> >> language to cover lower-level/native optimization, because we'll
> >> need that... and we need to accept the community cost as a
> >> consequence of that scope choice. Those are the only two options on
> >> the table.
> >>
> >> I'm actually not sure what is the best answer. But I'm convinced we,
> >> as a community, need to have a clear answer to that. We've been
> >> avoiding that clear answer until now, creating tension between the
> >> advocates of an ASF-like collection of tools and the advocates of a
> >> tighter-integrated "openstack" product. We have created silos and
> >> specialized areas as we got into the business of developing time-
> >> series databases or SDNs. As a result, it's not "one community"
> >> anymore. Should we further encourage that, or should we focus on
> >> what the core of our mission is, what we have in common, this
> >> integration engine that binds all those other open source projects
> >> into one programmable infrastructure solution ?
> >
> > You said the answer in your question. OpenStack isn't defined as an
> > integration engine[3]. The definition of OpenStack is whatever it
> > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > I mean that we've already gone to the effort of defining OpenStack. It's
> > our mission statement. We're all about building a cloud platform upon
> > which people can run their apps ("cloud-native" or otherwise), so we
> > write the software needed to do that.
> >
> > So where does OpenStack stop and the wider community start? OpenStack
> > includes the projects needed to fulfill its mission.
> 
> I'd totally agree with you if OpenStack was developed in a vacuum. But 
> there is a large number of open source projects and libraries that 
> OpenStack needs to fulfill its mission that are not in "OpenStack": they 
> are external open source projects we depend on. Python, MySQL, libvirt, 
> KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should 
> be included in OpenStack, and we are not NIHing replacements for those 
> in OpenStack either.
> 
> So it is not as clear-cut as you present it, and you can approach this 
> dependency question from two directions.
> 
> One is community-centric: "anything produced by our community is 
> OpenStack". If we are missing a lower-level piece to achieve our mission 
> and are developing it ourselves as a result, then it is OpenStack, even 
> if it ends up being a message queue or a database.
> 
> The other approach is product-centric: "lower-level pieces are OpenStack 
> dependencies, rather than OpenStack itself". If we are missing a 
> lower-level piece to achieve our mission and are developing it as a 
> result, it could be developed on OpenStack infrastructure by members of 
> the OpenStack community but it is not "OpenStack the product", it's an 
> OpenStack *dependency*. It is not governed by the TC, it can use any 
> language and tool deemed necessary.
> 
> On this second approach, there is the obvious question of where 
> "lower-level" starts, which as you explained above is not really 
> clear-cut. A good litmus test for it could be whenever Python is not 
> enough. If you can't develop it effectively with the language that is 
> currently sufficient for the rest of OpenStack, then developing it as an 
> OpenStack dependency in whatever language is appropriate might be the 
> solution...
> 
> That is what I mean by 'scope': where does "OpenStack" stop, and where 
> do "OpenStack dependencies" start ? It is a lot easier and a lot less 
> community-costly to allow additional languages in OpenStack dependencies 
> (we already have plenty there).
> 

I strongly agree with both of the points about what OpenStack is defined
as. We are  a set of projects attempting to fulfill our mission. In
doing so, we try to use outside dependencies to help us as much as
possible. Sometimes we cannot find an outside dependency to satisfy a
need whether due to optimization needs, licensing issues, usability
problems, or simply because an outside project doesn't exist. That is
when things become less clear-cut and we might need to develop software
not purely/directly related to fulfilling our mission.

In the product-centric approach I worry that we are going to be paying
many of the costs which the existing TC resolutions hoped to prevent. We
still will have to maintain and debug these 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Gregory Haynes]

On Mon, May 23, 2016, at 02:57 PM, Sean Dague wrote:
> On 05/23/2016 03:34 PM, Gregory Haynes wrote:
> > 
> > On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
> >> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> >>> On Mon, 23 May 2016, Doug Hellmann wrote:
>  Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> > I don't think language does (or should) have anything to do with it.
> >
> > The question is whether or not the tool (whether service or
> > dependent library) is useful to and usable outside the openstack-stack.
> > For example gnocchi is useful to openstack but you can use it with other
> > stuff, therefore _not_ openstack. More controversially: swift can be
> > usefully used all by its lonesome: _not_ openstack.
> 
> > 
> > Making a tool which is useful outside of the OpenStack context just
> > seems like good software engineering - it seems odd that we would try
> > and ensure our tools do not fit this description. Fortunately, many (or
> > even most) of the tools we create *are* useful outside of the OpenStack
> > world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
> > libraries. This is really a question of defining useful interfaces more
> > than anything else, not a statement of whether a tool is part of our
> > community.
> 
> Only if you are willing to pay the complexity and debt cost of having
> optional backends all over the place.
> 
> For instance, I think we're well beyond that point that Keystone being
> optional should be a thing anywhere (and it is a thing in a number of
> places). Keystone should be our auth system, all projects 100% depend on
> it, and if you have different site needs, put that into a Keystone
> backend.
> 

Services and Projects seem to be getting conflated here. IIUC Your two
points apply only to services - we certainly aren't paying any
complexity costs for making pbr optional and the same could be said for
many of our tools.

I don't have a ton of context for why some services are electing to pay
the cost of making Keystone optional. The point I was hoping to make is
that there is value in defining an interface which is useful outside of
OpenStack, and this is a very common pattern with many of our tools. I
completely agree that there are additional costs to doing so at times,
and obviously they have to be weighed against the benefits. That is
really a problem-specific issue, though.

> Most of the oslo libraries require other oslo libraries, which is fine.
> They aren't trying to solve the general purpose case of logging or
> configuration or db access. They are trying to solve a specific set of
> patterns that are applicable to OpenStack projects.
> 

This is true up to a point - there isn't any inherent value in
overfitting a problem to be OpenStack specific. To beat on the pbr
hammer some more - we created a tool that fulfills our needs and making
it in a way where others can use it didn't cost us anything. This isn't
always the case but sometimes it is, and there is absolutely value in
making a tool which others can use.

>   -Sean
> 
> -- 
> Sean Dague
> http://dague.net

Cheers,
Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Chris Dent]

On Mon, 23 May 2016, Gregory Haynes wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.


Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.


Defining what the community is is what we're trying to do here, yes?

Based on conversations I've had without people _outside_ the
community, those people think that OpenStack is a community building
a thing called OpenStack but they are universally confused about
what OpenStack _is_.

Despite the great usefulness of those things I'm pretty sure the
OpenStack those people are looking for is neither pbr, not git-review
nor diskimage-builder. Those tools are critical to the development
of OpenStack, our attention to them is critical. But now much to
they impact the definition of OpenStack?

So, yet another way to frame the original question (in a loaded way)
may be: Are we trying to come up with a way of defining the community
that lets us carry on doing what we've been doing, haphazardly, or
are we trying to get the process of defining the community to bring
us to a point where we have some useful constraints that allow us to
more effectively reach goal X?

Begging, of course: What's X?

(To me, an unfettered big tent is a great way to keep riding the
great OpenStack enterprise boondoggle, but I'm not sure it's
resulting in a great experience for humans who aren't on that
train.)

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Joshua Harlow]

Sean Dague wrote:

On 05/23/2016 03:34 PM, Gregory Haynes wrote:

On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:

On Mon, 23 May 2016, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.

Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.


Only if you are willing to pay the complexity and debt cost of having
optional backends all over the place.


We seem to do this quite well even without building tools/projects that 
work outside of openstack ;)


Or are you saying that using such library/project(s) u have to accept 
that there will be many optional backends that you will likely not/never 
use that exist in said library/project (and thus are akin to dead weight)?




For instance, I think we're well beyond that point that Keystone being
optional should be a thing anywhere (and it is a thing in a number of
places). Keystone should be our auth system, all projects 100% depend on
it, and if you have different site needs, put that into a Keystone backend.

Most of the oslo libraries require other oslo libraries, which is fine.
They aren't trying to solve the general purpose case of logging or
configuration or db access. They are trying to solve a specific set of
patterns that are applicable to OpenStack projects.


I just took a quick stab at annotating which ones (I think are) useable 
outside of openstack (without say bringing in the configuration 
ideology/pattern that oslo.config adds) and made the following:


automaton (useable)
cliff (useable)
cookiecutter (useable)
debtcollector (useable)
futurist (useable)
osprofiler (useable?)
oslo.cache
oslo.concurrency
oslo.context
oslo.config
oslo-cookiecutter
oslo.db
oslo.i18n
oslo.log
oslo.messaging
oslo.middleware
oslo.policy (useable?)
oslo.privsep (useable?)
oslo.reports
oslo.rootwrap
oslo.serialization (useable)
oslo.service
oslosphinx
oslotest (useable)
oslo.utils (useable)
oslo.versionedobjects
oslo.vmware
hacking (useable)
pbr (useable)
pyCADF (useable?)
stevedore (useable)
taskflow (useable)
tooz (useable)

So out of 33 that's about half (~17) that I think are useable outside 
without to many patterns/ideologies being forced on non-openstack folks 
(if your external project accepts the pattern of oslo.config then the 
number increases).




-Sean



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Sean Dague]

On 05/23/2016 03:34 PM, Gregory Haynes wrote:
> 
> On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
>> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
>>> On Mon, 23 May 2016, Doug Hellmann wrote:
 Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> I don't think language does (or should) have anything to do with it.
>
> The question is whether or not the tool (whether service or
> dependent library) is useful to and usable outside the openstack-stack.
> For example gnocchi is useful to openstack but you can use it with other
> stuff, therefore _not_ openstack. More controversially: swift can be
> usefully used all by its lonesome: _not_ openstack.

> 
> Making a tool which is useful outside of the OpenStack context just
> seems like good software engineering - it seems odd that we would try
> and ensure our tools do not fit this description. Fortunately, many (or
> even most) of the tools we create *are* useful outside of the OpenStack
> world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
> libraries. This is really a question of defining useful interfaces more
> than anything else, not a statement of whether a tool is part of our
> community.

Only if you are willing to pay the complexity and debt cost of having
optional backends all over the place.

For instance, I think we're well beyond that point that Keystone being
optional should be a thing anywhere (and it is a thing in a number of
places). Keystone should be our auth system, all projects 100% depend on
it, and if you have different site needs, put that into a Keystone backend.

Most of the oslo libraries require other oslo libraries, which is fine.
They aren't trying to solve the general purpose case of logging or
configuration or db access. They are trying to solve a specific set of
patterns that are applicable to OpenStack projects.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Gregory Haynes]

On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> > On Mon, 23 May 2016, Doug Hellmann wrote:
> > > Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> > >> I don't think language does (or should) have anything to do with it.
> > >>
> > >> The question is whether or not the tool (whether service or
> > >> dependent library) is useful to and usable outside the openstack-stack.
> > >> For example gnocchi is useful to openstack but you can use it with other
> > >> stuff, therefore _not_ openstack. More controversially: swift can be
> > >> usefully used all by its lonesome: _not_ openstack.
> > >

Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.

> > > Add keystone, cinder, and ironic to that list.
> > 
> > Hmmm. You can, but would people want to (that is, would it be a sound
> > choice?)? Or _do_ people? Maybe that's the distinction? As far as I
> 
> Yes, I'm aware of cases of each of those projects being used without
> "the rest" of OpenStack. I used keystone like that to secure some
> internal APIs myself.
> 

This has become  a very popular way of using Ironic as well. We even
have an OpenStack project (bifrost) which is used to deploy Ironic in
this fashion.

Cheers,
Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Doug Hellmann]

Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> On Mon, 23 May 2016, Doug Hellmann wrote:
> > Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> >> I don't think language does (or should) have anything to do with it.
> >>
> >> The question is whether or not the tool (whether service or
> >> dependent library) is useful to and usable outside the openstack-stack.
> >> For example gnocchi is useful to openstack but you can use it with other
> >> stuff, therefore _not_ openstack. More controversially: swift can be
> >> usefully used all by its lonesome: _not_ openstack.
> >
> > Add keystone, cinder, and ironic to that list.
> 
> Hmmm. You can, but would people want to (that is, would it be a sound
> choice?)? Or _do_ people? Maybe that's the distinction? As far as I

Yes, I'm aware of cases of each of those projects being used without
"the rest" of OpenStack. I used keystone like that to secure some
internal APIs myself.

> understand things, it's not unusual for standalone swift
> installations to exist and it's something of an openstack joke that
> there are "lots of rules about how things are done, but swift is
> special". Something like gnocchi was designed from the outset to be
> separable.
> 
> I don't really know. I'm firmly in the camp that OpenStack needs to
> be smaller and more tightly focused if a unitary thing called OpenStack
> expects to be any good. So I'm curious about and interested in
> strategies for figuring out where the boundaries are.
> 
> So that, of course, leads back to the original question: Is OpenStack
> supposed to be a unitary.
> 
> If it's _not_, they yes, we need some fairly arbitrary (as in, it's
> okay if we pull them out of thin air without relation to the quality
> of the product, because there is no product!) but concrete guidelines
> that delineate in or out. We can just choose to choose them.
> 
> > As you say, there are a lot of good reasons to strive for loose
> > coupling between components. On the other hand, whether or not an
> > individual component can or should be used by itself isn't a
> > sufficient line when we're talking about the nature of OpenStack
> > as a whole, especially if we want interoperability between deployments.
> 
> If we want interoperability between deployments (is that yet
> universally agreed?) then optionality is a useful guideline for

It's the basis of much if not all of the DefCore trademark work.
So there are still some folks who don't like it, but it's an official
goal.

> whether something is OpenStack[1] or not. If it's optional, then it
> can't be part of OpenStack because if one install has it and another
> does not, then interoperability is shot.

Not quite. With the different trademark programs in place, the
stance is "if you're calling your X API 'OpenStack' it needs to run
our code and it needs to pass the interop tests for the related
capabilities." Each trademark program has a different list of
projects to which it applies.  So it's more prix fixe with an option
of the number of courses rather than a la carte (I may need lunch).

> 
> So that seems like a good second question: Do we all agree that the
> thing called OpenStack at cloud A is the same thing as OpenStack
> at cloud B?
> 
> (There's plenty of talk about this concept, but I'm pretty sure
> there isn't agreement. If we're not working towards real agreement,
> and we're not going to follow that agreement if we ever reach it,
> what are we doing?)

The DefCore team is working to establish the definition, and has a
well-defined process for documenting and modifying the definition over
time.

Doug

> 
> [1] Being useful-to or usable-by OpenStack is a whole 'nother deal.
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Chris Dent]

On Mon, 23 May 2016, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.


Add keystone, cinder, and ironic to that list.


Hmmm. You can, but would people want to (that is, would it be a sound
choice?)? Or _do_ people? Maybe that's the distinction? As far as I
understand things, it's not unusual for standalone swift
installations to exist and it's something of an openstack joke that
there are "lots of rules about how things are done, but swift is
special". Something like gnocchi was designed from the outset to be
separable.

I don't really know. I'm firmly in the camp that OpenStack needs to
be smaller and more tightly focused if a unitary thing called OpenStack
expects to be any good. So I'm curious about and interested in
strategies for figuring out where the boundaries are.

So that, of course, leads back to the original question: Is OpenStack
supposed to be a unitary.

If it's _not_, they yes, we need some fairly arbitrary (as in, it's
okay if we pull them out of thin air without relation to the quality
of the product, because there is no product!) but concrete guidelines
that delineate in or out. We can just choose to choose them.


As you say, there are a lot of good reasons to strive for loose
coupling between components. On the other hand, whether or not an
individual component can or should be used by itself isn't a
sufficient line when we're talking about the nature of OpenStack
as a whole, especially if we want interoperability between deployments.


If we want interoperability between deployments (is that yet
universally agreed?) then optionality is a useful guideline for
whether something is OpenStack[1] or not. If it's optional, then it
can't be part of OpenStack because if one install has it and another
does not, then interoperability is shot.

So that seems like a good second question: Do we all agree that the
thing called OpenStack at cloud A is the same thing as OpenStack
at cloud B?

(There's plenty of talk about this concept, but I'm pretty sure
there isn't agreement. If we're not working towards real agreement,
and we're not going to follow that agreement if we ever reach it,
what are we doing?)

[1] Being useful-to or usable-by OpenStack is a whole 'nother deal.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Doug Hellmann]

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> On Fri, 20 May 2016, Thierry Carrez wrote:
> 
> > The other approach is product-centric: "lower-level pieces are OpenStack 
> > dependencies, rather than OpenStack itself". If we are missing a 
> > lower-level 
> > piece to achieve our mission and are developing it as a result, it could be 
> > developed on OpenStack infrastructure by members of the OpenStack community 
> > but it is not "OpenStack the product", it's an OpenStack *dependency*. It 
> > is 
> > not governed by the TC, it can use any language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where 
> > "lower-level" 
> > starts, which as you explained above is not really clear-cut. A good litmus 
> > test for it could be whenever Python is not enough. If you can't develop it 
> > effectively with the language that is currently sufficient for the rest of 
> > OpenStack, then developing it as an OpenStack dependency in whatever 
> > language 
> > is appropriate might be the solution...
> 
> I don't think language does (or should) have anything to do with it.
> 
> The question is whether or not the tool (whether service or
> dependent library) is useful to and usable outside the openstack-stack.
> For example gnocchi is useful to openstack but you can use it with other
> stuff, therefore _not_ openstack. More controversially: swift can be
> usefully used all by its lonesome: _not_ openstack.

Add keystone, cinder, and ironic to that list.

> Not being in OpenStack (where "in" means "of the product") is good
> for OpenStack, good for the project and good for opensource in general:
> 
> * Outside the OpenStack bubble, looking in, one can see a bunch of
>complexity and a bunch of bad architecture decisions but rarely
>see the good stuff that is actually there, so it is easy enough to walk
>away. Good stuff that a larger audience could benefit from may get
>dismissed, if that good stuff has an opportunity to have an
>independent identity, it can be useful.
> 
> * A project that is used by a larger and more diverse audience
>(people-wise and technology-wise) will of necessity be more
>robust.
> 
> * A project that defines itself as independent will be required to
>have strong and narrow contracts to satisfy its diverse audiences.
> 
> * A project that has those strong and narrow contracts can use what
>ever language it likes and still be useful and nobody really needs
>to care all that deeply except for the people making it. If they
>want to be in a language that infra doesn't want to support,
>that's fine: there are plenty of other ways to do CI.
> 
> * An openstack which is more narrow is far easier for people to
>comprehend and contemplate.
> 
> * A broad opensource world that has lots of nice things is a better
>one.

As you say, there are a lot of good reasons to strive for loose
coupling between components. On the other hand, whether or not an
individual component can or should be used by itself isn't a
sufficient line when we're talking about the nature of OpenStack
as a whole, especially if we want interoperability between deployments.

Most of our services add value on top of some other existing project
(either by integrating them or abstracting them or both).  What
would that story look like if we said that because keystone can be
used by itself, it is not part of OpenStack? Would it make sense
to still require deployments that want to use the trademark to use
keystone for authentication?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Dean Troyer]

On Fri, May 20, 2016 at 8:16 AM, Chris Dent  wrote:

> I don't think language does (or should) have anything to do with it.
>

^1024

Language is what finally forced this discussion (as a prerequisite), now
that we're here, lets finish the prerequisite before going back to what got
us here.


> The question is whether or not the tool (whether service or
> dependent library) is useful to and usable outside the openstack-stack.
> For example gnocchi is useful to openstack but you can use it with other
> stuff, therefore _not_ openstack. More controversially: swift can be
> usefully used all by its lonesome: _not_ openstack.
>

Heh, projects usually see this as a positive.

Not being in OpenStack (where "in" means "of the product") is good
> for OpenStack, good for the project and good for opensource in general:
>

[excellent list of project attributes removed]

I would really hope these could also apply to OpenStack projects, and
understand why they usually don't.  In or out of the tent, those attributes
have at least one strong common thread, the ability to say no when
necessary and follow a specific path.  Self-discipline.  In or out of the
tent, projects without that disintegrate eventually.  I'm posting that list
on the clubhouse wall.

Since we are, by definition in the mission statement, all things to all
people, we got really inclusive in the last year or two.  The pendulum does
appear to be swinging the other way however, somewhat due to the costs of
scale.  Maybe entry into the tent should come with a ticket price to help
defray some of those costs?  Not just direct $$$ (corporate foundation
membership), but dedicated X hours per cycle for documentation or Y hours
for Infra or Z core/time units of cloud resources.  Wait, we do something
like that already?  Maybe its time for a cost-of-living adjustment...

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Dean Troyer]

On Fri, May 20, 2016 at 7:48 AM, Thierry Carrez 
wrote:

> The other approach is product-centric: "lower-level pieces are OpenStack
> dependencies, rather than OpenStack itself". If we are missing a
> lower-level piece to achieve our mission and are developing it as a result,
> it could be developed on OpenStack infrastructure by members of the
> OpenStack community but it is not "OpenStack the product", it's an
> OpenStack *dependency*. It is not governed by the TC, it can use any
> language and tool deemed necessary.
>

I think we should include the degree of OpenStack-specificness here, as
something that may fit every other of your criteria for 'used by but not
part of OpenStack' is really only useful to OpenStack (Designate's DNS
code, for example, apparently reads the DB) should be part of OpenStack.
IIRC that has been used as one of the criteria for deciding if a new
library should be part of Oslo or stand-alone (independent of in-or-out of
OpenStack governance).


> That is what I mean by 'scope': where does "OpenStack" stop, and where do
> "OpenStack dependencies" start ? It is a lot easier and a lot less
> community-costly to allow additional languages in OpenStack dependencies
> (we already have plenty there).


So down the road when one of these dependencies that is very important to
OpenStack goes more dormant than we would like due to resource allocation
issues because it is not-Big Tent, will we adopt it like we have done with
other dependencies where that made more sense than re-writing around it?

I do hope that should the TC adopt the position of drawing the scope line
tighter around the core, that the tent-cleaning will follow in both
directions, down toward kernel-space and up toward end-user-space.  We are
historically bad at leaving that sort of debt lying and cleaning can make
some strides toward reducing the community cost of maintaining the current
ecosystem.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Chris Dent]

On Fri, 20 May 2016, Thierry Carrez wrote:

The other approach is product-centric: "lower-level pieces are OpenStack 
dependencies, rather than OpenStack itself". If we are missing a lower-level 
piece to achieve our mission and are developing it as a result, it could be 
developed on OpenStack infrastructure by members of the OpenStack community 
but it is not "OpenStack the product", it's an OpenStack *dependency*. It is 
not governed by the TC, it can use any language and tool deemed necessary.


On this second approach, there is the obvious question of where "lower-level" 
starts, which as you explained above is not really clear-cut. A good litmus 
test for it could be whenever Python is not enough. If you can't develop it 
effectively with the language that is currently sufficient for the rest of 
OpenStack, then developing it as an OpenStack dependency in whatever language 
is appropriate might be the solution...


I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.

Not being in OpenStack (where "in" means "of the product") is good
for OpenStack, good for the project and good for opensource in general:

* Outside the OpenStack bubble, looking in, one can see a bunch of
  complexity and a bunch of bad architecture decisions but rarely
  see the good stuff that is actually there, so it is easy enough to walk
  away. Good stuff that a larger audience could benefit from may get
  dismissed, if that good stuff has an opportunity to have an
  independent identity, it can be useful.

* A project that is used by a larger and more diverse audience
  (people-wise and technology-wise) will of necessity be more
  robust.

* A project that defines itself as independent will be required to
  have strong and narrow contracts to satisfy its diverse audiences.

* A project that has those strong and narrow contracts can use what
  ever language it likes and still be useful and nobody really needs
  to care all that deeply except for the people making it. If they
  want to be in a language that infra doesn't want to support,
  that's fine: there are plenty of other ways to do CI.

* An openstack which is more narrow is far easier for people to
  comprehend and contemplate.

* A broad opensource world that has lots of nice things is a better
  one.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Thierry Carrez]

John Dickinson wrote:

[...]
In the same vein, if we consider lower-level projects (which often
require such native optimization) as part of "OpenStack", rather
than as external open source projects that should be integrated by
"OpenStack", then we need a language like golang in our toolbelt.
There is basically no point in saying no to golang in OpenStack if
we need lower-level native optimization in OpenStack: we'll have to
accept the community cost that comes with such a community scope.


Defining "lower-level" is very hard. Since the Nova API[1] is
listening to a public network interface and coordinating with various
services in a cluster, is it low-level enough to need to consider
optimizations? Does the Nova API require optimization to handle a very
large number of connections using all of the hardware available on a
single server? If Nova is going to eek out every drop of performance
possible from a server, it probably does need to consider all kinds of
"low-level" optimizations.[2]


That is fair. There is no clear line between lower-level and 
higher-level (although one might argue there is a line between projects 
requiring Go and projects that don't require it). It's more of a gradient.



[...]

So the real question we need to answer is... where does OpenStack
stop, and where does the wider open source community start ? If
OpenStack is purely an "integration engine", glue code for other
lower-level technologies like hypervisors, databases, or distributed
block storage, then the scope is limited, Python should be plenty
enough, and we don't need to fragment our community. If OpenStack is
"whatever it takes to reach our mission", then yes we need to add one
language to cover lower-level/native optimization, because we'll
need that... and we need to accept the community cost as a
consequence of that scope choice. Those are the only two options on
the table.

I'm actually not sure what is the best answer. But I'm convinced we,
as a community, need to have a clear answer to that. We've been
avoiding that clear answer until now, creating tension between the
advocates of an ASF-like collection of tools and the advocates of a
tighter-integrated "openstack" product. We have created silos and
specialized areas as we got into the business of developing time-
series databases or SDNs. As a result, it's not "one community"
anymore. Should we further encourage that, or should we focus on
what the core of our mission is, what we have in common, this
integration engine that binds all those other open source projects
into one programmable infrastructure solution ?


You said the answer in your question. OpenStack isn't defined as an
integration engine[3]. The definition of OpenStack is whatever it
takes to fulfill our mission[4][5]. I don't mean that as a tautology.
I mean that we've already gone to the effort of defining OpenStack. It's
our mission statement. We're all about building a cloud platform upon
which people can run their apps ("cloud-native" or otherwise), so we
write the software needed to do that.

So where does OpenStack stop and the wider community start? OpenStack
includes the projects needed to fulfill its mission.


I'd totally agree with you if OpenStack was developed in a vacuum. But 
there is a large number of open source projects and libraries that 
OpenStack needs to fulfill its mission that are not in "OpenStack": they 
are external open source projects we depend on. Python, MySQL, libvirt, 
KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should 
be included in OpenStack, and we are not NIHing replacements for those 
in OpenStack either.


So it is not as clear-cut as you present it, and you can approach this 
dependency question from two directions.


One is community-centric: "anything produced by our community is 
OpenStack". If we are missing a lower-level piece to achieve our mission 
and are developing it ourselves as a result, then it is OpenStack, even 
if it ends up being a message queue or a database.


The other approach is product-centric: "lower-level pieces are OpenStack 
dependencies, rather than OpenStack itself". If we are missing a 
lower-level piece to achieve our mission and are developing it as a 
result, it could be developed on OpenStack infrastructure by members of 
the OpenStack community but it is not "OpenStack the product", it's an 
OpenStack *dependency*. It is not governed by the TC, it can use any 
language and tool deemed necessary.


On this second approach, there is the obvious question of where 
"lower-level" starts, which as you explained above is not really 
clear-cut. A good litmus test for it could be whenever Python is not 
enough. If you can't develop it effectively with the language that is 
currently sufficient for the rest of OpenStack, then developing it as an 
OpenStack dependency in whatever language is appropriate might be the 
solution...


That is what I mean by 'scope': where does "OpenStack" stop, and where 
do 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Fox, Kevin M]

+1. OpenStack does meet the requirements for being an Operating System at the 
data center level in my opinion. We just keep trying to beat around that bush 
for some reason... yes, we do that by Integrating a lot of things together, but 
thats not what its about.

From: Joshua Harlow [harlo...@fastmail.com]
Sent: Thursday, May 19, 2016 3:01 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

Morgan Fainberg wrote:
>
>
> On Thu, May 19, 2016 at 6:19 AM, Thierry Carrez <thie...@openstack.org
> <mailto:thie...@openstack.org>> wrote:
>
> Hi everyone,
>
> The discussion on the addition of golang focuses on estimating
> community costs vs. technical benefits, so that the TC can make the
> right call for "OpenStack". From that discussion, we established
> that the costs for cross-project teams (Infra...) seem to be
> workable. There is still significant community fragmentation effects
> as we add another language, creating language-expert silos,
> duplicating efforts, and losing some productivity as some needlessly
> rewrite things in golang. But we seem generally ready to accept that
> cost because we are missing a tool in our toolbelt: a language that
> lets us do such native optimization.
>
> We have a number of projects in OpenStack that are more low-level
> than others and which require such optimization, and for those
> projects (or subprojects) our current mix of language is not
> satisfactory. The Swift team in particular has spent a lot of time
> trying to get the I/O behavior they require with hard disk access
> using Python, and they certainly did not jump on the golang
> bandwagon lightly.
>
> I agree with this. it is totally reasonable to add golang (or a tool to
> fill the need for native optimizations). The fragmentation concerns are
> real, but filling this gap in the openstack tool-chain is important. It
> will garner more interest in the project as it opens the doors to
> examine the native level optimizations that are exceptionally hard in
> Python. While this all could be done in Python, I would argue that there
> are legitimate cases where a tool like golang is going to be a "better"
> fit for the job. I would also argue that the TC should provide a strong
> guidance on where golang should be used (initially). Specifically that
> it should be used in cases where Python can demonstrably be shown to be
> the inferior tool for the job and not as whole-sale replacement(s) for
> the core API/end-user interactions. The type of break that the swift
> team is advocating for is a great starting place. With the
> acknowledgement from the -infra team that this is reasonable and
> workable, I am willing (with my TC hat on), agree to include golang
> provided the recommendations are outlined for the initial use-cases. As
> we see where things shake out with these initial use-cases I am sure we
> will see expanded uses of golang that make sense. I do know that
> policing such uses is difficult at best, and I don't think it is
> something that should be done via technology but more of a social
> contract and documented as the recommended for initial cases of golang
> (or other similar tool-chain) inclusion.
>
> I believe the programming languages you need in OpenStack official
> projects are a function of the scope you define for OpenStack
> official projects. We wouldn't need to have JavaScript in the mix if
> we considered that web interfaces that purely consume OpenStack APIs
> are projects that consume OpenStack, rather than part of OpenStack
> itself.
>
> In the same vein, if we consider lower-level projects (which often
> require such native optimization) as part of "OpenStack", rather
> than as external open source projects that should be integrated by
> "OpenStack", then we need a language like golang in our toolbelt.
> There is basically no point in saying no to golang in OpenStack if
> we need lower-level native optimization in OpenStack: we'll have to
> accept the community cost that comes with such a community scope.
>
> So the real question we need to answer is... where does OpenStack
> stop, and where does the wider open source community start ? If
> OpenStack is purely an "integration engine", glue code for other
> lower-level technologies like hypervisors, databases, or distributed
> block storage, then the scope is limited, Python should be plenty
> enough, and we don't need to fragment our community. If OpenStack is
> "what

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Joshua Harlow]

John Dickinson wrote:

summary:
  * Defining the scope of OpenStack projects DOES NOT define the
languages needed to implement them. The considerations are
orthogonal.
  * We've already defined OpenStack--it's whatever it takes to
fulfill its mission statement.

On 19 May 2016, at 6:19, Thierry Carrez wrote:


Hi everyone,

The discussion on the addition of golang focuses on estimating
community costs vs. technical benefits, so that the TC can make the
right call for "OpenStack". From that discussion, we established
that the costs for cross-project teams (Infra...) seem to be
workable. There is still significant community fragmentation
effects as we add another language, creating language-expert silos,
duplicating efforts, and losing some productivity as some
needlessly rewrite things in golang. But we seem generally ready to
accept that cost because we are missing a tool in our toolbelt: a
language that lets us do such native optimization.

We have a number of projects in OpenStack that are more low-level
than others and which require such optimization, and for those
projects (or subprojects) our current mix of language is not
satisfactory. The Swift team in particular has spent a lot of time
trying to get the I/O behavior they require with hard disk access
using Python, and they certainly did not jump on the golang
bandwagon lightly.

I believe the programming languages you need in OpenStack official
projects are a function of the scope you define for OpenStack
official projects. We wouldn't need to have JavaScript in the mix if
we considered that web interfaces that purely consume OpenStack
APIs are projects that consume OpenStack, rather than part of
OpenStack itself.

In the same vein, if we consider lower-level projects (which often
require such native optimization) as part of "OpenStack", rather
than as external open source projects that should be integrated by
"OpenStack", then we need a language like golang in our toolbelt.
There is basically no point in saying no to golang in OpenStack if
we need lower-level native optimization in OpenStack: we'll have to
accept the community cost that comes with such a community scope.


Defining "lower-level" is very hard. Since the Nova API[1] is
listening to a public network interface and coordinating with various
services in a cluster, is it low-level enough to need to consider
optimizations? Does the Nova API require optimization to handle a very
large number of connections using all of the hardware available on a
single server? If Nova is going to eek out every drop of performance
possible from a server, it probably does need to consider all kinds of
"low-level" optimizations.[2]

Because deployers of any OpenStack project do want efficient software
that is performant, all parts of OpenStack need to consider what it
takes to meet the performance demanded. Most of the time this will not
require rewriting code in a different language (that's almost never
the right answer), but my point is that I believe you're wrong that
defining the scope of OpenStack projects also defines the languages
needed to implement them. The considerations are orthogonal.

[1] substitute your favorite OpenStack project

[2] 
http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html



So the real question we need to answer is... where does OpenStack
stop, and where does the wider open source community start ? If
OpenStack is purely an "integration engine", glue code for other
lower-level technologies like hypervisors, databases, or distributed
block storage, then the scope is limited, Python should be plenty
enough, and we don't need to fragment our community. If OpenStack is
"whatever it takes to reach our mission", then yes we need to add one
language to cover lower-level/native optimization, because we'll
need that... and we need to accept the community cost as a
consequence of that scope choice. Those are the only two options on
the table.

I'm actually not sure what is the best answer. But I'm convinced we,
as a community, need to have a clear answer to that. We've been
avoiding that clear answer until now, creating tension between the
advocates of an ASF-like collection of tools and the advocates of a
tighter-integrated "openstack" product. We have created silos and
specialized areas as we got into the business of developing time-
series databases or SDNs. As a result, it's not "one community"
anymore. Should we further encourage that, or should we focus on
what the core of our mission is, what we have in common, this
integration engine that binds all those other open source projects
into one programmable infrastructure solution ?


You said the answer in your question. OpenStack isn't defined as an
integration engine[3]. The definition of OpenStack is whatever it
takes to fulfill our mission[4][5]. I don't mean that as a tautology.
I mean that we've already gone to the effort of defining OpenStack. It's
our 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Joshua Harlow]

Morgan Fainberg wrote:



On Thu, May 19, 2016 at 6:19 AM, Thierry Carrez > wrote:

Hi everyone,

The discussion on the addition of golang focuses on estimating
community costs vs. technical benefits, so that the TC can make the
right call for "OpenStack". From that discussion, we established
that the costs for cross-project teams (Infra...) seem to be
workable. There is still significant community fragmentation effects
as we add another language, creating language-expert silos,
duplicating efforts, and losing some productivity as some needlessly
rewrite things in golang. But we seem generally ready to accept that
cost because we are missing a tool in our toolbelt: a language that
lets us do such native optimization.

We have a number of projects in OpenStack that are more low-level
than others and which require such optimization, and for those
projects (or subprojects) our current mix of language is not
satisfactory. The Swift team in particular has spent a lot of time
trying to get the I/O behavior they require with hard disk access
using Python, and they certainly did not jump on the golang
bandwagon lightly.

I agree with this. it is totally reasonable to add golang (or a tool to
fill the need for native optimizations). The fragmentation concerns are
real, but filling this gap in the openstack tool-chain is important. It
will garner more interest in the project as it opens the doors to
examine the native level optimizations that are exceptionally hard in
Python. While this all could be done in Python, I would argue that there
are legitimate cases where a tool like golang is going to be a "better"
fit for the job. I would also argue that the TC should provide a strong
guidance on where golang should be used (initially). Specifically that
it should be used in cases where Python can demonstrably be shown to be
the inferior tool for the job and not as whole-sale replacement(s) for
the core API/end-user interactions. The type of break that the swift
team is advocating for is a great starting place. With the
acknowledgement from the -infra team that this is reasonable and
workable, I am willing (with my TC hat on), agree to include golang
provided the recommendations are outlined for the initial use-cases. As
we see where things shake out with these initial use-cases I am sure we
will see expanded uses of golang that make sense. I do know that
policing such uses is difficult at best, and I don't think it is
something that should be done via technology but more of a social
contract and documented as the recommended for initial cases of golang
(or other similar tool-chain) inclusion.

I believe the programming languages you need in OpenStack official
projects are a function of the scope you define for OpenStack
official projects. We wouldn't need to have JavaScript in the mix if
we considered that web interfaces that purely consume OpenStack APIs
are projects that consume OpenStack, rather than part of OpenStack
itself.

In the same vein, if we consider lower-level projects (which often
require such native optimization) as part of "OpenStack", rather
than as external open source projects that should be integrated by
"OpenStack", then we need a language like golang in our toolbelt.
There is basically no point in saying no to golang in OpenStack if
we need lower-level native optimization in OpenStack: we'll have to
accept the community cost that comes with such a community scope.

So the real question we need to answer is... where does OpenStack
stop, and where does the wider open source community start ? If
OpenStack is purely an "integration engine", glue code for other
lower-level technologies like hypervisors, databases, or distributed
block storage, then the scope is limited, Python should be plenty
enough, and we don't need to fragment our community. If OpenStack is
"whatever it takes to reach our mission", then yes we need to add
one language to cover lower-level/native optimization, because we'll
need that... and we need to accept the community cost as a
consequence of that scope choice. Those are the only two options on
the table.


It would be disingenuous to say we are a pure integration engine. If
swift or a similar project is outside the scope of openstack, frankly, I
view Keystone as outside of the scope of openstack. Keystone (while in a
different position due to a lot of heavy requirements for auth in
openstack) is not a simple "integrate an underlying technology with
other things with some tracking of the resources". Keystone provides a
real added value, while perhaps not as low-level as swift, that extends
significantly beyond the scope of "glue" of it's individual components.
I do believe the lower level technologies have a place in "openstack's
big tent" but should be 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[John Dickinson]

summary:
 * Defining the scope of OpenStack projects DOES NOT define the
   languages needed to implement them. The considerations are
   orthogonal.
 * We've already defined OpenStack--it's whatever it takes to
   fulfill its mission statement.

On 19 May 2016, at 6:19, Thierry Carrez wrote:

> Hi everyone,
>
> The discussion on the addition of golang focuses on estimating
> community costs vs. technical benefits, so that the TC can make the
> right call for "OpenStack". From that discussion, we established
> that the costs for cross-project teams (Infra...) seem to be
> workable. There is still significant community fragmentation
> effects as we add another language, creating language-expert silos,
> duplicating efforts, and losing some productivity as some
> needlessly rewrite things in golang. But we seem generally ready to
> accept that cost because we are missing a tool in our toolbelt: a
> language that lets us do such native optimization.
>
> We have a number of projects in OpenStack that are more low-level
> than others and which require such optimization, and for those
> projects (or subprojects) our current mix of language is not
> satisfactory. The Swift team in particular has spent a lot of time
> trying to get the I/O behavior they require with hard disk access
> using Python, and they certainly did not jump on the golang
> bandwagon lightly.
>
> I believe the programming languages you need in OpenStack official
> projects are a function of the scope you define for OpenStack
> official projects. We wouldn't need to have JavaScript in the mix if
> we considered that web interfaces that purely consume OpenStack
> APIs are projects that consume OpenStack, rather than part of
> OpenStack itself.
>
> In the same vein, if we consider lower-level projects (which often
> require such native optimization) as part of "OpenStack", rather
> than as external open source projects that should be integrated by
> "OpenStack", then we need a language like golang in our toolbelt.
> There is basically no point in saying no to golang in OpenStack if
> we need lower-level native optimization in OpenStack: we'll have to
> accept the community cost that comes with such a community scope.

Defining "lower-level" is very hard. Since the Nova API[1] is
listening to a public network interface and coordinating with various
services in a cluster, is it low-level enough to need to consider
optimizations? Does the Nova API require optimization to handle a very
large number of connections using all of the hardware available on a
single server? If Nova is going to eek out every drop of performance
possible from a server, it probably does need to consider all kinds of
"low-level" optimizations.[2]

Because deployers of any OpenStack project do want efficient software
that is performant, all parts of OpenStack need to consider what it
takes to meet the performance demanded. Most of the time this will not
require rewriting code in a different language (that's almost never
the right answer), but my point is that I believe you're wrong that
defining the scope of OpenStack projects also defines the languages
needed to implement them. The considerations are orthogonal.

[1] substitute your favorite OpenStack project

[2] 
http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html


> So the real question we need to answer is... where does OpenStack
> stop, and where does the wider open source community start ? If
> OpenStack is purely an "integration engine", glue code for other
> lower-level technologies like hypervisors, databases, or distributed
> block storage, then the scope is limited, Python should be plenty
> enough, and we don't need to fragment our community. If OpenStack is
> "whatever it takes to reach our mission", then yes we need to add one
> language to cover lower-level/native optimization, because we'll
> need that... and we need to accept the community cost as a
> consequence of that scope choice. Those are the only two options on
> the table.
>
> I'm actually not sure what is the best answer. But I'm convinced we,
> as a community, need to have a clear answer to that. We've been
> avoiding that clear answer until now, creating tension between the
> advocates of an ASF-like collection of tools and the advocates of a
> tighter-integrated "openstack" product. We have created silos and
> specialized areas as we got into the business of developing time-
> series databases or SDNs. As a result, it's not "one community"
> anymore. Should we further encourage that, or should we focus on
> what the core of our mission is, what we have in common, this
> integration engine that binds all those other open source projects
> into one programmable infrastructure solution ?

You said the answer in your question. OpenStack isn't defined as an
integration engine[3]. The definition of OpenStack is whatever it
takes to fulfill our mission[4][5]. I don't mean that as a tautology.
I 

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Morgan Fainberg]

On Thu, May 19, 2016 at 6:19 AM, Thierry Carrez 
wrote:

> Hi everyone,
>
> The discussion on the addition of golang focuses on estimating community
> costs vs. technical benefits, so that the TC can make the right call for
> "OpenStack". From that discussion, we established that the costs for
> cross-project teams (Infra...) seem to be workable. There is still
> significant community fragmentation effects as we add another language,
> creating language-expert silos, duplicating efforts, and losing some
> productivity as some needlessly rewrite things in golang. But we seem
> generally ready to accept that cost because we are missing a tool in our
> toolbelt: a language that lets us do such native optimization.
>
> We have a number of projects in OpenStack that are more low-level than
> others and which require such optimization, and for those projects (or
> subprojects) our current mix of language is not satisfactory. The Swift
> team in particular has spent a lot of time trying to get the I/O behavior
> they require with hard disk access using Python, and they certainly did not
> jump on the golang bandwagon lightly.
>
>
I agree with this. it is totally reasonable to add golang (or a tool to
fill the need for native optimizations). The fragmentation concerns are
real, but filling this gap in the openstack tool-chain is important. It
will garner more interest in the project as it opens the doors to examine
the native level optimizations that are exceptionally hard in Python. While
this all could be done in Python, I would argue that there are legitimate
cases where a tool like golang is going to be a "better" fit for the job. I
would also argue that the TC should provide a strong guidance on where
golang should be used (initially). Specifically that it should be used in
cases where Python can demonstrably be shown to be the inferior tool for
the job and not as whole-sale replacement(s) for the core API/end-user
interactions. The type of break that the swift team is advocating for is a
great starting place. With the acknowledgement from the -infra team that
this is reasonable and workable, I am willing (with my TC hat on), agree to
include golang provided the recommendations are outlined for the initial
use-cases. As we see where things shake out with these initial use-cases I
am sure we will see expanded uses of golang that make sense. I do know that
policing such uses is difficult at best, and I don't think it is something
that should be done via technology but more of a social contract and
documented as the recommended for initial cases of golang (or other similar
tool-chain) inclusion.



> I believe the programming languages you need in OpenStack official
> projects are a function of the scope you define for OpenStack official
> projects. We wouldn't need to have JavaScript in the mix if we considered
> that web interfaces that purely consume OpenStack APIs are projects that
> consume OpenStack, rather than part of OpenStack itself.
>
> In the same vein, if we consider lower-level projects (which often require
> such native optimization) as part of "OpenStack", rather than as external
> open source projects that should be integrated by "OpenStack", then we need
> a language like golang in our toolbelt. There is basically no point in
> saying no to golang in OpenStack if we need lower-level native optimization
> in OpenStack: we'll have to accept the community cost that comes with such
> a community scope.
>
> So the real question we need to answer is... where does OpenStack stop,
> and where does the wider open source community start ? If OpenStack is
> purely an "integration engine", glue code for other lower-level
> technologies like hypervisors, databases, or distributed block storage,
> then the scope is limited, Python should be plenty enough, and we don't
> need to fragment our community. If OpenStack is "whatever it takes to reach
> our mission", then yes we need to add one language to cover
> lower-level/native optimization, because we'll need that... and we need to
> accept the community cost as a consequence of that scope choice. Those are
> the only two options on the table.
>
>
It would be disingenuous to say we are a pure integration engine. If swift
or a similar project is outside the scope of openstack, frankly, I view
Keystone as outside of the scope of openstack. Keystone (while in a
different position due to a lot of heavy requirements for auth in
openstack) is not a simple "integrate an underlying technology with other
things with some tracking of the resources". Keystone provides a real added
value, while perhaps not as low-level as swift, that extends significantly
beyond the scope of "glue" of it's individual components. I do believe the
lower level technologies have a place in "openstack's big tent" but should
be evaluated carefully before inclusion; I am not advocating we include
MySQL, PGSQL, or RabbitMQ as "part of the big tent". I think object storage