Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Clint Byrum]

Excerpts from Michael Barton's message of 2016-06-09 15:59:24 -0500:
> On Thu, Jun 9, 2016 at 2:49 PM, Clint Byrum  wrote:
> >
> > Agreed it isn't done in uvloop. But it is done in libuv and the uvloop
> > devs agree it should be done. So this is the kind of thing where the
> > community can invest in python + C to help solve problems thought only
> > solvable by other languages.
> 
> 
> I mean, if someone wants to figure out a file server in python that can
> compete in any way with a go version, I'm totally down for rewriting swift
> to target some other python-based architecture.
> 
> But personally, my desire to try to build a universe where such a thing is
> possible is pretty low.  Because I've been fighting with it for years, and
> go already works great and there's nothing wrong with it.
> 

Mike, the whole entire crux of this thread, and Monty's words, is that
this sort of sentiment is hard to ignore, but it's even harder to ignore
the massive amount of inertia and power there is in having a community
that can all work on each others' code without investing a lot of time
in learning a new language.

That inertia is entirely the reason why other languages have surpassed
Python in some areas like concurrency. It takes longer to turn a
massive community going really hard in one direction than it does to
just start off heading in that direction in the first place. But that
turn is starting, and I for one think it's worth everyone's time to take
a hard look at whether or not we can in fact get it done together.

Nobody will force you to, but what I think Monty, and the rest of the
TC members who have voted to stay the course, are asking us all to do,
is to try to throw what we can at python solutions for these problems.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Michael Barton]

On Thu, Jun 9, 2016 at 2:49 PM, Clint Byrum  wrote:
>
> Agreed it isn't done in uvloop. But it is done in libuv and the uvloop
> devs agree it should be done. So this is the kind of thing where the
> community can invest in python + C to help solve problems thought only
> solvable by other languages.


I mean, if someone wants to figure out a file server in python that can
compete in any way with a go version, I'm totally down for rewriting swift
to target some other python-based architecture.

But personally, my desire to try to build a universe where such a thing is
possible is pretty low.  Because I've been fighting with it for years, and
go already works great and there's nothing wrong with it.

- Mike
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Clint Byrum]

Excerpts from Michael Barton's message of 2016-06-09 14:01:11 -0500:
> On Thu, Jun 9, 2016 at 9:58 AM, Ben Meyer  wrote:
> 
> >
> > uvloop (first commit 2015-11-01) is newer than Swift's hummingbird
> > (2015-04-20, based on
> >
> > https://github.com/openstack/swift/commit/a0e300df180f7f4ca64fc1eaf3601a1a73fc68cb
> > and github network graph) so it would not have been part of the
> > consideration.
> >
> 
> And it still wouldn't be, since it doesn't solve the problem.
> 

Agreed it isn't done in uvloop. But it is done in libuv and the uvloop
devs agree it should be done. So this is the kind of thing where the
community can invest in python + C to help solve problems thought only
solvable by other languages.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Michael Barton]

On Thu, Jun 9, 2016 at 9:58 AM, Ben Meyer  wrote:

>
> uvloop (first commit 2015-11-01) is newer than Swift's hummingbird
> (2015-04-20, based on
>
> https://github.com/openstack/swift/commit/a0e300df180f7f4ca64fc1eaf3601a1a73fc68cb
> and github network graph) so it would not have been part of the
> consideration.
>

And it still wouldn't be, since it doesn't solve the problem.

- Mike
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Ben Meyer]

On 06/08/2016 11:05 PM, Chris Friesen wrote:
> On 06/07/2016 04:26 PM, Ben Meyer wrote:
>> On 06/07/2016 06:09 PM, Samuel Merritt wrote:
>>> On 6/7/16 12:00 PM, Monty Taylor wrote:
 [snip]

 I'd rather see us focus energy on Python3, asyncio and its pluggable
 event loops. The work in:

 http://magic.io/blog/uvloop-blazing-fast-python-networking/

 is a great indication in an actual apples-to-apples comparison of what
 can be accomplished in python doing IO-bound activities by using
 modern
 Python techniques. I think that comparing python2+eventlet to a fresh
 rewrite in Go isn't 100% of the story. A TON of work has gone in to
 Python that we're not taking advantage of because we're still
 supporting
 Python2. So what I've love to see in the realm of comparative
 experimentation is to see if the existing Python we already have
 can be
 leveraged as we adopt newer and more modern things.
>>>
>>> Asyncio, eventlet, and other similar libraries are all very good for
>>> performing asynchronous IO on sockets and pipes. However, none of them
>>> help for filesystem IO. That's why Swift needs a golang object server:
>>> the go runtime will keep some goroutines running even though some
>>> other goroutines are performing filesystem IO, whereas filesystem IO
>>> in Python blocks the entire process, asyncio or no asyncio.
>>
>> That can be modified. gevent has a tool
>> (http://www.gevent.org/gevent.fileobject.html) that enables the File IO
>> to be async  as well by putting the file into non-blocking mode. I've
>> used it, and it works and scales well.
>
> Arguably non-blocking isn't really async when it comes to reads.  I
> suspect what we really want is full-async where you issue a request
> and then get notified when it's done.

So when it comes to Swift or Glance where you have to transfer large
amounts of data between the HTTP client and HTTP serverunder WSGI,
the only way to make it truly cooperative for eventlet, gevent, etc is
to use non-blocking File I/O. These situations also reveal how
uncooperative green threads are too since if you can keep the data
pipeline full (e.g the read is continuous because the OS is able to
quickly service it) then one greed thread will take over and block the
others. Non-blocking I/O is only part of the solution, not the entire
solution.

Furthermore, the constraint of the web head having to provide the data
transfer makes the full-async offload with a notification impossible for
data transfer services like Swift. Tools like uvloop make great strides
in providing better environments for doing cooperative tasks than what
existed prior, and some of the articles comparing it to the various
tools - both alone and in combination - make for some fascinating
possibilities and solutions. Most likely the best solution here is going
to be going to Py3's asyncio+uvloop+non-blocking I/O on the files in
order to hit the throughput for Swift.

uvloop (first commit 2015-11-01) is newer than Swift's hummingbird
(2015-04-20, based on
https://github.com/openstack/swift/commit/a0e300df180f7f4ca64fc1eaf3601a1a73fc68cb
and github network graph) so it would not have been part of the
consideration.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Chris Friesen]

On 06/07/2016 04:26 PM, Ben Meyer wrote:

On 06/07/2016 06:09 PM, Samuel Merritt wrote:

On 6/7/16 12:00 PM, Monty Taylor wrote:

[snip]

I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.


Asyncio, eventlet, and other similar libraries are all very good for
performing asynchronous IO on sockets and pipes. However, none of them
help for filesystem IO. That's why Swift needs a golang object server:
the go runtime will keep some goroutines running even though some
other goroutines are performing filesystem IO, whereas filesystem IO
in Python blocks the entire process, asyncio or no asyncio.


That can be modified. gevent has a tool
(http://www.gevent.org/gevent.fileobject.html) that enables the File IO
to be async  as well by putting the file into non-blocking mode. I've
used it, and it works and scales well.


Arguably non-blocking isn't really async when it comes to reads.  I suspect what 
we really want is full-async where you issue a request and then get notified 
when it's done.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Gregory Haynes]

On Wed, Jun 8, 2016, at 03:46 AM, Thierry Carrez wrote:
> Another option (raised by dims) is to find a way to allow usage of 
> golang (or another language) in a more granular way: selectively allow 
> projects which really need another tool to use it. The benefit is that 
> it lets project teams make a case and use the best tool for the job, 
> while limiting the cross-project impact and without opening the 
> distraction floodgates of useless rewrites. The drawback is that 
> depending on how it's done, it may place the TC in the role of granting 
> "you're tall enough to use Go" badges, creating new endless discussions 
> and more "you're special" exceptions. That said, I'm still interested in 
> exploring that option, for one reason. I think that whenever a project 
> team considers adding a component or a rewrite in another language, they 
> are running into an interesting problem with Python, on which they 
> really could use advice from the rest of the OpenStack community. I'd 
> really like to see a cross-project team of Python performance experts to 
> look at the problem this specific team has that makes them want to use 
> another language. That seems like a great way to drive more practice 
> sharing and reduce fragmentation in "OpenStack" in general. We might 
> just need to put the bar pretty high so that we are not flooded by silly 
> rewrite requests.
> 

++.  There's a lot of value in these issues getting bubbled up to the
cross-project level: If we have identified a serious hurdle then this
knowledge really shouldn't live inside of a single project. Otherwise,
if we haven't identified such an issue, then the (the greater openstack
community) can offer some alternative solutions which is also a huge
win.

I completely understand the fear that we might be creating an endless
review stream for the TC by making them the review squad for getting
approval to use a new language, and I agree that we need to make sure
that doesn't happen. OTOH, I strongly believe that in almost all of the
cases which would be proposed some alternative solutions could be found.
I worry that if we just tell these folks 'the solution you thought of
isn't allowed' rather than offer an outlet for seriously investigating
the issue were likely to see teams try and find ways around that
restriction when really we want to identify another solution to the
problem. A perf team sounds like a great way to help both the tribal
knowledge problem and support the type of problem solving we are asking
for. Sign me up :).

Cheers,
Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Thierry Carrez]

Samuel Merritt wrote:

On 6/7/16 12:00 PM, Monty Taylor wrote:

[snip]

 >

I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.


Asyncio, eventlet, and other similar libraries are all very good for
performing asynchronous IO on sockets and pipes. However, none of them
help for filesystem IO. That's why Swift needs a golang object server:
the go runtime will keep some goroutines running even though some other
goroutines are performing filesystem IO, whereas filesystem IO in Python
blocks the entire process, asyncio or no asyncio.


As you probably know by now, there was a majority of TC members to deny 
the addition of Golang as another generally available tool in our 
toolbelt. The resolution (which was made to follow the guidelines the TC 
itself set as the process to follow in that specific case) was therefore 
rejected.


What made this decision pretty hard for everyone involved is that the 
Swift team built a compelling argument (see just above) on why they 
can't really solve a real user problem they have (scale and perform 
better) using the tools they are allowed to use in "OpenStack". There 
was probably also a majority of TC members agreeing that Swift needs to 
use another language to solve the specific filesystem IO problem they 
are running into. So we have been looking at various trade-off solutions 
to get the projects the tools they really need without making 
cross-project work significantly more complex, and without triggering 
the gigantic distraction of superfluous rewrites across OpenStack projects.


None of those middle-ground solutions is optimal. But I've been asked 
to list them for clarity.


One option is to develop generally-useful libraries or services outside 
of OpenStack that solve the specific issue you're running into. 
OpenStack relies on a lot of dependencies in the wider open source 
community, we don't have to develop *everything* as official "openstack 
projects" under TC governance. We even have a number of them that are 
using OpenStack infrastructure (git, gerrit, gate...). The benefit of 
this approach is that it lets you pick whatever language or framework is 
most appropriate, and forces clean contracts between the components, 
which makes replacing the external piece easier in the future. The 
drawback of this approach is that it creates local fragmentation at the 
project team level, as they have to artificially split the project into 
"OpenStack" and non-"OpenStack" bits. And it is also not optimal for 
Swift due to the way it's currently designed, you can't really slice it 
in this way.


Another option (raised by dims) is to find a way to allow usage of 
golang (or another language) in a more granular way: selectively allow 
projects which really need another tool to use it. The benefit is that 
it lets project teams make a case and use the best tool for the job, 
while limiting the cross-project impact and without opening the 
distraction floodgates of useless rewrites. The drawback is that 
depending on how it's done, it may place the TC in the role of granting 
"you're tall enough to use Go" badges, creating new endless discussions 
and more "you're special" exceptions. That said, I'm still interested in 
exploring that option, for one reason. I think that whenever a project 
team considers adding a component or a rewrite in another language, they 
are running into an interesting problem with Python, on which they 
really could use advice from the rest of the OpenStack community. I'd 
really like to see a cross-project team of Python performance experts to 
look at the problem this specific team has that makes them want to use 
another language. That seems like a great way to drive more practice 
sharing and reduce fragmentation in "OpenStack" in general. We might 
just need to put the bar pretty high so that we are not flooded by silly 
rewrite requests.


For completeness I'll also list the nuclear option: to reject the 
Technical Committee oversight altogether and decide that your project is 
better off as an autonomous project in the OpenStack ecosystem.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Jim Baker]

On Tue, Jun 7, 2016 at 4:26 PM, Ben Meyer  wrote:

> On 06/07/2016 06:09 PM, Samuel Merritt wrote:
> > On 6/7/16 12:00 PM, Monty Taylor wrote:
> >> [snip]
> > >
> >> I'd rather see us focus energy on Python3, asyncio and its pluggable
> >> event loops. The work in:
> >>
> >> http://magic.io/blog/uvloop-blazing-fast-python-networking/
> >>
> >> is a great indication in an actual apples-to-apples comparison of what
> >> can be accomplished in python doing IO-bound activities by using modern
> >> Python techniques. I think that comparing python2+eventlet to a fresh
> >> rewrite in Go isn't 100% of the story. A TON of work has gone in to
> >> Python that we're not taking advantage of because we're still supporting
> >> Python2. So what I've love to see in the realm of comparative
> >> experimentation is to see if the existing Python we already have can be
> >> leveraged as we adopt newer and more modern things.
> >
> > Asyncio, eventlet, and other similar libraries are all very good for
> > performing asynchronous IO on sockets and pipes. However, none of them
> > help for filesystem IO. That's why Swift needs a golang object server:
> > the go runtime will keep some goroutines running even though some
> > other goroutines are performing filesystem IO, whereas filesystem IO
> > in Python blocks the entire process, asyncio or no asyncio.
>
> That can be modified. gevent has a tool
> (http://www.gevent.org/gevent.fileobject.html) that enables the File IO
> to be async  as well by putting the file into non-blocking mode. I've
> used it, and it works and scales well.
>
> Sadly, Python doesn't offer this by default; perhaps OpenStack can get
> that changed.
>
> $0.02
>
> Ben
>
>
The uvloop library is very new, but libuv itself uses a thread pool, so it
should work just fine with files once this functionality is implemented;
see https://github.com/MagicStack/uvloop/issues/1 and
https://nikhilm.github.io/uvbook/filesystem.html

I don't see any difference here with how Golang would implement such async
support, in terms of mapping goroutines against its own thread pool. With
respect to latency, there could even be better performance for certain
workloads with uvloop than Go. This is because of CPython's refcounting, vs
Go's GC - even with all the recent improvements in Go (
https://blog.golang.org/go15gc), Go still does stop the world.

A key piece here is that uvloop does not hold the GIL in these ops - a big
advantage that Python C extensions enjoy, including libraries they wrap,
and Cython explicitly enables with nogil support. So in particular we have
the following code in the the core uvloop function, which in turn is just
calling into libuv with its own core function, uv_run:

```
# Although every UVHandle holds a reference to the loop,
# we want to do everything to ensure that the loop will
# never deallocate during the run -- so we do some
# manual refs management.
Py_INCREF(self)
with nogil:
err = uv.uv_run(self.uvloop, mode)
Py_DECREF(self)
```

More here on nogil support in Cython:
http://docs.cython.org/src/userguide/external_C_code.html#declaring-a-function-as-callable-without-the-gil

- Jim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Ben Meyer]

On 06/07/2016 06:09 PM, Samuel Merritt wrote:
> On 6/7/16 12:00 PM, Monty Taylor wrote:
>> [snip]
> >
>> I'd rather see us focus energy on Python3, asyncio and its pluggable
>> event loops. The work in:
>>
>> http://magic.io/blog/uvloop-blazing-fast-python-networking/
>>
>> is a great indication in an actual apples-to-apples comparison of what
>> can be accomplished in python doing IO-bound activities by using modern
>> Python techniques. I think that comparing python2+eventlet to a fresh
>> rewrite in Go isn't 100% of the story. A TON of work has gone in to
>> Python that we're not taking advantage of because we're still supporting
>> Python2. So what I've love to see in the realm of comparative
>> experimentation is to see if the existing Python we already have can be
>> leveraged as we adopt newer and more modern things.
>
> Asyncio, eventlet, and other similar libraries are all very good for
> performing asynchronous IO on sockets and pipes. However, none of them
> help for filesystem IO. That's why Swift needs a golang object server:
> the go runtime will keep some goroutines running even though some
> other goroutines are performing filesystem IO, whereas filesystem IO
> in Python blocks the entire process, asyncio or no asyncio.

That can be modified. gevent has a tool
(http://www.gevent.org/gevent.fileobject.html) that enables the File IO
to be async  as well by putting the file into non-blocking mode. I've
used it, and it works and scales well.

Sadly, Python doesn't offer this by default; perhaps OpenStack can get
that changed.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Samuel Merritt]

On 6/7/16 12:00 PM, Monty Taylor wrote:

[snip]

>

I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.


Asyncio, eventlet, and other similar libraries are all very good for 
performing asynchronous IO on sockets and pipes. However, none of them 
help for filesystem IO. That's why Swift needs a golang object server: 
the go runtime will keep some goroutines running even though some other 
goroutines are performing filesystem IO, whereas filesystem IO in Python 
blocks the entire process, asyncio or no asyncio.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[James Penick]

>rather than making progress on OpenStack, we'll spend the next 4 years
bikeshedding broadly about which bits, if any, should be rewritten in Go.

100% agreed, and well said.


On Tue, Jun 7, 2016 at 12:00 PM, Monty Taylor  wrote:

> This text is in my vote, but as I'm sure there are people who do not
> read all of the gerrit comments for governance changes, I'm posting it
> here so that my thoughts are clear.
>
> Please know that this has actually kept me up at night. I cast my vote
> on this neither glibly or superficially. I have talked to everyone I can
> possibly think of on the topic, and at the end, the only thing I can do
> is use my judgment and vote to the best of my ability. I apologize from
> the bottom of my heart to the people I find myself in disagreement with.
> I have nothing but the utmost respect for you all.
>
> I vote against allowing Go as an official language of OpenStack.
>
> "The needs of the many outweigh the needs of the few, or the one"
>
> I'm super unhappy about both possible votes here.
>
> I think go is a wonderful language. I think hummingbird is a well
> considered solution to a particular problem. I think that lack of
> flexibility is broadly speaking not a problem we have in OpenStack
> currently. I'm more worried about community cohesion in a post-Big Tent
> world than I am about specific optimization.
>
> I do not think that adding Go as a language to OpenStack today is enough
> of a win to justify the cost, so I don't like accepting it.
>
> I do not think that this should preclude serious thought about
> OpenStack's technology underpinnings, so I don't like rejecting it.
>
> "only a great fool would reach for what he was given..."
>
> I think that one of OpenStack's biggest and most loudly spoken about
> problems is too many per-project solutions and not enough holistic
> solutions. Because of that, and the six years of experience we have
> seeing where that gets us, I do not think that adding Go into the mix
> and "seeing what happens" is going to cause anything other than chaos.
>
> If we want to add Go, or any other language, into the mix for server
> projects, I think it should be done with the intent that we are going to
> do it because it's a markedly better choice across the board, that we
> are going to rewrite literally everything, and I believe that we should
> consider the cost associated with retraining 2000 developers as part of
> considering that. Before you think that that's me throwing the baby out
> with the bathwater...
>
> In a previous comment, Deklan says:
>
> "If Go was accepted as an officially supported language in the OpenStack
> community, I'd be the first to start to rewrite as much code as possible
> in Go."
>
> That is, in fact, EXACTLY the concern. That rather than making progress
> on OpenStack, we'll spend the next 4 years bikeshedding broadly about
> which bits, if any, should be rewritten in Go. It took Juju YEARS to
> rewrite from Python to Go and to hit feature parity. The size of that
> codebase was much smaller and they even had a BDFL (which people keep
> telling us makes things go quicker)
>
> It could be argued that we could exercise consideration about which
> things get rewritten in Go so as to avoid that, but I'm pretty sure that
> would just mean that the only conversation the TC would have for the
> next two years would be "should X be in Go or Python" - and we'd have
> strong proponents from each project on each side of the argument.
>
> David Goetz says "you aren’t doing the community any favors by deciding
> for them how they do their jobs". I get that, and can respect that point
> of view. However, for the most part, the negative feedback we get as
> members of the TC is actually that we're too lax, not that we're too
> strict.
>
> I know that it's a popular sentiment with some folks to say "let devs
> use whatever tool they want to." However, that has never been our
> approach with OpenStack. It has been suggested multiple times and
> aligning on limited chosen tech has always been the thing we've chosen.
> I tend to align in my personal thinking more with Dan McKinley in:
>
> http://mcfunley.com/choose-boring-technology
>
> I have effectively been arguing his point for as long as I've been
> involved in OpenStack governance - although probably not as well as he
> does. I don't see any reason to reverse myself now.
>
> I'd rather see us focus energy on Python3, asyncio and its pluggable
> event loops. The work in:
>
> http://magic.io/blog/uvloop-blazing-fast-python-networking/
>
> is a great indication in an actual apples-to-apples comparison of what
> can be accomplished in python doing IO-bound activities by using modern
> Python techniques. I think that comparing python2+eventlet to a fresh
> rewrite in Go isn't 100% of the story. A TON of work has gone in to
> Python that we're not taking advantage of because we're still supporting
> Python2. So what I've love to see in the realm of 

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Amrith Kumar]

Well put Monty. It is a tough choice and neither choice was inexpensive.

-amrith 

> -Original Message-
> From: Monty Taylor [mailto:mord...@inaugust.com]
> Sent: Tuesday, June 07, 2016 3:00 PM
> To: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] Reasoning behind my vote on the Go topic
> 
> This text is in my vote, but as I'm sure there are people who do not
> read all of the gerrit comments for governance changes, I'm posting it
> here so that my thoughts are clear.
> 
> Please know that this has actually kept me up at night. I cast my vote
> on this neither glibly or superficially. I have talked to everyone I can
> possibly think of on the topic, and at the end, the only thing I can do
> is use my judgment and vote to the best of my ability. I apologize from
> the bottom of my heart to the people I find myself in disagreement with.
> I have nothing but the utmost respect for you all.
> 
> I vote against allowing Go as an official language of OpenStack.
> 
> "The needs of the many outweigh the needs of the few, or the one"
> 
> I'm super unhappy about both possible votes here.
> 
> I think go is a wonderful language. I think hummingbird is a well
> considered solution to a particular problem. I think that lack of
> flexibility is broadly speaking not a problem we have in OpenStack
> currently. I'm more worried about community cohesion in a post-Big Tent
> world than I am about specific optimization.
> 
> I do not think that adding Go as a language to OpenStack today is enough
> of a win to justify the cost, so I don't like accepting it.
> 
> I do not think that this should preclude serious thought about
> OpenStack's technology underpinnings, so I don't like rejecting it.
> 
> "only a great fool would reach for what he was given..."
> 
> I think that one of OpenStack's biggest and most loudly spoken about
> problems is too many per-project solutions and not enough holistic
> solutions. Because of that, and the six years of experience we have
> seeing where that gets us, I do not think that adding Go into the mix
> and "seeing what happens" is going to cause anything other than chaos.
> 
> If we want to add Go, or any other language, into the mix for server
> projects, I think it should be done with the intent that we are going to
> do it because it's a markedly better choice across the board, that we
> are going to rewrite literally everything, and I believe that we should
> consider the cost associated with retraining 2000 developers as part of
> considering that. Before you think that that's me throwing the baby out
> with the bathwater...
> 
> In a previous comment, Deklan says:
> 
> "If Go was accepted as an officially supported language in the OpenStack
> community, I'd be the first to start to rewrite as much code as possible
> in Go."
> 
> That is, in fact, EXACTLY the concern. That rather than making progress
> on OpenStack, we'll spend the next 4 years bikeshedding broadly about
> which bits, if any, should be rewritten in Go. It took Juju YEARS to
> rewrite from Python to Go and to hit feature parity. The size of that
> codebase was much smaller and they even had a BDFL (which people keep
> telling us makes things go quicker)
> 
> It could be argued that we could exercise consideration about which
> things get rewritten in Go so as to avoid that, but I'm pretty sure that
> would just mean that the only conversation the TC would have for the
> next two years would be "should X be in Go or Python" - and we'd have
> strong proponents from each project on each side of the argument.
> 
> David Goetz says "you aren’t doing the community any favors by deciding
> for them how they do their jobs". I get that, and can respect that point
> of view. However, for the most part, the negative feedback we get as
> members of the TC is actually that we're too lax, not that we're too
> strict.
> 
> I know that it's a popular sentiment with some folks to say "let devs
> use whatever tool they want to." However, that has never been our
> approach with OpenStack. It has been suggested multiple times and
> aligning on limited chosen tech has always been the thing we've chosen.
> I tend to align in my personal thinking more with Dan McKinley in:
> 
> http://mcfunley.com/choose-boring-technology
> 
> I have effectively been arguing his point for as long as I've been
> involved in OpenStack governance - although probably not as well as he
> does. I don't see any reason to reverse myself now.
> 
> I'd rather see us focus energy on Python3, asyncio and its pluggable
> event loops. The work 

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Jay Pipes]

Well said and reasoned, Monty.

On 06/07/2016 03:00 PM, Monty Taylor wrote:

This text is in my vote, but as I'm sure there are people who do not
read all of the gerrit comments for governance changes, I'm posting it
here so that my thoughts are clear.

Please know that this has actually kept me up at night. I cast my vote
on this neither glibly or superficially. I have talked to everyone I can
possibly think of on the topic, and at the end, the only thing I can do
is use my judgment and vote to the best of my ability. I apologize from
the bottom of my heart to the people I find myself in disagreement with.
I have nothing but the utmost respect for you all.

I vote against allowing Go as an official language of OpenStack.

"The needs of the many outweigh the needs of the few, or the one"

I'm super unhappy about both possible votes here.

I think go is a wonderful language. I think hummingbird is a well
considered solution to a particular problem. I think that lack of
flexibility is broadly speaking not a problem we have in OpenStack
currently. I'm more worried about community cohesion in a post-Big Tent
world than I am about specific optimization.

I do not think that adding Go as a language to OpenStack today is enough
of a win to justify the cost, so I don't like accepting it.

I do not think that this should preclude serious thought about
OpenStack's technology underpinnings, so I don't like rejecting it.

"only a great fool would reach for what he was given..."

I think that one of OpenStack's biggest and most loudly spoken about
problems is too many per-project solutions and not enough holistic
solutions. Because of that, and the six years of experience we have
seeing where that gets us, I do not think that adding Go into the mix
and "seeing what happens" is going to cause anything other than chaos.

If we want to add Go, or any other language, into the mix for server
projects, I think it should be done with the intent that we are going to
do it because it's a markedly better choice across the board, that we
are going to rewrite literally everything, and I believe that we should
consider the cost associated with retraining 2000 developers as part of
considering that. Before you think that that's me throwing the baby out
with the bathwater...

In a previous comment, Deklan says:

"If Go was accepted as an officially supported language in the OpenStack
community, I'd be the first to start to rewrite as much code as possible
in Go."

That is, in fact, EXACTLY the concern. That rather than making progress
on OpenStack, we'll spend the next 4 years bikeshedding broadly about
which bits, if any, should be rewritten in Go. It took Juju YEARS to
rewrite from Python to Go and to hit feature parity. The size of that
codebase was much smaller and they even had a BDFL (which people keep
telling us makes things go quicker)

It could be argued that we could exercise consideration about which
things get rewritten in Go so as to avoid that, but I'm pretty sure that
would just mean that the only conversation the TC would have for the
next two years would be "should X be in Go or Python" - and we'd have
strong proponents from each project on each side of the argument.

David Goetz says "you aren’t doing the community any favors by deciding
for them how they do their jobs". I get that, and can respect that point
of view. However, for the most part, the negative feedback we get as
members of the TC is actually that we're too lax, not that we're too strict.

I know that it's a popular sentiment with some folks to say "let devs
use whatever tool they want to." However, that has never been our
approach with OpenStack. It has been suggested multiple times and
aligning on limited chosen tech has always been the thing we've chosen.
I tend to align in my personal thinking more with Dan McKinley in:

http://mcfunley.com/choose-boring-technology

I have effectively been arguing his point for as long as I've been
involved in OpenStack governance - although probably not as well as he
does. I don't see any reason to reverse myself now.

I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.

In summary, while I think that Go is a lovely language and the people
who work on it are lovely people, while I'm sure that hummingbird is
beneficial to the Cloud Files team in real ways and while I'm sure that

[openstack-dev] Reasoning behind my vote on the Go topic

[Monty Taylor]

This text is in my vote, but as I'm sure there are people who do not
read all of the gerrit comments for governance changes, I'm posting it
here so that my thoughts are clear.

Please know that this has actually kept me up at night. I cast my vote
on this neither glibly or superficially. I have talked to everyone I can
possibly think of on the topic, and at the end, the only thing I can do
is use my judgment and vote to the best of my ability. I apologize from
the bottom of my heart to the people I find myself in disagreement with.
I have nothing but the utmost respect for you all.

I vote against allowing Go as an official language of OpenStack.

"The needs of the many outweigh the needs of the few, or the one"

I'm super unhappy about both possible votes here.

I think go is a wonderful language. I think hummingbird is a well
considered solution to a particular problem. I think that lack of
flexibility is broadly speaking not a problem we have in OpenStack
currently. I'm more worried about community cohesion in a post-Big Tent
world than I am about specific optimization.

I do not think that adding Go as a language to OpenStack today is enough
of a win to justify the cost, so I don't like accepting it.

I do not think that this should preclude serious thought about
OpenStack's technology underpinnings, so I don't like rejecting it.

"only a great fool would reach for what he was given..."

I think that one of OpenStack's biggest and most loudly spoken about
problems is too many per-project solutions and not enough holistic
solutions. Because of that, and the six years of experience we have
seeing where that gets us, I do not think that adding Go into the mix
and "seeing what happens" is going to cause anything other than chaos.

If we want to add Go, or any other language, into the mix for server
projects, I think it should be done with the intent that we are going to
do it because it's a markedly better choice across the board, that we
are going to rewrite literally everything, and I believe that we should
consider the cost associated with retraining 2000 developers as part of
considering that. Before you think that that's me throwing the baby out
with the bathwater...

In a previous comment, Deklan says:

"If Go was accepted as an officially supported language in the OpenStack
community, I'd be the first to start to rewrite as much code as possible
in Go."

That is, in fact, EXACTLY the concern. That rather than making progress
on OpenStack, we'll spend the next 4 years bikeshedding broadly about
which bits, if any, should be rewritten in Go. It took Juju YEARS to
rewrite from Python to Go and to hit feature parity. The size of that
codebase was much smaller and they even had a BDFL (which people keep
telling us makes things go quicker)

It could be argued that we could exercise consideration about which
things get rewritten in Go so as to avoid that, but I'm pretty sure that
would just mean that the only conversation the TC would have for the
next two years would be "should X be in Go or Python" - and we'd have
strong proponents from each project on each side of the argument.

David Goetz says "you aren’t doing the community any favors by deciding
for them how they do their jobs". I get that, and can respect that point
of view. However, for the most part, the negative feedback we get as
members of the TC is actually that we're too lax, not that we're too strict.

I know that it's a popular sentiment with some folks to say "let devs
use whatever tool they want to." However, that has never been our
approach with OpenStack. It has been suggested multiple times and
aligning on limited chosen tech has always been the thing we've chosen.
I tend to align in my personal thinking more with Dan McKinley in:

http://mcfunley.com/choose-boring-technology

I have effectively been arguing his point for as long as I've been
involved in OpenStack governance - although probably not as well as he
does. I don't see any reason to reverse myself now.

I'd rather see us focus energy on Python3, asyncio and its pluggable
event loops. The work in:

http://magic.io/blog/uvloop-blazing-fast-python-networking/

is a great indication in an actual apples-to-apples comparison of what
can be accomplished in python doing IO-bound activities by using modern
Python techniques. I think that comparing python2+eventlet to a fresh
rewrite in Go isn't 100% of the story. A TON of work has gone in to
Python that we're not taking advantage of because we're still supporting
Python2. So what I've love to see in the realm of comparative
experimentation is to see if the existing Python we already have can be
leveraged as we adopt newer and more modern things.

In summary, while I think that Go is a lovely language and the people
who work on it are lovely people, while I'm sure that hummingbird is
beneficial to the Cloud Files team in real ways and while I'm sure that
if we were starting OpenStack from scratch today the conversations about
how to