subject:"Re\: \[openstack\-dev\] \[tc\] supporting Go"

Re: [openstack-dev] [tc] supporting Go

2016-05-20 Thread Clint Byrum

Excerpts from Thomas Goirand's message of 2016-05-20 12:42:09 +0200:
> On 05/11/2016 04:17 PM, Dean Troyer wrote:
> > The big difference with Go here is that the dependency work happens at
> > build time, not deploy/runtime in most cases.  That shifts much of the
> > burden to people (theoretically) better suited to manage that work.
> 
> I am *NOT* buying that doing static linking is a progress. We're back 30
> years in the past, before the .so format. It is amazing that some of us
> think it's better. It simply isn't. It's a huge regression, for package
> maintainers, system admins, production/ops, and our final users. The
> only group of people who like it are developers, because they just don't
> need to care about shared library API/ABI incompatibilities and
> regressions anymore.
> 

Static linking is _a_ model. Dynamic linking is _a_ model.

There are aspects of each model, and when you lay different values over
any model, it will appear better or worse depending on those values.

Debian values slow, massively reusable change. Everyone advances at
around the same pace, and as a result, the whole community has a net
positive improvement. This is fantastically useful and is not outdated
in any way IMO. I love my stable OS.

But there is more software being written now than ever before, and that
growth does not have a downward curve. As people write more software,
and the demands on them get more intense, they have less reasons to
reuse a wider set of libraries, and have more of a need to reuse a
narrow subset, in a specific way. This gives rise to the continuous
delivery model where one ships a narrow subset all together and tests it
deeply, rather than testing the broader tools in isolation. That means
sometimes they go faster than the rest of the community in one area,
and slower in others. They give up the broad long term efficiency for
short term agility.

That may sound crass, like it's just fast and loose with no regard for the
future. But without the agility, they will just get run over by somebody
else more agile. When somebody chooses this, they're choosing it because
they have to, not because they don't understand what they're giving up.

Whichever model is chosen, It doesn't mean one doesn't care about the
greater community. It simply means one has a set of challenges when
contributing along side those with conflicting values.

But it's not a regression, it is simply people with a different set of
values, finding the same old solutions useful again for different reasons.

So, I'd urge that we all seek to find some empathy with people in other
positions, and compromise when we can. Debian has done so already,
with a Built-Using helper now for go programs. When libraries update,
one can just rebuild the packages that are built using it. So, rather
than fighting this community, Debian seems to be embracing it. Only
tradition stands in the way of harmony in this case.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-20 Thread Fox, Kevin M

Containers are popular right now for the same reason statically linking is. But 
it gives you something half way in between. Static linking is really hard to do 
right. Even go's "static link all the things" is incomplete. if you run ldd on 
a go binary, it isn't actually static. Containers ensure the stuff inside 
really is statically linked.

So these days, I'd rather use docker for gaining the static linking like 
functionality rather then try really hard to get a single ".exe" that has 0 
dependencies. Its very hard.

That being said, the distro's have a big point about deps being handled 
dynamically so that the problem of vulnerability/bug patching is tractable. At 
the moment, Docker or any statically linking thing doesn't have a good solution 
to the problem. The importance of this issue can't be understated.

It can be fixed, but there isn't a project I'm aware of yet to commonly solve 
it.

Building containers using distro packages I think is a part of the solution. 
You can then much more easily find which containers are out of date/have 
vulnerabilities. but you still need machinery to scan for updates, rebuild 
containers, and a way to push them out to all the places that need updating.

Until those things are solved, or you have operators that very very carefully 
pay attention, traditional distro's have a strong thing going for them in the 
security realm.

Go's statically linking everything is an anti-feature in my opinion, better 
left to systems like Docker.

Thanks,
Kevin


From: Dean Troyer [dtro...@gmail.com]
Sent: Friday, May 20, 2016 5:48 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [tc] supporting Go

On Fri, May 20, 2016 at 5:42 AM, Thomas Goirand 
<z...@debian.org<mailto:z...@debian.org>> wrote:
I am *NOT* buying that doing static linking is a progress. We're back 30
years in the past, before the .so format. It is amazing that some of us
think it's better. It simply isn't. It's a huge regression, for package
maintainers, system admins, production/ops, and our final users. The
only group of people who like it are developers, because they just don't
need to care about shared library API/ABI incompatibilities and
regressions anymore.

I disagree, there are certainly places static linking is appropriate, however, 
I didn't mention that at all.  Much of the burden with Python dependency at 
install/run time is due to NO linking.  Even with C, you make choices at build 
time WRT what you link against, either statically or dynamically.  Even with 
shared libs, when the interface changes you have to re-link everything that 
uses that interface.  It is not as black and white as you suggest.

And I say that as a user, who so desperately wants an install process for OSC 
to match PuTTY on Windows: 1) copy an .exe; 2) run it.

dt

[Thomas, I have done _EVERY_ one of the jobs above that you listed, as a 
$DAY_JOB, and know exactly what it takes to run production-scale services built 
from everything from vendor packages to house-built source.  It would be nice 
if you refined your argument to stop leaning on static linking as the biggest 
problem since stack overflows.  There are other reasons this might be a bad 
idea, but I sense that you are losing traction fixating on only this one.]

--

Dean Troyer
dtro...@gmail.com<mailto:dtro...@gmail.com>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-20 Thread Adam Young


On 05/20/2016 08:48 AM, Dean Troyer wrote:
On Fri, May 20, 2016 at 5:42 AM, Thomas Goirand > wrote:


I am *NOT* buying that doing static linking is a progress. We're
back 30
years in the past, before the .so format. It is amazing that some
of us
think it's better. It simply isn't. It's a huge regression, for
package
maintainers, system admins, production/ops, and our final users. The
only group of people who like it are developers, because they just
don't
need to care about shared library API/ABI incompatibilities and
regressions anymore.


I disagree, there are certainly places static linking is appropriate, 
however, I didn't mention that at all. Much of the burden with Python 
dependency at install/run time is due to NO linking.  Even with C, you 
make choices at build time WRT what you link against, either 
statically or dynamically.  Even with shared libs, when the interface 
changes you have to re-link everything that uses that interface.  It 
is not as black and white as you suggest.


And I say that as a user, who so desperately wants an install process 
for OSC to match PuTTY on Windows: 1) copy an .exe; 2) run it.


dt

[Thomas, I have done _EVERY_ one of the jobs above that you listed, as 
a $DAY_JOB, and know exactly what it takes to run production-scale 
services built from everything from vendor packages to house-built 
source.  It would be nice if you refined your argument to stop leaning 
on static linking as the biggest problem since stack overflows.  There 
are other reasons this might be a bad idea, but I sense that you are 
losing traction fixating on only this one.]


Static linking Bad.  We can debate why elsewhere.

Go with dynamic linking is possible, and should be what the 
distributions target. This is a solvable problem.


/me burns bikeshed and installs a Hubcycle/Citibike kiosk.




--

Dean Troyer
dtro...@gmail.com 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-20 Thread Dean Troyer

On Fri, May 20, 2016 at 5:42 AM, Thomas Goirand  wrote:

> I am *NOT* buying that doing static linking is a progress. We're back 30
> years in the past, before the .so format. It is amazing that some of us
> think it's better. It simply isn't. It's a huge regression, for package
> maintainers, system admins, production/ops, and our final users. The
> only group of people who like it are developers, because they just don't
> need to care about shared library API/ABI incompatibilities and
> regressions anymore.
>

I disagree, there are certainly places static linking is appropriate,
however, I didn't mention that at all.  Much of the burden with Python
dependency at install/run time is due to NO linking.  Even with C, you make
choices at build time WRT what you link against, either statically or
dynamically.  Even with shared libs, when the interface changes you have to
re-link everything that uses that interface.  It is not as black and white
as you suggest.

And I say that as a user, who so desperately wants an install process for
OSC to match PuTTY on Windows: 1) copy an .exe; 2) run it.

dt

[Thomas, I have done _EVERY_ one of the jobs above that you listed, as a
$DAY_JOB, and know exactly what it takes to run production-scale services
built from everything from vendor packages to house-built source.  It would
be nice if you refined your argument to stop leaning on static linking as
the biggest problem since stack overflows.  There are other reasons this
might be a bad idea, but I sense that you are losing traction fixating on
only this one.]

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-20 Thread Thomas Goirand

On 05/11/2016 04:17 PM, Dean Troyer wrote:
> The big difference with Go here is that the dependency work happens at
> build time, not deploy/runtime in most cases.  That shifts much of the
> burden to people (theoretically) better suited to manage that work.

I am *NOT* buying that doing static linking is a progress. We're back 30
years in the past, before the .so format. It is amazing that some of us
think it's better. It simply isn't. It's a huge regression, for package
maintainers, system admins, production/ops, and our final users. The
only group of people who like it are developers, because they just don't
need to care about shared library API/ABI incompatibilities and
regressions anymore.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-19 Thread Ben Meyer

On 05/18/2016 06:42 PM, Eric Larson wrote:
>
> Dmitry Tantsur writes:
>
>> This is pretty subjective, I would say. I personally don't feel Go
>> (especially its approach to error handling) any natural (at least no
>> more than Rust or Scala, for example). If familiarity for Python
>> developers is an argument here, mastering Cython or making OpenStack
>> run on PyPy must be much easier for a random Python developer out
>> there to seriously bump the performance. And it would not require
>> introducing a completely new language to the picture.
>
> In one sense you are correct. It is easier for a Pythonista to pick up
> Cython and use that for performance specific areas of code. At the
> same time, I'd argue that OpenStack as a community is not the same as
> Python at large. There are packaging requirements and cross project
> standards that also come into play, not to mention operators that end
> up bearing the brunt of those decisions. For example, Debian will
> likely not package a PyPy only version of Designate along with all its
> requirements. Similarly, while 50% of operators use packaged
> versioned, that means 50% work from source control to build, test, and
> release OpenStack projects.
>
> You are correct that my position is subjective, but it is based on my
> experiences trying to operate and deploy OpenStack in addition to
> writing code. The draw of Go, in my experience, has been easily
> deploying a single binary I've been able to build and test
> consistently. The target system has doesn't require Go installed at
> all and it works on old distros. And it has been much faster.
True as long as your Kernel is new enough.

There are some issues with Go on older kernels, so you can only go so
far back.

For a product I'm working on (outside of OpenStack) I chose Go for an
updater so we could move forward against Python3 (our Python2-based
updater wasn't portable to Python-3, and no hope of making it so).
I had kept it simple, but after I finished discovered Go didn't
officially support CentOS/RHEL5 due to kernel issues - example
https://github.com/golang/go/issues/8445 (closed as "unfortunate"), see
also http://dave.cheney.net/2013/06/18/how-to-install-go-1-1-on-centos-5
which *might* help get things working, but is not guaranteed.

That said, my CentOS/RHEL binary was built on CentOS6, it ran on CentOS5
but I also didn't do anything very spectacular - just shell calls and an
IPC for the daemon, all very simple and not timing related. Something
more complex, like what has been discussed in this thread, will likely
run into a lot more issues.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-19 Thread Dmitry Tantsur


On 05/19/2016 12:58 PM, Robert Collins wrote:

On 19 May 2016 at 22:40, Dmitry Tantsur  wrote:



You are correct that my position is subjective, but it is based on my
experiences trying to operate and deploy OpenStack in addition to
writing code. The draw of Go, in my experience, has been easily
deploying a single binary I've been able to build and test consistently.
The target system has doesn't require Go installed at all and it works
on old distros. And it has been much faster.



.. this is something distributions would never do or encourage. Ask zigo for
reasons :)


Distros are having a hard time at the moment :) - much of their
/obvious/ value is no longer sought: for instance compile time is
cheap enough that folk are rebuilding distros just to check that the
binaries are actually from the same sources!

Further, the historical squashing of all dependencies into one version
becomes increasing fragile as dependency chains get larger: the
probability of a bug preventing a library being updated (best case -
caught by CI) or breaking something without warning (typical case,
little-to-no-CI of transitive reverse deps) goes up, not down. This is
one of the major reasons folks doing operations often bypass distro
packages (or roll their own isolated set with known-good
dependencies).

The idea of trusted-collections-of-packages made a lot more sense
before this explosion of software we have now, much of which is high
quality, and moving much much faster than distro release
cycles.Canonical/Ubuntu has at least partly got its head around this
with their focus for the last while on a vibrant app store ecosystem -
one where shipping a single binary with vendored, static dependencies
is actually viable.

So yeah, some distros are getting there, bit by bit :)


It's already an off-topic, but I don't believe in this model personally. 
Especially since how many times such models appeared already without 
apparent instant success. But let us see :)




-Rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-19 Thread Robert Collins

On 19 May 2016 at 22:40, Dmitry Tantsur  wrote:
>
>> You are correct that my position is subjective, but it is based on my
>> experiences trying to operate and deploy OpenStack in addition to
>> writing code. The draw of Go, in my experience, has been easily
>> deploying a single binary I've been able to build and test consistently.
>> The target system has doesn't require Go installed at all and it works
>> on old distros. And it has been much faster.
>
>
> .. this is something distributions would never do or encourage. Ask zigo for
> reasons :)

Distros are having a hard time at the moment :) - much of their
/obvious/ value is no longer sought: for instance compile time is
cheap enough that folk are rebuilding distros just to check that the
binaries are actually from the same sources!

Further, the historical squashing of all dependencies into one version
becomes increasing fragile as dependency chains get larger: the
probability of a bug preventing a library being updated (best case -
caught by CI) or breaking something without warning (typical case,
little-to-no-CI of transitive reverse deps) goes up, not down. This is
one of the major reasons folks doing operations often bypass distro
packages (or roll their own isolated set with known-good
dependencies).

The idea of trusted-collections-of-packages made a lot more sense
before this explosion of software we have now, much of which is high
quality, and moving much much faster than distro release
cycles.Canonical/Ubuntu has at least partly got its head around this
with their focus for the last while on a vibrant app store ecosystem -
one where shipping a single binary with vendored, static dependencies
is actually viable.

So yeah, some distros are getting there, bit by bit :)

-Rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-19 Thread Dmitry Tantsur


On 05/19/2016 12:42 AM, Eric Larson wrote:


Dmitry Tantsur writes:


This is pretty subjective, I would say. I personally don't feel Go
(especially its approach to error handling) any natural (at least no
more than Rust or Scala, for example). If familiarity for Python
developers is an argument here, mastering Cython or making OpenStack
run on PyPy must be much easier for a random Python developer out
there to seriously bump the performance. And it would not require
introducing a completely new language to the picture.


In one sense you are correct. It is easier for a Pythonista to pick up
Cython and use that for performance specific areas of code. At the same
time, I'd argue that OpenStack as a community is not the same as Python
at large. There are packaging requirements and cross project standards
that also come into play, not to mention operators that end up bearing
the brunt of those decisions. For example, Debian will likely not
package a PyPy only version of Designate along with all its
requirements. Similarly, while 50% of operators use packaged versioned,
that means 50% work from source control to build, test, and release
OpenStack projects.


Here you speak about distributions and packaging, and I rather agree 
with you, but then...




You are correct that my position is subjective, but it is based on my
experiences trying to operate and deploy OpenStack in addition to
writing code. The draw of Go, in my experience, has been easily
deploying a single binary I've been able to build and test consistently.
The target system has doesn't require Go installed at all and it works
on old distros. And it has been much faster.


.. this is something distributions would never do or encourage. Ask zigo 
for reasons :)




Coming from Python, the reason Go has been easy to get started with is
that it offers some protections that are useful such as memory
management. Features such as slices are extremely similar to Python and
go routines / channels allow supporting more complex patterns such as
generators. Yes, you are correct, error handling is controversial, but
at the same time, it is no better in C.

I'm not an expert in Go, but from what I've seen, Go has been easier to
build and deploy than Python, while being faster. Picking it up has been
trivial and becoming reasonably proficient has been a quick process.
When considered within the scope of OpenStack, it adds a minimal
overhead for testing, packaging and deployment, especially when compared
to C extensions, PyPy or Cython.

I hope that contextualizes my opinion a bit to make clear the subjective
aspects are based on OpenStack specific constraints.

--

Eric Larson | eric.lar...@rackspace.com Software Developer |
Cloud DNS | OpenStack Designate Rackspace Hosting   | Austin, Texas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-18 Thread Eric Larson



Dmitry Tantsur writes:

This is pretty subjective, I would say. I personally don't feel 
Go (especially its approach to error handling) any natural (at 
least no more than Rust or Scala, for example). If familiarity 
for Python developers is an argument here, mastering Cython or 
making OpenStack run on PyPy must be much easier for a random 
Python developer out there to seriously bump the 
performance. And it would not require introducing a completely 
new language to the picture.


In one sense you are correct. It is easier for a Pythonista to 
pick up Cython and use that for performance specific areas of 
code. At the same time, I'd argue that OpenStack as a community is 
not the same as Python at large. There are packaging requirements 
and cross project standards that also come into play, not to 
mention operators that end up bearing the brunt of those 
decisions. For example, Debian will likely not package a PyPy only 
version of Designate along with all its requirements. Similarly, 
while 50% of operators use packaged versioned, that means 50% work 
from source control to build, test, and release OpenStack 
projects.


You are correct that my position is subjective, but it is based on 
my experiences trying to operate and deploy OpenStack in addition 
to writing code. The draw of Go, in my experience, has been easily 
deploying a single binary I've been able to build and test 
consistently. The target system has doesn't require Go installed 
at all and it works on old distros. And it has been much faster.


Coming from Python, the reason Go has been easy to get started 
with is that it offers some protections that are useful such as 
memory management. Features such as slices are extremely similar 
to Python and go routines / channels allow supporting more complex 
patterns such as generators. Yes, you are correct, error handling 
is controversial, but at the same time, it is no better in C.


I'm not an expert in Go, but from what I've seen, Go has been 
easier to build and deploy than Python, while being 
faster. Picking it up has been trivial and becoming reasonably 
proficient has been a quick process. When considered within the 
scope of OpenStack, it adds a minimal overhead for testing, 
packaging and deployment, especially when compared to C 
extensions, PyPy or Cython.


I hope that contextualizes my opinion a bit to make clear the 
subjective aspects are based on OpenStack specific constraints.


--

Eric Larson | eric.lar...@rackspace.com Software Developer 
| Cloud DNS | OpenStack Designate Rackspace Hosting   | Austin, 
Texas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread Dmitry Tantsur


On 05/16/2016 05:21 PM, John Dickinson wrote:



On 16 May 2016, at 8:14, Dmitry Tantsur wrote:


On 05/16/2016 05:09 PM, Ian Cordasco wrote:



-Original Message-
From: Dmitry Tantsur <dtant...@redhat.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 16, 2016 at 09:55:27
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc] supporting Go


On 05/16/2016 04:35 PM, Adam Young wrote:

On 05/16/2016 05:23 AM, Dmitry Tantsur wrote:

On 05/14/2016 03:00 AM, Adam Young wrote:

On 05/13/2016 08:21 PM, Dieterly, Deklan wrote:

If we allow Go, then we should also consider allowing JVM based
languages.

Nope. Don't get me wrong, I've written more than my fair share of Java
in my career, and I like it, and I miss automated refactoring and real
threads. I have nothing against Java (I know a lot of you do).

Java fills the same niche as Python. We already have one of those, and
its very nice (according to John Cleese).


A couple of folks in this thread already stated that the primary
reason to switch from Python-based languages is the concurrency story.
JVM solves it and does it in the same manner as Go (at least that's my
assumption).

(not advocating for JVM, just trying to understand the objection)



So, what I think we are really saying here is "what is our Native
extension story going to be? Is it the traditional native languages, or
is it something new that has learned from them?"

Go is a complement to Python to fill in the native stuff. The
alternative is C or C++. Ok Flapper, or Rust.


C, C++, Rust, yes, I'd call them "native".

A language with a GC and green threads does not fall into "native"
category for me, rather the same as JVM.


MOre complex than just that. But Go does not have a VM, just put a lot
of effort into co-routines without taking context switches. Different
than green threads.


Ok, I think we have a different notion of "native" here. For me it's
being with as little magic happening behind the scenes as possible.


Have you written a lot of Rust?


Not a lot, but definitely some.


Rust handles the memory management for you as well. Certainly, you can 
determine the lifetime of something and tell the compiler how the underlying 
memory is shared, but Rust is far better than C in so much as you should never 
be able to write code that doubly frees the same memory unless you're 
explicitly using the unsafe features of the language that are infrequently 
needed.


I think we're in agreement here, not sure which bit you're arguing against :)



I'm with Flavio about preferring Rust personally, but I'm not a member of 
either of these teams and I understand the fact that most of the code is 
already written and has been shown to drastically improve performance in 
exactly the places where it's needed. With all of that in mind, I'm in favor of 
just agreeing already that Go is okay. I understand the concern that this will 
increase cognitive load on some developers and *might* have effects on our 
larger community but our community can only grow so long as our software is 
usable (performant) and useful (satisfies needs/requirements).


This Rust discussion is a bit offtopic, I was just stating that my notion of 
"nativeness" of Go is closer to one of JVM, not one of C/C++/Rust.

I guess my main question is whether folks seriously considered PyPy/Cython/etc.


Yes.

http://lists.openstack.org/pipermail/openstack-dev/2016-May/094960.html (which 
also references 
http://lists.openstack.org/pipermail/openstack-dev/2016-May/094549.html and 
http://lists.openstack.org/pipermail/openstack-dev/2016-May/094720.html


Thanks! Yeah, I've read the 1st, but somehow missed the other two, which 
actually provide a lot of interesting insights on how stuff works :) In 
this thread I was under a wrong impression, that people are hunting for 
running as "native" code as possible, while actually smart handling of 
IO operations is the critical point. I guess Go is really the best 
solution in this case.




--John




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread John Dickinson



On 16 May 2016, at 8:14, Dmitry Tantsur wrote:

> On 05/16/2016 05:09 PM, Ian Cordasco wrote:
>>
>>
>> -Original Message-
>> From: Dmitry Tantsur <dtant...@redhat.com>
>> Reply: OpenStack Development Mailing List (not for usage questions) 
>> <openstack-dev@lists.openstack.org>
>> Date: May 16, 2016 at 09:55:27
>> To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
>> Subject:  Re: [openstack-dev] [tc] supporting Go
>>
>>> On 05/16/2016 04:35 PM, Adam Young wrote:
>>>> On 05/16/2016 05:23 AM, Dmitry Tantsur wrote:
>>>>> On 05/14/2016 03:00 AM, Adam Young wrote:
>>>>>> On 05/13/2016 08:21 PM, Dieterly, Deklan wrote:
>>>>>>> If we allow Go, then we should also consider allowing JVM based
>>>>>>> languages.
>>>>>> Nope. Don't get me wrong, I've written more than my fair share of Java
>>>>>> in my career, and I like it, and I miss automated refactoring and real
>>>>>> threads. I have nothing against Java (I know a lot of you do).
>>>>>>
>>>>>> Java fills the same niche as Python. We already have one of those, and
>>>>>> its very nice (according to John Cleese).
>>>>>
>>>>> A couple of folks in this thread already stated that the primary
>>>>> reason to switch from Python-based languages is the concurrency story.
>>>>> JVM solves it and does it in the same manner as Go (at least that's my
>>>>> assumption).
>>>>>
>>>>> (not advocating for JVM, just trying to understand the objection)
>>>>>
>>>>>>
>>>>>> So, what I think we are really saying here is "what is our Native
>>>>>> extension story going to be? Is it the traditional native languages, or
>>>>>> is it something new that has learned from them?"
>>>>>>
>>>>>> Go is a complement to Python to fill in the native stuff. The
>>>>>> alternative is C or C++. Ok Flapper, or Rust.
>>>>>
>>>>> C, C++, Rust, yes, I'd call them "native".
>>>>>
>>>>> A language with a GC and green threads does not fall into "native"
>>>>> category for me, rather the same as JVM.
>>>>
>>>> MOre complex than just that. But Go does not have a VM, just put a lot
>>>> of effort into co-routines without taking context switches. Different
>>>> than green threads.
>>>
>>> Ok, I think we have a different notion of "native" here. For me it's
>>> being with as little magic happening behind the scenes as possible.
>>
>> Have you written a lot of Rust?
>
> Not a lot, but definitely some.
>
>> Rust handles the memory management for you as well. Certainly, you can 
>> determine the lifetime of something and tell the compiler how the underlying 
>> memory is shared, but Rust is far better than C in so much as you should 
>> never be able to write code that doubly frees the same memory unless you're 
>> explicitly using the unsafe features of the language that are infrequently 
>> needed.
>
> I think we're in agreement here, not sure which bit you're arguing against :)
>
>>
>> I'm with Flavio about preferring Rust personally, but I'm not a member of 
>> either of these teams and I understand the fact that most of the code is 
>> already written and has been shown to drastically improve performance in 
>> exactly the places where it's needed. With all of that in mind, I'm in favor 
>> of just agreeing already that Go is okay. I understand the concern that this 
>> will increase cognitive load on some developers and *might* have effects on 
>> our larger community but our community can only grow so long as our software 
>> is usable (performant) and useful (satisfies needs/requirements).
>
> This Rust discussion is a bit offtopic, I was just stating that my notion of 
> "nativeness" of Go is closer to one of JVM, not one of C/C++/Rust.
>
> I guess my main question is whether folks seriously considered 
> PyPy/Cython/etc.

Yes.

http://lists.openstack.org/pipermail/openstack-dev/2016-May/094960.html (which 
also references 
http://lists.openstack.org/pipermail/openstack-dev/2016-May/094549.html and 
http://lists.openstack.org/pipermail/openstack-dev/2016-May/094720.html

--John




signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread Dmitry Tantsur

On 05/16/2016 05:09 PM, Ian Cordasco wrote:

-Original Message-
From: Dmitry Tantsur <dtant...@redhat.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 16, 2016 at 09:55:27
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc] supporting Go

On 05/16/2016 04:35 PM, Adam Young wrote:

On 05/16/2016 05:23 AM, Dmitry Tantsur wrote:

On 05/14/2016 03:00 AM, Adam Young wrote:

On 05/13/2016 08:21 PM, Dieterly, Deklan wrote:

If we allow Go, then we should also consider allowing JVM based
languages.

Nope. Don't get me wrong, I've written more than my fair share of Java
in my career, and I like it, and I miss automated refactoring and real
threads. I have nothing against Java (I know a lot of you do).

Java fills the same niche as Python. We already have one of those, and
its very nice (according to John Cleese).

A couple of folks in this thread already stated that the primary
reason to switch from Python-based languages is the concurrency story.
JVM solves it and does it in the same manner as Go (at least that's my
assumption).

(not advocating for JVM, just trying to understand the objection)

So, what I think we are really saying here is "what is our Native
extension story going to be? Is it the traditional native languages, or
is it something new that has learned from them?"

Go is a complement to Python to fill in the native stuff. The
alternative is C or C++. Ok Flapper, or Rust.

C, C++, Rust, yes, I'd call them "native".

A language with a GC and green threads does not fall into "native"
category for me, rather the same as JVM.

MOre complex than just that. But Go does not have a VM, just put a lot
of effort into co-routines without taking context switches. Different
than green threads.

Ok, I think we have a different notion of "native" here. For me it's
being with as little magic happening behind the scenes as possible.

Have you written a lot of Rust?

Not a lot, but definitely some.

Rust handles the memory management for you as well. Certainly, you can 
determine the lifetime of something and tell the compiler how the underlying 
memory is shared, but Rust is far better than C in so much as you should never 
be able to write code that doubly frees the same memory unless you're 
explicitly using the unsafe features of the language that are infrequently 
needed.

I think we're in agreement here, not sure which bit you're arguing 
against :)

I'm with Flavio about preferring Rust personally, but I'm not a member of 
either of these teams and I understand the fact that most of the code is 
already written and has been shown to drastically improve performance in 
exactly the places where it's needed. With all of that in mind, I'm in favor of 
just agreeing already that Go is okay. I understand the concern that this will 
increase cognitive load on some developers and *might* have effects on our 
larger community but our community can only grow so long as our software is 
usable (performant) and useful (satisfies needs/requirements).

This Rust discussion is a bit offtopic, I was just stating that my 
notion of "nativeness" of Go is closer to one of JVM, not one of C/C++/Rust.

I guess my main question is whether folks seriously considered 
PyPy/Cython/etc.

--
Ian Cordasco

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread Ian Cordasco

-Original Message-
From: Dmitry Tantsur <dtant...@redhat.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: May 16, 2016 at 09:55:27
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc] supporting Go

> On 05/16/2016 04:35 PM, Adam Young wrote:
> > On 05/16/2016 05:23 AM, Dmitry Tantsur wrote:
> >> On 05/14/2016 03:00 AM, Adam Young wrote:
> >>> On 05/13/2016 08:21 PM, Dieterly, Deklan wrote:
> >>>> If we allow Go, then we should also consider allowing JVM based
> >>>> languages.
> >>> Nope. Don't get me wrong, I've written more than my fair share of Java
> >>> in my career, and I like it, and I miss automated refactoring and real
> >>> threads. I have nothing against Java (I know a lot of you do).
> >>>
> >>> Java fills the same niche as Python. We already have one of those, and
> >>> its very nice (according to John Cleese).
> >>
> >> A couple of folks in this thread already stated that the primary
> >> reason to switch from Python-based languages is the concurrency story.
> >> JVM solves it and does it in the same manner as Go (at least that's my
> >> assumption).
> >>
> >> (not advocating for JVM, just trying to understand the objection)
> >>
> >>>
> >>> So, what I think we are really saying here is "what is our Native
> >>> extension story going to be? Is it the traditional native languages, or
> >>> is it something new that has learned from them?"
> >>>
> >>> Go is a complement to Python to fill in the native stuff. The
> >>> alternative is C or C++. Ok Flapper, or Rust.
> >>
> >> C, C++, Rust, yes, I'd call them "native".
> >>
> >> A language with a GC and green threads does not fall into "native"
> >> category for me, rather the same as JVM.
> >
> > MOre complex than just that. But Go does not have a VM, just put a lot
> > of effort into co-routines without taking context switches. Different
> > than green threads.
>  
> Ok, I think we have a different notion of "native" here. For me it's
> being with as little magic happening behind the scenes as possible.

Have you written a lot of Rust? Rust handles the memory management for you as 
well. Certainly, you can determine the lifetime of something and tell the 
compiler how the underlying memory is shared, but Rust is far better than C in 
so much as you should never be able to write code that doubly frees the same 
memory unless you're explicitly using the unsafe features of the language that 
are infrequently needed.

I'm with Flavio about preferring Rust personally, but I'm not a member of 
either of these teams and I understand the fact that most of the code is 
already written and has been shown to drastically improve performance in 
exactly the places where it's needed. With all of that in mind, I'm in favor of 
just agreeing already that Go is okay. I understand the concern that this will 
increase cognitive load on some developers and *might* have effects on our 
larger community but our community can only grow so long as our software is 
usable (performant) and useful (satisfies needs/requirements).

--  
Ian Cordasco

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread Dmitry Tantsur

On 05/16/2016 04:35 PM, Adam Young wrote:

On 05/16/2016 05:23 AM, Dmitry Tantsur wrote:

On 05/14/2016 03:00 AM, Adam Young wrote:

On 05/13/2016 08:21 PM, Dieterly, Deklan wrote:

If we allow Go, then we should also consider allowing JVM based
languages.

Nope. Don't get me wrong, I've written more than my fair share of Java
in my career, and I like it, and I miss automated refactoring and real
threads. I have nothing against Java (I know a lot of you do).

Java fills the same niche as Python. We already have one of those, and
its very nice (according to John Cleese).

A couple of folks in this thread already stated that the primary
reason to switch from Python-based languages is the concurrency story.
JVM solves it and does it in the same manner as Go (at least that's my
assumption).

(not advocating for JVM, just trying to understand the objection)

So, what I think we are really saying here is "what is our Native
extension story going to be? Is it the traditional native languages, or
is it something new that has learned from them?"

Go is a complement to Python to fill in the native stuff. The
alternative is C or C++. Ok Flapper, or Rust.

C, C++, Rust, yes, I'd call them "native".

A language with a GC and green threads does not fall into "native"
category for me, rather the same as JVM.

MOre complex than just that. But Go does not have a VM, just put a lot
of effort into co-routines without taking context switches. Different
than green threads.

Ok, I think we have a different notion of "native" here. For me it's
being with as little magic happening behind the scenes as possible.

http://programmers.stackexchange.com/questions/222642/are-go-langs-goroutine-pools-just-green-threads

You can do userland level co-routines in C, C++ and Erlang, probably
even Rust (even if it is not written yet, no idea). Which is different
than putting in a code translation layer.

We are not talking about a replacement for Python here. We are talking
about a language to use for native optimizations when Python's
concurrency model or other overhead gets in the way.

In scientific computing, using a language like R or Python to then call
into a linear algebra or messaging library is the norm for just this
reason. Since we are going to be writing the native code here, the
question is what language to use for it.

We are not getting rid of python, and we are not bringing in Java. Those
are different questions.

The question is "How do we take performance sensitive sections in
OpenStack and optimize to native?"

The list of answers that map to the question here as I see it include:

1. We are not doing native code, stop asking.
2. Stick with C
3. C or C++ is Ok
4. Fortran (OK I just put this in to see if you are paying attention).
5. Go
6. Rust (Only put in to keep Flapper off my back)

Count me in here as well, though of course I have no voting rights
within this issue :)

We have two teams asking for Go, and Flapper asking for Rust. No one
has suggested new native code in C or C++, instead those types of
projects seem to be kept out of OpenStack proper.

This is coming from someone that has done Kernel stuff. I did C++ in
both the Windows and Linux worlds. I've written inversion of control
stuff in C++ template metaprogramming. I am not personally afraid of
writing code in either language. But I don't want to inflict that on
OpenStack. Its a question of reducing complexity, not increasing it.

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

2016-05-16 Thread Adam Young