Re: [openstack-dev] [tc] supporting Go
Dmitry Tantsur writes: This is pretty subjective, I would say. I personally don't feel Go (especially its approach to error handling) any natural (at least no more than Rust or Scala, for example). If familiarity for Python developers is an argument here, mastering Cython or making OpenStack run on PyPy must be much easier for a random Python developer out there to seriously bump the performance. And it would not require introducing a completely new language to the picture. In one sense you are correct. It is easier for a Pythonista to pick up Cython and use that for performance specific areas of code. At the same time, I'd argue that OpenStack as a community is not the same as Python at large. There are packaging requirements and cross project standards that also come into play, not to mention operators that end up bearing the brunt of those decisions. For example, Debian will likely not package a PyPy only version of Designate along with all its requirements. Similarly, while 50% of operators use packaged versioned, that means 50% work from source control to build, test, and release OpenStack projects. You are correct that my position is subjective, but it is based on my experiences trying to operate and deploy OpenStack in addition to writing code. The draw of Go, in my experience, has been easily deploying a single binary I've been able to build and test consistently. The target system has doesn't require Go installed at all and it works on old distros. And it has been much faster. Coming from Python, the reason Go has been easy to get started with is that it offers some protections that are useful such as memory management. Features such as slices are extremely similar to Python and go routines / channels allow supporting more complex patterns such as generators. Yes, you are correct, error handling is controversial, but at the same time, it is no better in C. I'm not an expert in Go, but from what I've seen, Go has been easier to build and deploy than Python, while being faster. Picking it up has been trivial and becoming reasonably proficient has been a quick process. When considered within the scope of OpenStack, it adds a minimal overhead for testing, packaging and deployment, especially when compared to C extensions, PyPy or Cython. I hope that contextualizes my opinion a bit to make clear the subjective aspects are based on OpenStack specific constraints. -- Eric Larson | eric.lar...@rackspace.com Software Developer | Cloud DNS | OpenStack Designate Rackspace Hosting | Austin, Texas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][oslo][designate][zaqar][nova][swift] using pylibmc instead of python-memcached
Monty Taylor writes: On 05/13/2016 08:23 AM, Mehdi Abaakouk wrote: On Fri, May 13, 2016 at 02:58:08PM +0200, Julien Danjou wrote: What's wrong with pymemcache, that we picked for tooz and are using for 2 years now? https://github.com/pinterest/pymemcache Looks like a good alternative. Honestly, nobody should be using pymemcache or python-memcached or pylibmc for anything caching related in OpenStack. People should be using oslo.cache - however, if that needs work before it's usable, people should be using dogpile.cache, which is what oslo.cache uses on the backend. dogpile is pluggable, so it means that the backend used for caching can be chosen in a much broader manner. As morgan mentions elsewhere, that means that people who want to use a different memcache library just need to write a dogpile driver. Please don't anybody directly use memcache libraries for caching in OpenStack. Please. Using dogpile doesn't remove the decision of what caching backend is used. Dogpile has support (I think) for all the libraries mentioned here: https://bitbucket.org/zzzeek/dogpile.cache/src/87965ada186f9b3a4eb7ff033a2e31437d5e9bc6/dogpile/cache/backends/memcached.py Oslo cache would need to be the one making decision as to what backend is used if we need to have something consistent. With that said, it is important that we understand what projects have specific requirements or have experienced issues, otherwise there is a good chance teams will hit an issue down the line and have to work around it. Eric Monty __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Eric Larson | eric.lar...@rackspace.com Software Developer | Cloud DNS | OpenStack Designate Rackspace Hosting | Austin, Texas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc] supporting Go
Jim Rollenhagen writes: On Wed, May 11, 2016 at 03:36:09PM +0200, Thomas Goirand wrote: On 05/11/2016 02:41 PM, Jim Rollenhagen wrote: >> Installing from $language manager instead of distro >> packages, be it in containers or not, will almost always >> make you download random blobs from the Internet, which are >> of course changing over time without any notice, loosing the >> above 3 important features. > > Unless you pin the versions of your dependencies. Pinning versions doesn't change the fact that you'll have to trust a large amount of providers, with some of the files stored in a single location on the Internet. Yes, you can add a cache, etc. but these are band-aids... Well, if we're talking about python, it all comes from PyPI. For Go, the recommendation is for everything to come from Github, but you can choose other sources if you desire. To clarify, Go best practices are to checkout the repo into a vendor directory that must be updated explicitly. While not everyone commits the vendored deps, I'd argue it is a reasonable practice, which means that at build time within a CI system, there should be *NO* dependencies resolved. Tools such as glide (https://github.com/Masterminds/glide) also create a `glide.lock` that provides the dependencies from the latest build, that can be checked into source control. -- Eric Larson | eric.lar...@rackspace.com Software Developer | Cloud DNS | OpenStack Designate Rackspace Hosting | Austin, Texas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc] supporting Go
Flavio Percoco writes: On 11/05/16 09:47 -0500, Dean Troyer wrote: On Tue, May 10, 2016 at 5:54 PM, Flavio Percoco <fla...@redhat.com> wrote: [language mixing bits were here] The above is my main concern with this proposal. I've mentioned this in the upstream review and I'm glad to have found it here as well. The community impact of this change is perhaps not being discussed enough and I believe, in the long run, it'll bite us. Agreed, but to do nothing instead is so not what we are about. The change from integrated/incubated to Big Tent was done to address some issues knowing we did not have all of the answers up front and would learn some things along the way. We did learn some things, both good and bad. I do believe that we can withstand the impact of a new language, particularly when we do it intentionally and knowing where some of the pitfalls are. Also, the specific request is coming from the oldest of all OpenStack projects, and one that has a history of not making big changes without _really_ good reasons. Yes it opens a door, but it will be opened with what I believe to be a really solid model to build upon in other parts of the OpenStack community. I would MUCH rather do it this way then with a new Go-only project that is joining OpenStack from scratch in more than just the implementation language. So, one thing that was mentioned during the last TC meeting is to decide this in a project basis. Don't open the door entirely but let projects sign up for this. This will give us a more contained growth as far as projects with go-code go but it does mean we'll have to do a technical analysis on every project willing to sign up and it kinda goes against the principles of the big tent. The feedback from the Horizon community has been that it's been impossible to avoid a community split and that's what I'd like to avoid. I do think part of this is also due to the differences in the problem domain of client/browser-side and server-side. I believe there is a similar issue with devs writing SQL, the overlap in expertise between the two is way smaller than we all wish it was. Exactly! This separation of domains is the reason why opening the door for JS code was easier. The request was for browser apps that can't be written in Python. And for the specific Python-Golang overlap, it feels to me like more Python devs have (at least talked about) working in Go than in other newish languages. There are worse choices to test the waters with. Just to stress this a bit more, I don't think the problem is the language per se. There are certainly technical issues related to it (packaging, CI, etc) but the main discussion is currently going around the impact this change will have in the community and other areas. I'm sure we can figure the technical issues out. One thing to consider regarding the community's ability to task switch is how Go is much easier than other languages and techniques. For example, one common tactic people suggest when Python becomes too slow is to rewrite the slow parts in C. In designate's case, rewriting the dns wire protocol aspects in C could be beneficial, but it would be very difficult as well. We would need to write an implementation that is able to safely parse dns wire format in a reasonably thread safe fashion that also will work well when those threads have been patched by eventlet, all while writing C code that is compatible with Python internals. To contrast that, the go POC was able to use a well tested go DNS library and implement the same documented interface that was then testable via the same functional tests. It also allowed an extremely simple deployment and had a minimal impact for our CI systems. Finally, as other go code has been written on our small team, getting Python developers up to speed has been trivial. Memory management, built in concurrency primitives, and similar language constructs have made using Go feel natural. This experience is different from JavaScript because there are very specific silos between the UI and the backend. I'd expect that, even though JavaScript is an accepted language in OpenStack, writing a node.js service would prevent a whole host of new complexity the project would similarly debate. Fortunately, on a technical level, I believe we can try Go without its requirements putting a large burden on the CI team resources. Eric Flavio dt -- Dean Troyer dtro...@gmail.com -- Eric Larson | eric.lar...@rackspace.com Software Developer | Cloud DNS | OpenStack Designate Rackspace Hosting | Austin, Texas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
