Re: [openstack-dev] [swift] Go! Swift!
Excerpts from Clay Gerrard's message of 2015-05-07 18:35:23 -0700: On Thu, May 7, 2015 at 3:48 PM, Clint Byrum cl...@fewbar.com wrote: I'm still very curious to hear if anybody has been willing to try to make Swift work on pypy. yeah, Alex Gaynor was helping out with it for awhile. It worked. And it helped. A little bit. Probably still worth looking at if you're curious, but I'm not aware of anyone who's currently working aggressively to productionize swift running on pypy. So if I take your phrase A little bit to mean Not enough to matter then I can imagine there isn't much more that can be done. It sounds like there are really deep architectural issues in Swift that need addressing, not just make code run faster, but get closer to the metal type efficiencies that are being sought. This should be interesting indeed. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 05/08/2015 12:13 AM, Clint Byrum wrote: Excerpts from Clay Gerrard's message of 2015-05-07 18:35:23 -0700: On Thu, May 7, 2015 at 3:48 PM, Clint Byrum cl...@fewbar.com wrote: I'm still very curious to hear if anybody has been willing to try to make Swift work on pypy. yeah, Alex Gaynor was helping out with it for awhile. It worked. And it helped. A little bit. Probably still worth looking at if you're curious, but I'm not aware of anyone who's currently working aggressively to productionize swift running on pypy. So if I take your phrase A little bit to mean Not enough to matter then I can imagine there isn't much more that can be done. It sounds like there are really deep architectural issues in Swift that need addressing, not just make code run faster, but get closer to the metal type efficiencies that are being sought. I understood it to be not so much get closer to the metal, but more deal efficiently with parallel storage I/O. Chris __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
I think most are missing the point a bit. The question that should really be asked is, what is right for Swift to continue to scale. Since the inception of Openstack, Swift has had to solve for problems of scale that generally are not shared with the rest of Openstack. When we first set out to write Swift, we had set, what we thought at the time were pretty lofty goals for ourselves: * 100 Billion objects * 100 Petabytes of data * 100 K requests/second * 100 Gb/s throughput We started with Python figuring that when we hit major bottlenecks, we would look at other options. We have been surprised at how far we have been able to push Python and have met most if not all of the goals above. As we look toward the future, we realize that we are now looking for how we will support trillions of objects, 100's of petabytes to exabytes of data, etc. We feel that we have finally hit that point that we need more than incremental improvements, and that we are running out of incremental improvements that can be made with Python. What started as a simple experiment by Mike Barton, has turned into quite a significant improvement in performance and builds a base that can be built off of for future improvements. This wasn't built because of it being shiny but out of direct need, and is currently being tested with great results on production workloads. I applaud the team that has worked on this at Rackspace, and hope the community can look at the current needs of Swift, and the merits of the work that has been accomplished, rather than the politics of shiny. Thanks, -- Chuck On Thu, Apr 30, 2015 at 11:45 AM John Dickinson m...@not.mn wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. --John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, May 7, 2015 at 7:05 PM, Adam Lawson alaw...@aqorn.com wrote: Chuck (and/or others who understand tor have experienced the limits of Python) I found this comment of yours incredibly intriguing: we are running out of incremental improvements that can be made with Python. Given your work with Swift thus far, what sort of limitations have you discovered that had to do specifically with the fact we're using Python? I haven't run into real-life limitations specific to a particular language before (I usually run into issues with my approach rather than limitations with the language itself) so i find this to be a fascinating (and perhaps accidental) consideration. Well, Swift is sort of different from provisioning services like most Openstack projects. We handle hundreds of times as many requests as big Nova installations, and the backend servers like this one handle some multiplier on top of that. Our users care a lot about performance because it affects things like their page load times. Then Python turns out to be kind of a bad choice for writing a high-performance file server. It's slow. Its concurrency model really punishes you for having workloads that mix disk and network i/o and CPU. Swift's mix of worker processes, eventlet, and thread pools mostly works but it's complicated and inefficient. Blocking disk operations and CPU-heavy tasks are still prone to either locking up event loops or thrashing the GIL. Python 3 and pypy would both make some aspects of that better, but not fix it (yet). - Mike __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Chuck (and/or others who understand tor have experienced the limits of Python) I found this comment of yours incredibly intriguing: we are running out of incremental improvements that can be made with Python. Given your work with Swift thus far, what sort of limitations have you discovered that had to do specifically with the fact we're using Python? I haven't run into real-life limitations specific to a particular language before (I usually run into issues with my approach rather than limitations with the language itself) so i find this to be a fascinating (and perhaps accidental) consideration. *Adam Lawson* AQORN, Inc. 427 North Tatnall Street Ste. 58461 Wilmington, Delaware 19801-2230 Toll-free: (844) 4-AQORN-NOW ext. 101 International: +1 302-387-4660 Direct: +1 916-246-2072 On Thu, May 7, 2015 at 3:48 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Chuck Thier's message of 2015-05-07 13:10:13 -0700: I think most are missing the point a bit. The question that should really be asked is, what is right for Swift to continue to scale. Since the inception of Openstack, Swift has had to solve for problems of scale that generally are not shared with the rest of Openstack. When we first set out to write Swift, we had set, what we thought at the time were pretty lofty goals for ourselves: * 100 Billion objects * 100 Petabytes of data * 100 K requests/second * 100 Gb/s throughput We started with Python figuring that when we hit major bottlenecks, we would look at other options. We have been surprised at how far we have been able to push Python and have met most if not all of the goals above. As we look toward the future, we realize that we are now looking for how we will support trillions of objects, 100's of petabytes to exabytes of data, etc. We feel that we have finally hit that point that we need more than incremental improvements, and that we are running out of incremental improvements that can be made with Python. What started as a simple experiment by Mike Barton, has turned into quite a significant improvement in performance and builds a base that can be built off of for future improvements. This wasn't built because of it being shiny but out of direct need, and is currently being tested with great results on production workloads. I applaud the team that has worked on this at Rackspace, and hope the community can look at the current needs of Swift, and the merits of the work that has been accomplished, rather than the politics of shiny. Chuck, much respect to you and the team for everything accomplished. I'm still very curious to hear if anybody has been willing to try to make Swift work on pypy. This is pretty much why pypy exists, and making pypy work for Swift could mean some really nice residual benefits to the other projects that haven't gone as far as to experiment with a compiled language like Go yet. There's also the other benefit that pypy would gain some eyeballs and improvements that we could feed back into it. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, May 7, 2015 at 3:48 PM, Clint Byrum cl...@fewbar.com wrote: I'm still very curious to hear if anybody has been willing to try to make Swift work on pypy. yeah, Alex Gaynor was helping out with it for awhile. It worked. And it helped. A little bit. Probably still worth looking at if you're curious, but I'm not aware of anyone who's currently working aggressively to productionize swift running on pypy. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Excerpts from Chuck Thier's message of 2015-05-07 13:10:13 -0700: I think most are missing the point a bit. The question that should really be asked is, what is right for Swift to continue to scale. Since the inception of Openstack, Swift has had to solve for problems of scale that generally are not shared with the rest of Openstack. When we first set out to write Swift, we had set, what we thought at the time were pretty lofty goals for ourselves: * 100 Billion objects * 100 Petabytes of data * 100 K requests/second * 100 Gb/s throughput We started with Python figuring that when we hit major bottlenecks, we would look at other options. We have been surprised at how far we have been able to push Python and have met most if not all of the goals above. As we look toward the future, we realize that we are now looking for how we will support trillions of objects, 100's of petabytes to exabytes of data, etc. We feel that we have finally hit that point that we need more than incremental improvements, and that we are running out of incremental improvements that can be made with Python. What started as a simple experiment by Mike Barton, has turned into quite a significant improvement in performance and builds a base that can be built off of for future improvements. This wasn't built because of it being shiny but out of direct need, and is currently being tested with great results on production workloads. I applaud the team that has worked on this at Rackspace, and hope the community can look at the current needs of Swift, and the merits of the work that has been accomplished, rather than the politics of shiny. Chuck, much respect to you and the team for everything accomplished. I'm still very curious to hear if anybody has been willing to try to make Swift work on pypy. This is pretty much why pypy exists, and making pypy work for Swift could mean some really nice residual benefits to the other projects that haven't gone as far as to experiment with a compiled language like Go yet. There's also the other benefit that pypy would gain some eyeballs and improvements that we could feed back into it. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, May 7, 2015 at 5:05 PM, Adam Lawson alaw...@aqorn.com wrote: what sort of limitations have you discovered that had to do specifically with the fact we're using Python? Python is great. Conscious decision to optimize for developer wall time over cpu cycles has made it a great language for 20 years - and probably will for another 20 at *least* (IMHO). I don't think you would pick out anything to point at as a limitation of python that you couldn't point at any dynamic interpreted language, but my list is something like this: - Dynamic Interpreted Runtime overhead - Eventlet non-blocking hub is NOT OK for blocking operations (cpu, disk) - OTOH, dispatch to threads has overhead AND GIL - non-byte-aligned buffers restricts access to O_DIRECT and asyncio *So often* this kinda stuff just doesn't matter. Or even lots of times even when it *does* matter - it doesn't matter that much in the grand scheme of things. Or maybe it matters a non-trivial amount, *but* there's still other things that just mater more *right now*. I think Swift has been in that last case for a long time, maybe we still are - great thing about open-source is redbo can publish an experiment on a feature branch in gerrit and in-between the hard work of testing it - we can pontificate about it on the mailing list! ;) FWIW, I don't think anyone should find it particularly surprising that a mature data-path project would naturally gravitate closer to the metal in the critical paths - it shouldn't be a big deal - unless it all works out - and it's $%^! tons faster - then BOOYAH! ;) But I'd suggest you be very careful not to draw any assumptions in general about a great language like python even if this one time this one project thought maybe they should find out if some part of the distributed system might be better by some measure in something not-python. ;) Cheers, -Clay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Well put, Clay. Totally Agree. On Fri, May 8, 2015 at 10:19 AM, Clay Gerrard clay.gerr...@gmail.com wrote: On Thu, May 7, 2015 at 5:05 PM, Adam Lawson alaw...@aqorn.com wrote: what sort of limitations have you discovered that had to do specifically with the fact we're using Python? Python is great. Conscious decision to optimize for developer wall time over cpu cycles has made it a great language for 20 years - and probably will for another 20 at *least* (IMHO). I don't think you would pick out anything to point at as a limitation of python that you couldn't point at any dynamic interpreted language, but my list is something like this: Dynamic Interpreted Runtime overhead Eventlet non-blocking hub is NOT OK for blocking operations (cpu, disk) OTOH, dispatch to threads has overhead AND GIL non-byte-aligned buffers restricts access to O_DIRECT and asyncio *So often* this kinda stuff just doesn't matter. Or even lots of times even when it *does* matter - it doesn't matter that much in the grand scheme of things. Or maybe it matters a non-trivial amount, *but* there's still other things that just mater more *right now*. I think Swift has been in that last case for a long time, maybe we still are - great thing about open-source is redbo can publish an experiment on a feature branch in gerrit and in-between the hard work of testing it - we can pontificate about it on the mailing list! ;) FWIW, I don't think anyone should find it particularly surprising that a mature data-path project would naturally gravitate closer to the metal in the critical paths - it shouldn't be a big deal - unless it all works out - and it's $%^! tons faster - then BOOYAH! ;) But I'd suggest you be very careful not to draw any assumptions in general about a great language like python even if this one time this one project thought maybe they should find out if some part of the distributed system might be better by some measure in something not-python. ;) Cheers, -Clay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Regards Huang Zhiteng __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, May 7, 2015 at 2:10 PM, Chuck Thier cth...@gmail.com wrote: What started as a simple experiment by Mike Barton, has turned into quite a significant improvement in performance and builds a base that can be built off of for future improvements. This wasn't built because of it being shiny but out of direct need, and is currently being tested with great results on production workloads. Excellent and congrats to the team. Also very happy that it has been posted and not made secret sauce. Much appreciated. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Monty Taylor wrote: On 04/30/2015 08:06 PM, John Dickinson wrote: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. I'm excited to see where this discussion goes. If we decide that a portion of swift being in Go (or C++ or Rust or nim) is a good idea, (just as we've decided that devstack being in shell and portions of horizon and tuskar being in Javascript is a good idea) I'd like to caution people from thinking that must necessarily mean that our general policy of python is dead. The stance has always been python unless there is a compelling reason otherwise. It sounds like there may be a compelling reason otherwise here. Also: http://mcfunley.com/choose-boring-technology I'm pretty much with Monty on this one. There was (and still is) community benefits in sharing the same language and development culture. One of the reasons that people that worked on one OpenStack project continue to work on OpenStack (but on another project) is because we share so much (language, values, CI...) between projects. Now it's always been a trade-off -- unless there is a compelling reason otherwise. JavaScript is for example already heavily used in OpenStack GUI development. We just need to make sure the trade-off is worth it. That the technical benefit is compelling enough to outweigh the community / network drawbacks or the fragmentation risks. That said, of all the languages we could add, I think Go is one that makes the most sense community-wise (due to its extensive use in the container world). -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 30/04/15 10:54 -0700, Clint Byrum wrote: * +1 for compiled languages that are less-daunting than say C++, but still help find problems early. Anybody up for a Rust version too? ;-) /me is always up for some Rust -- @flaper87 Flavio Percoco pgpe5UOvbPWsd.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 04/05/15 12:17 +0200, Thierry Carrez wrote: Monty Taylor wrote: On 04/30/2015 08:06 PM, John Dickinson wrote: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. I'm excited to see where this discussion goes. If we decide that a portion of swift being in Go (or C++ or Rust or nim) is a good idea, (just as we've decided that devstack being in shell and portions of horizon and tuskar being in Javascript is a good idea) I'd like to caution people from thinking that must necessarily mean that our general policy of python is dead. The stance has always been python unless there is a compelling reason otherwise. It sounds like there may be a compelling reason otherwise here. Also: http://mcfunley.com/choose-boring-technology I'm pretty much with Monty on this one. There was (and still is) community benefits in sharing the same language and development culture. One of the reasons that people that worked on one OpenStack project continue to work on OpenStack (but on another project) is because we share so much (language, values, CI...) between projects. Now it's always been a trade-off -- unless there is a compelling reason otherwise. JavaScript is for example already heavily used in OpenStack GUI development. We just need to make sure the trade-off is worth it. That the technical benefit is compelling enough to outweigh the community / network drawbacks or the fragmentation risks. TBH, I'm a bit torn. I'm always cheering for innovation, for using the right tool, etc, but I also agree with Monty, the linked post and some of the arguments that have been made in this thread. To some extent, I believe it'd be fair to say that as long as all the other aspects are maintained by the project itself, it should be fine for projects to do this. To be more precise, I don't think our infra team should reply to the request of having a Go/Rust/Nim CI unless there are enough cases that would make this worth it for them to offer this service. This means, swift needs to run their own CI for the Go code, provide tools for deplying it, etc. One question that raises (naturally?) is whether Swift will end up being completely rewritten in Go? I wouldn't discard this option. That said, of all the languages we could add, I think Go is one that makes the most sense community-wise (due to its extensive use in the container world). Not going to get into language wars but the above is also arguable. I'm not against Go itself, it's just that choosing a language to use for a task is hardly that simple. Cheers, Flavio -- @flaper87 Flavio Percoco pgpyc53lM9bBb.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On May 4, 2015, at 5:26 AM, Flavio Percoco fla...@redhat.com wrote: On 04/05/15 12:17 +0200, Thierry Carrez wrote: Monty Taylor wrote: On 04/30/2015 08:06 PM, John Dickinson wrote: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. I'm excited to see where this discussion goes. If we decide that a portion of swift being in Go (or C++ or Rust or nim) is a good idea, (just as we've decided that devstack being in shell and portions of horizon and tuskar being in Javascript is a good idea) I'd like to caution people from thinking that must necessarily mean that our general policy of python is dead. The stance has always been python unless there is a compelling reason otherwise. It sounds like there may be a compelling reason otherwise here. Also: http://mcfunley.com/choose-boring-technology I'm pretty much with Monty on this one. There was (and still is) community benefits in sharing the same language and development culture. One of the reasons that people that worked on one OpenStack project continue to work on OpenStack (but on another project) is because we share so much (language, values, CI...) between projects. Now it's always been a trade-off -- unless there is a compelling reason otherwise. JavaScript is for example already heavily used in OpenStack GUI development. We just need to make sure the trade-off is worth it. That the technical benefit is compelling enough to outweigh the community / network drawbacks or the fragmentation risks. TBH, I'm a bit torn. I'm always cheering for innovation, for using the right tool, etc, but I also agree with Monty, the linked post and some of the arguments that have been made in this thread. To some extent, I believe it'd be fair to say that as long as all the other aspects are maintained by the project itself, it should be fine for projects to do this. To be more precise, I don't think our infra team should reply to the request of having a Go/Rust/Nim CI unless there are enough cases that would make this worth it for them to offer this service. This means, swift needs to run their own CI for the Go code, provide tools for deplying it, etc. One question that raises (naturally?) is whether Swift will end up being completely rewritten in Go? I wouldn't discard this option. Just as a point of clarity (since I've seen it mentioned a few times now in this thread and elsewhere): At the current time, we are NOT planning to rewrite everything in Go. We are exploring one specific question: is a compiled language object server worth it?. Or restated with more words: Is there a specific part of Swift that we can make more efficient by implementing in a different language such that the benefits gained outweigh the costs of using another language? And additionally, how can we as a community explore this question in the open rather than pushing this work to be done apart from the community and behind closed doors? There's already an ecosystem out there of people who have written stuff for and around Swift (ie middleware and DiskFile implementations). One of the first questions that comes up in the what if it's not Python discussions is what about WSGI middleware. That right there is one reason that, for now, there are no plans to rewrite everything in Go, and should *anything* be written in something other than Python in the master branch and in a release, it will be done only after much careful consideration. --John That said, of all the languages we could add, I think Go is one that makes the most sense community-wise (due to its extensive use in the container world). Not going to get into language wars but the above is also arguable. I'm not against Go itself, it's just that choosing a language to use for a task is hardly that simple. Cheers, Flavio -- @flaper87 Flavio Percoco __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On May 4, 2015, at 7:26 AM, Flavio Percoco fla...@redhat.com wrote: TBH, I'm a bit torn. I'm always cheering for innovation, for using the right tool, etc, but I also agree with Monty, the linked post and some of the arguments that have been made in this thread. Using something different isn't innovating. Finding a way to do something significantly better is innovating. -- Ed Leafe signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 04/05/15 11:33 -0500, Ed Leafe wrote: On May 4, 2015, at 7:26 AM, Flavio Percoco fla...@redhat.com wrote: TBH, I'm a bit torn. I'm always cheering for innovation, for using the right tool, etc, but I also agree with Monty, the linked post and some of the arguments that have been made in this thread. Using something different isn't innovating. Finding a way to do something significantly better is innovating. FWIW, I didn't say using something different is innovating. It's what you do with that tool that makes it so. However, you can also use innovative tools for common tasks and I could get really philosophical about this so I'll stop here :) Flavio -- @flaper87 Flavio Percoco pgpi23bwYkvVe.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 05/04/2015 11:42 AM, Flavio Percoco wrote: Using something different isn't innovating. Finding a way to do something significantly better is innovating. FWIW, I didn't say using something different is innovating. It's what you do with that tool that makes it so. However, you can also use innovative tools for common tasks and I could get really philosophical about this so I'll stop here :) Oh, I know you know. I was just commenting on the common reaction to Oooh, shiny! as a persuasive technical argument. :) - -- - -- Ed Leafe -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - https://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCgAGBQJVR6ozAAoJEKMgtcocwZqLPaQP/jKYecECgzPU0l1ANLXiZB8y DcNiRpSR30L+7sFfuesBUKl5dnLYjExgUIH9eBonjaJLh7XFTnhAUYnt0/TkxKrQ gz8PcA2iYMUY6uOi6JGp/4vdvXSa9pIYNCpEU3YHh71IlgtXkifXU0D/rVVwqyxZ te3tR3B+Kx3X/smVRNlCxzbFuEcZL7f2lez3I4rr8rgBpSuILeK5H7rfuTrOXFle JCYXjkDu8MKE5hMTOOVU49Fwl4GpYDI8g1570UutqDs5hn37H6KKenaJC83xnD4I 6Hi4M8hpCfJsrVHzVlgnuDceYCSBxcKfW8gN/efGcmN5RZtwXq0uMKGtEGS5/f3i SOSFTJ+PMnUz/gTv7MPrlBLma3Ntr/HHk+Cu+axZ9mn69DCLtjJkPpLU6yIvYd/c qNm6QYoz7YFxbzuL/NltCnywcniTt8hMtjcoKH21bR2HF6cfL8XGeOXh6mtMBfZX OzNQwuwRvQnxFYFpsE5Voy9MAGlAr00goKpRCeZFRSDZKbP8nVEeL5GfRbSv2xXD v3VoiPMuEJZQBO1DiEvM1gWfDtb7vwW4LYJyS9IQVts94RuIA7INXRHaGBr8ng1t YX0TuX3g0OOLcW99MnqWmbpLKMUne0W9/JQEdn76qiKpL1IQHH/rPzPGFlD5jP53 ++YWvY60Fy827VLe0+ki =HQql -END PGP SIGNATURE- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, Apr 30, 2015 at 12:54 PM, Clint Byrum cl...@fewbar.com wrote: * Go's import and build system is rather odd. Python's weird distribution issues are at least well known in the OpenStack community. This is actually the main reason I've never gravitated toward Go, as I feel it is trying to be magical rather than logical. I imagine there are others who are also nervous about that. Honestly I don't think Go's dep system, which is still developing, is any worse than Python's, which is also still developing. The big win for us (devs, ops, everyone!) is that the hassles are build-time, not install/run-time. Once you have a build, it will run. I dream of a time when we are free of the madness that pip leaves us in, at every turn. dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 04/30/2015 01:54 PM, Clint Byrum wrote: * Go's import and build system is rather odd. Python's weird distribution issues are at least well known in the OpenStack community. This is actually the main reason I've never gravitated toward Go, as I feel it is trying to be magical rather than logical. I imagine there are others who are also nervous about that. I don't think it tries to be magical, it tries to be developer friendly by default, and lets your VCS handle versioning your dependencies. It just so happens version control systems are great at that. -- Ryan Brown / Software Engineer, Openstack / Red Hat, Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 04/30/2015 12:52 PM, Jay Pipes wrote: On 04/30/2015 12:40 PM, John Dickinson wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Although I have come to like python in certain ways, here is my take on advantages: 1. Performance 2. Code understandability, when paired with a good IDE. Particularly with folks new to a code base. With static typing: a) You can hover over any variable and know its type (and value when in the debugger) b) You can find definitions and references discriminated using real name scopes, not getting false hits for different variables with the same name c) You can examine static call graphs at any point in the code d) The IDE can do refactoring for you without worrying about variables names that are the same even though they are in different scopes e) check the features of any real modern IDE that uses static typing and type inference to understand the code This sort of question has spawned many religious wars of course. Statically typed languages like Java can be very clunky to use with a lot of boilerplate code to write. It is easier to prototype things using a language like Python because you do not have to determine type structure up front. This is a double-edged sword IMO, when you are not prototyping. Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. Awesome discussion topic. I've long argued that OpenStack should be the API, not the implementation, to allow for experimentation in other languages such as Golang. I have always thought there were valid arguments on both sides for the API vs. implementation debate, but I don't think it is necessary to resolve that to proceed in such a direction. Seems this is more about the current OpenStack position that all implementations must be written in Python. There would be nothing (other than vociferous objections from some folks :-) ) to stop us from saying that OpenStack is an implementation andnot an API, but not all OpenStack project implementations are required to use Python. -David Kudos to the Rackspace Cloud Files team for this effort. I'll definitely dig into the code. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [swift] Go! Swift!
Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. --John signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 04/30/2015 12:40 PM, John Dickinson wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. Awesome discussion topic. I've long argued that OpenStack should be the API, not the implementation, to allow for experimentation in other languages such as Golang. Kudos to the Rackspace Cloud Files team for this effort. I'll definitely dig into the code. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Apr 30, 2015, at 9:52 AM, Jay Pipes jaypi...@gmail.com wrote: On 04/30/2015 12:40 PM, John Dickinson wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. Awesome discussion topic. I've long argued that OpenStack should be the API, not the implementation, to allow for experimentation in other languages such as Golang. Just a small clarification on the words here: this isn't a re-implementation of Swift. This is part of Swift. It is code that is proposed to and managed by the governance structures set up by OpenStack and the OpenStack Swift community, therefore it is Swift. The important part is that this is done in the open. That's why I'm happy it's been proposed upstream. Kudos to the Rackspace Cloud Files team for this effort. I'll definitely dig into the code. +1 Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Excerpts from John Dickinson's message of 2015-04-30 09:40:02 -0700: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. First, great job getting this out there at such an early stage. It's not an easy thing to decide when is right to bring something out into the light. I would hope we can have many discussions on the mailing list first, as it is so broad in scope, every single developer would have interest in it, and unless we commandeer the plenary space, we probably won't be very comfortable stacked 6 high in a fishbowl room. ;) Some points I'd like to see discussed here, in no particular order: * I'd also think that we need to open this up to the operators too, as python was chosen to be friendly to ops as well. Go is pretty ops friendly too, so I don't think it would be a problem, but I think this needs to be shared with operators before getting too far down the road. * Hummingbird claims to be an attempt to improve performance dramatically. I wonder if there was any attempt to use pypy to achieve that with the existing code base, and if you could publicize the results of such an attempt. If not, I think it would be quite interesting to see comparisons and worthwhile to attempt a pypy vs. cpython vs. hummingbird comparison. * Even if the performance is better, will the maturity be regained quick enough to have that matter? There are some past lessons to be learned here: [1] * Go's import and build system is rather odd. Python's weird distribution issues are at least well known in the OpenStack community. This is actually the main reason I've never gravitated toward Go, as I feel it is trying to be magical rather than logical. I imagine there are others who are also nervous about that. * +1 for compiled languages that are less-daunting than say C++, but still help find problems early. Anybody up for a Rust version too? ;-) [1] http://www.joelonsoftware.com/articles/fog69.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Apr 30, 2015, at 10:54 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from John Dickinson's message of 2015-04-30 09:40:02 -0700: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. First, great job getting this out there at such an early stage. It's not an easy thing to decide when is right to bring something out into the light. I would hope we can have many discussions on the mailing list first, as it is so broad in scope, every single developer would have interest in it, and unless we commandeer the plenary space, we probably won't be very comfortable stacked 6 high in a fishbowl room. ;) Some points I'd like to see discussed here, in no particular order: * I'd also think that we need to open this up to the operators too, as python was chosen to be friendly to ops as well. Go is pretty ops friendly too, so I don't think it would be a problem, but I think this needs to be shared with operators before getting too far down the road. * Hummingbird claims to be an attempt to improve performance dramatically. I wonder if there was any attempt to use pypy to achieve that with the existing code base, and if you could publicize the results of such an attempt. If not, I think it would be quite interesting to see comparisons and worthwhile to attempt a pypy vs. cpython vs. hummingbird comparison. * Even if the performance is better, will the maturity be regained quick enough to have that matter? There are some past lessons to be learned here: [1] * Go's import and build system is rather odd. Python's weird distribution issues are at least well known in the OpenStack community. This is actually the main reason I've never gravitated toward Go, as I feel it is trying to be magical rather than logical. I imagine there are others who are also nervous about that. * +1 for compiled languages that are less-daunting than say C++, but still help find problems early. Anybody up for a Rust version too? ;-) You raise a lot of good questions, and I can think of about a dozen more. Now that we see this in the open community, we can start talking about it. There don't need to be 1000 cooks in the kitchen, though. We'll be going over it in Vancouver and in the normal Swift dev community channels. Right now there are more questions than answers. We'll answer them. To your first point, yes, operators are a huge part. I know that the Cloud Files devs are themselves responsible for maintaining production clusters, as are most of the active contributors to Swift. I'm confident that Swift will continue to be something that continues to be operator-friendly. I'm excited to see what's happening, and we, the Swift community, will continue to deliver high-quality, tested, scalable, manageable code for object storage. --John [1] http://www.joelonsoftware.com/articles/fog69.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 1 May 2015 at 09:04, Dean Troyer dtro...@gmail.com wrote: On Thu, Apr 30, 2015 at 3:28 PM, Robert Collins robe...@robertcollins.net wrote: Blaming pip for our problems, when we haven't been willing to go and contribute fixes to pip is massively unfair to the pip developers. So I _really_ do not blame pip, it is by far the best of the options, it is just (still) incomplete for our needs. dstufft has been _very_ responsive, as you know first hand. And he has brought pip far from where it was even a year ago. Absolutely - kudos to dstufft et al :) My issue was more that we have approx 1000 developers being impacted by this limitation in the core, and we've collectively failed to organise even a tenth of a percent of our effort to assist the developers of this tool we depend so heavily on. Thats something I think we need to change. I guess I was commenting more on the general state of packaging in Python. When I first came to this community I never expected to find something (or multiple somethings!) that made CPAN look so good. For an ecosystem as old as this one that surprised me. I see I misread your mail - sorry! -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 04/30/2015 08:06 PM, John Dickinson wrote: On Apr 30, 2015, at 9:52 AM, Jay Pipes jaypi...@gmail.com wrote: On 04/30/2015 12:40 PM, John Dickinson wrote: Swift is a scalable and durable storage engine for storing unstructured data. It's been proven time and time again in production in clusters all over the world. We in the Swift developer community are constantly looking for ways to improve the codebase and deliver a better quality codebase to users everywhere. During the past year, the Rackspace Cloud Files team has been exploring the idea of reimplementing parts of Swift in Go. Yesterday, they released some of this code, called hummingbird, for the first time. It's been proposed to a feature/hummingbird branch in Swift's source repo. https://review.openstack.org/#/c/178851 I am very excited about this work being in the greater OpenStack Swift developer community. If you look at the patch above, you'll see that there are various parts of Swift reimplemented in Go. During the next six months (i.e. before Tokyo), I would like us to answer this question: What advantages does a compiled-language object server bring, and do they outweigh the costs of using a different language? Of course, there are a ton of things we need to explore on this topic, but I'm happy that we'll be doing it in the context of the open community instead of behind closed doors. We will have a fishbowl session in Vancouver on this topic. I'm looking forward to the discussion. I'm excited to see where this discussion goes. If we decide that a portion of swift being in Go (or C++ or Rust or nim) is a good idea, (just as we've decided that devstack being in shell and portions of horizon and tuskar being in Javascript is a good idea) I'd like to caution people from thinking that must necessarily mean that our general policy of python is dead. The stance has always been python unless there is a compelling reason otherwise. It sounds like there may be a compelling reason otherwise here. Also: http://mcfunley.com/choose-boring-technology Awesome discussion topic. I've long argued that OpenStack should be the API, not the implementation, I have and continue to disagree... to allow for experimentation in other languages such as Golang. As John points out more politely below, OpenStack can experiment with language without turning ourselves into a feckless standards body. Just a small clarification on the words here: this isn't a re-implementation of Swift. This is part of Swift. It is code that is proposed to and managed by the governance structures set up by OpenStack and the OpenStack Swift community, therefore it is Swift. The important part is that this is done in the open. That's why I'm happy it's been proposed upstream. ++ Kudos to the Rackspace Cloud Files team for this effort. I'll definitely dig into the code. This I agree with. :) -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBCAAGBQJVQoS8AAoJEDOQ22gEGhLwU6sP/0EBqCJTpzLl6Vt9HIY8BQbK Y6jB5lHYsDUOfpZdDBdZ/WZ9QomuW7F8n8neJRCp2Q+1Bee09Vk8YQMSHPBpJHEN 565nCM0gWEznRcXhKgXeNA9q04Q5mj3kJWEC5eaMiLXMy1QLtTpo3OxBaxgbj+UN G/i0a6bIuhW5BIogbxnWDj7AoF+RA/3InhDy3VLmdNLgZ9z3NQKF2os2cJLvOYTB AdwblMREGtHqvE+ANkp5HNezCDT03weIfwPZXan+7up1CN80yt03WavoSqPuOyoj 5rlOgz3AaEM6yQpnCG+kqOAm1EL5i0yzOdZYPAPoFXPbWvSdHEO/YFN9JcSh76AD V5WAiyu4MW/q18nsWly8Rcrf+fdKAcOvLaflQCDcIrblwBM1vYuyoXPSvOpXKlTg S3kVdwnhD85pO9WZxW2PJf1nGekcjwUwvmKJevoJRsXWxMJMUkXU/erXpaIkFg2c w76YUSgqjFr1lQDdLe493/rT13WEnMk1ZTqIs6BZJMxyRVRFzpggxBBLP+ZDUW7D TDbUSZNwYtPacRX0QDKXfL3Kvywdt7hlc6iixCFt2qg6xQ//2r9uuF0xArM0iwv0 MfSLYoUkHsV4lf5OVG8800fC0dgWS5TYzCrL6uXcw3wT7rHHQ8xxAdQBBtnogj6P F5m5K6nFX6KfbxCk5Y6B =ljsb -END PGP SIGNATURE- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On 1 May 2015 at 06:11, Dean Troyer dtro...@gmail.com wrote: Honestly I don't think Go's dep system, which is still developing, is any worse than Python's, which is also still developing. The big win for us (devs, ops, everyone!) is that the hassles are build-time, not install/run-time. Once you have a build, it will run. I dream of a time when we are free of the madness that pip leaves us in, at every turn. A word of caution though : we are guilty of not contributing fixes to pip though: we're a demanding user - many projects, many branches, high accidental failure rates in changes - and rather than fixing the plumbing we've been papering over it. Blaming pip for our problems, when we haven't been willing to go and contribute fixes to pip is massively unfair to the pip developers. Any technology adoption where we try to solely work on top is going to have similar symptoms: issues that arise won't get fixed unless we're willing to work up the dependency graph and fix things at the right place. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
Last I used it, they only had static compiling, which meant you had to rebuild all the go apps when one shared piece needed updating. That sort of thing is usually distasteful to the distro folks... Docker does the same thing in a way, and that's gaining a lot of ground these days, so maybe its less of an issue... Thanks, Kevin From: Ryan Brown [rybr...@redhat.com] Sent: Thursday, April 30, 2015 11:18 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [swift] Go! Swift! On 04/30/2015 01:54 PM, Clint Byrum wrote: * Go's import and build system is rather odd. Python's weird distribution issues are at least well known in the OpenStack community. This is actually the main reason I've never gravitated toward Go, as I feel it is trying to be magical rather than logical. I imagine there are others who are also nervous about that. I don't think it tries to be magical, it tries to be developer friendly by default, and lets your VCS handle versioning your dependencies. It just so happens version control systems are great at that. -- Ryan Brown / Software Engineer, Openstack / Red Hat, Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Go! Swift!
On Thu, Apr 30, 2015 at 3:28 PM, Robert Collins robe...@robertcollins.net wrote: Blaming pip for our problems, when we haven't been willing to go and contribute fixes to pip is massively unfair to the pip developers. So I _really_ do not blame pip, it is by far the best of the options, it is just (still) incomplete for our needs. dstufft has been _very_ responsive, as you know first hand. And he has brought pip far from where it was even a year ago. I guess I was commenting more on the general state of packaging in Python. When I first came to this community I never expected to find something (or multiple somethings!) that made CPAN look so good. For an ecosystem as old as this one that surprised me. dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
