Unless there's a big piece of GCD I'm missing, it does not serve as a replacement for Fibers. Enqueuing jobs for execution within a thread pool and having the system intelligently manage the pool is great. But fibers are unique in their ability to pause execution at any point.
That is, if I have a web server with 100,000 active requests, they cannot be blocked on I/O in the traditional sense or else there would be 100,000 active threads. I'd have to architect my app from the ground up so that all points where it could possibly perform a blocking call, it instead enqueues something in a GCD queue and unwinds it thread. Of course, then I can't use any 3rd party libraries because they'd all have to be rearchitected to revolve around work queues as well. So for a large app that uses many 3rd party libs (my relatively humble rails app currently depends directly on 50+ gems), that's HUGE development and maintenance effort that's largely avoidable using fibers. Do you have a link to the Ruby doc indicating their resource usage? I'm looking at http://ruby-doc.org/ruby-1.9/classes/Fiber.html. 4kB for the stack, plus, say, another 8kB for heap allocated data structures (of course would vary per application) would require 1.2GB for 100k concurrent connections, practically nothing for a modern server. The state-of-the-art for handling highly concurrent loads is to use an event-driven I/O framework. Evented I/O is difficult for the same reasons using a GCD queue is difficult, the application and all dependencies have to be designed from the ground up to unwind the stack at every point where I/O may occur (no blocking calls). For an extra 4kB/request, Fibers mean you don't have to rewrite your application and all its dependencies from the ground up. As someone who's worked with many event-I/O frameworks over the years, I can say it's a HUGE productivity boost. I think that's all I can say on the subject, you can believe fibers are useful in such a scenario or not; if you think not, I encourage you to attempt to write software typically authored with a thread-per-request model instead in an evented/non-blocking model and compare the levels of effort. As I think Easco mentioned, we can conclude that because Fibers have unique properties not present in Threads or GCD, they are useful when those properties add value. One would use a different abstraction otherwise. So if you believe Fibers exist in the API for a reason, they should exhibit the same properties as they do on other platforms for them to be truly useful (and not just spec-compliance useful). I hope that happens some day, as I look forwards to switching to MacRuby for that class of application. Logan Bowers On Aug 13, 2010, at 11:05 AM, macr...@djc.net wrote: > >> The use case for me is somewhat more Easco is somewhat more practical. Most >> servers on the Internet use a thread-per-request model, which is good for >> concurrent workloads up to a few hundred concurrent requests. Yes, >> technically you can allocate more threads on a modern system, but rarely >> have I seen the number go much higher for a complex workload. Many of these >> servers are I/O bound, so increasing the concurrency would increase >> throughput. I'd like to take some of my workloads to 100,000+ concurrent >> requests. Threads cannot help me at that scale. > > Exactly so: GCD exists in part to solve this exact issue. > Without GCD, you must design you application "smartly" - i.e. not spawn a > brand new thread per request, but that can also be done. > I would argue you don't want 100,000+ concurrent Fibers on a "standard" > server, either (see ruby-doc.org / impl : they are not nearly as lightweight > as you might think). > > So: while I understand Easco's example, I don't see Fibers as unique, or nec. > top choice, for a clean design of this scenario. > > Perhaps this topic should go offline, since this is leaving the land of > MacRuby [direct].... > > -Daniel > > >> That may sound like an esoteric use case, but with websockets and Comet, >> it's actually pretty common. Sent from my iPhone >> On Aug 13, 2010, at 10:15 AM, "Ernest N. Prabhakar, Ph.D." >> <prabh...@apple.com> wrote: >>> Hi Easco, >>> On Aug 13, 2010, at 8:15 AM, easco wrote: >>>> Instead I was curious to know if there was "a plan" for implementing >>>> Fibers and if so, I was curious to know what operating system technology >>>> they would be built on. It appears that the problem has not been looked >>>> at, in-depth, yet and I am satisfied with that. Nor does it appear that >>>> there is any OS level technology that would particularly support the >>>> creation of Fibers (i.e. Mac OS X doesn't really have any built-in support >>>> for cooperatively scheduled multitasking... outside of deprecated >>>> technologies like the Carbon Thread Manager and makecontext/swapcontext). >>> Actually, I'm confused by that statement. My understanding is that >>> cooperatively scheduled threads is a just a subset of the system-scheduled >>> model. >>> In particular, I'm pretty sure you can emulate them on top of GCD using >>> semaphores. Should I explain further, or did you already consider that and >>> discover it was unworkable? >>> -- Ernie P. >>> _______________________________________________ >>> MacRuby-devel mailing list >>> MacRuby-devel@lists.macosforge.org >>> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel >> _______________________________________________ >> MacRuby-devel mailing list >> MacRuby-devel@lists.macosforge.org >> http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel > _______________________________________________ MacRuby-devel mailing list MacRuby-devel@lists.macosforge.org http://lists.macosforge.org/mailman/listinfo.cgi/macruby-devel
