On Fri, Oct 2, 2009 at 6:58 AM, Philipp <[email protected]> wrote: > > David, > > > My immediate problem is: > http://groups.google.com/group/liftweb/browse_thread/thread/b3783e24b... > > > > This has been a persistent problem with Scala Actors and I identified it > > last year in November or December. > > This is indeed supposed to be fixed in Scala 2.7.7 (to be released > later today). It replaces the old (as opposed to the new one for JDK7) > FJ framework with java.util.concurrent.ThreadPoolExecutor. >
Just to be very, very, very clear, I will not upgrade Lift to use any Scala release that has not gone through a reasonable release candidate testing cycle. While EPFL may choose to have a failed release policy, I will not subject users of Lift to such a policy. I will address the balance of the email when I have more time. > > > Philipp did the 2.7.4 release which did not address the issue. The 2.7.5 > > release was supposed to address the issue, but the use of Lift Actors > masked > > the issue until the above issue was raised. I left my 2.7.5 related > > discussions with Philipp with the impression that the > java.util.concurrent > > library was being used for thread pooling rather than the FJ library. On > > this, I backed out the Lift Actor changes from powering Lift's > CometActors > > (Lift Actors power the long polling part of Lift). > > So, I guess there was a mis-understanding. 2.7.5 tried to fix several > issues related to memory leakage; however, it did not replace the use > of the FJ library. This was, however, done in trunk a couple of weeks > before 2.7.5 was released. At the time of JavaOne (beginning of June), > the plan was still to release (a beta of) Scala 2.8 in July. > > > > . The ForkJoinPool in 2.8 is completely different from FJTask in > > > 2.7.5; it's the version that's going into JDK7. It has been > > > battle-tested and should not suffer from any memory leaks. > > > > > The reason why Scala actors use the FJ framework is performance, in > > > particular on multi-core hardware. So we do not think it's a good idea > > > to go back to java.util.concurrent, except maybe for applications with > > > very specialized demands. > > > > Do you have specific benchmarks that justify the tradeoff of baking in an > > external library (that could be as buggy as the one that's currently > baked > > into Scala) versus java.util.concurrent? > > We did some experiments that show that, indeed, a work-stealing thread > pool is better than a java.util.concurrent.Executor with a central > entry queue when running on an 8-core machine. You find a plot in this > tech report: > http://lamp.epfl.ch/~phaller/doc/haller07actorsunify.pdf > > I am talking about figure 7 on page 16. It shows the performance of a > heat diffusion simulation implemented using actors. In the "global" > version, actors use a thread pool with a global task queue (like > java.util.concurrent.Executor). The "local" version uses FJ for task > execution (each pool thread has its own local queue). One can see that > when using more than 4 hardware threads, the performance of "global" > decreases because the global task queue is contended. This is just one > concrete experiment that I did. There is of course a lot more work out > there that shows experimental results on the performance of work- > stealing (for instance, Cilk, data-parallel Haskell, etc). > > So, I firmly believe that the ForkJoinPool of JDK 7 is the way to go. > Doug Lea told me in person that the new version has virtually no line > of code in common with the old FJTask library; it has been completely > redesigned and rewritten. However, by default it compiles and runs > only on JDK 1.6. It is possible to backport it to JDK 1.5; however, if > your 1.5 JVM is not up-to-date, you can encounter unexpected > InterruptedExceptions being thrown. That's the main reason why I am > not in favor of using it on JDK 1.5. > > > > The > > > result is that people have not noticed the changes. For example, most > > > of the issues that Erik raises in his blog post no longer apply to > > > Scala 2.8. Initially we wanted 2.8 to be out by now, but it's taken > > > much longer than we have foreseen, because some of the problems were > > > harder than initially thought. We are sorry to have left the 2.7 > > > branch relatively unattended for so long. It's difficult for us, > > > though, to provide the resources to support two diverging branches in > > > parallel. More community support with backports etc could help. > > > > While this is a nice thought, given the choice between debating with EPFL > as > > to whether my changes are good enough or doing a Lift Actor > implementation, > > I will opt for continuing to develop and maintain the Lift Actor library. > > This gives Lift users the assurance that bugs will be fixed in a timely > > manner, that we can add features based on community need, and that we can > > work with other library authors to insure common interfaces. > > I'd be happier if more people would debate with us. :-) > > > But there is a difference between a research vehicle and something that > can > > be used in production systems. What Philipp has done is delivered a > > prototype and a vision of concurrency to the Scala community. What Jonas > > and I have done is built production-quality systems based on that vision, > > our experience, and other systems we have used or observed. > > I second Martin's view. You and Jonas are doing really great work. > But, I also think that it is important to have a core set of > concurrency abstractions in the standard library that can be used in > production systems. There are several people in the Scala community > that could help achieve that. > > > I would much prefer to free Philipp to do his pi calculus work (or > whatever > > else inspires him) than maintain a library for me. I want to see the > next > > great thing he comes up with rather than making him my bug-fixing slave. > > I'd certainly like to spend more time trying to do that. (I'm really > excited about my current project; but that's for another mail.) > > Thanks, > Philipp > > > > -- Lift, the simply functional web framework http://liftweb.net Beginning Scala http://www.apress.com/book/view/1430219890 Follow me: http://twitter.com/dpp Surf the harmonics --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~----------~----~----~----~------~----~------~--~---
