On Sun, Jul 22, 2012 at 09:27:17AM -0700, Phil Steitz wrote: > On 7/21/12 6:17 AM, Gilles Sadowski wrote: > > Hi. > > > > My previous post (with subject "Synchronisation") made me think (again) that > > it might be useful to start considering how to take advantage of > > multi-threading in Commons Math. > > Indeed, it seems that some parts of the library might end up not being used > > anymore because their performance simply cannot match competing > > implementations that do benefit form parallelization. [The recent example > > that comes to mind is the FFT.] > > This is an interesting question. I am also -1 on adding > dependencies, but it would be a good idea to look at how others have > solved the problem of how to support parallel execution by multiple > threads without managing threads directly. Lots of [math] > algorithms could be parallelized. The question is how to > effectively coordinate the work without owning or creating the > workers. I would be -0 to any suggestion that involved [math] > itself spawning threads,
I certainly do mean that, although threads are to be managed by the utilities in package "java.util.concurrent". > since that 0) creates management headeaches If it does, then it's too complex for CM. But it shouldn't in readily paralellizable tasks (i.e. a processing that can be cut into independeant sub-tasks). > 1) may violate some container contracts The usage of multiple cores would be a user setting (i.e. how many tasks can run in parallel). > and 2) forces execution > threads to be in the same process. I don't understand that. > I think it is worth thinking > about how we might support parallel execution by externally managed > workers. An obvious thing to look at is how to break our > parallelizable algorithms into pieces that could be executed in > Hadoop Map/Reduce jobs. I don't know what that is. > Step 0) is the breaking up part. Then step > 1) might be either some examples added to the user guide or custom > Pig functions (or examples of how to code them). I don't know about that either. I was rather thinking of using the utilities readily available in the Java language standard e.g.: http://docs.oracle.com/javase/tutorial/essential/concurrency/forkjoin.html Regards, Gilles --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
