Yep, agreed, try whatever works, given the env! If in the course of debugging, someone can tell where I did something dumb with the mutlithreading, I'd be happy to dig in and fix it, but multhreaded *performance* is one of those things that's really hard to fix with a unit test, so I've never really gotten to the bottom of why num_train_threads doesn't seem to speed it up as much as it should.
On Thu, Jun 13, 2013 at 2:37 PM, Sebastian Schelter <[email protected]> wrote: > > I'm not too much of a fan of stealing control of the whole box - my local > > hadoop admin would really not like me. :) > > Completely agree. Our implementations should not do this, thats why ALS > runs per default with a single thread per mapper. > > Just wanted to point out that there are some tricks one can play to > greatly enhance performance, given the environment (e.g. spawned cluster > on EC2, research setting) supports them. > > -sebastian > > > -- -jake
