I mean you would have to write one and modify the code to use it. I don't know this job well enough to know whether it's possible or not though. At least, this is getting directly at reducing the amount of data spilled, rather than reducing the intermediate I/O needed to sort it.
Doesn't io.sort.* also affect the mapper? I was sure it did. Maybe it only matters when a combiner is in play on the mapper side. On Wed, May 22, 2013 at 2:17 PM, Jakub Pawłowski <[email protected]> wrote: > Yes, I was manipulating io.sort.factor too, it speeds up reducer, values > around 30 gives good result for me. > But my problem is not reducer, my problem is Bt-job map taks that spills to > drive. > > You mentioned Combiner, how can I turn it on ? I'm running my job from > console like that > > mahout ssvd --rank 400 --computeU true --computeV true --reduceTasks 3 > --input ${INPUT} --output ${OUTPUT} -ow --tempDir /tmp/ssvdtmp/ > > document at > https://cwiki.apache.org/MAHOUT/stochastic-singular-value-decomposition.data/SSVD-CLI.pdf > doesn't mention anything about combiner. > > Thanks for your answer.
