I will get back on this after my vacation. At this point I haven't tested the code with u0. Also one of the jobs must have one reducer (the parallelism of this job is heavily vested on use of combiners and high degree of aggregated output), this may have been broken by one of those attempts to unify how mr gets configured in Mahout.
One of my mahout-frozen branches works fine with cdh3, although not with u0 yet, that's what it was originally written for. You can try to take one of those from my virgin account if there's any sense of urgency, they have been verified with cdh3b2 I think. On Aug 5, 2011 1:55 PM, "Eshwaran Vijaya Kumar" <[email protected]> wrote: > All, > I am trying to test Stochastic SVD and am facing some errors where it would be great if someone could clarifying what is going on. I am trying to feed the solver a DistributedRowMatrix with the exact same parameters that the test in LocalSSVDSolverSparseSequentialTest uses, i.e, Generate a 1000 X 100 DRM with SequentialSparseVectors and then ask for blockHeight 251, p (oversampling) = 60, k (rank) = 40. I get the following error: > > Exception in thread "main" java.io.IOException: Unexpected overrun in upper triangular matrix files > at org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.loadUpperTriangularMatrix(SSVDSolver.java:471) > at org.apache.mahout.math.hadoop.stochasticsvd.SSVDSolver.run(SSVDSolver.java:268) > at com.mozilla.SSVDCli.run(SSVDCli.java:89) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at com.mozilla.SSVDCli.main(SSVDCli.java:129) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > > Also, I am using CDH3 with Mahout recompiled to work with CDH3 jars. > > Thanks > Esh >
