[
https://issues.apache.org/jira/browse/MAHOUT-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986825#action_12986825
]
Danny Bickson edited comment on MAHOUT-542 at 1/25/11 10:05 PM:
----------------------------------------------------------------
Hi!
I tried to install patch MAHOUT-542-3 against a clean svn checkout from Mahout
trunk.
it seems that the file AlternateLeastSquaresSolver.java was already inserted
into svn, so the code got doubled.
I tried to edit manually changes. I got several compilation errors. I attach a
patch I created (named unifiedpatch)
after fixing the compilation errors.
The code compiles, but fails some tests.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.mahout.math.jet.random.NormalTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.437 sec
Running org.apache.mahout.math.TestMatrixView
Tests run: 49, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.283 sec
Running org.apache.mahout.math.TestDenseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.134 sec
Running org.apache.mahout.math.TestSparseMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 sec
Running org.apache.mahout.math.jet.stat.ProbabilityTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.mahout.math.VectorListTest
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.291 sec
Running org.apache.mahout.math.stats.LogLikelihoodTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running org.apache.mahout.math.jet.random.engine.MersenneTwisterTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec
Running org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.071 sec
Running org.apache.mahout.math.TestDenseMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec
Running org.apache.mahout.math.jet.random.ExponentialTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.707 sec
Running org.apache.mahout.math.jet.stat.GammaTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.193 sec
Running org.apache.mahout.math.stats.OnlineSummarizerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 sec
Running org.apache.mahout.math.VectorTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.048 sec
Running org.apache.mahout.common.RandomUtilsTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec
Running org.apache.mahout.math.TestSparseColumnMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.104 sec
Running org.apache.mahout.math.TestSequentialAccessSparseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec
Running org.apache.mahout.math.QRDecompositionTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec
Running org.apache.mahout.math.TestSparseRowMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec
Running org.apache.mahout.math.TestRandomAccessSparseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec
Running org.apache.mahout.math.TestOrderedIntDoubleMapping
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running org.apache.mahout.math.als.AlternateLeastSquaresSolverTest
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec <<<
FAILURE!
Running org.apache.mahout.math.jet.random.NegativeBinomialTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running org.apache.mahout.math.TestVectorView
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec
Running org.apache.mahout.math.jet.random.GammaTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.72 sec
Running org.apache.mahout.math.TestSingularValueDecomposition
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
Running org.apache.mahout.math.decomposer.lanczos.TestLanczosSolver
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.875 sec
Results :
Failed tests:
createRiIiMaybeTransposedExceptionOnNonSequentialVector(org.apache.mahout.math.als.AlternateLeastSquaresSolverTest)
ubuntu@ip-10-195-226-63:/usr/local/mahout-0.4$ cat
math/target/surefire-reports/org.apache.mahout.math.als.AlternateLeastSquaresSolverTest.txt
-------------------------------------------------------------------------------
Test set: org.apache.mahout.math.als.AlternateLeastSquaresSolverTest
-------------------------------------------------------------------------------
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec <<<
FAILURE!
createRiIiMaybeTransposedExceptionOnNonSequentialVector(org.apache.mahout.math.als.AlternateLeastSquaresSolverTest)
Time elapsed: 0.002 sec <<< FAILURE!
java.lang.AssertionError:
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.fail(Assert.java:98)
at
org.apache.mahout.math.als.AlternateLeastSquaresSolverTest.createRiIiMaybeTransposedExceptionOnNonSequentialVector(AlternateLeastSquaresSolverTest.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at
org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Can you please take a look and instruct me what to do?
Thanks,
Danny Bickson
was (Author: danny.bickson):
Hi!
I tried to install patch MAHOUT-542-3 against a clean svn checkout from Mahout
trunk.
it seems that the file AlternateLeastSquaresSolver.java was already inserted
into svn, so the code got doubled.
I tried to edit manually changes, and I attach a patch I created (named
unifiedpatch)
The code compiles, but fails some tests.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.mahout.math.jet.random.NormalTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.437 sec
Running org.apache.mahout.math.TestMatrixView
Tests run: 49, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.283 sec
Running org.apache.mahout.math.TestDenseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.134 sec
Running org.apache.mahout.math.TestSparseMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 sec
Running org.apache.mahout.math.jet.stat.ProbabilityTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.mahout.math.VectorListTest
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.291 sec
Running org.apache.mahout.math.stats.LogLikelihoodTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running org.apache.mahout.math.jet.random.engine.MersenneTwisterTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec
Running org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.071 sec
Running org.apache.mahout.math.TestDenseMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec
Running org.apache.mahout.math.jet.random.ExponentialTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.707 sec
Running org.apache.mahout.math.jet.stat.GammaTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.193 sec
Running org.apache.mahout.math.stats.OnlineSummarizerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 sec
Running org.apache.mahout.math.VectorTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.048 sec
Running org.apache.mahout.common.RandomUtilsTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec
Running org.apache.mahout.math.TestSparseColumnMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.104 sec
Running org.apache.mahout.math.TestSequentialAccessSparseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec
Running org.apache.mahout.math.QRDecompositionTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec
Running org.apache.mahout.math.TestSparseRowMatrix
Tests run: 58, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec
Running org.apache.mahout.math.TestRandomAccessSparseVector
Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec
Running org.apache.mahout.math.TestOrderedIntDoubleMapping
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Running org.apache.mahout.math.als.AlternateLeastSquaresSolverTest
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec <<<
FAILURE!
Running org.apache.mahout.math.jet.random.NegativeBinomialTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec
Running org.apache.mahout.math.TestVectorView
Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec
Running org.apache.mahout.math.jet.random.GammaTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.72 sec
Running org.apache.mahout.math.TestSingularValueDecomposition
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
Running org.apache.mahout.math.decomposer.lanczos.TestLanczosSolver
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.875 sec
Results :
Failed tests:
createRiIiMaybeTransposedExceptionOnNonSequentialVector(org.apache.mahout.math.als.AlternateLeastSquaresSolverTest)
ubuntu@ip-10-195-226-63:/usr/local/mahout-0.4$ cat
math/target/surefire-reports/org.apache.mahout.math.als.AlternateLeastSquaresSolverTest.txt
-------------------------------------------------------------------------------
Test set: org.apache.mahout.math.als.AlternateLeastSquaresSolverTest
-------------------------------------------------------------------------------
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec <<<
FAILURE!
createRiIiMaybeTransposedExceptionOnNonSequentialVector(org.apache.mahout.math.als.AlternateLeastSquaresSolverTest)
Time elapsed: 0.002 sec <<< FAILURE!
java.lang.AssertionError:
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.fail(Assert.java:98)
at
org.apache.mahout.math.als.AlternateLeastSquaresSolverTest.createRiIiMaybeTransposedExceptionOnNonSequentialVector(AlternateLeastSquaresSolverTest.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at
org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Can you please take a look and instruct me what to do?
Thanks,
Danny Bickson
> MapReduce implementation of ALS-WR
> ----------------------------------
>
> Key: MAHOUT-542
> URL: https://issues.apache.org/jira/browse/MAHOUT-542
> Project: Mahout
> Issue Type: New Feature
> Components: Collaborative Filtering
> Affects Versions: 0.5
> Reporter: Sebastian Schelter
> Attachments: MAHOUT-452.patch, MAHOUT-542-2.patch,
> MAHOUT-542-3.patch, unifiedpatch
>
>
> As Mahout is currently lacking a distributed collaborative filtering
> algorithm that uses matrix factorization, I spent some time reading through a
> couple of the Netflix papers and stumbled upon the "Large-scale Parallel
> Collaborative Filtering for the Netflix Prize" available at
> http://www.hpl.hp.com/personal/Robert_Schreiber/papers/2008%20AAIM%20Netflix/netflix_aaim08(submitted).pdf.
> It describes a parallel algorithm that uses "Alternating-Least-Squares with
> Weighted-λ-Regularization" to factorize the preference-matrix and gives some
> insights on how the authors distributed the computation using Matlab.
> It seemed to me that this approach could also easily be parallelized using
> Map/Reduce, so I sat down and created a prototype version. I'm not really
> sure I got the mathematical details correct (they need some optimization
> anyway), but I wanna put up my prototype implementation here per Yonik's law
> of patches.
> Maybe someone has the time and motivation to work a little on this with me.
> It would be great if someone could validate the approach taken (I'm willing
> to help as the code might not be intuitive to read) and could try to
> factorize some test data and give feedback then.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
