Github user dlwh commented on the pull request:
https://github.com/apache/spark/pull/5005#issuecomment-84507139
It's probably just HotSpot warming up. I wouldn't worry about it.
On Sat, Mar 21, 2015 at 7:34 PM, Debasish Das <[email protected]>
wrote:
> Even after cleaning up iterator, adding in-place gemv and create the state
> and re-use the memory, still the first iteration of Breeze NNLS is slower
> than mllib NNLS...Rest iterations are fine:
>
> Breeze NNLS:
>
> ./bin/spark-submit --master spark://TUSCA09LMLVT00C.local:7077 --class
> org.apache.spark.examples.mllib.MovieLensALS --jars
> ~/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
> --total-executor-cores 1
> ./examples/target/spark-examples_2.10-1.3.0-SNAPSHOT.jar --rank 50
> --numIterations 2 --nonNegative ~/datasets/ml-1m/ratings.dat
>
> Got 1000209 ratings from 6040 users on 3706 movies.
> Training: 800702, test: 199507.
> Running Breeze NNLSSolver
> Test RMSE = 1.9630818565404409.
> TUSCA09LMLVT00C:spark-brznnls v606014$ grep solveTime
> ./work/app-20150321192419-0000/0/stderr
> 15/03/21 19:24:29 INFO ALS: solveTime 249.982 ms
> 15/03/21 19:24:29 INFO ALS: solveTime 83.31 ms
> 15/03/21 19:24:30 INFO ALS: solveTime 91.942 ms
> 15/03/21 19:24:30 INFO ALS: solveTime 92.131 ms
> 15/03/21 19:24:31 INFO ALS: solveTime 58.648 ms
> 15/03/21 19:24:31 INFO ALS: solveTime 54.959 ms
> 15/03/21 19:24:32 INFO ALS: solveTime 93.302 ms
> 15/03/21 19:24:33 INFO ALS: solveTime 110.504 ms
> 15/03/21 19:24:33 INFO ALS: solveTime 57.124 ms
> 15/03/21 19:24:34 INFO ALS: solveTime 55.7 ms
>
> mllib NNLS:
>
> export solver=mllib; ./bin/spark-submit --master
> spark://TUSCA09LMLVT00C.local:7077 --class
> org.apache.spark.examples.mllib.MovieLensALS --jars
> ~/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
> --total-executor-cores 1
> ./examples/target/spark-examples_2.10-1.3.0-SNAPSHOT.jar --rank 50
> --numIterations 2 --nonNegative ~/datasets/ml-1m/ratings.dat
>
> Got 1000209 ratings from 6040 users on 3706 movies.
> Training: 800702, test: 199507.
> Test RMSE = 1.9630818565404409.
> TUSCA09LMLVT00C:spark-brznnls v606014$ grep solveTime
> ./work/app-20150321192
> app-20150321192419-0000/ app-20150321192553-0001/
> TUSCA09LMLVT00C:spark-brznnls v606014$ grep solveTime
> ./work/app-20150321192553-0001/0/stderr
> 15/03/21 19:26:02 INFO ALS: solveTime 88.237 ms
> 15/03/21 19:26:02 INFO ALS: solveTime 61.216 ms
> 15/03/21 19:26:03 INFO ALS: solveTime 88.628 ms
> 15/03/21 19:26:03 INFO ALS: solveTime 76.532 ms
> 15/03/21 19:26:04 INFO ALS: solveTime 44.945 ms
> 15/03/21 19:26:04 INFO ALS: solveTime 44.895 ms
> 15/03/21 19:26:05 INFO ALS: solveTime 82.933 ms
> 15/03/21 19:26:06 INFO ALS: solveTime 83.018 ms
> 15/03/21 19:26:06 INFO ALS: solveTime 48.138 ms
> 15/03/21 19:26:07 INFO ALS: solveTime 49.3 ms
>
> This is the version I will push to Breeze...It will be great if you guys
> could take a look at the breeze nnls and give some pointers on the first
> iteration...
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/5005#issuecomment-84499918>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
