[
https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994434#comment-14994434
]
ASF GitHub Bot commented on MAHOUT-1786:
----------------------------------------
Github user dlyubimov commented on the pull request:
https://github.com/apache/mahout/pull/174#issuecomment-154542945
Michel:
Just to be clear: can you or can you not confirm that this change
(1) is known to work correctly (as in all unit tests pass in a
java-serialized session); and
(2) that it is actually indeed faster than the same things in a kryo
session?
I think those are the main things to confirm in this issue.
On Fri, Nov 6, 2015 at 9:37 AM, Michel Lemay <[email protected]>
wrote:
> I leave it to you to find a way to use that new codegen optimizations in
> spark.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/mahout/pull/174#issuecomment-154481151>.
>
> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
> Key: MAHOUT-1786
> URL: https://issues.apache.org/jira/browse/MAHOUT-1786
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.11.0
> Reporter: Michel Lemay
> Priority: Minor
> Labels: performance
>
> Spark 1.5 comes with a new very efficient serializer that uses code
> generation. It is twice as fast as kryo. When using mahout, we have to set
> KryoSerializer because some classes aren't serializable otherwise.
> I suggest to declare Math classes as "implements Serializable" where needed.
> For instance, to use coocurence package in spark 1.5, we had to modify
> AbstractMatrix, AbstractVector, DenseVector and SparseRowMatrix to make it
> work without Kryo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
