[
https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998721#comment-14998721
]
ASF GitHub Bot commented on MAHOUT-1786:
----------------------------------------
Github user michellemay commented on the pull request:
https://github.com/apache/mahout/pull/174#issuecomment-155447141
I'm not 100% sure.. I still have other tests to do but it looks like
default serializer of spark 1.5.1 is 19% SLOWER than kryo when performing large
matrice AtA.
That is really sad since I see huge gains elsewhere in our spark tasks.
Is there any possibility to have best of both world ?
> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
> Key: MAHOUT-1786
> URL: https://issues.apache.org/jira/browse/MAHOUT-1786
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.11.0
> Reporter: Michel Lemay
> Priority: Minor
> Labels: performance
>
> Spark 1.5 comes with a new very efficient serializer that uses code
> generation. It is twice as fast as kryo. When using mahout, we have to set
> KryoSerializer because some classes aren't serializable otherwise.
> I suggest to declare Math classes as "implements Serializable" where needed.
> For instance, to use coocurence package in spark 1.5, we had to modify
> AbstractMatrix, AbstractVector, DenseVector and SparseRowMatrix to make it
> work without Kryo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
