[
https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994036#comment-14994036
]
ASF GitHub Bot commented on MAHOUT-1786:
----------------------------------------
Github user dlyubimov commented on the pull request:
https://github.com/apache/mahout/pull/174#issuecomment-154478671
although i am not sure about efficiency point said in the original issue --
bytecode generated something in spark 1.5... i am dubious it will work well
without custom serialization algorithms. needs a benchmark imo. there is still
may be a significant difference between between serializing data structure vs.
serializing data structure iterators and rebuilding a data structure.
> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
> Key: MAHOUT-1786
> URL: https://issues.apache.org/jira/browse/MAHOUT-1786
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.11.0
> Reporter: Michel Lemay
> Priority: Minor
> Labels: performance
>
> Spark 1.5 comes with a new very efficient serializer that uses code
> generation. It is twice as fast as kryo. When using mahout, we have to set
> KryoSerializer because some classes aren't serializable otherwise.
> I suggest to declare Math classes as "implements Serializable" where needed.
> For instance, to use coocurence package in spark 1.5, we had to modify
> AbstractMatrix, AbstractVector, DenseVector and SparseRowMatrix to make it
> work without Kryo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
