[
https://issues.apache.org/jira/browse/FLINK-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326196#comment-14326196
]
Fay Beligianni edited comment on FLINK-1421 at 2/18/15 5:13 PM:
----------------------------------------------------------------
Hey Stefan,
Thank you very much for the response! Indeed that was the problem.\\
Regarding the Java serialization, yes it is not efficient but when we tried to
use the Kryo serializer we were encountering problems with the tuples that we
are streaming between the invokables.\\
Specifically, because we are streaming Tuple3<String,UserDefinedObject,String>
elements, where UserDefinedClass is a custom implementation of the Samoa
"ContentEvent" interface, we couldn't pass the TypeInformation of the custom
object to the serializer, thus we had to use Java serialization.\\
For sure though we will try to come up with a more efficient solution for that
issue.\\
P.S.: Thanks for the formatting hint!
was (Author: fobeligi):
Hey Stefan,
Thank you very much for the response! Indeed that was the problem.
Regarding the Java serialization, yes it is not efficient but when we tried to
use the Kryo serializer we were encountering problems with the tuples that we
are streaming between the invokables.
Specifically, because we are streaming Tuple3<String,UserDefinedObject,String>
elements, where UserDefinedClass is a custom implementation of the Samoa
"ContentEvent" interface, we couldn't pass the TypeInformation of the custom
object to the serializer, thus we had to use Java serialization.
For sure though we will try to come up with a more efficient solution for that
issue.
P.S.: Thanks for the formatting hint!
> Implement a SAMOA Adapter for Flink Streaming
> ---------------------------------------------
>
> Key: FLINK-1421
> URL: https://issues.apache.org/jira/browse/FLINK-1421
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Reporter: Paris Carbone
> Assignee: Paris Carbone
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> Yahoo's Samoa is an experimental incremental machine learning library that
> builds on an abstract compositional data streaming model to write streaming
> algorithms. The task is to provide an adapter from SAMOA topologies to
> Flink-streaming job graphs in order to support Flink as a backend engine for
> SAMOA tasks.
> A statup guide can be viewed here :
> https://docs.google.com/document/d/18glDJDYmnFGT1UGtZimaxZpGeeg1Ch14NgDoymhPk2A/pub
> The main working branch of the adapter :
> https://github.com/senorcarbone/samoa/tree/flink
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
