The suggestion wasn't to serialize an operator (which was already designed to work with Kryo), but those fields in the operator that may only be Java serializable. Please use the annotation with those fields.
On Wed, Jan 13, 2016 at 11:22 PM, Tushar Gosavi <[email protected]> wrote: > One approach I found is that we could annotate operator > with @DefaultSerializer(JavaSerializer.class), but when operator is > de-serialized the constructor is not called, and port objects in operators > are left uninitialized (null). To handle this you will have to provide > readObject method which will initialize port object after object is read. > > something like below code > ``` > @DefaultSerializer(JavaSerializer.class) > public abstract class BaseSinglePortOperator<A,B> extends BaseOperator > implements Serializable > { > public transient DefaultOutputPort<B> output; > public transient DefaultInputPort<A> input; > > private void init() { > output = new DefaultOutputPort<>(); > > input = new DefaultInputPort<A>() { > @Override > public void process(A tuple) > { > processTuple(tuple); > } > }; > } > > protected abstract void processTuple(A tuple); > > public BaseSinglePortOperator() { > init(); > } > > private void readObject(ObjectInputStream in) throws IOException, > ClassNotFoundException > { > in.defaultReadObject(); > init(); > } > } > ``` > > - Tushar. > > > On Thu, Jan 14, 2016 at 10:51 AM, Gaurav Gupta <[email protected]> > wrote: > > > Bhupesh, > > > > You can get more details here > > > http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception > > < > > > http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception > > > > > > > Thanks > > - Gaurav > > > > > On Jan 12, 2016, at 9:36 AM, Gaurav Gupta <[email protected]> > > wrote: > > > > > > Bhupesh, > > > > > > There are following two ways > > > > > > 1. If you can change the classes, add default constructor to these > > classes. > > > 2. If you can’t change the classes, you can use custom serializer for > > these classes using Kryo’s @Bind annotation > > > > > > @Bind(JavaSerializer.class) > > > SetMultimap<String, String> someMap; > > > > > > This will work when there is an existing alternative serializer for the > > > type in question. > > > > > > Thanks > > > - Gaurav > > > > > >> On Jan 12, 2016, at 5:00 AM, Bhupesh Chawda <[email protected] > > <mailto:[email protected]>> wrote: > > >> > > >> Hi All, > > >> > > >> I am facing an issue with Kryo where some classes (which are a part of > > >> operators in Dag) in the imported jars do not have a zero-argument > > >> constructor. This results in a KryoException. > > >> > > >> Any suggestions on how to handle this? > > >> In general how can we deal with classes which do not have default > > >> constructors, without modifying them? > > >> > > >> Thanks. > > >> -Bhupesh > > >> > > >> On Thu, Oct 29, 2015 at 10:10 PM, Vlad Rozov <[email protected] > > <mailto:[email protected]>> > > >> wrote: > > >> > > >>> +1 > > >>> > > >>> > > >>> On 10/29/15 08:47, Pramod Immaneni wrote: > > >>> > > >>>> +1 > > >>>> > > >>>> On Thu, Oct 29, 2015 at 8:34 AM, Amol Kekre <[email protected] > > <mailto:[email protected]>> wrote: > > >>>> > > >>>> Samoa can be used to test iteration support as that feature gets > > >>>>> developed. > > >>>>> > > >>>>> Amol > > >>>>> > > >>>>> > > >>>>> On Thu, Oct 29, 2015 at 5:12 AM, Bhupesh Chawda < > > [email protected] <mailto:[email protected]> > > >>>>>> > > >>>>> wrote: > > >>>>> > > >>>>> Yes, iteration support will be needed for quite a few algorithms. > > >>>>>> > > >>>>>> Thanks. > > >>>>>> Bhupesh > > >>>>>> > > >>>>>> On Wed, Oct 28, 2015 at 7:20 PM, Sandesh Hegde < > > [email protected] <mailto:[email protected]> > > >>>>>>> > > >>>>>> wrote: > > >>>>>> > > >>>>>> Does it need iteration support? Good idea to discuss this feature > > in > > >>>>>>> > > >>>>>> both > > >>>>>> > > >>>>>>> the mailing list together. > > >>>>>>> > > >>>>>>> Adding Samoa mailing list. > > >>>>>>> > > >>>>>>> On Wed, Oct 28, 2015, 4:28 AM Sandeep Deshmukh < > > >>>>>>> > > >>>>>> [email protected] <mailto:[email protected]>> > > >>>>> > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> +1 > > >>>>>>>> > > >>>>>>>> Regards > > >>>>>>>> Sandeep > > >>>>>>>> > > >>>>>>>> Regards, > > >>>>>>>> Sandeep > > >>>>>>>> > > >>>>>>>> On Wed, Oct 28, 2015 at 11:36 AM, Amol Kekre < > > [email protected] <mailto:[email protected]>> > > >>>>>>>> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> +1 > > >>>>>>>>> > > >>>>>>>>> Amol > > >>>>>>>>> > > >>>>>>>>> On Tue, Oct 27, 2015 at 10:27 PM, Bhupesh Chawda < > > >>>>>>>>> > > >>>>>>>> [email protected] <mailto:[email protected]>> > > >>>>>>>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>> Hi All, > > >>>>>>>>>> > > >>>>>>>>>> Apache Samoa <https://samoa.incubator.apache.org/ < > > https://samoa.incubator.apache.org/>> is a > > >>>>>>>>>> > > >>>>>>>>> distributed > > >>>>>> > > >>>>>>> streaming machine learning framework that contains a programming > > >>>>>>>>>> abstraction for distributed streaming machine learning > > >>>>>>>>>> > > >>>>>>>>> algorithms. > > >>>>> > > >>>>>> Apache > > >>>>>>>> > > >>>>>>>>> SAMOA enables development of new ML algorithms without directly > > >>>>>>>>>> > > >>>>>>>>> dealing > > >>>>>>> > > >>>>>>>> with the complexity of underlying distributed stream processing > > >>>>>>>>>> > > >>>>>>>>> engines > > >>>>>>> > > >>>>>>>> (DSPEe, such as Apache Storm, Apache S4, and Apache Samza). > > >>>>>>>>>> > > >>>>>>>>> Apache > > >>>>> > > >>>>>> SAMOA > > >>>>>>>> > > >>>>>>>>> users can develop distributed streaming ML algorithms once and > > >>>>>>>>>> > > >>>>>>>>> execute > > >>>>>>> > > >>>>>>>> them > > >>>>>>>>> > > >>>>>>>>>> on multiple DSPEs. > > >>>>>>>>>> > > >>>>>>>>>> Apache Samoa currently has integrations with Apache Storm, > > Apache > > >>>>>>>>>> > > >>>>>>>>> Flink, > > >>>>>>>> > > >>>>>>>>> Apache S4 and Apache Samza. This means the ML algorithms > > >>>>>>>>>> > > >>>>>>>>> developed > > >>>>> > > >>>>>> on > > >>>>>> > > >>>>>>> Apache Samoa can run on these platforms without any change in the > > >>>>>>>>>> algorithms. > > >>>>>>>>>> It would be a good idea to integrate Apache Apex as a > > distributed > > >>>>>>>>>> > > >>>>>>>>> stream > > >>>>>>>> > > >>>>>>>>> processing engine (DSPE) into Apache Samoa which would allow > > >>>>>>>>>> > > >>>>>>>>> users > > >>>>> > > >>>>>> to > > >>>>>> > > >>>>>>> run > > >>>>>>>> > > >>>>>>>>> ML algorithms developed in Samoa on Apache Apex. > > >>>>>>>>>> > > >>>>>>>>>> Here is the Apex JIRA for integration work: > > >>>>>>>>>> https://malhar.atlassian.net/browse/APEX-202 < > > https://malhar.atlassian.net/browse/APEX-202> > > >>>>>>>>>> Also, here is the JIRA in SAMOA project: > > >>>>>>>>>> https://issues.apache.org/jira/browse/SAMOA-49 > > >>>>>>>>>> > > >>>>>>>>>> Thanks. > > >>>>>>>>>> > > >>>>>>>>>> > > >>> > > > > > > > >
