Thanks all for the replies. @Bind seems to be working for my case. -Bhupesh
On Thu, Jan 14, 2016 at 1:03 PM, Thomas Weise <[email protected]> wrote: > The suggestion wasn't to serialize an operator (which was already designed > to work with Kryo), but those fields in the operator that may only be Java > serializable. Please use the annotation with those fields. > > On Wed, Jan 13, 2016 at 11:22 PM, Tushar Gosavi <[email protected]> > wrote: > > > One approach I found is that we could annotate operator > > with @DefaultSerializer(JavaSerializer.class), but when operator is > > de-serialized the constructor is not called, and port objects in > operators > > are left uninitialized (null). To handle this you will have to provide > > readObject method which will initialize port object after object is read. > > > > something like below code > > ``` > > @DefaultSerializer(JavaSerializer.class) > > public abstract class BaseSinglePortOperator<A,B> extends BaseOperator > > implements Serializable > > { > > public transient DefaultOutputPort<B> output; > > public transient DefaultInputPort<A> input; > > > > private void init() { > > output = new DefaultOutputPort<>(); > > > > input = new DefaultInputPort<A>() { > > @Override > > public void process(A tuple) > > { > > processTuple(tuple); > > } > > }; > > } > > > > protected abstract void processTuple(A tuple); > > > > public BaseSinglePortOperator() { > > init(); > > } > > > > private void readObject(ObjectInputStream in) throws IOException, > > ClassNotFoundException > > { > > in.defaultReadObject(); > > init(); > > } > > } > > ``` > > > > - Tushar. > > > > > > On Thu, Jan 14, 2016 at 10:51 AM, Gaurav Gupta <[email protected]> > > wrote: > > > > > Bhupesh, > > > > > > You can get more details here > > > > > > http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception > > > < > > > > > > http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception > > > > > > > > > > Thanks > > > - Gaurav > > > > > > > On Jan 12, 2016, at 9:36 AM, Gaurav Gupta <[email protected]> > > > wrote: > > > > > > > > Bhupesh, > > > > > > > > There are following two ways > > > > > > > > 1. If you can change the classes, add default constructor to these > > > classes. > > > > 2. If you can’t change the classes, you can use custom serializer for > > > these classes using Kryo’s @Bind annotation > > > > > > > > @Bind(JavaSerializer.class) > > > > SetMultimap<String, String> someMap; > > > > > > > > This will work when there is an existing alternative serializer for > the > > > > type in question. > > > > > > > > Thanks > > > > - Gaurav > > > > > > > >> On Jan 12, 2016, at 5:00 AM, Bhupesh Chawda < > [email protected] > > > <mailto:[email protected]>> wrote: > > > >> > > > >> Hi All, > > > >> > > > >> I am facing an issue with Kryo where some classes (which are a part > of > > > >> operators in Dag) in the imported jars do not have a zero-argument > > > >> constructor. This results in a KryoException. > > > >> > > > >> Any suggestions on how to handle this? > > > >> In general how can we deal with classes which do not have default > > > >> constructors, without modifying them? > > > >> > > > >> Thanks. > > > >> -Bhupesh > > > >> > > > >> On Thu, Oct 29, 2015 at 10:10 PM, Vlad Rozov < > [email protected] > > > <mailto:[email protected]>> > > > >> wrote: > > > >> > > > >>> +1 > > > >>> > > > >>> > > > >>> On 10/29/15 08:47, Pramod Immaneni wrote: > > > >>> > > > >>>> +1 > > > >>>> > > > >>>> On Thu, Oct 29, 2015 at 8:34 AM, Amol Kekre <[email protected] > > > <mailto:[email protected]>> wrote: > > > >>>> > > > >>>> Samoa can be used to test iteration support as that feature gets > > > >>>>> developed. > > > >>>>> > > > >>>>> Amol > > > >>>>> > > > >>>>> > > > >>>>> On Thu, Oct 29, 2015 at 5:12 AM, Bhupesh Chawda < > > > [email protected] <mailto:[email protected]> > > > >>>>>> > > > >>>>> wrote: > > > >>>>> > > > >>>>> Yes, iteration support will be needed for quite a few algorithms. > > > >>>>>> > > > >>>>>> Thanks. > > > >>>>>> Bhupesh > > > >>>>>> > > > >>>>>> On Wed, Oct 28, 2015 at 7:20 PM, Sandesh Hegde < > > > [email protected] <mailto:[email protected]> > > > >>>>>>> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>> Does it need iteration support? Good idea to discuss this > feature > > > in > > > >>>>>>> > > > >>>>>> both > > > >>>>>> > > > >>>>>>> the mailing list together. > > > >>>>>>> > > > >>>>>>> Adding Samoa mailing list. > > > >>>>>>> > > > >>>>>>> On Wed, Oct 28, 2015, 4:28 AM Sandeep Deshmukh < > > > >>>>>>> > > > >>>>>> [email protected] <mailto:[email protected]>> > > > >>>>> > > > >>>>>> wrote: > > > >>>>>>> > > > >>>>>>> +1 > > > >>>>>>>> > > > >>>>>>>> Regards > > > >>>>>>>> Sandeep > > > >>>>>>>> > > > >>>>>>>> Regards, > > > >>>>>>>> Sandeep > > > >>>>>>>> > > > >>>>>>>> On Wed, Oct 28, 2015 at 11:36 AM, Amol Kekre < > > > [email protected] <mailto:[email protected]>> > > > >>>>>>>> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> +1 > > > >>>>>>>>> > > > >>>>>>>>> Amol > > > >>>>>>>>> > > > >>>>>>>>> On Tue, Oct 27, 2015 at 10:27 PM, Bhupesh Chawda < > > > >>>>>>>>> > > > >>>>>>>> [email protected] <mailto:[email protected]>> > > > >>>>>>>> > > > >>>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>> Hi All, > > > >>>>>>>>>> > > > >>>>>>>>>> Apache Samoa <https://samoa.incubator.apache.org/ < > > > https://samoa.incubator.apache.org/>> is a > > > >>>>>>>>>> > > > >>>>>>>>> distributed > > > >>>>>> > > > >>>>>>> streaming machine learning framework that contains a > programming > > > >>>>>>>>>> abstraction for distributed streaming machine learning > > > >>>>>>>>>> > > > >>>>>>>>> algorithms. > > > >>>>> > > > >>>>>> Apache > > > >>>>>>>> > > > >>>>>>>>> SAMOA enables development of new ML algorithms without > directly > > > >>>>>>>>>> > > > >>>>>>>>> dealing > > > >>>>>>> > > > >>>>>>>> with the complexity of underlying distributed stream > processing > > > >>>>>>>>>> > > > >>>>>>>>> engines > > > >>>>>>> > > > >>>>>>>> (DSPEe, such as Apache Storm, Apache S4, and Apache Samza). > > > >>>>>>>>>> > > > >>>>>>>>> Apache > > > >>>>> > > > >>>>>> SAMOA > > > >>>>>>>> > > > >>>>>>>>> users can develop distributed streaming ML algorithms once > and > > > >>>>>>>>>> > > > >>>>>>>>> execute > > > >>>>>>> > > > >>>>>>>> them > > > >>>>>>>>> > > > >>>>>>>>>> on multiple DSPEs. > > > >>>>>>>>>> > > > >>>>>>>>>> Apache Samoa currently has integrations with Apache Storm, > > > Apache > > > >>>>>>>>>> > > > >>>>>>>>> Flink, > > > >>>>>>>> > > > >>>>>>>>> Apache S4 and Apache Samza. This means the ML algorithms > > > >>>>>>>>>> > > > >>>>>>>>> developed > > > >>>>> > > > >>>>>> on > > > >>>>>> > > > >>>>>>> Apache Samoa can run on these platforms without any change in > the > > > >>>>>>>>>> algorithms. > > > >>>>>>>>>> It would be a good idea to integrate Apache Apex as a > > > distributed > > > >>>>>>>>>> > > > >>>>>>>>> stream > > > >>>>>>>> > > > >>>>>>>>> processing engine (DSPE) into Apache Samoa which would allow > > > >>>>>>>>>> > > > >>>>>>>>> users > > > >>>>> > > > >>>>>> to > > > >>>>>> > > > >>>>>>> run > > > >>>>>>>> > > > >>>>>>>>> ML algorithms developed in Samoa on Apache Apex. > > > >>>>>>>>>> > > > >>>>>>>>>> Here is the Apex JIRA for integration work: > > > >>>>>>>>>> https://malhar.atlassian.net/browse/APEX-202 < > > > https://malhar.atlassian.net/browse/APEX-202> > > > >>>>>>>>>> Also, here is the JIRA in SAMOA project: > > > >>>>>>>>>> https://issues.apache.org/jira/browse/SAMOA-49 > > > >>>>>>>>>> > > > >>>>>>>>>> Thanks. > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>> > > > > > > > > > > > > >
