+1 for util function from me too. Thks Amol
On Mon, Jan 25, 2016 at 4:59 PM, Timothy Farkas <[email protected]> wrote: > +1 for util function > > Thanks, > Tim > > On Mon, Jan 25, 2016 at 4:52 PM, Siyuan Hua <[email protected]> > wrote: > > > Sandesh and I found an issue in almost all the operators that use Kryo to > > clone operator itself in definePartition method. They all use default > > Kryo instance without setting context classloader from the thread that > > execute the code. It would fail in local mode if it tries to load class > > that only in app package classpath. > > > > When I'm trying to fix it in all the places, I found repeated pattern in > > definePartitions. And it's very error-prone since no one actually did it > in > > right way until we found the issue. I'm thinking to make a Util functin > to > > create instance from operator itself and also hide the Kryo dependency > from > > the operator. Do you have any better suggestions to do this? > > > > Thanks! > > > > Siyuan > > >
