+1 for util function Thanks, Tim
On Mon, Jan 25, 2016 at 4:52 PM, Siyuan Hua <[email protected]> wrote: > Sandesh and I found an issue in almost all the operators that use Kryo to > clone operator itself in definePartition method. They all use default > Kryo instance without setting context classloader from the thread that > execute the code. It would fail in local mode if it tries to load class > that only in app package classpath. > > When I'm trying to fix it in all the places, I found repeated pattern in > definePartitions. And it's very error-prone since no one actually did it in > right way until we found the issue. I'm thinking to make a Util functin to > create instance from operator itself and also hide the Kryo dependency from > the operator. Do you have any better suggestions to do this? > > Thanks! > > Siyuan >
