Here is the jira for the issue https://issues.apache.org/jira/browse/APEXMALHAR-1984
It affects only local mode not on the cluster. I was trying it on DataTorrent Sandbox, in that machine local mode failed and passed on yarn. On Tue, Jan 26, 2016 at 7:13 AM Shubham Pathak <[email protected]> wrote: > Could we have some more details as to what exactly is the issue and why is > it failing in local mode ? > Also, i couldn't get why only local mode is mentioned specifically ? Does > it not affect when operator runs on the cluster ? > > Thanks, > Shubham > > On Tue, Jan 26, 2016 at 7:40 PM, Pramod Immaneni <[email protected]> > wrote: > > > Siyuan can you explain the statement some more "It would fail in local > mode > > if it tries to load class that only in app package classpath." > > > > Thanks > > > > On Mon, Jan 25, 2016 at 4:52 PM, Siyuan Hua <[email protected]> > > wrote: > > > > > Sandesh and I found an issue in almost all the operators that use Kryo > to > > > clone operator itself in definePartition method. They all use default > > > Kryo instance without setting context classloader from the thread that > > > execute the code. It would fail in local mode if it tries to load class > > > that only in app package classpath. > > > > > > When I'm trying to fix it in all the places, I found repeated pattern > in > > > definePartitions. And it's very error-prone since no one actually did > it > > in > > > right way until we found the issue. I'm thinking to make a Util > functin > > to > > > create instance from operator itself and also hide the Kryo dependency > > from > > > the operator. Do you have any better suggestions to do this? > > > > > > Thanks! > > > > > > Siyuan > > > > > >
