The JIRA does not explain why it happens. Can you add the explanation. It might be useful if similar situations arise in future in other scenarios.
On Tue, Jan 26, 2016 at 7:17 AM, Sandesh Hegde <[email protected]> wrote: > Here is the jira for the issue > https://issues.apache.org/jira/browse/APEXMALHAR-1984 > > It affects only local mode not on the cluster. I was trying it on > DataTorrent Sandbox, in that machine local mode failed and passed on yarn. > > On Tue, Jan 26, 2016 at 7:13 AM Shubham Pathak <[email protected]> > wrote: > > > Could we have some more details as to what exactly is the issue and why > is > > it failing in local mode ? > > Also, i couldn't get why only local mode is mentioned specifically ? Does > > it not affect when operator runs on the cluster ? > > > > Thanks, > > Shubham > > > > On Tue, Jan 26, 2016 at 7:40 PM, Pramod Immaneni <[email protected] > > > > wrote: > > > > > Siyuan can you explain the statement some more "It would fail in local > > mode > > > if it tries to load class that only in app package classpath." > > > > > > Thanks > > > > > > On Mon, Jan 25, 2016 at 4:52 PM, Siyuan Hua <[email protected]> > > > wrote: > > > > > > > Sandesh and I found an issue in almost all the operators that use > Kryo > > to > > > > clone operator itself in definePartition method. They all use > default > > > > Kryo instance without setting context classloader from the thread > that > > > > execute the code. It would fail in local mode if it tries to load > class > > > > that only in app package classpath. > > > > > > > > When I'm trying to fix it in all the places, I found repeated pattern > > in > > > > definePartitions. And it's very error-prone since no one actually did > > it > > > in > > > > right way until we found the issue. I'm thinking to make a Util > > functin > > > to > > > > create instance from operator itself and also hide the Kryo > dependency > > > from > > > > the operator. Do you have any better suggestions to do this? > > > > > > > > Thanks! > > > > > > > > Siyuan > > > > > > > > > >
