Hi, I checked the application https://github.com/chaithu14/AppThreadLocal
In this example, exception from downstream operator is thrown in a different thread in AbstractReconciler operator. And the rethrow to main operator thread is done in handleIdleTime. This function is not guaranteed to be invoked in every window. In Thread_local locality I checked that handleIdleTime did not get invoked. So, the exception did not get rethrown. The exception thrown from a different thread other than the main operator thread are not caught by Application Master. Something we can probably add to troubleshooting guide to add a rethrow in the main thread. I verified that if downstream operator throws exception in the main thread, it is caught appropriately by application master even in thread local case. Thanks, Isha On Thu, Feb 25, 2016 at 9:57 PM, Chaitanya Chebolu < [email protected]> wrote: > Hi All, > > Created Sample application for THREAD_LOCAL issue. Application is here > <https://github.com/chaithu14/AppThreadLocal>. > Application has the following DAG: > > RandomEventGenerator -> OuputOperator. > > Both the operators are THREAD_LOCAL. > > In OutputOperator, throwing exceptions at every committed window. So, > AppMaster supposed to kill container at every committed window. This is > expected behavior. > But, this is not happening with the current Apex. > > One more observation is, If the upstream operator throws exception at > every committed window, then AppMaster is killing the container > continuously. But, this is not happening with the downstream operator. > > Created JIRA for this issue: APEXCORE-357 > > Regards, > Chaitanya > > On Thu, Feb 25, 2016 at 12:36 PM, Chaitanya Chebolu < > [email protected]> wrote: > > > Hi , > > > > I am facing issues in Thread_Local. Two operators which are thread > local > > and out of which, the downstream operator throws exceptions. But, > AppMaster > > is not catching those exceptions. I was unable to figure out why > > application is not working. > > If both the operators are deployed on different containers, then the > > container is killed continuously by AppMaster. This is expected behavior. > > > > For Example, Let's say the dag be op1 -> op2 where op1, op2 are two > > operators which are of them thread local. Throws an exception from the > > downstream operator op2, AppMaster is not catching exceptions. I will > > create a JIRA for this issue. Please some one help on this. > > > > Regards, > > Chaitanya > > >
