Stephan,

Thanks for looking at this. Could you elaborate on the misbehavior in the 
accumulator? I'd like to fix it if it's incorrect.

Dave


> On January 20, 2017 at 4:29 AM Stephan Ewen <[email protected]> wrote:
> 
>     Hi!
> 
>     It seems that the accumulator behaves in a non-standard way, but the 
> JobManager should also catch that (log a warning or debug message) and simply 
> continue (not crash).
> 
>     I'll try to add a patch that the JobManager tolerates these kinds of 
> issues in the accumulators.
> 
>     Stephan
> 
> 
>     On Thu, Jan 19, 2017 at 7:26 PM, Dave Marion <[email protected] 
> mailto:[email protected] > wrote:
> 
>         > > 
> >         Noticed I didn't cc the user list.
> > 
> >             > -------- Original Message ----------
>             From: Dave Marion <[email protected] 
> mailto:[email protected] >
>             To: Ted Yu <[email protected] mailto:[email protected] >
>             Date: January 19, 2017 at 12:13 PM
>             Subject: Re: NPE in JobManager
> 
> 
>             That might take some time. Here is a hand typed top N lines. If 
> that is not enough let me know and I will start the process of getting the 
> full stack trace.
> 
> 
>             NullPointerException
> 
>             at JobManager$$updateAccumulators$1.apply(JobManager.scala:1790)
> 
>             at JobManager$$updateAccumulators$1.apply(JobManager.scala:1788)
> 
>             at 
> scala.collection.mutable.ResizableArray$class.forEach(ArrayBuffer.scala:48)
> 
>             at 
> scala.collection.mutable.ArrayBuffer.forEach(ArrayBuffer.scala:48)
> 
>             at org.apache.flink.runtime.jobmanager.JobManager.org 
> http://org.apache.flink.runtime.jobmanager.JobManager.org 
> $apache$flink$runtime$jobmanager$JobManager$$updateAccumulators(JobManager.scala:1788)
> 
>             at 
> org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1.applyOrElse(JobManager.scala:967)
> 
>             at 
> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
> 
>             at 
> org.apache.flink.runtime.LeaderSessionMassageFilter$$anonfun$receive$1.applyOrEslse(LeaderSessionMessageFilter.scala:44)
> 
>             at 
> scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
> 
>             at 
> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:33)
> 
>             at 
> org.apache.flink.runtime.LogMessages$$anon$1.apply(LogMessages.scala:28)
> 
>             at 
> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
> 
>             at 
> org.apache.flink.runtime.LogMesages$$anon$1.applyOrElse(LogMessages.scala:28)
> 
> 
>                 > > On January 19, 2017 at 11:58 AM Ted Yu 
> <[email protected] mailto:[email protected] > wrote:
> > 
> >                 Can you pastebin the complete stack trace for the NPE ?
> > 
> >                 Thanks
> > 
> >                 On Thu, Jan 19, 2017 at 8:57 AM, Dave Marion 
> > <[email protected] mailto:[email protected] > wrote:
> > 
> >                     > > > 
> > >                     I'm running flink-1.1.4-bin-hadoop27-scala_2.11 and 
> > > I'm running into an issue where after some period of time (measured in 1 
> > > - 3 hours) the JobManager gets an NPE and shuts itself down. The failure 
> > > is at JobManager$$updateAccumulators$1.apply(JobManager.scala:1790). I'm 
> > > using a custom accumulator[1], but can't tell from the JobManager code 
> > > whether the issue is in my Accumulator, or is a bug in the JobManager.
> > > 
> > > 
> > >                     [1] 
> > > https://github.com/NationalSecurityAgency/timely/blob/master/analytics/src/main/java/timely/analytics/flink/SortedStringAccumulator.java
> > >  
> > > https://github.com/NationalSecurityAgency/timely/blob/master/analytics/src/main/java/timely/analytics/flink/SortedStringAccumulator.java
> > > 
> > >                 > > 
> > 
> >             > 
>         


 

Reply via email to