Re: hanging map reduce processes

2011-11-17 Thread Henning Blohm
Hi Sudhan, well that was just an example. In reality the situation is slightly more complex, as its not a single thread but some serious amount of third party libs that are not fully under my control. While there is a chance that I can shut these things down in most cases, experience shows that s

Re: hanging map reduce processes

2011-10-27 Thread Sudharsan Sampath
Hi Henning, I feel it's the non-daemon thread that's causing the issue. A JVM will not exit until all its non-daemon threads have finished. Is there a reason why you want this thread to be non-daemon? If unavoidable, then can you exit this thread when the reducer's job is completed? Thanks Sudhan

Re: hanging map reduce processes

2011-10-27 Thread Henning Blohm
Hi Harsh, here's the simplest example I could come up with: Add     protected void setup(Context context) throws IOException ,InterruptedException {     // start some non-deamon thread     Thread t = new Thread(new Runnable() {   

Re: hanging map reduce processes

2011-10-27 Thread Harsh J
Hey Henning, What version of Hadoop are you running, and can we have a dumbed down sample to reproduce? On Thu, Oct 27, 2011 at 3:28 PM, Henning Blohm wrote: > Hi, > > found that several people have run into this issue, but I was not able to > find a solution yet. > > We have reduce tasks that l

hanging map reduce processes

2011-10-27 Thread Henning Blohm
Hi, found that several people have run into this issue, but I was not able to find a solution yet. We have reduce tasks that leave a hanging "child" process. The implementation uses a lot of third party stuff and leave Timer threads running (as you can readily see in thread dumps). Which is