Jsvc start/stop and threads

[Andrea Lo Pumo]

I am creating a thread in init(), and starting it in start().
In the user guide, there is written:
"void stop(): Inform the Thread to terminate the run(), close the
ServerSockets".
So in stop() I am interrupting my thread.
In the apidocs, there is written "the container may restart the Daemon by
calling start() after stop().". But, if my thread has finished, it cannot
be restarted again. (Thread.start() cannot be called after the thread has
completed execution).
So, what should I do?

Thanks.

[Re] Proper use of Executors

[Siegfried Goeschl]

Hi Chris,

any new findings from your side?

Thanks in advance,

Siegfried Goeschl

> On 28 May 2017, at 21:41, Siegfried Goeschl  
> wrote:
> 
> Hi Chris,
> 
> there are couple of things to consider
> 
> * You are using a PumpStreamHander but the STDERR is not consumed. Each 
> process has an internal buffer (size depends on the OS) and when the buffer 
> is full any write to STDERR is blocked
> * That could happen if the process being executed actually writes some error 
> messages :-)
> * Are you 100% sure that the processes will terminate? See ExecuteWatchdog
> * You might habe a look at ProcessDestroyer to cleanup during shutdown
> 
> Thanks in advance,
> 
> Siegfried Goeschl
> 
> 
>> On 27 May 2017, at 14:27, Chris Gamache  wrote:
>> 
>> Hi all,
>> 
>> I'm using org.apache.commons:commons-exec:1.3 on Java 8.
>> 
>> I'm having an issue where my Tomcat server is bleeding out hundreds of
>> threads and all of the memory in the form of Executors that I'm running but
>> don't seem to be closing down ... When the server finally grinds to a halt
>> I have to restart. When I do it looks like this at shutdown time:
>> 
>> 
>> 
>> 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11]
>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
>> web application [ROOT##000252] appears to have started a thread named [Exec
>> Default Executor] but has failed to stop it. This is very likely to create
>> a memory leak. Stack trace of thread:
>> java.lang.Object.wait(Native Method)
>> java.lang.Object.wait(Object.java:502)
>> java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
>> org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364)
>> org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
>> org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
>> java.lang.Thread.run(Thread.java:745)
>> 
>> 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11]
>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
>> web application [ROOT##000252] appears to have started a thread named [Exec
>> Stream Pumper] but has failed to stop it. This is very likely to create a
>> memory leak. Stack trace of thread:
>> java.io.FileInputStream.readBytes(Native Method)
>> java.io.FileInputStream.read(FileInputStream.java:255)
>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
>> java.io.BufferedInputStream.read(BufferedInputStream.java:345)
>> java.io.FilterInputStream.read(FilterInputStream.java:107)
>> org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107)
>> java.lang.Thread.run(Thread.java:745)
>> 
>> 
>> 
>> And my thread dump is a mile long.
>> 
>> I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the
>> relevant code. It is called from a regular method in a regular class,
>> nothing fancy:
>> 
>> CommandLine cmdLine = CommandLine.parse(command.toString());
>> DefaultExecutor executor = new DefaultExecutor();
>> PumpStreamHandler esh = new PumpStreamHandler(os,null,is);
>> executor.setStreamHandler(esh);
>> executor.execute(cmdLine);
>> 
>> `is` and `os` are passed in on the constructor. Their opens and closes are
>> managed well and cleaned up on the outside of this class...
>> Are there further steps I'm missing to ensure the threads I'm creating are
>> getting shut down properly and the resources they are using are being
>> returned?
>> 
>> Any help is much appreciated.
> 


-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Jsvc start/stop and threads

[Bernd Eckenfels]

These are just examples, depending on your use case. But I would agree that 
thread is a bad one, as it is quite normal to create and start them in start.

Gruss
Bernd
--
http://bernd.eckenfels.net

From: Andrea Lo Pumo 
Sent: Monday, June 5, 2017 12:30:41 PM
To: Commons Users List
Subject: Re: Jsvc start/stop and threads

So I need to create the thread inside start() and not in init().
Why in the documentation it is suggested to create threads in init()?
"void init(String[] arguments): Here open configuration files, create a
trace file, create ServerSockets, Threads"

Thanks for the prompt reply.

2017-06-05 12:27 GMT+02:00 Greg Thomas :

> If you're terminating your thread, you'll simply need to start a new one.
>
> Greg
>
> On 5 June 2017 at 11:24, Andrea Lo Pumo  wrote:
>
> > I am creating a thread in init(), and starting it in start().
> > In the user guide, there is written:
> > "void stop(): Inform the Thread to terminate the run(), close the
> > ServerSockets".
> > So in stop() I am interrupting my thread.
> > In the apidocs, there is written "the container may restart the Daemon by
> > calling start() after stop().". But, if my thread has finished, it cannot
> > be restarted again. (Thread.start() cannot be called after the thread has
> > completed execution).
> > So, what should I do?
> >
> > Thanks.
> >
>

Re: [Re] Proper use of Executors

[Chris Gamache]

Hi Siegfried,

I have implemented the executor watchdogs and shutdown hooks. I will know if 
our efforts have been fruitful when we make our first restart which I'm sure 
will be soon. I promise I'll write back and let you know how it goes. 

Thank you vary much for checking in with me!

> On Jun 5, 2017, at 11:45 AM, Siegfried Goeschl 
>  wrote:
> 
> Hi Chris,
> 
> any new findings from your side?
> 
> Thanks in advance,
> 
> Siegfried Goeschl
> 
>> On 28 May 2017, at 21:41, Siegfried Goeschl  
>> wrote:
>> 
>> Hi Chris,
>> 
>> there are couple of things to consider
>> 
>> * You are using a PumpStreamHander but the STDERR is not consumed. Each 
>> process has an internal buffer (size depends on the OS) and when the buffer 
>> is full any write to STDERR is blocked
>> * That could happen if the process being executed actually writes some error 
>> messages :-)
>> * Are you 100% sure that the processes will terminate? See ExecuteWatchdog
>> * You might habe a look at ProcessDestroyer to cleanup during shutdown
>> 
>> Thanks in advance,
>> 
>> Siegfried Goeschl
>> 
>> 
>>> On 27 May 2017, at 14:27, Chris Gamache  wrote:
>>> 
>>> Hi all,
>>> 
>>> I'm using org.apache.commons:commons-exec:1.3 on Java 8.
>>> 
>>> I'm having an issue where my Tomcat server is bleeding out hundreds of
>>> threads and all of the memory in the form of Executors that I'm running but
>>> don't seem to be closing down ... When the server finally grinds to a halt
>>> I have to restart. When I do it looks like this at shutdown time:
>>> 
>>> 
>>> 
>>> 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11]
>>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
>>> web application [ROOT##000252] appears to have started a thread named [Exec
>>> Default Executor] but has failed to stop it. This is very likely to create
>>> a memory leak. Stack trace of thread:
>>> java.lang.Object.wait(Native Method)
>>> java.lang.Object.wait(Object.java:502)
>>> java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
>>> org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364)
>>> org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
>>> org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
>>> java.lang.Thread.run(Thread.java:745)
>>> 
>>> 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11]
>>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The
>>> web application [ROOT##000252] appears to have started a thread named [Exec
>>> Stream Pumper] but has failed to stop it. This is very likely to create a
>>> memory leak. Stack trace of thread:
>>> java.io.FileInputStream.readBytes(Native Method)
>>> java.io.FileInputStream.read(FileInputStream.java:255)
>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345)
>>> java.io.FilterInputStream.read(FilterInputStream.java:107)
>>> org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107)
>>> java.lang.Thread.run(Thread.java:745)
>>> 
>>> 
>>> 
>>> And my thread dump is a mile long.
>>> 
>>> I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the
>>> relevant code. It is called from a regular method in a regular class,
>>> nothing fancy:
>>> 
>>> CommandLine cmdLine = CommandLine.parse(command.toString());
>>> DefaultExecutor executor = new DefaultExecutor();
>>> PumpStreamHandler esh = new PumpStreamHandler(os,null,is);
>>> executor.setStreamHandler(esh);
>>> executor.execute(cmdLine);
>>> 
>>> `is` and `os` are passed in on the constructor. Their opens and closes are
>>> managed well and cleaned up on the outside of this class...
>>> Are there further steps I'm missing to ensure the threads I'm creating are
>>> getting shut down properly and the resources they are using are being
>>> returned?
>>> 
>>> Any help is much appreciated.
>> 
> 

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Jsvc start/stop and threads

[Andrea Lo Pumo]

So I need to create the thread inside start() and not in init().
Why in the documentation it is suggested to create threads in init()?
"void init(String[] arguments): Here open configuration files, create a
trace file, create ServerSockets, Threads"

Thanks for the prompt reply.

2017-06-05 12:27 GMT+02:00 Greg Thomas :

> If you're terminating your thread, you'll simply need to start a new one.
>
> Greg
>
> On 5 June 2017 at 11:24, Andrea Lo Pumo  wrote:
>
> > I am creating a thread in init(), and starting it in start().
> > In the user guide, there is written:
> > "void stop(): Inform the Thread to terminate the run(), close the
> > ServerSockets".
> > So in stop() I am interrupting my thread.
> > In the apidocs, there is written "the container may restart the Daemon by
> > calling start() after stop().". But, if my thread has finished, it cannot
> > be restarted again. (Thread.start() cannot be called after the thread has
> > completed execution).
> > So, what should I do?
> >
> > Thanks.
> >
>

Re: [EXEC] Question

[Siegfried Goeschl]

Hi Dan,

some thoughts along the line

* Just to make sure - you are running NOT on Windows?
* are you redirecting streams within your Java code and the Bash scripts? If 
so, can you get rid of them? I vaguely remember issues where a process was 
killed by the ExecuteWatchdog but hang on a re-directed stream
* do you have more information regarding the “ExecuteWatchdog” problem - maybe 
some code snippet to reproduce the issues

Thanks in advance,

Siegfried Goeschl


> On 1 Jun 2017, at 18:26, Dan C  wrote:
> 
> Thanks for the reply!!
> 
> Here’s my situation:
> I’m using the Apache Commons Exec version 1.3 code to call a bash script 
> (which in turn calls one or more bash scripts).  In most cases, everything 
> works and the calling Java code gets a response.  However, on some occasions, 
> the call to DefaultExecutor’s execute method never returns (I’m calling one 
> of the synchronous execute methods) …
> 
> I have tracing in the bash scripts which indicate that they have finished.  
> Also, I am using the ExecuteWatchdog class, but it doesn’t recognize the fact 
> that my timeout has been exceeded.
> 
> So, I’m looking for the following :
> - Some help (tips/tricks/what to look for/at) with how to debug my problem.
> - A good clear example of how to use the Apache Commons Exec for
>- a synchronous call to a bash script
>- an asynchronous call to a bash script   
> 
> Thanks in advance for any help!!
> 
>> On Jun 1, 2017, at 11:26 AM, Amey Jadiye  wrote:
>> 
>> Hi Dan,
>> 
>> You can post the question directly on mailing list. With questions you can
>> give some references like Github gist or link to github commits url.
>> 
>> Regards,
>> Amey
>> 
>> On Thu, Jun 1, 2017 at 8:33 PM, Dan C  wrote:
>> 
>>> Now that I think I’m a registered user, is the the proper way to submit a
>>> question on the Apache Commons Exec code??
>> 
>> 
>> 
>> 
>> -- 
>> 
>> -
>> 
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> 
>> For additional commands, e-mail: dev-h...@commons.apache.org
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
> 


-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org