Re: [Re] Proper use of Executors
Ok so, it's hard to tell if this made much of a difference. There might be
a few less messages? Hard to be sure.
I'm still getting those messages, but I'm now suspecting Selenium is
generating the majority of these messages. It is behaving badly when it
comes to killing its spawned phantomjs processes, thereby leaving the
executors open. Seems like a recurring theme, eh Siegfried? ;)
Selenium is co-opting the Apache Commons Exec. It looks like it is using
ExecuteWatchdog.INFINITE_TIMEOUT, so no help there. I see they are making a
pretty good effort to kill the processes. I won't rule-out that I'm doing
it wrong, but I'm pretty sure
} finally {
driver.quit();
}
is all I'm responsible for if I want Selenium to close phantomjs processes.
I do have some room to upgrade Selenium and PhantomJS which might help. I
didn't see any meaningful changes in Selenium that would lead me to believe
that an upgrade would fix the issue. But, it's a good practice and any
patches will have to be made at the head of the project anyway.
Thanks for all the help!
CG
On Mon, Jun 19, 2017 at 4:11 PM, Siegfried Goeschl <
siegfried.goes...@it20one.com> wrote:
> Staying tuned :-)
>
> > On 5 Jun 2017, at 13:59, Chris Gamache <cgama...@gmail.com> wrote:
> >
> > Hi Siegfried,
> >
> > I have implemented the executor watchdogs and shutdown hooks. I will
> know if our efforts have been fruitful when we make our first restart which
> I'm sure will be soon. I promise I'll write back and let you know how it
> goes.
> >
> > Thank you vary much for checking in with me!
> >
> >> On Jun 5, 2017, at 11:45 AM, Siegfried Goeschl <
> siegfried.goes...@it20one.com> wrote:
> >>
> >> Hi Chris,
> >>
> >> any new findings from your side?
> >>
> >> Thanks in advance,
> >>
> >> Siegfried Goeschl
> >>
> >>> On 28 May 2017, at 21:41, Siegfried Goeschl <
> siegfried.goes...@it20one.com> wrote:
> >>>
> >>> Hi Chris,
> >>>
> >>> there are couple of things to consider
> >>>
> >>> * You are using a PumpStreamHander but the STDERR is not consumed.
> Each process has an internal buffer (size depends on the OS) and when the
> buffer is full any write to STDERR is blocked
> >>> * That could happen if the process being executed actually writes some
> error messages :-)
> >>> * Are you 100% sure that the processes will terminate? See
> ExecuteWatchdog
> >>> * You might habe a look at ProcessDestroyer to cleanup during shutdown
> >>>
> >>> Thanks in advance,
> >>>
> >>> Siegfried Goeschl
> >>>
> >>>
> >>>> On 27 May 2017, at 14:27, Chris Gamache <cgama...@gmail.com> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> I'm using org.apache.commons:commons-exec:1.3 on Java 8.
> >>>>
> >>>> I'm having an issue where my Tomcat server is bleeding out hundreds of
> >>>> threads and all of the memory in the form of Executors that I'm
> running but
> >>>> don't seem to be closing down ... When the server finally grinds to a
> halt
> >>>> I have to restart. When I do it looks like this at shutdown time:
> >>>>
> >>>>
> >>>>
> >>>> 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11]
> >>>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads
> The
> >>>> web application [ROOT##000252] appears to have started a thread named
> [Exec
> >>>> Default Executor] but has failed to stop it. This is very likely to
> create
> >>>> a memory leak. Stack trace of thread:
> >>>> java.lang.Object.wait(Native Method)
> >>>> java.lang.Object.wait(Object.java:502)
> >>>> java.lang.UNIXProcess.waitFor(UNIXProcess.java:396)
> >>>> org.apache.commons.exec.DefaultExecutor.executeInternal(
> DefaultExecutor.java:364)
> >>>> org.apache.commons.exec.DefaultExecutor.access$200(
> DefaultExecutor.java:48)
> >>>> org.apache.commons.exec.DefaultExecutor$1.run(
> DefaultExecutor.java:200)
> >>>> java.lang.Thread.run(Thread.java:745)
> >>>>
> >>>> 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11]
> >>>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads
> The
> >>>> web application [ROOT##000252] appears to have started a thread named
> [Exec
> >>>> Stream Pumper] but has failed to stop it.
Re: [Re] Proper use of Executors
Hi Siegfried, I have implemented the executor watchdogs and shutdown hooks. I will know if our efforts have been fruitful when we make our first restart which I'm sure will be soon. I promise I'll write back and let you know how it goes. Thank you vary much for checking in with me! > On Jun 5, 2017, at 11:45 AM, Siegfried Goeschl > <siegfried.goes...@it20one.com> wrote: > > Hi Chris, > > any new findings from your side? > > Thanks in advance, > > Siegfried Goeschl > >> On 28 May 2017, at 21:41, Siegfried Goeschl <siegfried.goes...@it20one.com> >> wrote: >> >> Hi Chris, >> >> there are couple of things to consider >> >> * You are using a PumpStreamHander but the STDERR is not consumed. Each >> process has an internal buffer (size depends on the OS) and when the buffer >> is full any write to STDERR is blocked >> * That could happen if the process being executed actually writes some error >> messages :-) >> * Are you 100% sure that the processes will terminate? See ExecuteWatchdog >> * You might habe a look at ProcessDestroyer to cleanup during shutdown >> >> Thanks in advance, >> >> Siegfried Goeschl >> >> >>> On 27 May 2017, at 14:27, Chris Gamache <cgama...@gmail.com> wrote: >>> >>> Hi all, >>> >>> I'm using org.apache.commons:commons-exec:1.3 on Java 8. >>> >>> I'm having an issue where my Tomcat server is bleeding out hundreds of >>> threads and all of the memory in the form of Executors that I'm running but >>> don't seem to be closing down ... When the server finally grinds to a halt >>> I have to restart. When I do it looks like this at shutdown time: >>> >>> >>> >>> 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11] >>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The >>> web application [ROOT##000252] appears to have started a thread named [Exec >>> Default Executor] but has failed to stop it. This is very likely to create >>> a memory leak. Stack trace of thread: >>> java.lang.Object.wait(Native Method) >>> java.lang.Object.wait(Object.java:502) >>> java.lang.UNIXProcess.waitFor(UNIXProcess.java:396) >>> org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364) >>> org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) >>> org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) >>> java.lang.Thread.run(Thread.java:745) >>> >>> 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11] >>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The >>> web application [ROOT##000252] appears to have started a thread named [Exec >>> Stream Pumper] but has failed to stop it. This is very likely to create a >>> memory leak. Stack trace of thread: >>> java.io.FileInputStream.readBytes(Native Method) >>> java.io.FileInputStream.read(FileInputStream.java:255) >>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) >>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) >>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) >>> java.io.FilterInputStream.read(FilterInputStream.java:107) >>> org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107) >>> java.lang.Thread.run(Thread.java:745) >>> >>> >>> >>> And my thread dump is a mile long. >>> >>> I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the >>> relevant code. It is called from a regular method in a regular class, >>> nothing fancy: >>> >>> CommandLine cmdLine = CommandLine.parse(command.toString()); >>> DefaultExecutor executor = new DefaultExecutor(); >>> PumpStreamHandler esh = new PumpStreamHandler(os,null,is); >>> executor.setStreamHandler(esh); >>> executor.execute(cmdLine); >>> >>> `is` and `os` are passed in on the constructor. Their opens and closes are >>> managed well and cleaned up on the outside of this class... >>> Are there further steps I'm missing to ensure the threads I'm creating are >>> getting shut down properly and the resources they are using are being >>> returned? >>> >>> Any help is much appreciated. >> > - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Proper use of Executors
Hi all, I'm using org.apache.commons:commons-exec:1.3 on Java 8. I'm having an issue where my Tomcat server is bleeding out hundreds of threads and all of the memory in the form of Executors that I'm running but don't seem to be closing down ... When the server finally grinds to a halt I have to restart. When I do it looks like this at shutdown time: 27-May-2017 07:56:21.631 WARNING [localhost-startStop-11] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT##000252] appears to have started a thread named [Exec Default Executor] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) java.lang.UNIXProcess.waitFor(UNIXProcess.java:396) org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:364) org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) java.lang.Thread.run(Thread.java:745) 27-May-2017 07:56:21.633 WARNING [localhost-startStop-11] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT##000252] appears to have started a thread named [Exec Stream Pumper] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: java.io.FileInputStream.readBytes(Native Method) java.io.FileInputStream.read(FileInputStream.java:255) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.FilterInputStream.read(FilterInputStream.java:107) org.apache.commons.exec.StreamPumper.run(StreamPumper.java:107) java.lang.Thread.run(Thread.java:745) And my thread dump is a mile long. I am certainly willing to concede I'm Doing It Wrong(tm) ... Here's the relevant code. It is called from a regular method in a regular class, nothing fancy: CommandLine cmdLine = CommandLine.parse(command.toString()); DefaultExecutor executor = new DefaultExecutor(); PumpStreamHandler esh = new PumpStreamHandler(os,null,is); executor.setStreamHandler(esh); executor.execute(cmdLine); `is` and `os` are passed in on the constructor. Their opens and closes are managed well and cleaned up on the outside of this class... Are there further steps I'm missing to ensure the threads I'm creating are getting shut down properly and the resources they are using are being returned? Any help is much appreciated.
Re: Help needed: commons-exec CLOSE_WAIT problem
I saw that! I couldn't have done it without you. Many thanks! CG Sent from my iPhone > On Feb 5, 2016, at 12:24 PM, Siegfried Goeschl <siegfried.goes...@gmail.com> > wrote: > > Hi Chris, > > according to https://github.com/SeleniumHQ/selenium/pull/1545 the PR is > merged and closed :-) > > Cheers, > > Siegfried Goeschl > >> On 27.01.2016, at 14:52, Siegfried Göschl <siegfried.goes...@it20one.com> >> wrote: >> >> Hi Chris, >> >> glad to help - please note that I took the freedom to post my findings at >> https://github.com/SeleniumHQ/selenium/issues/1080 - could you do the same >> :-) >> >> Cheers, >> >> Siegfried Goeschl >> >> >> - Ursprüngliche Mail - >> Von: "Chris Gamache" <cgama...@gmail.com> >> An: "Commons Users List" <user@commons.apache.org> >> Gesendet: Mittwoch, 27. Januar 2016 14:43:32 >> Betreff: Re: Help needed: commons-exec CLOSE_WAIT problem >> >> I really appreciate the thorough inspection you have given. With your help >> I think I've targeted the real problem... >> >> GhostDriver can't exist outside of phantomjs. GhostDriver is an >> implementation of WebDriver Wire Protocol which runs on phantomjs's >> JavaScript engine. It can't run without phantomjs. >> >> We can rest easy that commons-exec is not holding open the file handle. >> That's good. I think the PhantomJSDriver is firing a copy of phantomjs up >> using commons-exec, but it's not communicating with the service through >> commons-exec. Upon closer inspection selenium-java is communicating through >> tcp/ip to a WebDriver server phantomjs is popping up on random available >> ports. This happens to be an instance of some class that implements >> org.apache.http.client.HttpClient, by default it looks to >> be org.apache.http.impl.client.CloseableHttpClient ... I think that when >> commons-exec terminates phantomjs, thereby terminating the GhostDriver >> server, the HttpClient is still attached to that port. >> >> Here's my take on the real problem: >> >> I think the selenium folks didn't consider port closing >> (org.apache.http.client.HttpClient doesn't have a close method to >> implement) a necessary step and they figured that if whatever service the >> HttpClient was attached to shut itself down that the port would >> automatically close itself down also. The standard use for selenium usually >> doesn't require it to run for days and days on end. I can't fault them for >> missing it. >> >> That being said, they made this difficult to override in consumer classes >> by keeping a lot of the guts of the connections hidden behind private >> classes and private member data. >> >> So, I'll need to dig into selenium-java, change a few interfaces to make >> sure that whatever HttpClient is used will have a close() hook, and then >> make sure that whatever is driving the HttpClient has access to the close >> hook on shutdown, and fires it. When the HttpClient is closed, the file >> descriptor should disappear. >> >> Thank you again for your help. It was indispensable! >> >> CG >> >> >> On Tue, Jan 26, 2016 at 4:07 PM, Siegfried Göschl < >> siegfried.goes...@it20one.com> wrote: >> >>> Hi Chris, >>> >>> I played with your GitHub repo [ >>> https://github.com/cgamache/openfile](https://github.com/cgamache/openfile) >>> >>> ### 1. First Take >>> >>> Your test program actually starts a JVM and the Selenium library start >>> multiple "phantomjs" executable to run the test as shown below >>> >>> ``` >>> application> ps -ef | grep phantom >>> 501 1021 460 0 9:26PM ttys0000:00.71 java -jar openfile.jar >>> /usr/local/Cellar/phantomjs/2.1.0/bin/phantomjs >>> 501 1073 1062 0 9:26PM ttys0000:00.08 >>> /usr/local/Cellar/phantomjs/2.1.0/bin/phantomjs --webdriver=36418 >>> --webdriver-logfile=/Users/sgoeschl/work/github/cgamache/openfile/target/phantomjsdriver.log >>> ``` >>> >>> So commons-exec is used under the hood to run the "phantomjs" binary and >>> make sure nothing hangs (output & error thread pumping, watchdog, ..) >>> >>> When the "WebDriver" has finished its work there >>> >>> * is no ***phantomjs*** process running >>> * is only your program running as shown below >>> >>> ``` >>> application> ps -ef
Re: Help needed: commons-exec CLOSE_WAIT problem
I just did the same. Thank you once more! On Wed, Jan 27, 2016 at 8:52 AM, Siegfried Göschl < siegfried.goes...@it20one.com> wrote: > Hi Chris, > > glad to help - please note that I took the freedom to post my findings at > https://github.com/SeleniumHQ/selenium/issues/1080 - could you do the > same :-) > > Cheers, > > Siegfried Goeschl > > > ----- Ursprüngliche Mail - > Von: "Chris Gamache" <cgama...@gmail.com> > An: "Commons Users List" <user@commons.apache.org> > Gesendet: Mittwoch, 27. Januar 2016 14:43:32 > Betreff: Re: Help needed: commons-exec CLOSE_WAIT problem > > I really appreciate the thorough inspection you have given. With your help > I think I've targeted the real problem... > > GhostDriver can't exist outside of phantomjs. GhostDriver is an > implementation of WebDriver Wire Protocol which runs on phantomjs's > JavaScript engine. It can't run without phantomjs. > > We can rest easy that commons-exec is not holding open the file handle. > That's good. I think the PhantomJSDriver is firing a copy of phantomjs up > using commons-exec, but it's not communicating with the service through > commons-exec. Upon closer inspection selenium-java is communicating through > tcp/ip to a WebDriver server phantomjs is popping up on random available > ports. This happens to be an instance of some class that implements > org.apache.http.client.HttpClient, by default it looks to > be org.apache.http.impl.client.CloseableHttpClient ... I think that when > commons-exec terminates phantomjs, thereby terminating the GhostDriver > server, the HttpClient is still attached to that port. > > Here's my take on the real problem: > > I think the selenium folks didn't consider port closing > (org.apache.http.client.HttpClient doesn't have a close method to > implement) a necessary step and they figured that if whatever service the > HttpClient was attached to shut itself down that the port would > automatically close itself down also. The standard use for selenium usually > doesn't require it to run for days and days on end. I can't fault them for > missing it. > > That being said, they made this difficult to override in consumer classes > by keeping a lot of the guts of the connections hidden behind private > classes and private member data. > > So, I'll need to dig into selenium-java, change a few interfaces to make > sure that whatever HttpClient is used will have a close() hook, and then > make sure that whatever is driving the HttpClient has access to the close > hook on shutdown, and fires it. When the HttpClient is closed, the file > descriptor should disappear. > > Thank you again for your help. It was indispensable! > > CG > > > On Tue, Jan 26, 2016 at 4:07 PM, Siegfried Göschl < > siegfried.goes...@it20one.com> wrote: > > > Hi Chris, > > > > I played with your GitHub repo [ > > > https://github.com/cgamache/openfile](https://github.com/cgamache/openfile) > > > > ### 1. First Take > > > > Your test program actually starts a JVM and the Selenium library start > > multiple "phantomjs" executable to run the test as shown below > > > > ``` > > application> ps -ef | grep phantom > > 501 1021 460 0 9:26PM ttys0000:00.71 java -jar openfile.jar > > /usr/local/Cellar/phantomjs/2.1.0/bin/phantomjs > > 501 1073 1062 0 9:26PM ttys0000:00.08 > > /usr/local/Cellar/phantomjs/2.1.0/bin/phantomjs --webdriver=36418 > > > --webdriver-logfile=/Users/sgoeschl/work/github/cgamache/openfile/target/phantomjsdriver.log > > ``` > > > > So commons-exec is used under the hood to run the "phantomjs" binary and > > make sure nothing hangs (output & error thread pumping, watchdog, ..) > > > > When the "WebDriver" has finished its work there > > > > * is no ***phantomjs*** process running > > * is only your program running as shown below > > > > ``` > > application> ps -ef | grep phantom > > 501 1021 460 0 9:23PM ttys0000:00.72 java -jar openfile.jar > > /usr/local/Cellar/phantomjs/2.1.0/bin/phantomjs > > 501 1034 469 0 9:23PM ttys0010:00.00 grep phantom > > ``` > > > > And as described we have a couple of ports in CLOSE_WAIT stemming from > > your program > > > > ``` > > application> lsof -p 1021 | grep CLOSE_WAIT > > java1021 sgoeschl 16uIPv6 0x78418468470e25c7 0t0 TCP > > localhost:50581->localhost:40560 (CLOSE_WAIT) > > java1021 sgoeschl 18uIPv6 0x784184685efd95a7 0t0 TCP > > localhost:50599->localhost:4531
Re: Help needed: commons-exec CLOSE_WAIT problem
e
>
> ```
> public static final int REPS = 5;
>
> for (; i < REPS; i++) {
> WebDriver driver = new PhantomJSDriver(caps);
> driver.get("http://www.apache.org;);
> driver.quit();
> }
> ```
>
> In other words each ***phantomjs*** process leaves exactly on port behind
> :-)
>
>
> ### 2. Second Take
>
> So where is this port coming from? The next session - starting the test
> programm leaves the 5 ports behind
>
> ```
> application> lsof -p 1329 | grep CLOSE_WAIT
> java1329 sgoeschl 16uIPv6 0x784184686037bb07 0t0 TCP
> localhost:51358->localhost:30177 (CLOSE_WAIT)
> java1329 sgoeschl 18uIPv6 0x784184685f5c3b47 0t0 TCP
> localhost:51379->localhost:cadkey-tablet (CLOSE_WAIT)
> java1329 sgoeschl 19uIPv6 0x784184686037f0c7 0t0 TCP
> localhost:51400->localhost:43648 (CLOSE_WAIT)
> java1329 sgoeschl 20uIPv6 0x784184686037c5c7 0t0 TCP
> localhost:51418->localhost:34505 (CLOSE_WAIT)
> java1329 sgoeschl 21uIPv6 0x78418468470e1047 0t0 TCP
> localhost:51438->localhost:30905 (CLOSE_WAIT)
> ```
>
> All of ports are actually found in phantomjsdriver.log (cadkey-tablet is
> actually port 1400)
>
> ```
> [INFO - 2016-01-26T20:51:19.697Z] GhostDriver - Main - running on port
> 30177
> [INFO - 2016-01-26T20:51:21.685Z] GhostDriver - Main - running on port
> 1400
> [INFO - 2016-01-26T20:51:23.427Z] GhostDriver - Main - running on port
> 43648
> [INFO - 2016-01-26T20:51:25.156Z] GhostDriver - Main - running on port
> 34505
> [INFO - 2016-01-26T20:51:26.887Z] GhostDriver - Main - running on port
> 30905
> ```
>
> So the ports are actually used/opened for ***GhostDriver*** and left there
> even when ***driver.quit()*** was called
>
>
> ### 3. Conclusion
>
> * ASFAIK it is not commons-exec issue since all ***phantomjs*** were
> cleanly terminated
> * Each ***PhantomJSDriver*** invocation leaves a port in ***CLOSE_WAIT***
> behind
> * Those ports are linked to ***GhostDriver - Main - running on port
> 30177***
>
>
> Cheers,
>
> Siegfried Goeschl
>
>
>
> - Ursprüngliche Mail -
> Von: "Chris Gamache" <cgama...@gmail.com>
> An: "Commons Users List" <user@commons.apache.org>
> Gesendet: Dienstag, 26. Januar 2016 04:02:41
> Betreff: Re: Help needed: commons-exec CLOSE_WAIT problem
>
> Most defnintely. This was tested on OSX, but should work properly on Linux
> also. You'll need to install phantomjs.
>
> You can clone this repo (https://github.com/cgamache/openfile) and build
> with Maven. The com.codeborne:phantomjsdriver artifact has all of selenium
> packaged inside.
>
> To run,
>
> $ java -jar openfile.jar /path/to/phantomjs
>
> To examine,
>
> $ lsof -p 38270 | grep CLOSE_WAIT | wc -l
>
> Jan 25, 2016 9:46:17 PM
> org.openqa.selenium.phantomjs.PhantomJSDriverService
>
> ... super chatty
>
> [INFO - 2016-01-26T02:46:41.169Z] ShutdownReqHand - _handle - About to
> shutdown
>
>
> Sleeping for 30 seconds so you can examine file handles... you should see 5
>
> Then you can use the lsof command at the top of the output, complete with
> process ID, to check the CLOSE_WAIT file handles.
>
> Thanks for taking a look!
>
> CG
>
>
>
> On Mon, Jan 25, 2016 at 5:28 PM, Siegfried Göschl <
> siegfried.goes...@it20one.com> wrote:
>
> > Hi Chris,
> >
> > there could be couple of reasons for this behaviour - is there a minimal
> > setup to reproduce the problem?
> >
> > Thanks in advance
> >
> > Siegfried Goeschl
> >
> > - Ursprüngliche Mail -
> > Von: "Chris Gamache" <cgama...@gmail.com>
> > An: user@commons.apache.org
> > Gesendet: Montag, 25. Januar 2016 22:34:34
> > Betreff: Help needed: commons-exec CLOSE_WAIT problem
> >
> > Hi commons-exec folks,
> >
> > Hoping you can help me figure this out. Selenium Java uses commons-exec
> 1.3
> > under the hood to communicate with phantomjs. When it fires up it opens
> up
> > a pipe that you can see with lsof:
> >
> > # lsof -p 19947
> >
> > ...
> > java19947 user 64 PIPE 0xd2cd00ccca85f9d 16384
> > ->0xd2cd00ca9fbbf9d
> > java19947 user 66 PIPE 0xd2cd00ca31d445d 16384
> > ->0xd2cd00ca31d4c9d
> >
> > Then we get this in lsof as selenium is driving phantomjs:
> >
> > java19947 user 62uIPv6 0xd2cd00cc904879d 0t0 TCP
> > localhost:49757->localhost:23795 (ESTABLISHED)
> >
> > Then aft
Help needed: commons-exec CLOSE_WAIT problem
Hi commons-exec folks, Hoping you can help me figure this out. Selenium Java uses commons-exec 1.3 under the hood to communicate with phantomjs. When it fires up it opens up a pipe that you can see with lsof: # lsof -p 19947 ... java19947 user 64 PIPE 0xd2cd00ccca85f9d 16384 ->0xd2cd00ca9fbbf9d java19947 user 66 PIPE 0xd2cd00ca31d445d 16384 ->0xd2cd00ca31d4c9d Then we get this in lsof as selenium is driving phantomjs: java19947 user 62uIPv6 0xd2cd00cc904879d 0t0 TCP localhost:49757->localhost:23795 (ESTABLISHED) Then after selenium closes and terminates the executor-- properly, as I observed by stepping through the code as it executes, but maybe someone knows otherwise-- we can see in lsof: java19947 user 62uIPv6 0xd2cd00cc904879d 0t0 TCP localhost:49757->localhost:23795 (CLOSE_WAIT) And each successive web driver instance that gets created cause those to build up and build up and build up until you run out of file handles. ... So are there any considerations for using commons-exec that the selenium folks might not be addressing during process destruction which might manifest themselves in these file handles just hanging out, taking up space? This can't be the correct/uncorrectable behavior. I happen to be using Selenium Java 2.49.1 which is the latest version as of this moment, and Java 8. It seems like this has been broken for quite some time though -- https://github.com/SeleniumHQ/selenium/issues/1080 Please advise! Thanks! CG
Re: Help needed: commons-exec CLOSE_WAIT problem
Most defnintely. This was tested on OSX, but should work properly on Linux also. You'll need to install phantomjs. You can clone this repo (https://github.com/cgamache/openfile) and build with Maven. The com.codeborne:phantomjsdriver artifact has all of selenium packaged inside. To run, $ java -jar openfile.jar /path/to/phantomjs To examine, $ lsof -p 38270 | grep CLOSE_WAIT | wc -l Jan 25, 2016 9:46:17 PM org.openqa.selenium.phantomjs.PhantomJSDriverService ... super chatty [INFO - 2016-01-26T02:46:41.169Z] ShutdownReqHand - _handle - About to shutdown Sleeping for 30 seconds so you can examine file handles... you should see 5 Then you can use the lsof command at the top of the output, complete with process ID, to check the CLOSE_WAIT file handles. Thanks for taking a look! CG On Mon, Jan 25, 2016 at 5:28 PM, Siegfried Göschl < siegfried.goes...@it20one.com> wrote: > Hi Chris, > > there could be couple of reasons for this behaviour - is there a minimal > setup to reproduce the problem? > > Thanks in advance > > Siegfried Goeschl > > ----- Ursprüngliche Mail - > Von: "Chris Gamache" <cgama...@gmail.com> > An: user@commons.apache.org > Gesendet: Montag, 25. Januar 2016 22:34:34 > Betreff: Help needed: commons-exec CLOSE_WAIT problem > > Hi commons-exec folks, > > Hoping you can help me figure this out. Selenium Java uses commons-exec 1.3 > under the hood to communicate with phantomjs. When it fires up it opens up > a pipe that you can see with lsof: > > # lsof -p 19947 > > ... > java19947 user 64 PIPE 0xd2cd00ccca85f9d 16384 > ->0xd2cd00ca9fbbf9d > java19947 user 66 PIPE 0xd2cd00ca31d445d 16384 > ->0xd2cd00ca31d4c9d > > Then we get this in lsof as selenium is driving phantomjs: > > java19947 user 62uIPv6 0xd2cd00cc904879d 0t0 TCP > localhost:49757->localhost:23795 (ESTABLISHED) > > Then after selenium closes and terminates the executor-- properly, as I > observed by stepping through the code as it executes, but maybe someone > knows otherwise-- we can see in lsof: > > java19947 user 62uIPv6 0xd2cd00cc904879d 0t0 TCP > localhost:49757->localhost:23795 (CLOSE_WAIT) > > And each successive web driver instance that gets created cause those to > build up and build up and build up until you run out of file handles. > > ... So are there any considerations for using commons-exec that the > selenium folks might not be addressing during process destruction which > might manifest themselves in these file handles just hanging out, taking up > space? This can't be the correct/uncorrectable behavior. > > I happen to be using Selenium Java 2.49.1 which is the latest version as of > this moment, and Java 8. It seems like this has been broken for quite some > time though -- https://github.com/SeleniumHQ/selenium/issues/1080 > > Please advise! Thanks! > > CG > > - > To unsubscribe, e-mail: user-unsubscr...@commons.apache.org > For additional commands, e-mail: user-h...@commons.apache.org > >
