[
https://issues.apache.org/jira/browse/HTTPCORE-631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098061#comment-17098061
]
Ben Tindall commented on HTTPCORE-631:
--------------------------------------
Thanks for getting back to me [~olegk] and [~michael-o]. I can also confirm
[~olegk]'s proposal. My observations are:
# If you don't call client.start() before client.close() file descriptors will
leak every time
# If I add a delay of 100ms between client.start() and client.close() there
are no FD leaks
Here is my new test which successfully executes until the end and in which
client.close() closes the FDs successfully every time:
{code:java}
for (int i = 0; i < 1000; i++) {
CloseableHttpAsyncClient client = null;
try {
System.out.println("Attempt " + i + "...");
client = HttpAsyncClientBuilder.create().build();
client.start();
System.out.println("Attempt " + i + " succeeded");
Thread.sleep(100); //sleep to let client start up properly
} catch (Throwable e) {
System.out.println("Attempt " + i + " failed: " + e.getMessage());
e.printStackTrace();
} finally {
if (client != null)
client.close();
}
Thread.sleep(10); //sleep before the next iteration
}
System.out.println("Now sleeping");
Thread.sleep(300000);
{code}
During the test:
{code:java}
➜ ~ lsof -p 27598 | grep KQUEUE | wc -l
12 {code}
During the sleep period at the end of the test:
{code:java}
➜ ~ lsof -p 27598 | grep KQUEUE | wc -l
0 {code}
Do you want me to raise a ticket to look at the leak occurring when you close a
client that has not been started yet? I'm happy to also try to fix this as I
have consumed a lot of your time already and am happy to contribute in any way
I can :)
Also what could be nice is to have a callback when the ioReactor.start() has
finished initialising, so that we know when the client is ready for requests.
But as Oleg said, this is a minor thing.
Thanks again,
Ben
> HttpAsyncClient 5.0 file descriptor leak
> ----------------------------------------
>
> Key: HTTPCORE-631
> URL: https://issues.apache.org/jira/browse/HTTPCORE-631
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore NIO
> Affects Versions: 5.0
> Reporter: Ben Tindall
> Priority: Minor
> Fix For: 5.0.1
>
> Attachments: apacheleaktest.tar
>
>
> It seems that HttpAsyncClient doesn't close all its file descriptors with the
> close() operation. Consider the following test:
> {code:java}
> @Test
> public void leakApache() throws Exception {
> for (int i = 0; i < 10000; i++) {
> CloseableHttpAsyncClient client = null;
> try {
> client = HttpAsyncClientBuilder.create().build();
> System.out.println(i);
> } catch (Throwable e) {
> e.printStackTrace();
> } finally {
> if (client != null)
> client.close();
> }
> Thread.sleep(10);
> }
> Thread.sleep(300000);
> }
> {code}
> After around 280 iterations I start seeing this error message
> (java.io.IOException: *Too many open files*):
> {code:java}
> java.lang.IllegalStateException: Unexpected failure opening I/O selector
> at
> org.apache.hc.core5.reactor.AbstractSingleCoreIOReactor.<init>(AbstractSingleCoreIOReactor.java:60)
> at
> org.apache.hc.core5.reactor.SingleCoreIOReactor.<init>(SingleCoreIOReactor.java:81)
> at
> org.apache.hc.core5.reactor.DefaultConnectingIOReactor.<init>(DefaultConnectingIOReactor.java:71)
> at
> org.apache.hc.client5.http.impl.async.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:924)
> at com.mimecast.micro.rest.BenTest.leakApache(BenTest.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> Caused by: java.io.IOException: Too many open files
> at sun.nio.ch.IOUtil.makePipe(Native Method)
> at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
> at
> sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
> at java.nio.channels.Selector.open(Selector.java:227)
> at
> org.apache.hc.core5.reactor.AbstractSingleCoreIOReactor.<init>(AbstractSingleCoreIOReactor.java:58)
> ... 26 more {code}
> If I run lsof i see around 10k files. These files do not get released until
> the whole test closes.
> Is there something I'm missing?
> Thanks,
> Ben
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
