[
https://issues.apache.org/jira/browse/CAMEL-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616441#comment-14616441
]
Grzegorz Grzybek commented on CAMEL-8860:
-----------------------------------------
[~himanshu], I checked again with camel 2.16-SNAPSHOT and even when I used
passive mode and NO tempFileName, I had connection per write.
I've additionally set a range of passive ports to be used in ftp server
(mina-ftp used in camel ftp tests) to {{40000-40020}} and I wrote 1000 files.
{{ss -napt | wc -l}} showed me 1000 sockets in TIME-WAIT state like this:
{noformat}
...
TIME-WAIT 0 0 ::ffff:127.0.0.1:60184 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60185 ::ffff:127.0.0.1:40006
TIME-WAIT 0 0 ::ffff:127.0.0.1:60190 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60193 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60197 ::ffff:127.0.0.1:40006
TIME-WAIT 0 0 ::ffff:127.0.0.1:60212 ::ffff:127.0.0.1:40006
TIME-WAIT 0 0 ::ffff:127.0.0.1:60217 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60220 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60235 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60241 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60259 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60274 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60277 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60284 ::ffff:127.0.0.1:40006
TIME-WAIT 0 0 ::ffff:127.0.0.1:60290 ::ffff:127.0.0.1:40006
TIME-WAIT 0 0 ::ffff:127.0.0.1:60295 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60301 ::ffff:127.0.0.1:40003
TIME-WAIT 0 0 ::ffff:127.0.0.1:60322 ::ffff:127.0.0.1:40003
...
{noformat}
So no problem with reusing TIME-WAIT sockets.
Also I've used my own socket factory to be used in ftp endpoint:
{code:java}
@Override
protected JndiRegistry createRegistry() throws Exception {
JndiRegistry jndi = super.createRegistry();
SocketFactory sf = new AuditingSocketFactory();
jndi.bind("sf", sf);
return jndi;
}
{code}
I used this parameter in FTP endpoint: {{ftpClient.socketFactory=#sf}}. My
socket factory just logged information about opening and closing sockets. I had
1000 blocks of such logs:
{noformat}
2015-07-07 11:49:38,264 [main ] DEBUG ProducerCache
- >>>>
Endpoint[ftp://admin@localhost:21000/out/?disconnect=false&fileName=%24%7Bfile%3Aname%7D&ftpClient.socketFactory=%23sf&passiveMode=true&password=xxxxxx&tempFileName=%2Ftemp%2Fhello.tmp]
Exchange[Message: Hello World 00018]
2015-07-07 11:49:38,266 [main ] INFO NameProducerConnectionLeakTest
- Creating socket 298317613
2015-07-07 11:49:38,266 [main ] INFO NameProducerConnectionLeakTest
- connecting socket 298317613 to /127.0.0.1:40014
2015-07-07 11:49:38,267 [main ] INFO NameProducerConnectionLeakTest
- closing socket 298317613
2015-07-07 11:49:38,267 [main ] INFO NameProducerConnectionLeakTest
- closing socket 298317613
2015-07-07 11:49:38,307 [main ] INFO NameProducerConnectionLeakTest
- Creating socket 1597289593
2015-07-07 11:49:38,307 [main ] INFO NameProducerConnectionLeakTest
- connecting socket 1597289593 to /127.0.0.1:40001
2015-07-07 11:49:38,308 [main ] INFO NameProducerConnectionLeakTest
- closing socket 1597289593
2015-07-07 11:49:38,308 [main ] INFO NameProducerConnectionLeakTest
- closing socket 1597289593
2015-07-07 11:49:38,350 [main ] DEBUG FtpOperations
- About to store file: hello.tmp using stream:
java.io.ByteArrayInputStream@7d34a4c3
2015-07-07 11:49:38,351 [main ] INFO NameProducerConnectionLeakTest
- Creating socket 187514477
2015-07-07 11:49:38,351 [main ] INFO NameProducerConnectionLeakTest
- connecting socket 187514477 to /127.0.0.1:40003
2015-07-07 11:49:38,352 [main ] INFO NameProducerConnectionLeakTest
- closing socket 187514477
2015-07-07 11:49:38,352 [main ] INFO NameProducerConnectionLeakTest
- closing socket 187514477
2015-07-07 11:49:38,391 [main ] DEBUG FtpOperations
- Took 0.041 seconds (41 millis) to store file: hello.tmp and FTP client
returned: true
2015-07-07 11:49:38,393 [main ] DEBUG RemoteFileProducer
- Wrote [out//temp/hello.tmp] to
[Endpoint[ftp://admin@localhost:21000/out/?disconnect=false&fileName=%24%7Bfile%3Aname%7D&ftpClient.socketFactory=%23sf&passiveMode=true&password=xxxxxx&tempFileName=%2Ftemp%2Fhello.tmp]]
2015-07-07 11:49:38,393 [main ] DEBUG FtpOperations
- Renaming file: out//temp/hello.tmp to: out/file-00018.txt
{noformat}
> Too many connection in TIME_WAIT when tempfilename is set in the FTP producer
> URI.
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-8860
> URL: https://issues.apache.org/jira/browse/CAMEL-8860
> Project: Camel
> Issue Type: Bug
> Components: camel-ftp
> Affects Versions: 2.14.0
> Reporter: Himanshu Mittal
> Assignee: Grzegorz Grzybek
> Attachments: NetStat_NoTempFIle.PNG, NetStat_Passive_False.PNG,
> NetStat_Passive_true.PNG, Passive_false.log, Passive_false_NoTempFile.log,
> Passive_true.log, ResultWithOutTempFileName.log, ResultWithTempfileName.log,
> Result_With_tempfileName_FTPServer_Connection_Count.PNG,
> Result_Without_tempfilename.PNG, TestFTPConnectionLeak.java
>
>
> So many connections remain in TIME_WAIT state while configuring a FTP
> producer route with tempfilename property in FTP producer.
> Files are writen using producer template.
> Producertemplate.sendBodyAndHeader(producerURI, Exchange.FILE_NAME, fileName)
> to write file to destination.
> Producer URI :
> {{ftp://<USER@MACHINE_ADDRESS>//?password=RAW(PASSWORD)&binary=true&passiveMode=false&delete=true&pollStrategy=#ConsumerPollStrategy&disconnect=false&tempFileName=/temp/hello.tmp&initialDelay=10&throwExceptionOnConnectFailed=true&reconnectDelay=10&maximumReconnectAttempts=0&maxMessagesPerPoll=100&readLock=changed&readLockCheckInterval=500&readLockMinLength=1&sendEmptyMessageWhenIdle=true}}
> fileName: {{file.tmp}}
> We are writing ~ 2000 files . First files are written to temp directory then
> moved to destination. Connections opened for tempfiles are not re used to
> write temporary files and we run out of connection after writing 1000 files.
> Route resumes once connections are released.
> Is there a way we can reuse connection to write temporary files also.
> Once tempfilename is removed from the configuration then single connection is
> used for ftp transfer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
