I've searched the web, and through the archives, looking for similar problems. The most common cause seems to be too few file
descriptors.
We seem to have lots of file descriptors:
cat /proc/sys/fs/file-max
406054
And not many are being used (this is with the code running):
cat /proc/sys/fs/file-nr
2730 0 406054
This happens in a long-running program (it processes images, and keeps going
till there are no more - there are usually a few
thousand in the queue waiting to be downloaded).
A group of images is retrieved via http (wget), resized (via
image-magik-convert), and then ftp'd up to other servers.
The exception always occurs in the ftp "storeFile" portion of the process.
The code is,
try
{
File remoteFile = new File(strRemoteDirectory + f.getName());
boolean bOneSucceeded = false;
for (int iServerCount = 0; iServerCount < ftpConnections.length;
iServerCount++)
{
FTPClient ftpClient = ftpConnections[iServerCount];
if ( !ftpClient.changeWorkingDirectory(strRemoteDirectory) )
{
m_sLogger.info("\t" + new java.util.Date() + "
ListingExchangeImageProcessor: run: could not change directories
(on ftp server) to " + strRemoteDirectory);
continue;
}
fInputStream = new FileInputStream(f);
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
if ( ftpClient.storeFile(f.getName(), fInputStream) )
{
bOneSucceeded = true;
}
}
if ( !bOneSucceeded )
{
m_sLogger.info("\t" + new java.util.Date() + "
ListingExchangeImageProcessor: run: unable to upload to a single ftp
server...");
return;
}
}
catch (Exception e)
{
m_sLogger.info("\t" + new java.util.Date() + "
ListingExchangeImageProcessor: run: exception uploading file: " + e.toString());
e.printStackTrace();
return;
}
finally
{
try
{
fInputStream.close();
}
catch (Exception e) {}
}
The stack trace for the exception is,
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:502)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:388)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1388)
at
listingexchange.ListingExchangeImageProcessor.run(ListingExchangeImageProcessor.java:1143)
at
listingexchange.ListingExchangeImageProcessor.main(ListingExchangeImageProcessor.java:204)
Any insight would be helpful.
David
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
