[JSch-users] Problem transferring files larger than 4GB via SFTP

[Nicolas Kukolja]

Hi,

 

I am implementing a java application that transfers (large) files to an ftp-server, using sftp (JSch).

 

Everything works fine so far with small files. But if a file is larger, the transfer hangs always at about 4GB - 4,1GB and doesn't move on.

After 10 minutes, I get an "java.io.IOException: Pipe closed". The "rekey"-Param (http://www.proftpd.org/docs/contrib/mod_sftp.html#SFTPRekey) of the server is configured to 500MB, and the key exchange seems to work fine, as the server log tells me.

 

Does anyone have a suggestion, what I am doing wrong?

If you need any more information, please give me a hint...

 

Kind regards,

Nicolas

 

 

 

Environment:

Client:

- Windows 2008 Server

- Java 1.6.0_45 x64

- Java application using JSch 0.1.50.jar

 

Server:

- Solaris 10

- FTP-Server: http://www.proftpd.org

 

 

 

My connect-method:

   public boolean connect( Long orderId )
   {
      try
      {
         this.jSch = new JSch();

         logger.info( "[Order:" + orderId + "] Connecting to SFTP-Server: " + this.username + ":" + this.password + "@" + this.host + ":" + this.port );
         // create session
         this.sftpSession = this.jSch.getSession( this.username, this.host, this.port );

         // build config
         Hashtable<String,String> config = new Hashtable<String,String>();
         config.put( "StrictHostKeyChecking", "no" );
         this.sftpSession.setConfig( config );
         this.sftpSession.setPassword( this.password );

         // establish connection
         this.sftpSession.connect();

         this.sftpChannel = ( ChannelSftp ) this.sftpSession.openChannel( "sftp" );
         this.sftpChannel.connect();

         logger.info( "[Order:" + orderId + "] Current directory on SFTP-Server (pwd): " + this.sftpChannel.pwd() );
         if( this.ftpSubdir != null && this.ftpSubdir.length() > 0 )
         {
            logger.info( "[Order:" + orderId + "] Changing to subdirectory on SFTP-Server (cd): " + this.ftpSubdir );
            this.sftpChannel.cd( this.ftpSubdir );
            logger.info( "[Order:" + orderId + "] Current directory on SFTP-Server (pwd): " + this.sftpChannel.pwd() );
         }
      }
      catch( JSchException e )
      {
         logger.error( "[Order:" + orderId + "] Error connecting to SFTP-Server... (uri: '" + this.sftpUri + "')", e );
         return false;
      }
      catch( SftpException e )
      {
         logger.error( "[Order:" + orderId + "] Error connecting to SFTP-Server... (uri: '" + this.sftpUri + "')", e );
         return false;
      }
      return this.sftpChannel.isConnected();
   }

 

My upload-method:

   public void uploadSynchronized( String source, String target, Long filesize, Long orderId )
   {
      long overallFilesize = filesize == null ? 0 : filesize;
      long overallUploadedBytes = 0;
      long uploadedBytes = 0;

      try
      {
         if( !checkIfFileExists( source ) )
         {
            OutputStream tOut = this.sftpChannel.put( target );
            FileInputStream in = new FileInputStream( source );
            byte[] bytes = new byte[ this.chunkSize ];
            int count = in.read( bytes );
            try
            {
               while( count != -1 && count <= this.chunkSize && !this.abort )
               {
                  tOut.write( bytes, 0, count );

                  uploadedBytes = uploadedBytes + this.chunkSize;
                  if( uploadedBytes >= BYTES_TO_LOG )
                  {

                     overallUploadedBytes = overallUploadedBytes + uploadedBytes;
                     uploadedBytes = 0;
                     logger.info( "[Order:" + orderId + "] " + overallUploadedBytes / MEGA_BYTE + "MB of " + overallFilesize / MEGA_BYTE
                           + "MB uploaded (source: '" + source + "', target: '" + target + "')" );
                  }
                  count = in.read( bytes );
               }
            }
            finally
            {
               in.close();
               tOut.close();
            }
         }
         else
         {
            this.success = false;
            logger.error( "[Order:" + orderId + "] Error uploading file... (source: '" + source + "', target: '" + target
                  + "'): File already exists on SFTP-Server" );
         }
      }
      catch( IOException e )
      {
         this.success = false;
         logger.error( "[Order:" + orderId + "] Error uploading file... (source: '" + source + "', target: '" + target + "')", e );

      }
      catch( SftpException e )
      {
         this.success = false;
         logger.error( "[Order:" + orderId + "] Error uploading file... (source: '" + source + "', target: '" + target + "')", e );
      }
      catch( Exception e )
      {
         this.success = false;
         logger.error( "[Order:" + orderId + "] Error uploading file... (source: '" + source + "', target: '" + target + "')", e );
      }
   }

------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk
_______________________________________________
JSch-users mailing list
JSch-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jsch-users