tomaswolf commented on issue #485:

   After some analysis, here's what's going on:
   transferTo/transferFrom, as well as the FileChannel.write() operations, are 
_positional_ operations. `readableChannel.transferTo(0, length, 
writeableChannel)` will essentially read 8kB `ByteBuffer`s from the file and 
then call `writeableChannel.write()` for each buffer.
   However, `SftpRemotePathChannel.write()` doesn't know that it is being 
called essentially for a sequential copy operation, and so it doesn't employ a 
number of optimizations. The result is the slow transfer.
   If you change the logic and use `writeableChannel.transferFrom()`, then the 
`SftpRemotePathChannel` drives the operation, and it knows that it is going to 
sequentially read buffers. Hence it can employ these optimizations.
   When you use OutputStream/InputStream as in my `Files.copy()` examples, then 
it is known that a sequential data transfer occurs, and the SFTP implementation 
can employ its optimizations unconditionally.
   Finally, transferTo/transferFrom by default copy data in 8kB chunks. With 
streams, the chunks are about 32kB. This difference causes the 25% slowdown.
   - In general, using streams is the simplest for downloading and uploading 
files, and gives good performance.
   - If you want to use `FileChannel`s:
     - Always let the remote channel drive the operation. Use transferFrom for 
uploading and transferTo for downloading.
     - Execute transferTo/From in a loop until all data has been transferred.
     - Increase the transfer buffer size via 
`SftpModuleProperties.COPY_BUF_SIZE.set(session, 32 * 1024);`
   It might be possible to improve our implementation to handle the case you 
stumbled upon better, but I'm not sure yet. 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail:

For queries about this service, please contact Infrastructure at:

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to