[
https://issues.apache.org/jira/browse/SSHD-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250495#comment-17250495
]
min yun law commented on SSHD-1078:
-----------------------------------
test using mina 2.6.0 with sftp with three big files uploading to the same host
at the same time, and a 10 loops, look like it pass without issue. look like it
fix the issue in 2.6.0
-rw-r--r--. 1 root root 5.2G Dec 16 12:34
p30032947_187000DBRU_Linux-x86-64.zip.sftp
-rw-r--r--. 1 root root 8.1G Dec 16 12:37
p30397820_188000GIRU_Linux-x86-64.zip.sftp
-rw-r--r--. 1 root root 8.1G Dec 16 12:37
p30032946_187000GIRU_Linux-x86-64.zip.sftp
retry count: 0
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 288s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 391s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 391s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 1
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 291s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 403s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 405s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 2
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 292s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 419s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 425s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 3
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 330s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 444s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 444s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 4
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 285s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 399s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 400s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 5
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 331s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 444s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 445s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 6
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 376s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 497s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 497s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
retry count: 7
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 286s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 408s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 409s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 8
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 298s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 409s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 412s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
retry count: 9
Using SFTP for upload....
Upload file p30032947_187000DBRU_Linux-x86-64.zip in 356s
*****done for p30032947_187000DBRU_Linux-x86-64.zip*************
Upload file p30397820_188000GIRU_Linux-x86-64.zip in 499s
*****done for p30397820_188000GIRU_Linux-x86-64.zip*************
Upload file p30032946_187000GIRU_Linux-x86-64.zip in 500s
*****done for p30032946_187000GIRU_Linux-x86-64.zip*************
> SCP/SFTP failed for big file upload
> -----------------------------------
>
> Key: SSHD-1078
> URL: https://issues.apache.org/jira/browse/SSHD-1078
> Project: MINA SSHD
> Issue Type: Bug
> Affects Versions: 2.5.1
> Reporter: min yun law
> Priority: Major
>
> Trying to merge product from jsch to mina sshd that keep the same SSH
> architecture (lets call SessionPool) which for each remote node, there is
> only one session object and limited channels (<=8) for executing commands and
> file operations. The SessionPool manages the session/channels for nodes and
> make sure only 8 threads(ssh channels) per node that can be run concurrently.
> This architecture works for Jsch for many years, but when using mina sshd,
> this architecture wont work when with upload big file, we always see
> following exception:
> {code:java}
> java.io.EOFException: Channel is being closed
> at
> org.apache.sshd.common.channel.AbstractChannel$2.<init>(AbstractChannel.java:794)
> at
> org.apache.sshd.common.channel.AbstractChannel.writePacket(AbstractChannel.java:792)
> at
> org.apache.sshd.common.channel.ChannelAsyncOutputStream.doWriteIfPossible(ChannelAsyncOutputStream.java:168)
> at
> org.apache.sshd.common.channel.ChannelAsyncOutputStream.writePacket(ChannelAsyncOutputStream.java:70)
> at
> org.apache.sshd.client.subsystem.sftp.impl.DefaultSftpClient.send(DefaultSftpClient.java:292)
> at
> org.apache.sshd.client.subsystem.sftp.impl.SftpOutputStreamAsync.flush(SftpOutputStreamAsync.java:210)
> at
> org.apache.sshd.client.subsystem.sftp.impl.SftpOutputStreamAsync.write(SftpOutputStreamAsync.java:141)
> {code}
> From SSH server log, we see follow error:
> {code:java}
> Aug 19 15:56:38 xxxx sftp-server[202217]: error: Unknown message 0
> Aug 19 15:56:38 xxxx sftp-server[202217]: error: msg_len 0 < consumed 1
> {code}
> other client side exceptions by mina sshd as:
> org.apache.sshd.common.channel.WindowClosedException: Already closed:
> Window[client/remote](ChannelExec[id=79, recipient=1]-ClientSessionImp
>
> ----
> After many tries, finally I found that for big file upload/download, I have
> to give its own SshClient/ClientSession objects and then there will no such
> channel closing exception during file upload.
> That means we cannot share the same SshClient and ClientSession object per
> remote node to execute commands and file operations, and there may has bug in
> ClientSession when handle channels in this situation.
> the reason that we like to share the same SshClient/ClientSession for the
> same remote node is to manage the session/channels and close them when no
> usage, also to reduce the cost to allocate those objects.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
