[
https://issues.apache.org/jira/browse/NET-686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17244465#comment-17244465
]
Vitaliy Khudenko edited comment on NET-686 at 12/5/20, 11:30 AM:
-----------------------------------------------------------------
Looks like I've encountered this issue too.
My setup is an Android app as an FTPS client running on a physical device
(Google Nexus, Android 8.1) or an emulator (Android 10) from Android Studio
(Win 10). VPN.
On the FTPS server I have 3 zip files (1, 4 and 74 KBytes).
Small files (1 and 4 KB) are never causing troubles. However for the largest
one the issue happens with ~25-33% probability.
The exact file size is 75'743 bytes. But it happens that only 75'741 bytes or
75'740 bytes is downloaded (missing just a few bytes) with no errors reported
by {{FTPSClient}}.
Here is my code (Kotlin):
{code:java}
private val CONNECT_TIMEOUT_MILLIS = 120_000 // 2 minutes
private val SO_TIMEOUT_MILLIS = 120_000 // 2 minutes
private val BUFFER_SIZE = 1024 // 1 KB
val client: FTPSClient = FTPSClient("TLS", false)
client.connectTimeout = CONNECT_TIMEOUT_MILLIS
client.connect(hostname, port)
client.soTimeout = SO_TIMEOUT_MILLIS
if (!FTPReply.isPositiveCompletion(client.replyCode)) {
throw Exception("FTP server refused connection")
}
if (!client.login(username, password)) {
throw Exception("FTP server refused to authenticate $username")
}
FileOutputStream(localTargetFile).use { outputStream: FileOutputStream ->
val status = client.retrieveFile(ftpFilePath, outputStream)
outputStream.flush()
check(status) { "negative retrieveFile status" } // throws Exception if
status is false, but this is always true
}
{code}
I tried to mitigate this with a retry mechanism where I would resume
downloading for the number of the missing bytes only (e.g. the second attempt
only fetches the last missing few bytes). But it still results in a corrupted
zip file, despite the total number of bytes is correct. Looks like the first
pass/attempt already brings a corrupted chunk, so resumed download does not
help:
{code:java}
repeat(ATTEMPTS_TO_DOWNLOAD) {
if (localTargetFile.exists()) {
client.restartOffset = localTargetFile.length()
FileOutputStream(localTargetFile, true)
} else {
FileOutputStream(localTargetFile)
}.use { outputStream: FileOutputStream ->
val status = client.retrieveFile(ftpFilePath, outputStream)
outputStream.flush()
check(status) { "negative retrieveFile status" } // throws Exception if
status is false, but this is always true
}
}{code}
The odd things I noticed (which is maybe due to FTPS server misconfig/missetup,
but FTPS server setup is beyond my control):
* {{FTPClient.mlistFile()}} always returns {{null}}
* I am not able to browse the FTPS content via WinSCP
was (Author: vkhudenko):
Looks like I've encountered this issue too.
My setup is an Android app as an FTPS client running on a physical device
(Google Nexus, Android 8.1) or an emulator (Android 10) from Android Studio
(Win 10). VPN.
On the FTPS server I have 3 zip files (1, 4 and 74 KBytes).
Small files (1 and 4 KB) are never causing troubles. However for the largest
one the issue happens with ~25-33% probability.
The exact file size is 75'743 bytes. But it happens that only 75'741 bytes or
75'740 bytes is downloaded (missing just a few bytes) with no errors reported
by {{FTPSClient}}.
Here is my code (Kotlin):
{code:java}
private val CONNECT_TIMEOUT_MILLIS = 120_000 // 2 minutes
private val SO_TIMEOUT_MILLIS = 120_000 // 2 minutes
private val BUFFER_SIZE = 1024 // 1 KB
val client: FTPSClient = FTPSClient("TLS", false)
client.connectTimeout = CONNECT_TIMEOUT_MILLIS
client.connect(config.hostname, config.port)
client.soTimeout = SO_TIMEOUT_MILLIS
if (!FTPReply.isPositiveCompletion(client.replyCode)) {
throw Exception("FTP server refused connection")
}
if (!client.login(username, password)) {
throw Exception("FTP server refused to authenticate $username")
}
FileOutputStream(localTargetFile).use { outputStream: FileOutputStream ->
val status = client.retrieveFile(ftpFilePath, outputStream)
outputStream.flush()
check(status) { "negative retrieveFile status" } // throws Exception if
status is false, but this is always true
}
{code}
I tried to mitigate this with a retry mechanism where I would resume
downloading for the number of the missing bytes only (e.g. the second attempt
only fetches the last missing few bytes). But it still results in a corrupted
zip file, despite the total number of bytes is correct. Looks like the first
pass/attempt already brings a corrupted chunk, so resumed download does not
help:
{code:java}
repeat(ATTEMPTS_TO_DOWNLOAD) {
if (localTargetFile.exists()) {
client.restartOffset = localTargetFile.length()
FileOutputStream(localTargetFile, true)
} else {
FileOutputStream(localTargetFile)
}.use { outputStream: FileOutputStream ->
val status = client.retrieveFile(ftpFilePath, outputStream)
outputStream.flush()
check(status) { "negative retrieveFile status" } // throws Exception if
status is false, but this is always true
}
}{code}
The odd things I noticed (which is maybe due to FTPS server misconfig/missetup,
but FTPS server setup is beyond my control):
* {{FTPClient.mlistFile()}} always returns {{null}}
* I am not able to browse the FTPS content via WinSCP
> Most files aren't downloaded completely from an FTP server
> ----------------------------------------------------------
>
> Key: NET-686
> URL: https://issues.apache.org/jira/browse/NET-686
> Project: Commons Net
> Issue Type: Bug
> Components: FTP
> Affects Versions: 3.6, 3.7.2
> Environment: Win 10
> Java 8
> Android Studio 3.6.1 (min SDK 24, target SDK 27)
> Reporter: JRRR
> Priority: Major
> Attachments: 2a-original.png, 2b-corrupt.png, 2c-corrupt.png,
> 5a-original.jpg, 5b-corrupt.jpg, 5c-corrupt.jpg, DownloadProblem.java
>
>
> About a month ago I opened another
> [issue|https://issues.apache.org/jira/browse/NET-684] that was closed because
> it wasn't reproducible with macOS and a public FTP server.
> Short summary: Downloading files from an FTP server results in these files
> randomly missing bytes. It looks like the download always "completes" and
> there are no error messages/exceptions but random bytes in random files are
> simply skipped. Images (jpg & png) are usually affected more (up to 30, maybe
> 40, bytes smaller than the original), and are then also visibly corrupt, than
> text files (usually only 2-3 bytes smaller, rarely more).
> I'm working on an Android app (Win 10, Java 8, Android Studio 3.6.1, min SDK
> 24, target SDK 27), which I'm testing with FTP servers in the same network
> (1x Win 10, 1x Linux, both accessed via IP - "10.1.1.xxx"). No matter what
> method in the library I use (retrieveFile, retrieveFileStream,
> sendCommand(FTPCmd.RETRIEVE, filename)), most of the time there's at least a
> single file that's corrupted.
> I also tested the same code with public servers and even though I didn't have
> a lot of time because those servers regularely delete uploaded files, I never
> experienced said problem with them.
> I even wrote my own mini-library (just for login/logout and download) using
> Java's default "Socket" but I still had the same problem on Android Studio's
> simulator/a real device. BUT: When I used the same code to create a small
> Windows/Swing/Java app, there were no more corrupted files.
> It looks like this bug is only affecting a very specific combination of
> OS,...:
> Android (emulator/real device) + Java (8) + FTP server in the same network
> (accessed via IP)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
