[
https://issues.apache.org/jira/browse/CAMEL-25757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25757 started by shashank.
----------------------------------------
> camel-azure-files - resumeDownload never completes an interrupted download:
> the 206 Partial Content answer to the ranged download is taken as a failure,
> and the file is then never routed
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25757
> URL: https://issues.apache.org/jira/browse/CAMEL-25757
> Project: Camel
> Issue Type: Bug
> Components: camel-azure
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> With {{resumeDownload=true}} and a {{localWorkDirectory}}, an interrupted
> download leaves {{<name>.inprogress}} in the local work directory. On the
> next poll {{FilesOperations.retrieveFileToFileInLocalWorkDirectory}} (main
> ddca15435112, lines 376-387) asks the file service for the rest of the file
> and accepts only 200:
> {code:java}
> ShareFileRange range = new ShareFileRange(0);
> if (resumeDownload && existingSize > 0) {
> range = new ShareFileRange(existingSize);
> }
> var ret = cwd().getFileClient(remoteName).downloadWithResponse(os, range,
> null, endpoint.getDataTimeout(), Context.NONE);
> result = ret.getStatusCode() == HTTP_OK;
> {code}
> With a range that does not start at 0 the SDK sends {{x-ms-range:
> bytes=<existingSize>-}} ({{ShareFileRange.toHeaderValue()}}), and Azure Files
> answers a ranged Get File with *206 Partial Content*. The Azure Files REST
> reference ("Specifying the range header for FileREST operations", format
> {{bytes=startByte-}}) says: "If the offset is valid and does not exceed the
> file's total length, the request will return a status code 206 (Partial
> Content)." The SDK's generated {{FilesService.download}} also declares 200
> and 206 as the expected answers. So:
> # poll 1 after the interruption: the rest of the file is appended to
> {{<name>.inprogress}} (which now holds the whole file), but {{result}} is
> {{false}}: the file is not renamed, the consumer reports "Cannot retrieve
> file" and does not route it;
> # every later poll: the range now starts at the end of the file; the service
> answers 416 (Requested Range Not Satisfiable, as HTTP range rules give for a
> start at or after the end), the download throws, and the {{.inprogress}} file
> is kept because {{resumeDownload=true}}. (Even if the service answered such a
> range with an empty 206, it is still not 200 and the result is the same.)
> The file is never routed again, and an error is logged on every poll, until
> someone deletes the {{.inprogress}} file by hand. This is the only case
> {{resumeDownload}} exists for, so the option never works. A download without
> a range (no {{.inprogress}} file, or {{resumeDownload=false}}) is answered
> with 200 and is not affected.
> *Fix*: a 206 answer to a download whose range starts after 0 is a successful
> retrieve ({{HTTP_PARTIAL_CONTENT}} next to {{HTTP_OK}}). The status of the
> other downloads is still checked against 200 only.
> *Tests*: new {{FilesOperationsResumeDownloadTest}} (2 tests). The real
> {{FilesOperations}} and the real Azure SDK client, with an {{HttpClient}}
> that answers Get File like Azure Files (200 without a range, 206 with a range
> inside the file, 416 with a range at or after its end).
> {{resumedDownloadIsCompletedAndRenamed}}: a 4-byte {{hello.txt.inprogress}}
> of a 10-byte file; the SDK asks for {{bytes=4-}}; on main {{retrieveFile}}
> returns false ({{expected: <true> but was: <false>}}); with the fix it
> returns true and the local work file holds the 10 bytes.
> {{downloadWithoutPartialFileIsRetrieved}} (no {{.inprogress}} file) passes on
> main and with the fix; it shows that the normal download is unchanged.
> camel-azure-files unit tests: 30 run, 0 failures.
> *How it was found*: a Z3 model of the byte ranges of a resumed download
> across polls (size of the {{.inprogress}} file before each poll, remote size
> S, the status the service gives for the range, the {{result}} check, the
> rename). On the code of main the formula "the file is routed in one of the 6
> polls after an interruption at byte k" is unsatisfiable for every S > 0 and 0
> < k < S (counterexample S=2, k=1: in-progress sizes 1, 2, 2, 2, ...); with
> the fix the file is routed on the first poll and the routed size is always S.
> The JUnit test above is the evidence.
> Not changed: a {{.inprogress}} file that already holds the whole file (the
> JVM stopped after the last byte and before the rename) still gets 416 on the
> next poll; that needs a comparison with the remote size and is a separate
> case.
> *Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x
> (the same lines; read, not run).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
