[
https://issues.apache.org/jira/browse/CAMEL-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25906 started by shashank.
----------------------------------------
> camel-ftp / sftp / smb / azure-files - checksumFileAlgorithm writes no
> checksum file since 4.18, or the checksum of an earlier file producer's
> payload
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25906
> URL: https://issues.apache.org/jira/browse/CAMEL-25906
> Project: Camel
> Issue Type: Bug
> Components: camel-file, camel-ftp
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> {{checksumFileAlgorithm}} ("calculate a checksum from the file that has been
> written ... and store the result in the CamelFileChecksum header";
> {{checksumWriteFile}} writes it as {{<file>.<algorithm>}}) is declared on
> {{GenericFileEndpoint}}, so it is an option of every file component: file,
> ftp, ftps, sftp, mina-sftp, scp, smb and azure-files. CAMEL-20172 (4.3) added
> it for "camel-file/ftp": {{GenericFileProducer.writeChecksumFile}} calculated
> the checksum from the body for every producer.
> CAMEL-22846 (4.18) moved the calculation into {{FileOperations.storeFile}} (a
> digest while it writes the local file), and
> {{GenericFileProducer.writeChecksumFile}} (main 74ee085f7d64, line 290) now
> only writes the header that the operations set:
> {code:java}
> String hash = exchange.getMessage().getHeader(FileConstants.FILE_CHECKSUM,
> String.class);
> if (hash != null) {
> ... operations.storeFileDirectly(checksumFileName, hash) ...
> }
> {code}
> Only {{FileOperations}} sets {{CamelFileChecksum}}. So since 4.18:
> * an ftp, ftps, sftp, mina-sftp, scp, smb or azure-files producer with
> {{checksumFileAlgorithm}} uploads the file and writes no checksum file and no
> header, without an error or a log line. A receiver that waits for
> {{<file>.sha256}} (or verifies it) never gets it; an upgrade from 4.14 LTS to
> 4.18 LTS loses the checksum files silently;
> * when an earlier file producer of the route set the header, e.g.
> {{.to("file:archive?checksumFileAlgorithm=sha256").to("sftp:...?checksumFileAlgorithm=sha256")}},
> the remote producer writes that checksum: the checksum of the archive copy,
> which is not the uploaded file as soon as the route changed the body in
> between, or the remote producer encodes it with its {{charset}}.
> *Fix* (in {{GenericFileProducer}}, one place for all components): before the
> write, a {{CamelFileChecksum}} header set earlier is removed; for operations
> other than {{FileOperations}} the checksum is calculated of the bytes the
> remote operations upload. All of them read the body as an {{InputStream}}
> ({{getMandatoryBody(InputStream.class)}}), or through
> {{GenericFileHelper.toInputStream}} when the endpoint has a {{charset}}. A
> stream body that can be read once is wrapped in a {{DigestInputStream}} that
> the upload reads, so its checksum takes no second pass and no memory; with a
> {{charset}} such a stream is kept as a stream cache first (memory, or a spool
> file when spooling is enabled). Other bodies (a file, bytes, text, a stream
> cache) are read once more for the checksum: for a large file that is a second
> read from disk, the price of keeping the fix in one place instead of a digest
> in the upload code of each of the seven operations classes. With
> {{fileExist=Ignore}} and an existing file nothing is uploaded and no checksum
> is written, as with the file component. {{FileOperations}} is unchanged.
> Upgrade note in the 4.24 guide.
> *Tests*: new {{FtpProducerChecksumFileAlgorithmIT}} (5 tests, camel-ftp, the
> embedded FTP server of camel-test-infra-ftp, no Docker): the default
> ({{checksumWriteFile=true}}) with a String body, a one-shot stream body, a
> String and a one-shot stream with {{charset=ISO-8859-1}},
> {{checksumWriteFile=false}} (header only), and a file producer earlier in the
> route. On main all 5 fail: {{Checksum file should exist ==> expected: <true>
> but was: <false>}} (3), {{expected: <a591a6d4...> but was: <null>}} (header),
> and {{expected: <01285a4c...> but was: <a591a6d4...>}} (the checksum of the
> archived {{Hello World}} written for an uploaded {{Bye World}}). With the
> fix: camel-ftp unit tests 81 run, 0 failures (9 skipped); camel-ftp producer
> ITs ({{*Producer*IT}}: 42 FTP and 4 SFTP classes with the embedded servers)
> 86 run, 0 failures (3 skipped); camel-core
> {{org.apache.camel.component.file}} tests 474 run, 0 failures (12 skipped);
> camel-file 28 run.
> *How it was found*: following up the producer paths of
> {{checksumFileAlgorithm}} while fixing its charset path in camel-file
> (separate ticket): the option is declared on the shared endpoint, the
> calculation lives in one subclass of {{GenericFileOperations}}.
> *Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.18.x (read, not run:
> {{writeChecksumFile}} reads only the header, and no remote operations set
> it). camel-4.14.x calculated the checksum for every producer.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
