[
https://issues.apache.org/jira/browse/CAMEL-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-25906:
--------------------------------
Fix Version/s: 4.24.0
> camel-ftp / sftp / smb / azure-files - checksumFileAlgorithm writes no
> checksum file since 4.18, or the checksum of an earlier file producer's
> payload
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25906
> URL: https://issues.apache.org/jira/browse/CAMEL-25906
> Project: Camel
> Issue Type: Bug
> Components: camel-file, camel-ftp
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> {{checksumFileAlgorithm}} ("calculate a checksum from the file that has been
> written ... and store the result in the CamelFileChecksum header";
> {{checksumWriteFile}} writes it as {{<file>.<algorithm>}}) is declared on
> {{GenericFileEndpoint}}, so it is an option of every file component: file,
> ftp, ftps, sftp, mina-sftp, scp, smb and azure-files. CAMEL-20172 (4.3) added
> it for "camel-file/ftp": {{GenericFileProducer.writeChecksumFile}} calculated
> the checksum from the body for every producer.
> CAMEL-22846 (4.18) moved the calculation into {{FileOperations.storeFile}} (a
> digest while it writes the local file), and
> {{GenericFileProducer.writeChecksumFile}} (main 74ee085f7d64, line 290) now
> only writes the header that the operations set:
> {code:java}
> String hash = exchange.getMessage().getHeader(FileConstants.FILE_CHECKSUM,
> String.class);
> if (hash != null) {
> ... operations.storeFileDirectly(checksumFileName, hash) ...
> }
> {code}
> Only {{FileOperations}} sets {{CamelFileChecksum}}. So since 4.18:
> * an ftp, ftps, sftp, mina-sftp, scp, smb or azure-files producer with
> {{checksumFileAlgorithm}} uploads the file and writes no checksum file and no
> header, without an error or a log line. A receiver that waits for
> {{<file>.sha256}} (or verifies it) never gets it; an upgrade from 4.14 LTS to
> 4.18 LTS loses the checksum files silently;
> * when an earlier file producer of the route set the header, e.g.
> {{.to("file:archive?checksumFileAlgorithm=sha256").to("sftp:...?checksumFileAlgorithm=sha256")}},
> the remote producer writes that checksum: the checksum of the archive copy,
> which is not the uploaded file as soon as the route changed the body in
> between, or the remote producer encodes it with its {{charset}}.
> *Fix* (in {{GenericFileProducer}}, one place for all components): before the
> write, a {{CamelFileChecksum}} header set earlier is removed; for operations
> other than {{FileOperations}} the checksum is calculated of the bytes the
> remote operations upload. ftp, ftps, sftp, mina-sftp and smb read the body as
> an {{InputStream}} ({{getMandatoryBody(InputStream.class)}}), or through
> {{GenericFileHelper.toInputStream}} when the endpoint has a {{charset}}; scp
> and azure-files ignore {{charset}} (the upgrade note says not to combine it
> with {{checksumFileAlgorithm}} there). A body that gives the same bytes each
> time (text, bytes, a file, a stream cache) is read once more for the
> checksum; any other body is turned into a stream cache first when it can be
> (a {{Reader}} becomes a reader cache), so the checksum does not use it up
> before the upload. An {{InputStream}} that can be read once is wrapped in a
> {{DigestInputStream}} that the upload reads, so its checksum takes no second
> pass and no memory; with a {{charset}} such a stream is kept as a stream
> cache first (memory, or a spool file when spooling is enabled). The
> {{CamelFileChecksum}} header is set once the file is in place (after the
> upload and the rename of a temporary file). Reading a file body twice: for a
> large file that is a second read from disk, the price of keeping the fix in
> one place instead of a digest in the upload code of each of the seven
> operations classes. With {{fileExist=Ignore}} and an existing file nothing is
> uploaded and no checksum is written, as with the file component.
> {{FileOperations}} is unchanged. Upgrade note in the 4.24 guide.
> *Tests*: new {{FtpProducerChecksumFileAlgorithmIT}} (7 tests, camel-ftp, the
> embedded FTP server of camel-test-infra-ftp, no Docker): the default
> ({{checksumWriteFile=true}}) with a String body, a one-shot stream body, a
> String and a one-shot stream with {{charset=ISO-8859-1}},
> {{checksumWriteFile=false}} (header only), and a file producer earlier in the
> route. On main all 5 fail: {{Checksum file should exist ==> expected: <true>
> but was: <false>}} (3), {{expected: <a591a6d4...> but was: <null>}} (header),
> and {{expected: <01285a4c...> but was: <a591a6d4...>}} (the checksum of the
> archived {{Hello World}} written for an uploaded {{Bye World}}).
> {{testChecksumOfReaderBody}} (a {{StringReader}} body, with and without
> {{charset}}) and {{testNoChecksumHeaderWhenTheExistingFileIsKept}}
> ({{tempFileName}}, {{fileExist=Ignore}}, {{eagerDeleteTargetFile=false}},
> existing target) were added in review; they fail on the first version of the
> fix ({{Stream closed}}, and a checksum header for a file that was not
> written). With the fix: camel-ftp unit tests and FTP producer ITs ({{*Test}},
> {{FtpProducer*IT}}, embedded server) 156 run, 0 failures (12 skipped);
> camel-core {{org.apache.camel.component.file}} tests 474 run, 0 failures (12
> skipped); camel-file 28 run.
> *How it was found*: following up the producer paths of
> {{checksumFileAlgorithm}} while fixing its charset path in camel-file
> (separate ticket): the option is declared on the shared endpoint, the
> calculation lives in one subclass of {{GenericFileOperations}}.
> *Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.18.x (read, not run:
> {{writeChecksumFile}} reads only the header, and no remote operations set
> it). camel-4.14.x calculated the checksum for every producer.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
