[
https://issues.apache.org/jira/browse/CAMEL-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-24089:
-------------------------------------
Attachment: FileConsumerFileLockHugeFileTruncationIssueTest.java
> camel-file: file-to-file copy with readLock=fileLock silently truncates files
> larger than 2 GB on Linux (single transferTo, return value ignored)
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24089
> URL: https://issues.apache.org/jira/browse/CAMEL-24089
> Project: Camel
> Issue Type: Bug
> Components: camel-file
> Affects Versions: 4.21.0
> Reporter: Federico Mariani
> Priority: Critical
> Labels: code-review
> Attachments: FileConsumerFileLockHugeFileTruncationIssueTest.java
>
>
> h3. Problem
> Route {{from("file:in?readLock=fileLock").to("file:out")}} keeps the
> read-lock {{FileChannel}} in an exchange property;
> {{FileOperations.writeFileByFile}} (FileOperations.java:515-532) then copies
> via:
> {code:java}
> channel.transferTo(0, channel.size(), out); // single call, return value
> discarded
> {code}
> {{FileChannel.transferTo}} is documented to possibly transfer fewer bytes
> than requested, and the JDK caps each call at {{Integer.MAX_VALUE}} bytes (~2
> 147 479 552 with sendfile on Linux). Any source file larger than ~2 GB
> produces a silently truncated target while the producer reports success - the
> done file is still written and the commit deletes/moves the source.
> *Platform note:* the truncation manifests on Linux, where the JDK takes the
> direct sendfile/copy_file_range path capped at {{Integer.MAX_VALUE}}-ish
> bytes per call ({{sun.nio.ch.FileChannelImpl.transferTo}}: {{int icount =
> (int) Math.min(count, nd.maxDirectTransferSize())}}). On macOS the direct
> transfer is unsupported for file-to-file targets and the JDK falls back to
> {{transferToTrustedChannel}}, an internal loop that copies the full count -
> verified empirically (a single {{transferTo}} of a 3 GB file returned
> 3221225472 on macOS). Relying on that fallback is still a contract violation:
> {{transferTo}} is documented to possibly transfer fewer bytes than requested.
> h3. Failure scenario
> {{readLock=fileLock}} consumer + file producer + source file > 2 GB on Linux:
> target is truncated at ~2 GB, no exception, source is gone after commit. The
> non-locked path ({{Files.copy}}) is unaffected.
> h3. History
> Introduced by CAMEL-13680 (2019). Notably the pre-2017 code this area evolved
> from *did* loop: {{while (position < size) position +=
> in.transferTo(position, bufferSize, out);}} (removed in d8c0a53b6787).
> h3. Suggested fix
> Loop until {{channel.size()}} bytes are transferred (standard transferTo
> loop).
> h3. Reproducer
> Attached JUnit test (place in
> {{core/camel-core/src/test/java/org/apache/camel/component/file/}}, where the
> file component tests live). Uses a sparse 3 GB source file; needs ~3 GB free
> transient disk space. Asserts target length equals source length. It FAILS on
> Linux (target truncated to ~2147483647 bytes) and passes on macOS due to the
> JDK fallback described above. The fix - the standard loop until all bytes are
> transferred - is correct on all platforms.
> ----
> _This issue was researched and filed by Claude Code on behalf of [~croway]
> (GitHub: Croway), as part of a deep code review of camel-file and camel-ftp.
> A JUnit reproducer is attached; it fails deterministically on Linux on
> current main (4.22.0-SNAPSHOT)._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
