shashank created CAMEL-25723:
--------------------------------
Summary: camel-zipfile - ZipAggregationStrategy silently replaces
a zip entry when a later message has the same entry name, e.g. two orders.csv
from different folders of a recursive file consumer, so the earlier file is lost
Key: CAMEL-25723
URL: https://issues.apache.org/jira/browse/CAMEL-25723
Project: Camel
Issue Type: Bug
Components: camel-zipfile
Reporter: shashank
Assignee: shashank
Since CAMEL-13399 (Camel 3.0) {{ZipAggregationStrategy}} adds each message to
the zip file through a {{ZipFileSystem}}: {{Files.copy(file, dest,
REPLACE_EXISTING)}} for a file body (line 319) and {{Files.write(dest, buffer,
CREATE, TRUNCATE_EXISTING)}} for other bodies (line 334, main 826a664c5f16).
When the zip already has an entry with that name, the entry is replaced and the
content of the earlier message is gone, without an error or a log.
With the default strategy ({{new ZipAggregationStrategy()}},
{{preserveFolderStructure=false}}) the entry name of a file is its name only. A
recursive file consumer that polls {{in/a/orders.csv}} ("from A") and
{{in/b/orders.csv}} ("from B") and aggregates them with the documented
{{completionFromBatchConsumer().eagerCheckCompletion()}} produces one zip with
one entry {{orders.csv}} = "from B". {{useFilenameHeader=true}} with a repeated
{{CamelFileName}} behaves the same. Folders per day or per sender with the same
file names are common.
Trigger: default options of {{ZipAggregationStrategy}}; two messages of one
group with the same entry name (file bodies with the same file name, or the
same {{CamelFileName}} with {{useFilenameHeader=true}}).
Why CAMEL-13399 used these options: the issue was about speed (the strategy
rebuilt the whole zip for every message). The commit "Optimized
ZipAggregationStrategy to use ZipFileSystem" wrote with {{REPLACE_EXISTING}},
and the follow-up "Truncate zip entry, if already exists" added
{{TRUNCATE_EXISTING}} so that a shorter content does not leave the tail of the
old one in the entry. Neither commit nor the issue discusses two messages with
the same name; before 3.0 the {{ZipOutputStream}} rebuild failed with
"duplicate entry" in that case.
{{TarAggregationStrategy}} on main does not lose the content but does not
separate the names either: it rebuilds the tar for each message, copies the
existing entries ({{copyExistingEntries}}, TarAggregationStrategy.java 285-292)
and appends the new one under the same name ({{addNewEntry}} 277-283,
{{createNewEntry}} 321-327). The tar then holds two entries {{orders.csv}}; a
usual extraction writes both to the same path and keeps only the last.
h3. Reproduction
New test {{ZipAggregationStrategySameFileNameTest}} (7 tests). Without the fix
6 fail, for example:
* a recursive file route over only {{a/orders.csv}} and {{b/orders.csv}} with
{{completionFromBatchConsumer().eagerCheckCompletion()}}: {{expected: <[orders
(1).csv=from B, orders.csv=from A]> but was: <[orders.csv=from B]>}};
* the same route with a third file {{c/other.csv}}: {{but was:
<[orders.csv=from B, other.csv=from C]>}};
* the strategy called directly with {{a/orders.csv}} then {{b/orders.csv}}, and
with {{useFilenameHeader=true}} and two byte bodies named {{report.txt}}: the
zip holds only the second content;
* five files, four named {{orders.csv}} and one really named {{orders
(1).csv}}: main keeps 2 of the 5 contents;
* names without an extension ({{README}}, {{.profile}}).
The seventh test checks that {{preserveFolderStructure=true}} still gives
{{a/orders.csv}} and {{b/orders.csv}}; it passes on main as well and is there
to show that the fix does not rename entries whose names differ.
h3. Proposed fix
When the zip file already has an entry with the name, write the new entry under
the first free name with " (n)" before the extension: {{orders.csv}}, {{orders
(1).csv}}, {{orders (2).csv}}. A leading dot is part of the name ({{.profile
(1)}}), a name without an extension gets the number at the end ({{README
(1)}}), and with {{preserveFolderStructure}} only the file name part changes.
The free name is checked against the entries of the zip file itself (one lookup
in the open {{ZipFileSystem}} when the name is free, one more per earlier entry
of the same name otherwise), so a file really named {{orders (1).csv}} is not
overwritten either. A name with two dots gets the number before the last one
({{data.tar (1).gz}}). The write options of CAMEL-13399 stay as they are.
A rename was chosen over failing the second message: with
{{eagerCheckCompletion}} the batch completion is checked before the
aggregation, so when the duplicate is the last file of the poll it completes
the batch and then fails; the group with the first file is never completed and
the duplicate fails again at every poll. A rename needs no rollback and keeps
every message in the zip of its batch.
Behaviour change: entry names can now differ from the file names (or the
{{CamelFileName}} header) when names repeat within a group. A note goes into
the 4.24 upgrade guide and the zipfile data format page (and its catalog copy).
With the fix the camel-zipfile tests pass: 56 run, 0 failures.
{{ZipSplitterRouteTest.testSplitterLargeFileWithSpoolEnabled}} (zip splitter,
stream cache spool directory not yet empty when checked) is flaky in this
environment with and without the fix: it does not use the aggregation strategy,
failed on main's code in 2 of 2 runs here (passed on a surefire rerun), and
failed in 3 runs and passed in the last one with the fix.
Affected: main (tested); camel-4.23.x, camel-4.22.x, camel-4.18.x and
camel-4.14.x have the same {{REPLACE_EXISTING}} / {{TRUNCATE_EXISTING}} lines
at 319 / 334 (read, not run).
Duplicate check (2026-10-17): JIRA text "ZipAggregationStrategy" (12 issues,
none about entries with the same name), "zip aggregation same name"; GitHub
pull requests "ZipAggregationStrategy", "zipfile aggregation": none.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
