[
https://issues.apache.org/jira/browse/CAMEL-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25723 started by shashank.
----------------------------------------
> camel-zipfile - ZipAggregationStrategy silently replaces a zip entry when a
> later message has the same entry name, e.g. two orders.csv from different
> folders of a recursive file consumer, so the earlier file is lost
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25723
> URL: https://issues.apache.org/jira/browse/CAMEL-25723
> Project: Camel
> Issue Type: Bug
> Components: camel-zipfile
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> Since CAMEL-13399 (Camel 3.0) {{ZipAggregationStrategy}} adds each message to
> the zip file through a {{ZipFileSystem}}: {{Files.copy(file, dest,
> REPLACE_EXISTING)}} for a file body (line 319) and {{Files.write(dest,
> buffer, CREATE, TRUNCATE_EXISTING)}} for other bodies (line 334, main
> 826a664c5f16). When the zip already has an entry with that name, the entry is
> replaced and the content of the earlier message is gone, without an error or
> a log.
> With the default strategy ({{new ZipAggregationStrategy()}},
> {{preserveFolderStructure=false}}) the entry name of a file is its name only.
> A recursive file consumer that polls {{in/a/orders.csv}} ("from A") and
> {{in/b/orders.csv}} ("from B") and aggregates them with the documented
> {{completionFromBatchConsumer().eagerCheckCompletion()}} produces one zip
> with one entry {{orders.csv}} = "from B". {{useFilenameHeader=true}} with a
> repeated {{CamelFileName}} behaves the same. Folders per day or per sender
> with the same file names are common.
> Trigger: default options of {{ZipAggregationStrategy}}; two messages of one
> group with the same entry name (file bodies with the same file name, or the
> same {{CamelFileName}} with {{useFilenameHeader=true}}).
> Why CAMEL-13399 used these options: the issue was about speed (the strategy
> rebuilt the whole zip for every message). The commit "Optimized
> ZipAggregationStrategy to use ZipFileSystem" wrote with {{REPLACE_EXISTING}},
> and the follow-up "Truncate zip entry, if already exists" added
> {{TRUNCATE_EXISTING}} so that a shorter content does not leave the tail of
> the old one in the entry. Neither commit nor the issue discusses two messages
> with the same name; before 3.0 the {{ZipOutputStream}} rebuild failed with
> "duplicate entry" in that case.
> {{TarAggregationStrategy}} on main does not lose the content but does not
> separate the names either: it rebuilds the tar for each message, copies the
> existing entries ({{copyExistingEntries}}, TarAggregationStrategy.java
> 285-292) and appends the new one under the same name ({{addNewEntry}}
> 277-283, {{createNewEntry}} 321-327). The tar then holds two entries
> {{orders.csv}}; a usual extraction writes both to the same path and keeps
> only the last.
> h3. Reproduction
> New test {{ZipAggregationStrategySameFileNameTest}} (7 tests). Without the
> fix 6 fail, for example:
> * a recursive file route over only {{a/orders.csv}} and {{b/orders.csv}} with
> {{completionFromBatchConsumer().eagerCheckCompletion()}}: {{expected:
> <[orders (1).csv=from B, orders.csv=from A]> but was: <[orders.csv=from B]>}};
> * the same route with a third file {{c/other.csv}}: {{but was:
> <[orders.csv=from B, other.csv=from C]>}};
> * the strategy called directly with {{a/orders.csv}} then {{b/orders.csv}},
> and with {{useFilenameHeader=true}} and two byte bodies named {{report.txt}}:
> the zip holds only the second content;
> * five files, four named {{orders.csv}} and one really named {{orders
> (1).csv}}: main keeps 2 of the 5 contents;
> * names without an extension ({{README}}, {{.profile}}).
> The seventh test checks that {{preserveFolderStructure=true}} still gives
> {{a/orders.csv}} and {{b/orders.csv}}; it passes on main as well and is there
> to show that the fix does not rename entries whose names differ.
> h3. Proposed fix
> When the zip file already has an entry with the name, write the new entry
> under the first free name with " (n)" before the extension: {{orders.csv}},
> {{orders (1).csv}}, {{orders (2).csv}}. A leading dot is part of the name
> ({{.profile (1)}}), a name without an extension gets the number at the end
> ({{README (1)}}), and with {{preserveFolderStructure}} only the file name
> part changes. The free name is checked against the entries of the zip file
> itself (one lookup in the open {{ZipFileSystem}} when the name is free, one
> more per earlier entry of the same name otherwise), so a file really named
> {{orders (1).csv}} is not overwritten either. A name with two dots gets the
> number before the last one ({{data.tar (1).gz}}). The write options of
> CAMEL-13399 stay as they are.
> A rename was chosen over failing the second message: with
> {{eagerCheckCompletion}} the batch completion is checked before the
> aggregation, so when the duplicate is the last file of the poll it completes
> the batch and then fails; the group with the first file is never completed
> and the duplicate fails again at every poll. A rename needs no rollback and
> keeps every message in the zip of its batch.
> Behaviour change: entry names can now differ from the file names (or the
> {{CamelFileName}} header) when names repeat within a group. A note goes into
> the 4.24 upgrade guide and the zipfile data format page (and its catalog
> copy).
> With the fix the camel-zipfile tests pass: 56 run, 0 failures.
> {{ZipSplitterRouteTest.testSplitterLargeFileWithSpoolEnabled}} (zip splitter,
> stream cache spool directory not yet empty when checked) is flaky in this
> environment with and without the fix: it does not use the aggregation
> strategy, failed on main's code in 2 of 2 runs here (passed on a surefire
> rerun), and failed in 3 runs and passed in the last one with the fix.
> Affected: main (tested); camel-4.23.x, camel-4.22.x, camel-4.18.x and
> camel-4.14.x have the same {{REPLACE_EXISTING}} / {{TRUNCATE_EXISTING}} lines
> at 319 / 334 (read, not run).
> Duplicate check (2026-10-17): JIRA text "ZipAggregationStrategy" (12 issues,
> none about entries with the same name), "zip aggregation same name"; GitHub
> pull requests "ZipAggregationStrategy", "zipfile aggregation": none.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
