[
https://issues.apache.org/jira/browse/CAMEL-25722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25722.
---------------------------------
Resolution: Fixed
Merged in https://github.com/apache/camel/pull/28030 for 4.24.0.
_Claude Code on behalf of Claus Ibsen_
> camel-flatpack - marshal drops the first row of the List<Map> body with the
> default ignoreFirstRecord=true, so the first record is silently lost and
> unmarshal followed by marshal does not round trip
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25722
> URL: https://issues.apache.org/jira/browse/CAMEL-25722
> Project: Camel
> Issue Type: Bug
> Components: camel-flatpack
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> {{FlatpackDataFormat.marshal}} (main 9fb9bc911f78) writes the header line
> with {{writer.printHeader()}} and then loops over the maps of the body, but
> skips the first map when {{ignoreFirstRecord}} is true, which is the default:
> {code:java}
> writer.printHeader();
> for (Map<String, Object> row : data) {
> if (ignoreFirstRecord && first) {
> // skip first row
> first = false;
> continue;
> }
> ...
> {code}
> The header line does not come from the first map: without a definition the
> column names are the keys of the first map, with a definition they come from
> the pzmap file. So the skipped map is a data row. Marshalling {{[ {ID=1,
> NAME=alice}, {ID=2, NAME=bob} ]}} with a default {{new FlatpackDataFormat()}}
> gives:
> {noformat}
> ID,NAME
> 2,bob
> {noformat}
> There is no error and no log. {{unmarshal}} returns a {{DataSetList}} without
> the header line, so {{unmarshal().flatpack()}} followed by
> {{marshal().flatpack()}} loses the first record of every file. The option is
> documented as "Whether the first line is ignored for delimited files (for the
> column headers)", which is what unmarshal does; nothing documents that
> marshal drops a data row. The component page shows a marshal example that
> sets {{ignoreFirstRecord(false)}} for a fixed-length format, without saying
> why.
> Trigger: default options; {{marshal().flatpack()}} (delimited without or with
> a definition, or fixed-length with a definition) of a {{List<Map<String,
> Object>>}}.
> h3. Reproduction
> Found by an executed round-trip property (random rows, 400 cases: every case
> lost its first row). New test {{FlatpackMarshalKeepsFirstRowTest}} (3 tests),
> all fail on main:
> * default data format, 2 maps: expected {{[ID,NAME, 1,alice, 2,bob]}}, main
> {{[ID,NAME, 2,bob]}};
> * data format with the {{INVENTORY-Delimited.pzmap.xml}} definition, 2 maps:
> expected 3 lines, main 2;
> * unmarshal of {{ID,NAME / 1,alice / 2,bob / 3,carol}} then marshal: main
> {{[ID,NAME, 2,bob, 3,carol]}}.
> h3. Proposed fix
> Marshal writes every map of the list; {{ignoreFirstRecord}} keeps its meaning
> for unmarshal. With the fix the camel-flatpack tests pass: 41 run, 0
> failures, 0 errors, 0 skipped. A route that added a dummy first map to work
> around the defect now gets that map in the output; a note goes into the 4.24
> upgrade guide.
> Affected: main (tested); camel-4.23.x, camel-4.22.x, camel-4.18.x and
> camel-4.14.x have the same lines (read, not run).
> Duplicate check (2026-10-17): JIRA text "flatpack marshal", "flatpack first
> row", "flatpack ignoreFirstRecord", camel-flatpack issues since 2024: only
> CAMEL-24170 (item M18 lists this defect as Medium; the two PRs that resolved
> the umbrella, #24946 and #24952, do not touch Flatpack). GitHub pull requests
> "flatpack": none about this.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
