shashank created CAMEL-25722:
--------------------------------
Summary: camel-flatpack - marshal drops the first row of the
List<Map> body with the default ignoreFirstRecord=true, so the first record is
silently lost and unmarshal followed by marshal does not round trip
Key: CAMEL-25722
URL: https://issues.apache.org/jira/browse/CAMEL-25722
Project: Camel
Issue Type: Bug
Components: camel-flatpack
Reporter: shashank
Assignee: shashank
{{FlatpackDataFormat.marshal}} (main 9fb9bc911f78) writes the header line with
{{writer.printHeader()}} and then loops over the maps of the body, but skips
the first map when {{ignoreFirstRecord}} is true, which is the default:
{code:java}
writer.printHeader();
for (Map<String, Object> row : data) {
if (ignoreFirstRecord && first) {
// skip first row
first = false;
continue;
}
...
{code}
The header line does not come from the first map: without a definition the
column names are the keys of the first map, with a definition they come from
the pzmap file. So the skipped map is a data row. Marshalling {{[ {ID=1,
NAME=alice}, {ID=2, NAME=bob} ]}} with a default {{new FlatpackDataFormat()}}
gives:
{noformat}
ID,NAME
2,bob
{noformat}
There is no error and no log. {{unmarshal}} returns a {{DataSetList}} without
the header line, so {{unmarshal().flatpack()}} followed by
{{marshal().flatpack()}} loses the first record of every file. The option is
documented as "Whether the first line is ignored for delimited files (for the
column headers)", which is what unmarshal does; nothing documents that marshal
drops a data row. The component page shows a marshal example that sets
{{ignoreFirstRecord(false)}} for a fixed-length format, without saying why.
Trigger: default options; {{marshal().flatpack()}} (delimited without or with a
definition, or fixed-length with a definition) of a {{List<Map<String,
Object>>}}.
h3. Reproduction
Found by an executed round-trip property (random rows, 400 cases: every case
lost its first row). New test {{FlatpackMarshalKeepsFirstRowTest}} (3 tests),
all fail on main:
* default data format, 2 maps: expected {{[ID,NAME, 1,alice, 2,bob]}}, main
{{[ID,NAME, 2,bob]}};
* data format with the {{INVENTORY-Delimited.pzmap.xml}} definition, 2 maps:
expected 3 lines, main 2;
* unmarshal of {{ID,NAME / 1,alice / 2,bob / 3,carol}} then marshal: main
{{[ID,NAME, 2,bob, 3,carol]}}.
h3. Proposed fix
Marshal writes every map of the list; {{ignoreFirstRecord}} keeps its meaning
for unmarshal. With the fix the camel-flatpack tests pass: 41 run, 0 failures,
0 errors, 0 skipped. A route that added a dummy first map to work around the
defect now gets that map in the output; a note goes into the 4.24 upgrade guide.
Affected: main (tested); camel-4.23.x, camel-4.22.x, camel-4.18.x and
camel-4.14.x have the same lines (read, not run).
Duplicate check (2026-10-17): JIRA text "flatpack marshal", "flatpack first
row", "flatpack ignoreFirstRecord", camel-flatpack issues since 2024: only
CAMEL-24170 (item M18 lists this defect as Medium; the two PRs that resolved
the umbrella, #24946 and #24952, do not touch Flatpack). GitHub pull requests
"flatpack": none about this.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
