[
https://issues.apache.org/jira/browse/CAMEL-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-24170:
--------------------------------
Fix Version/s: 4.22.0
> camel dataformats - Umbrella: remaining high/medium findings from the July
> 2026 data format review
> --------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24170
> URL: https://issues.apache.org/jira/browse/CAMEL-24170
> Project: Camel
> Issue Type: Task
> Reporter: Federico Mariani
> Assignee: Federico Mariani
> Priority: Major
> Labels: ai-assisted-review, dataformat
> Fix For: 4.22.0
>
> Attachments: camel-dataformats-review-2026-07-public.md
>
>
> This umbrella collects the remaining findings from a July 2026 review of the
> Camel *data format* components (JSON: Jackson/JacksonXML + Jackson3 twins,
> Gson, Fastjson, Jsonb, CBOR; XML: JAXB, SOAP; CSV: commons-csv,
> univocity-parsers; fixed/flat: Bindy, Flatpack; YAML: SnakeYAML; binary:
> Avro, Protobuf, Jackson-Avro/Protobuf; archives: zipfile, tarfile;
> compression/encoding: base64, lzf, zip-deflater), plus the DSL model ->
> reifier -> data-format option wiring.
> The critical finding and the eight highest-impact highs are filed separately
> with attached failing reproducers: C1=CAMEL-24152, H1=CAMEL-24162,
> H3=CAMEL-24163, H4=CAMEL-24164, H5=CAMEL-24165, H7=CAMEL-24166,
> H8=CAMEL-24167, H12=CAMEL-24168, H14=CAMEL-24169. The full findings document
> (with all file:line references, root-cause traces and the medium/low tail) is
> attached to this issue as {{camel-dataformats-review-2026-07-public.md}}.
> A recurring theme: commit {{5dfc5c2857e9}} (CAMEL-22354, "align data formats
> getters/setters to model", released 4.15.0) is the root cause of several
> regressions here (the SnakeYAML critical, the CSV {{format}} regression, the
> YAML {{typeFilter}} builder bug, and the SOAP {{ignoreUnmarshalledHeaders}}
> gaps). A focused audit of everything that commit touched is worthwhile.
> h2. Remaining HIGH findings (not filed individually)
> * *H2 - camel-jacksonxml {xmlMapper} DSL option requires an undocumented {#}
> prefix.* {JacksonXMLDataFormatReifier} does {properties.put("xmlMapper",
> definition.getXmlMapper())} without {asRef()} (contrast
> {JsonDataFormatReifier} for {objectMapper}). {<jacksonXml
> xmlMapper="myXmlMapper"/>} fails with {NoTypeConversionAvailableException:
> String -> XmlMapper}; only {"#myXmlMapper"} works. Same reifier serves
> camel-jackson3xml. Runtime-reproduced.
> * *H9 - camel-univocity-parsers {lazyLoad=true} + {asMap=true} uses a {static
> ThreadLocal} for headers.* {HeaderRowProcessor} stores captured headers in a
> static ThreadLocal; consuming the lazy iterator on another thread (parallel
> split, seda/kafka hand-off) NPEs, and a pooled thread can carry stale headers
> from a different exchange, silently mapping values to wrong keys.
> Runtime-reproduced (cross-thread NPE).
> * *H10 - camel-univocity-parsers {lazyLoad=true} leaks the input stream* when
> the iterator is not fully consumed. No on-completion hook (camel-csv fixed
> the identical problem via {addOnCompletion}, CAMEL-15105). Abandoned split /
> partial read / downstream error leaks the reader/FileInputStream.
> * *H11 - camel-bindy KVP unmarshal truncates any value containing the
> key-value separator.* {BindyKeyValuePairFactory} does
> {s.split(getKeyValuePairSeparator())} (no limit) and keeps only element [1];
> {58=a=b} -> {a}. Fix: {split(sep, 2)}. Runtime-reproduced.
> * *H13 - camel-bindy fixed-length {UnicodeHelper.indexOf} off-by-one.* Loop
> {index + len < length()} should be {<=}; a delimiter at end-of-record is
> unreachable, then {substring(0, -1)} -> {IndexOutOfBoundsException}.
> Runtime-reproduced.
> * *H15 - camel-protobuf JSON unmarshal uses the platform default charset*
> while marshal writes UTF-8 ({new InputStreamReader(inputStream)} vs
> {StandardCharsets.UTF_8}). Asymmetric round-trip on non-UTF-8 JVMs. Fix: pass
> UTF-8.
> * *H16 - camel-avro {contentTypeHeader} option silently ignored.* The model
> {AvroDataFormat} declares the attribute (default true) but does not implement
> {ContentTypeHeaderAware}, so the generic reifier never transfers it; {<avro
> contentTypeHeader="false"/>} is dead for the default {avroJackson} library.
> Fix: implement {ContentTypeHeaderAware}.
> * *H17 - camel-core YAMLDataFormat.Builder.typeFilter(Class...) joins names
> with "." instead of ",".* {typeFilter(Foo.class, Bar.class)} yields one bogus
> pattern {com.x.Foo.com.y.Bar} matching neither class -> every unmarshal
> throws. Fails closed. Fix: {new StringJoiner(",")}. Runtime-reproduced.
> (CAMEL-22354.)
> h2. MEDIUM findings
> _(One additional medium-severity security-hardening item identified in this
> review is intentionally omitted from this public issue and is being handled
> through the private ASF security process per SECURITY.md.)_
> * *M2 - Gson unmarshal InputStream fallback ignores {Exchange.CHARSET_NAME}*
> ({new InputStreamReader(is)}) while marshal honors it. Same class also in
> JAXB filtered-unmarshal fallback and Jsonb/Fastjson.
> * *M3 - Jsonb never sets Content-Type; {contentTypeHeader} is a no-op* (does
> not implement {DataFormatContentTypeHeader}).
> * *M4 - Jsonb {Jsonb} instance (AutoCloseable) never closed on stop* (doStop
> is a no-op) - resource leak on restart.
> * *M5 - {timezone} silently ignored for avroJackson and protobufJackson*
> (neither reifier Jackson branch transfers it; JSON does).
> * *M6 - {avro()} fluent Builder default library is ApacheAvro while the model
> default is Jackson* (CAMEL-20502 leftover) - same logical config resolves
> different implementations per DSL.
> * *M7 - Avro/Protobuf schema resolvers use {Class.forName} instead of the
> Camel class resolver* - breaks in Spring Boot fat-jar / Quarkus / OSGi.
> * *M8 - camel-avro marshalling a {GenericData.Record} without a configured
> schema throws* instead of using the record's own {GenericContainer} schema.
> * *M9 - SOAP fault unmarshalling NPE / UnsupportedOperationException* when
> the fault detail is unknown or a SOAP 1.2 {<Reason>} is missing
> (peer-controlled input; no null-guard, lookup outside try/catch).
> * *M10 - JAXB {filterNonXmlChars} corrupts valid supplementary-plane
> characters* (emoji, CJK-ext) - treats surrogate pairs as invalid, replaces
> with two spaces.
> * *M11 - CBOR marshal buffers the whole output in memory*
> ({writeValueAsBytes} then copy) instead of {writeValue(stream, graph)}.
> * *M12 - univocity fixed-width {recordEndsOnNewline=false} still enables the
> feature* (hardcoded {setRecordEndsOnNewline(true)}).
> * *M13 - camel-csv {captureHeaderRecord=true} ignored with {lazyLoad=true}*,
> and the header row is consumed and lost (only the bulk unmarshaller sets the
> header).
> * *M14 - camel-bindy CSV blank lines manufacture phantom empty records* (or
> fail if a field is required), unlike KVP which skips blank lines.
> Runtime-reproduced.
> * *M15 - camel-bindy CSV {defaultValue} not applied when exactly one trailing
> field is missing* (off-by-one {pos < totalFields});
> {setDefaultValuesForFields} also NPEs on gapped positions. Runtime-reproduced.
> * *M16 - camel-bindy CSV marshal of a null {@OneToMany} list drops the child
> columns* (dead code passes {Field.class} as the element type).
> Runtime-reproduced.
> * *M17 - camel-bindy {@CsvRecord(allowEmptyStream=true)} discards real data
> when {InputStream.available()==0}* (a legal return for non-empty streams);
> two different {allowEmptyStream} flags. Runtime-reproduced.
> * *M18 - camel-flatpack marshal drops the first data row with the default
> {ignoreFirstRecord=true}* ({printHeader()} already emits headers;
> unmarshal->marshal round trip loses record #1).
> h2. LOW findings
> See the attached document for the low-severity tail (base64 urlSafe CRLF
> chunking and decode-side {lineSeparator} validation; Flatpack
> {getDataFormatName()} typo {"flatback"}; Bindy {DefaultFactoryRegistry}
> HashMap hot-path race and other Bindy quoting/offset/locale nits; camel-csv
> bulk reader leak on parser-construct failure;
> {useOrderedMaps}+{ignoreHeaderCase} ordering; fixed-width reifier
> {Integer.parseInt("null")}; jackson {useList} clobbering {collectionType};
> jackson3 type-converter module-loss and dropped {canSerialize} guard; jackson
> {enableJacksonTypeConverter} dead property; non-mandatory {unmarshalTypeName}
> resolution; tarfile-vs-zipfile twin drift - empty-tar throw, {hasNext()}
> non-idempotence dropping zero-byte entries, marshal NPE on
> {CamelFileName="/"}; Avro resolver unconditional {URLDecoder.decode} and dead
> {validate} option; JAXB WARN-level payload logging; SnakeYAML
> lifecycle/type-filter widening; zip-deflater compressionLevel late
> validation).
> h2. Verified NON-issues (no action)
> JAXB XXE (all unmarshal paths use the hardened {StaxConverter}); fastjson2
> autotype off by default; jackson3 missing
> {BLOCK_UNSAFE_POLYMORPHIC_BASE_TYPES} (constant removed upstream); zip-slip
> ({jailStartingDirectory=true}); SnakeYAML default {SafeConstructor} posture
> and billion-laughs limits.
> ----
> _Reported by Claude Code on behalf of Federico Mariani (Croway). Full
> findings document attached._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
