Federico Mariani created CAMEL-24170:
----------------------------------------
Summary: camel dataformats - Umbrella: remaining high/medium
findings from the July 2026 data format review
Key: CAMEL-24170
URL: https://issues.apache.org/jira/browse/CAMEL-24170
Project: Camel
Issue Type: Task
Reporter: Federico Mariani
Attachments: camel-dataformats-review-2026-07-public.md
This umbrella collects the remaining findings from a July 2026 review of the
Camel *data format* components (JSON: Jackson/JacksonXML + Jackson3 twins,
Gson, Fastjson, Jsonb, CBOR; XML: JAXB, SOAP; CSV: commons-csv,
univocity-parsers; fixed/flat: Bindy, Flatpack; YAML: SnakeYAML; binary: Avro,
Protobuf, Jackson-Avro/Protobuf; archives: zipfile, tarfile;
compression/encoding: base64, lzf, zip-deflater), plus the DSL model -> reifier
-> data-format option wiring.
The critical finding and the eight highest-impact highs are filed separately
with attached failing reproducers: C1=CAMEL-24152, H1=CAMEL-24162,
H3=CAMEL-24163, H4=CAMEL-24164, H5=CAMEL-24165, H7=CAMEL-24166, H8=CAMEL-24167,
H12=CAMEL-24168, H14=CAMEL-24169. The full findings document (with all
file:line references, root-cause traces and the medium/low tail) is attached to
this issue as {{camel-dataformats-review-2026-07-public.md}}.
A recurring theme: commit {{5dfc5c2857e9}} (CAMEL-22354, "align data formats
getters/setters to model", released 4.15.0) is the root cause of several
regressions here (the SnakeYAML critical, the CSV {{format}} regression, the
YAML {{typeFilter}} builder bug, and the SOAP {{ignoreUnmarshalledHeaders}}
gaps). A focused audit of everything that commit touched is worthwhile.
h2. Remaining HIGH findings (not filed individually)
* *H2 - camel-jacksonxml {xmlMapper} DSL option requires an undocumented {#}
prefix.* {JacksonXMLDataFormatReifier} does {properties.put("xmlMapper",
definition.getXmlMapper())} without {asRef()} (contrast {JsonDataFormatReifier}
for {objectMapper}). {<jacksonXml xmlMapper="myXmlMapper"/>} fails with
{NoTypeConversionAvailableException: String -> XmlMapper}; only
{"#myXmlMapper"} works. Same reifier serves camel-jackson3xml.
Runtime-reproduced.
* *H9 - camel-univocity-parsers {lazyLoad=true} + {asMap=true} uses a {static
ThreadLocal} for headers.* {HeaderRowProcessor} stores captured headers in a
static ThreadLocal; consuming the lazy iterator on another thread (parallel
split, seda/kafka hand-off) NPEs, and a pooled thread can carry stale headers
from a different exchange, silently mapping values to wrong keys.
Runtime-reproduced (cross-thread NPE).
* *H10 - camel-univocity-parsers {lazyLoad=true} leaks the input stream* when
the iterator is not fully consumed. No on-completion hook (camel-csv fixed the
identical problem via {addOnCompletion}, CAMEL-15105). Abandoned split /
partial read / downstream error leaks the reader/FileInputStream.
* *H11 - camel-bindy KVP unmarshal truncates any value containing the key-value
separator.* {BindyKeyValuePairFactory} does
{s.split(getKeyValuePairSeparator())} (no limit) and keeps only element [1];
{58=a=b} -> {a}. Fix: {split(sep, 2)}. Runtime-reproduced.
* *H13 - camel-bindy fixed-length {UnicodeHelper.indexOf} off-by-one.* Loop
{index + len < length()} should be {<=}; a delimiter at end-of-record is
unreachable, then {substring(0, -1)} -> {IndexOutOfBoundsException}.
Runtime-reproduced.
* *H15 - camel-protobuf JSON unmarshal uses the platform default charset* while
marshal writes UTF-8 ({new InputStreamReader(inputStream)} vs
{StandardCharsets.UTF_8}). Asymmetric round-trip on non-UTF-8 JVMs. Fix: pass
UTF-8.
* *H16 - camel-avro {contentTypeHeader} option silently ignored.* The model
{AvroDataFormat} declares the attribute (default true) but does not implement
{ContentTypeHeaderAware}, so the generic reifier never transfers it; {<avro
contentTypeHeader="false"/>} is dead for the default {avroJackson} library.
Fix: implement {ContentTypeHeaderAware}.
* *H17 - camel-core YAMLDataFormat.Builder.typeFilter(Class...) joins names
with "." instead of ",".* {typeFilter(Foo.class, Bar.class)} yields one bogus
pattern {com.x.Foo.com.y.Bar} matching neither class -> every unmarshal throws.
Fails closed. Fix: {new StringJoiner(",")}. Runtime-reproduced. (CAMEL-22354.)
h2. MEDIUM findings
_(One additional medium-severity security-hardening item identified in this
review is intentionally omitted from this public issue and is being handled
through the private ASF security process per SECURITY.md.)_
* *M2 - Gson unmarshal InputStream fallback ignores {Exchange.CHARSET_NAME}*
({new InputStreamReader(is)}) while marshal honors it. Same class also in JAXB
filtered-unmarshal fallback and Jsonb/Fastjson.
* *M3 - Jsonb never sets Content-Type; {contentTypeHeader} is a no-op* (does
not implement {DataFormatContentTypeHeader}).
* *M4 - Jsonb {Jsonb} instance (AutoCloseable) never closed on stop* (doStop is
a no-op) - resource leak on restart.
* *M5 - {timezone} silently ignored for avroJackson and protobufJackson*
(neither reifier Jackson branch transfers it; JSON does).
* *M6 - {avro()} fluent Builder default library is ApacheAvro while the model
default is Jackson* (CAMEL-20502 leftover) - same logical config resolves
different implementations per DSL.
* *M7 - Avro/Protobuf schema resolvers use {Class.forName} instead of the Camel
class resolver* - breaks in Spring Boot fat-jar / Quarkus / OSGi.
* *M8 - camel-avro marshalling a {GenericData.Record} without a configured
schema throws* instead of using the record's own {GenericContainer} schema.
* *M9 - SOAP fault unmarshalling NPE / UnsupportedOperationException* when the
fault detail is unknown or a SOAP 1.2 {<Reason>} is missing (peer-controlled
input; no null-guard, lookup outside try/catch).
* *M10 - JAXB {filterNonXmlChars} corrupts valid supplementary-plane
characters* (emoji, CJK-ext) - treats surrogate pairs as invalid, replaces with
two spaces.
* *M11 - CBOR marshal buffers the whole output in memory* ({writeValueAsBytes}
then copy) instead of {writeValue(stream, graph)}.
* *M12 - univocity fixed-width {recordEndsOnNewline=false} still enables the
feature* (hardcoded {setRecordEndsOnNewline(true)}).
* *M13 - camel-csv {captureHeaderRecord=true} ignored with {lazyLoad=true}*,
and the header row is consumed and lost (only the bulk unmarshaller sets the
header).
* *M14 - camel-bindy CSV blank lines manufacture phantom empty records* (or
fail if a field is required), unlike KVP which skips blank lines.
Runtime-reproduced.
* *M15 - camel-bindy CSV {defaultValue} not applied when exactly one trailing
field is missing* (off-by-one {pos < totalFields}); {setDefaultValuesForFields}
also NPEs on gapped positions. Runtime-reproduced.
* *M16 - camel-bindy CSV marshal of a null {@OneToMany} list drops the child
columns* (dead code passes {Field.class} as the element type).
Runtime-reproduced.
* *M17 - camel-bindy {@CsvRecord(allowEmptyStream=true)} discards real data
when {InputStream.available()==0}* (a legal return for non-empty streams); two
different {allowEmptyStream} flags. Runtime-reproduced.
* *M18 - camel-flatpack marshal drops the first data row with the default
{ignoreFirstRecord=true}* ({printHeader()} already emits headers;
unmarshal->marshal round trip loses record #1).
h2. LOW findings
See the attached document for the low-severity tail (base64 urlSafe CRLF
chunking and decode-side {lineSeparator} validation; Flatpack
{getDataFormatName()} typo {"flatback"}; Bindy {DefaultFactoryRegistry} HashMap
hot-path race and other Bindy quoting/offset/locale nits; camel-csv bulk reader
leak on parser-construct failure; {useOrderedMaps}+{ignoreHeaderCase} ordering;
fixed-width reifier {Integer.parseInt("null")}; jackson {useList} clobbering
{collectionType}; jackson3 type-converter module-loss and dropped
{canSerialize} guard; jackson {enableJacksonTypeConverter} dead property;
non-mandatory {unmarshalTypeName} resolution; tarfile-vs-zipfile twin drift -
empty-tar throw, {hasNext()} non-idempotence dropping zero-byte entries,
marshal NPE on {CamelFileName="/"}; Avro resolver unconditional
{URLDecoder.decode} and dead {validate} option; JAXB WARN-level payload
logging; SnakeYAML lifecycle/type-filter widening; zip-deflater
compressionLevel late validation).
h2. Verified NON-issues (no action)
JAXB XXE (all unmarshal paths use the hardened {StaxConverter}); fastjson2
autotype off by default; jackson3 missing {BLOCK_UNSAFE_POLYMORPHIC_BASE_TYPES}
(constant removed upstream); zip-slip ({jailStartingDirectory=true}); SnakeYAML
default {SafeConstructor} posture and billion-laughs limits.
----
_Reported by Claude Code on behalf of Federico Mariani (Croway). Full findings
document attached._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
