shashank created CAMEL-25150:
--------------------------------
Summary: camel-bindy - with quotingEscaped=true a CSV field whose
value ends with a quote (He said "hi", 12") is not closed on unmarshal, so it
swallows the next fields and the columns shift (regression of CAMEL-22068)
Key: CAMEL-25150
URL: https://issues.apache.org/jira/browse/CAMEL-25150
Project: Camel
Issue Type: Bug
Components: camel-bindy
Reporter: shashank
With {{@CsvRecord(quoting = true, quotingEscaped = true)}} Bindy writes a quote
inside a field as a backslash and the quote (CAMEL-7519): the value {{He said
"hi"}} is written {{"He said \"hi\""}}. Unmarshal cannot read this field back
when the quote character is the double quote and the value ends with an odd
number of quotes:
{noformat}
line: "123","He said \"hi\"","10"
expected: firstField=123, secondField=He said "hi", number=10
actual: firstField=123, secondField=He said "hi"",10 (the third column is
merged into the second), number=null
{noformat}
The cause is the loop that CAMEL-22068 added to
{{BindyCsvDataFormat.unquoteTokens}} to support RFC 4180 doubled quotes: for a
token that ends with the quote, it counts the quotes in front of the final one,
skipping backslashes, and treats the final quote as escaped when the count is
odd. That is the RFC 4180 rule ({{""}} is one quote in the value), but it also
runs when {{quotingEscaped=true}}, where the value's quote is written {{\"}}.
For {{...\""}} the loop skips the backslash, counts one quote, decides that the
closing quote is escaped, and keeps the field open until a later token ends
with a quote. The result is silent data corruption: the value gets the
separator and the following columns appended, the following fields are shifted
or empty, and a typed field (number, date) that receives the wrong text fails.
Values with an even number of trailing quotes (the value {{""foo""}} of the
existing test) and values without a trailing quote are read correctly, which is
why the existing tests pass. The single quote as quote character is not
affected, because the loop only runs for the double quote.
Before 4.8.8 / 4.10.5 / 4.12.0 a token ending with the quote always closed the
field, and these values were read back correctly. A brute force over all values
of up to 3 characters from {{a , " \ space}} in each of three columns (marshal,
then unmarshal): 120 of 468 records fail today, 33 with the unquote code of
4.10.0, 33 with the fix below. The 33 remaining ones are values with an escaped
quote directly in front of the separator ({{a",b}}), which already failed
before CAMEL-22068 and are not a regression.
h3. Reproduction
{code:java}
@CsvRecord(separator = ",", quote = "\"", quoting = true, quotingEscaped = true)
public static class Row {
@DataField(pos = 1) private String firstField;
@DataField(pos = 2) private String secondField;
@DataField(pos = 3, pattern = "########.##") private BigDecimal number;
}
// marshal of (123, He said "hi", 10) gives "123","He said \"hi\"","10"
// unmarshal of that line gives secondField = He said "hi"",10 and number = null
{code}
A unit test fails on main (three runs) for the values {{He said "hi"}},
{{12"}}, {{"}} and {{a "b" "c"}}. A small formal model (Lean 4) of the loop
shows that for every value that ends with {{n}} quotes after another character
the field is closed exactly when {{n}} is even, and that with the fix every
quoted field without a separator in it is read back as written.
h3. Affected versions
The loop is in 4.8.8, 4.10.5, 4.12.0 and every later release up to main; 4.8.7
and 4.10.4 do not have it.
h3. Proposed fix
Apply the doubled-quote rule only when {{quotingEscaped}} is false:
{code:java}
if (quote.equals("\"") && !quotingEscaped) {
// RFC 4180 doubled quotes
...
}
{code}
With {{quotingEscaped=true}} a token that ends with the quote closes the field
again, as before CAMEL-22068. RFC 4180 input ({{quotingEscaped=false}}, the
default) is read exactly as today, including the cases of CAMEL-22068. Values
that end with a backslash ({{C:\temp\}}) keep working.
The one input that is read differently is RFC 4180 style input in
{{quotingEscaped=true}} mode with a separator inside the quotes and an odd
number of quotes in front of it ({{"x"",y"}}): today it stays one field (with
the doubled quote kept, since this mode does not unescape {{""}}), with the fix
it is split, as in 4.8.7 and 4.10.4. Such input is not the format
{{quotingEscaped}} describes.
Duplicate check (2026-09-30): JIRA "quotingEscaped", "bindy" with "escape" and
"quote": CAMEL-7519, CAMEL-12274, CAMEL-22068, CAMEL-19143, all fixed, none
about this case. No open pull request touches {{unquoteTokens}}.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
