[
https://issues.apache.org/jira/browse/CAMEL-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25150.
---------------------------------
Fix Version/s: 4.23.0
Resolution: Fixed
The fix is merged on main, so it is in Camel 4.23.0:
* cb8769029093 CAMEL-25150: camel-bindy - with quotingEscaped a CSV field
ending with a quote is closed again
Resolving, as the ticket was not updated when the PR was merged.
_Claude Code on behalf of Claus Ibsen_
> camel-bindy - with quotingEscaped=true a CSV field whose value ends with a
> quote (He said "hi", 12") is not closed on unmarshal, so it swallows the next
> fields and the columns shift (regression of CAMEL-22068)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25150
> URL: https://issues.apache.org/jira/browse/CAMEL-25150
> Project: Camel
> Issue Type: Bug
> Components: camel-bindy
> Reporter: shashank
> Priority: Minor
> Labels: regression
> Fix For: 4.23.0
>
>
> With {{@CsvRecord(quoting = true, quotingEscaped = true)}} Bindy writes a
> quote inside a field as a backslash and the quote (CAMEL-7519): the value
> {{He said "hi"}} is written {{"He said \"hi\""}}. Unmarshal cannot read this
> field back when the quote character is the double quote and the value ends
> with an odd number of quotes:
> {noformat}
> line: "123","He said \"hi\"","10"
> expected: firstField=123, secondField=He said "hi", number=10
> actual: firstField=123, secondField=He said "hi"",10 (the third column is
> merged into the second), number=null
> {noformat}
> The cause is the loop that CAMEL-22068 added to
> {{BindyCsvDataFormat.unquoteTokens}} to support RFC 4180 doubled quotes: for
> a token that ends with the quote, it counts the quotes in front of the final
> one, skipping backslashes, and treats the final quote as escaped when the
> count is odd. That is the RFC 4180 rule ({{""}} is one quote in the value),
> but it also runs when {{quotingEscaped=true}}, where the value's quote is
> written {{\"}}. For {{...\""}} the loop skips the backslash, counts one
> quote, decides that the closing quote is escaped, and keeps the field open
> until a later token ends with a quote. The result is silent data corruption:
> the value gets the separator and the following columns appended, the
> following fields are shifted or empty, and a typed field (number, date) that
> receives the wrong text fails.
> Values with an even number of trailing quotes (the value {{""foo""}} of the
> existing test) and values without a trailing quote are read correctly, which
> is why the existing tests pass. The single quote as quote character is not
> affected, because the loop only runs for the double quote.
> Before 4.8.8 / 4.10.5 / 4.12.0 a token ending with the quote always closed
> the field, and these values were read back correctly. A brute force over all
> values of up to 3 characters from {{a , " \ space}} in each of three columns
> (marshal, then unmarshal): 120 of 468 records fail today, 33 with the unquote
> code of 4.10.0, 33 with the fix below. The 33 remaining ones are values with
> an escaped quote directly in front of the separator ({{a",b}}), which already
> failed before CAMEL-22068 and are not a regression.
> h3. Reproduction
> {code:java}
> @CsvRecord(separator = ",", quote = "\"", quoting = true, quotingEscaped =
> true)
> public static class Row {
> @DataField(pos = 1) private String firstField;
> @DataField(pos = 2) private String secondField;
> @DataField(pos = 3, pattern = "########.##") private BigDecimal number;
> }
> // marshal of (123, He said "hi", 10) gives "123","He said \"hi\"","10"
> // unmarshal of that line gives secondField = He said "hi"",10 and number =
> null
> {code}
> A unit test fails on main (three runs) for the values {{He said "hi"}},
> {{12"}}, {{"}} and {{a "b" "c"}}. A small formal model (Lean 4) of the loop
> shows that for every value that ends with {{n}} quotes after another
> character the field is closed exactly when {{n}} is even, and that with the
> fix every quoted field without a separator in it is read back as written.
> h3. Affected versions
> The loop is in 4.8.8, 4.10.5, 4.12.0 and every later release up to main;
> 4.8.7 and 4.10.4 do not have it.
> h3. Proposed fix
> Apply the doubled-quote rule only when {{quotingEscaped}} is false:
> {code:java}
> if (quote.equals("\"") && !quotingEscaped) {
> // RFC 4180 doubled quotes
> ...
> }
> {code}
> With {{quotingEscaped=true}} a token that ends with the quote closes the
> field again, as before CAMEL-22068. RFC 4180 input ({{quotingEscaped=false}},
> the default) is read exactly as today, including the cases of CAMEL-22068.
> Values that end with a backslash ({{C:\temp\}}) keep working.
> The one input that is read differently is RFC 4180 style input in
> {{quotingEscaped=true}} mode with a separator inside the quotes and an odd
> number of quotes in front of it ({{"x"",y"}}): today it stays one field (with
> the doubled quote kept, since this mode does not unescape {{""}}), with the
> fix it is split, as in 4.8.7 and 4.10.4. Such input is not the format
> {{quotingEscaped}} describes.
> Duplicate check (2026-09-30): JIRA "quotingEscaped", "bindy" with "escape"
> and "quote": CAMEL-7519, CAMEL-12274, CAMEL-22068, CAMEL-19143, all fixed,
> none about this case. No open pull request touches {{unquoteTokens}}.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
