[
https://issues.apache.org/jira/browse/CASSANDRA-21381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18082172#comment-18082172
]
Jens Geyer edited comment on CASSANDRA-21381 at 5/19/26 10:49 PM:
------------------------------------------------------------------
Apprteciated. I basically tried to come up with a patch myself since it still
was in "needs triage" status since January and no reaction happened. Going to
have a look, thanks!
was (Author: jensg):
Apprteciated. I basically tried to come up with a patch myself since it still
was in "needs traige" status since January and no reaction happened. Going to
have a look, thanks!
> CSV COPY TO corrupts control characters (newline, null byte, etc.) in text
> values
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-21381
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21381
> Project: Apache Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
> Reporter: Jens Geyer
> Assignee: Arvind Kandpal
> Priority: Normal
> Time Spent: 40m
> Remaining Estimate: 0h
>
> h2. Problem
> During COPY TO, control characters in text column values are replaced with
> their Python repr() notation by
> UNICODE_CONTROLCHARS_RE.sub(_show_control_chars, ...) in
> {{format_value_text}} ({{pylib/cqlshlib/formatting.py}}).
> Examples:
> * A stored newline (0x0A) becomes the two-character sequence {{\n}} in the
> CSV; after COPY FROM it is stored as {{\n}} (backslash + n) -- a different
> value.
> * A null byte (0x00) becomes {{\x00}} (six characters).
> The regex {{UNICODE_CONTROLCHARS_RE = re.compile(r"[\x00-\x1f\x7f-\xa0]")}}
> matches all ASCII control characters (0x00-0x1F: newline, tab, carriage
> return, BEL, etc.) and Latin-1 supplement control characters (0x7F-0xA0).
> This substitution is correct for terminal display of SELECT results (where
> invisible characters need a human-readable representation). It is incorrect
> in the *CSV export path*, where {{csv.writer}} handles control characters
> correctly via field quoting -- no pre-processing is needed.
> h2. Affected Versions
> All Cassandra versions with {{format_value_text}} containing the
> {{UNICODE_CONTROLCHARS_RE}} substitution (at minimum 3.x through trunk).
> h2. Root Cause
> {{format_value_text}} is shared between the terminal display path (SELECT
> output) and the CSV export path (COPY TO). The {{UNICODE_CONTROLCHARS_RE}}
> substitution converts control characters to their Python repr-string for
> display, but this transformation is *not reversible* via the CSV import path.
> This bug is *independent of, but in the same function as*, the
> backslash-doubling bug fixed in CASSANDRA-21131. Applying the CASSANDRA-21131
> patch does NOT fix this issue.
> h2. Expected Fix
> In the CSV export path, skip the {{UNICODE_CONTROLCHARS_RE.sub(...)}} call.
> An {{escape_control_chars}} parameter (analogous to the {{escape_backslash}}
> parameter introduced by CASSANDRA-21131) can conditionally suppress the
> substitution when calling {{format_value_text}} from the CSV export path.
> h2. Related
> CASSANDRA-21131 -- backslash-doubling bug in the same code path, already
> patched.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
