[
https://issues.apache.org/jira/browse/CASSANDRA-21131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18091383#comment-18091383
]
Brad Schoening edited comment on CASSANDRA-21131 at 6/25/26 4:48 AM:
---------------------------------------------------------------------
A clean conceptual fix is to recognize that the backslash-doubling belongs to
{_}display rendering{_}, not to representing the canonical value in plain text.
Formatting currently has two modes, Plain and Color. Instead of adding a new
boolean parameter, I suggest recognizing we have different output modes and use
an enum.
{code:java}
class OutputMode(Enum):
COLOR_DISPLAY = auto() # ANSI palette (still needs the colormap dict),
single backslash, FormattedValue
PLAIN_DISPLAY = auto() # no ANSI, backslash doubled to disambiguate, bare
str
TEXT = auto() # no ANSI, raw bytes, tbd how to handle
U+0000–U+001F{code}
The consumer or producer of CSV may be an unknown external parser or exporter,
not just COPY TO/COPY FROM round-tripping. We shouldn't rely on a private
contract between cqlsh's writer and reader — the bytes have to be safe and
unambiguous to a stranger.
JSON is unambiguous precisely where RFC 4180 is silent. Per RFC 8259, a JSON
string's {{unescaped}} production starts at U+0020 — every control character
U+0000–U+001F _must_ be escaped. A literal NUL or backspace inside a JSON
string is invalid JSON; conformant parsers reject it, lenient ones tolerate it.
JSON does not allow raw backspaces or nulls.
was (Author: bschoeni):
A clean conceptual fix is to recognize that the backslash-doubling belongs to
{_}display rendering{_}, not to representing the canonical value in plain text.
Formatting currently has two modes, Plain and Color. Instead of adding a new
boolean parameter, I suggest recognizing we have different output modes and use
an enum.
{code:java}
class OutputMode(Enum):
COLOR_DISPLAY = auto() # ANSI palette (still needs the colormap dict),
single backslash, FormattedValue
PLAIN_DISPLAY = auto() # no ANSI, backslash doubled to disambiguate, bare
str
TEXT = auto() # no ANSI, raw bytes, tbd how to handle
U+0000–U+001F{code}
The consumer or producer of CSV may be an unknown external parser or exporter,
not just COPY TO/COPY FROM round-tripping. We shouldn't rely on a private
contract between cqlsh's writer and reader — the bytes have to be safe and
unambiguous to a stranger.
> CSV COPY TO/FROM double-escapes backslashes in text columns, corrupting data
> on round-trip
> ------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-21131
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21131
> Project: Apache Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
> Reporter: Jens Geyer
> Priority: Normal
> Attachments: console.txt, cyclist.csv, test_cassandra_21131.py,
> test_cassandra_21131_collections.py
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> I'm following the example from the docs, just slightly changed:
> https://docs.datastax.com/en/cql-oss/3.x/cql/cql_reference/cqlshCopy.html
> {code}
> CREATE KEYSPACE cycling WITH REPLICATION = { 'class' :
> 'NetworkTopologyStrategy', 'datacenter1' : 1 } ;
> CREATE TABLE cycling.cyclist_name ( id UUID PRIMARY KEY, lastname text,
> firstname text ) ;
> INSERT INTO cycling.cyclist_name (id, lastname, firstname)
> VALUES (5b6962dd-3f90-4c93-8f61-eabfa4a803e2, 'VOS','Marianne');
> INSERT INTO cycling.cyclist_name (id, lastname, firstname)
> VALUES (88888888-8888-8888-8888-888888888888, 'V\S','\"Marianne"\');
> SELECT * FROM cycling.cyclist_name ;
> {code}
> This gives:
> {code}
> id | firstname | lastname
> --------------------------------------+----------------+----------
> 5b6962dd-3f90-4c93-8f61-eabfa4a803e2 | Marianne | VOS
> 88888888-8888-8888-8888-888888888888 | \\"Marianne"\\ | V\\S
> (2 rows)
> {code}
> Now export to a file, then import from the very same file:
> {code}
> COPY cycling.cyclist_name TO 'cyclist.csv' WITH HEADER = TRUE ;
> TRUNCATE cycling.cyclist_name ;
> COPY cycling.cyclist_name FROM 'cyclist.csv' WITH HEADER = TRUE ;
> SELECT * FROM cycling.cyclist_name ;
> {code}
> Now the result is this:
> {code}
> id | firstname | lastname
> --------------------------------------+--------------------+----------
> 5b6962dd-3f90-4c93-8f61-eabfa4a803e2 | Marianne | VOS
> 88888888-8888-8888-8888-888888888888 | \\\\"Marianne"\\\\ | V\\\\S
> (2 rows)
> {code}
> I did not change any data in between. Just export and reimport.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
