I have trouble inserting Chinese characters into an Oracle database with
SQLAlchemy (and to some extent with cx_Oracle). I don't specifically need
Chinese characters but I used them in a unit test to see if my code
(hopefully) can handle "any" unicode characters.
The setup:
- SQLAlchemy 1.1.15 and 1.2
- Oracle database 11.2
- cx_Oracle 6.1
- environment variable set from the shell ( export
NLS_LANG="German_Germany.UTF8" ) and in the code
Database table:
CREATE TABLE UTF8_TEST (
TEXT NVARCHAR2(10)
)
1) SQLAlchemy, version 1.1.15 and 1.2
import os
os.environ["NLS_LANG"] = "German_Germany.UTF8"
import sqlalchemy as sa
metadata = sa.MetaData()
test_table = sa.Table("utf8_test", metadata, sa.Column("text",
sa.Unicode(10)))
def test(host, port, service_name, user, password):
connect_string =
"oracle+cx_oracle://{}:{}@{}:{}/?service_name={}".format(user, password,
host, port, service_name)
engine = sa.engine.create_engine(connect_string, encoding="utf8")
insert = test_table.insert().values(text="\u7535\u8111")
with engine.connect() as connection:
connection.execute(insert)
if __name__ == "__main__":
db_config = ...
test(db_config.host, db_config.port, db_config.service_name,
db_config.user, db_config.password)
When this code is executed, it inserts something into the database, but it
shows up as two upside-down question marks when viewed with Oracle SQL
Developer. This doesn't seem to be an encoding/rendering problem in SQL
Developer (see below for code snippet 3).
2) cx_Oracle, without `Cursor.setinputsizes`
import os
os.environ["NLS_LANG"] = "German_Germany.UTF8"
import cx_Oracle
def test(host, port, service_name, user, password):
dsn = cx_Oracle.makedsn(host, port, service_name=service_name)
conn = cx_Oracle.connect(user, password, dsn=dsn, encoding="utf8",
nencoding="utf8")
cur = conn.cursor()
cur.execute("INSERT INTO UTF8_TEST (TEXT) VALUES (:t)", {"t":
"\u7535\u8111"})
cur.close()
conn.commit()
if __name__ == "__main__":
db_config = ...
test(db_config.host, db_config.port, db_config.service_name,
db_config.user, db_config.password)
This code has the same effect as the SQLAlchemy code above.
3) cx_Oracle with `Cursor.setinputsizes`
import os
os.environ["NLS_LANG"] = "German_Germany.UTF8"
import cx_Oracle
def test(host, port, service_name, user, password):
dsn = cx_Oracle.makedsn(host, port, service_name=service_name)
conn = cx_Oracle.connect(user, password, dsn=dsn, encoding="utf8",
nencoding="utf8")
cur = conn.cursor()
* # Using `setinputsizes` makes the difference. cur.execute("SELECT
TEXT FROM UTF8_TEST") description = cur.description # `t` is name of
bind parameter below. cur.setinputsizes(t=description[0][1])*
cur.execute("INSERT INTO UTF8_TEST (TEXT) VALUES (:t)", {"t":
"\u7535\u8111"})
cur.close()
conn.commit()
if __name__ == "__main__":
db_config = ...
test(db_config.host, db_config.port, db_config.service_name,
db_config.user, db_config.password)
With the `setinputsizes` call, the Chinese characters appear in Oracle SQL
Developer as they should.
In case it matters, I tried to use some German special characters (the
string "äß") and this works for all three variants. These two characters
are encoded in UTF-8 to one byte each whereas the Chinese characters are
encoded to several bytes each.
Should the above code for SQLAlchemy work as-is and insert the Chinese
charaters (I expected it)?
Is there a workaround for now to have SQLAlchemy insert the correct
characters? For example, is there a way to tell SQLAlchemy to use
`Cursor.setinputsizes` for inserts? (I assume this would apply to updates,
too.)
Best regards,
Stefan
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
