[issue13676] sqlite3: Zero byte truncates string contents
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 2e13011b3719 by Petri Lehtinen in branch '3.2': sqlite3: Handle strings with embedded zeros correctly http://hg.python.org/cpython/rev/2e13011b3719 New changeset 93ac4b12a750 by Petri Lehtinen in branch '2.7': sqlite3: Handle strings with embedded zeros correctly http://hg.python.org/cpython/rev/93ac4b12a750 New changeset 6f4044afa600 by Petri Lehtinen in branch 'default': Merge branch 3.2 http://hg.python.org/cpython/rev/6f4044afa600 -- nosy: +python-dev resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Petri Lehtinen pe...@digip.org added the comment: Attached an updated patch. The custom text_factory case is now fixed, and bytes, bytearray and custom factory are all tested. I also added back the pysqlite_unicode_from_string() function, as this makes the patch a bit smaller. It also seems to me (only by looking at the code) that the sqlite3.OptimizedUnicode factory isn't currently working as documented. Antoine: Do you happen to know what's the status of the OptimizeUnicode thingie? Has it been changed for a reason or is it just an error that happened during the py3k transition? -- Added file: http://bugs.python.org/file24122/sqlite3_zero_byte_v3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Changes by Petri Lehtinen pe...@digip.org: Removed file: http://bugs.python.org/file24122/sqlite3_zero_byte_v3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Petri Lehtinen pe...@digip.org added the comment: (Whoops, I didn't mean to change the magic source coding comment. Updating the patch once again.) -- Added file: http://bugs.python.org/file24123/sqlite3_zero_byte_v3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Antoine Pitrou pit...@free.fr added the comment: Attached an updated patch. The custom text_factory case is now fixed, and bytes, bytearray and custom factory are all tested. Thanks, looks good to me. Antoine: Do you happen to know what's the status of the OptimizeUnicode thingie? Has it been changed for a reason or is it just an error that happened during the py3k transition? It looks obsolete in 3.x to me. If you look at the 2.7 source code, it had a real meaning there. Probably we could simplify the 3.x source code by removing that option (but better to do it in a separate patch). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Petri Lehtinen pe...@digip.org added the comment: What? Don't you SEE that it works correctly? :) Attached an updated patch with a test case. FTR, I also tried to make it possible to have the SQL statement include a zero byte, but it seems that sqlite3_prepare() (and also the newer sqlite3_prepare_v2()) always stops reading at the zero byte. See: http://www.sqlite.org/c3ref/prepare.html -- Added file: http://bugs.python.org/file24111/sqlite3_zero_byte_v2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Antoine Pitrou pit...@free.fr added the comment: It would be nice to also have tests for the bytes and bytearray cases. It also seems the generic case hasn't been fixed (PyObject_CallFunction(self-connection-text_factory, y, val_str)). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
New submission from Petri Lehtinen pe...@digip.org:
Inserting a string with embedded zero byte only inserts the string up to the
first zero byte:
import sqlite3
connection = sqlite3.connect(':memory:')
cursor = connection.cursor()
cursor.execute('CREATE TABLE test (value TEXT)')
cursor.execute('INSERT INTO test (value) VALUES (?)', ('foo\x00bar',))
cursor.execute('SELECT value FROM test')
print(cursor.fetchone())
# expected output: (u'foo\x00bar',)
# actual output: (u'foo',)
Also, if there's already data inserted to a table like above with embedded zero
bytes, the sqlite-API-to-Python-string conversion truncates the strings to just
before the first zero byte.
Attaching a patch against 3.3 that fixes the problem. Basically, it uses
PyUnicode_AsStringAndSize and PyUnicode_FromStringAndSize instead of the
non-size variants.
Please review, as I'm not sure it covers each possible case.
--
components: Library (Lib)
files: sqlite3_zero_byte.patch
keywords: needs review, patch
messages: 150329
nosy: ghaering, petri.lehtinen
priority: normal
severity: normal
status: open
title: sqlite3: Zero byte truncates string contents
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file24104/sqlite3_zero_byte.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13676
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13676] sqlite3: Zero byte truncates string contents
Antoine Pitrou pit...@free.fr added the comment: Where are the tests? :) -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13676 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
