[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Aviv Palivoda added the comment: In my patch I use sqlite3_create_function_v2 which was added in sqlite 3.7.3 (2010-10-08). There were a few problems after adding sqlite3_stmt_readonly in 284676cf2ac8 as can be seen in issue #29355. sqlite3_stmt_readonly was added in 3.7.4 (2010-12-07) so I

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Aviv Palivoda added the comment: After looking more into the _pysqlite_set_result function fail in Ingo example I think this is the expected behavior. The PyUnicode_AsUTF8 fails but this is expected as the value is an invalid UTF-8. -- ___ Python

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Aviv Palivoda added the comment: Actually had a small mistake in the patch I uploaded. Uploading a fixed one. -- Added file: http://bugs.python.org/file46166/29021-fixed.patch ___ Python tracker

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Aviv Palivoda added the comment: I actually was wrong and there is no problem with the type that is returned from the text_factory as it passes directly as argument for the create_function function. Attached is a patch that change create_function and create_aggregate to use the text_factory

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Berker Peksag added the comment: We can just fallback to the current behavior (i.e. return None) if the return type of text_factory(val_str) is not str, bytes or bytearray. I think this is also somewhat similar to what we do for the return values of sqlite3_column_text(). -- nosy:

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Aviv Palivoda added the comment: The problem is in _pysqlite_build_py_params function at connection.c. In case we have text we do the following code: case SQLITE_TEXT: val_str = (const char*)sqlite3_value_text(cur_value); cur_py_value = PyUnicode_FromString(val_str); /* TODO:

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

Changes by Ned Deily : -- nosy: +ghaering ___ Python tracker ___ ___ Python-bugs-list

[issue29021] Custom functions in sqlite receive None on invalid UTF-8

New submission from Ingo Ruhnke: When a sqlite database contains invalid UTF-8 code in a TEXT column, Python can query that data normally when .text_factory is set appropriately. However when a custom function is created with .create_function() and applied to that column the custom function