[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +25081
pull_request: https://github.com/python/cpython/pull/26485

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

> Erlend, check out 
> https://github.com/python/cpython/pull/26206#discussion_r643910263. After 
> that we can close this issue

See msg393857: I'll add a PR for _pysqlite_connection_begin as well. After 
that, we can close this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset fbf25b8c0dd1e62db59117d53bbd2d4131a06867 by Erlend Egeberg 
Aasland in branch 'main':
bpo-44165: pysqlite_statement_create now returns a Py object, not an int 
(GH-26484)
https://github.com/python/cpython/commit/fbf25b8c0dd1e62db59117d53bbd2d4131a06867


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +25080
pull_request: https://github.com/python/cpython/pull/26484

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Erlend, check out 
https://github.com/python/cpython/pull/26206#discussion_r643910263. After that 
we can close this issue

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset a384b6c04054a2c5050a99059836175cf73e2016 by Erlend Egeberg 
Aasland in branch 'main':
bpo-44165: Optimise sqlite3 statement preparation by passing string size 
(GH-26206)
https://github.com/python/cpython/commit/a384b6c04054a2c5050a99059836175cf73e2016


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

SQLITE_TOOBIG is currently mapped to sqlite3.DataError. In order to keep the 
current behaviour, DataError must be raised.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
Removed message: https://bugs.python.org/msg393977

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Regarding the maximum length of an SQL string, quoting from 
https://sqlite.org/limits.html:
"The current implementation will only support a string or BLOB length up to 
2^31-1 or 2147483647. And some built-in functions such as hex() might fail well 
before that point. In security-sensitive applications it is best not to try to 
increase the maximum string and blob length. In fact, you might do well to 
lower the maximum string and blob length to something more in the range of a 
few million if that is possible."

The size returned from functions such as PyUnicode_AsUTF8AndSize is Py_ssize_t. 
I suggest checking the passed SQL string size and raising OverflowError if the 
SQL string is larger than SQLITE_MAX_LENGTH.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Regarding the maximum length of an SQL string, quoting from 
https://sqlite.org/limits.html:
"The current implementation will only support a string or BLOB length up to 
2^31-1 or 2147483647. And some built-in functions such as hex() might fail well 
before that point. In security-sensitive applications it is best not to try to 
increase the maximum string and blob length. In fact, you might do well to 
lower the maximum string and blob length to something more in the range of a 
few million if that is possible."

The size returned from functions such as PyUnicode_AsUTF8AndSize is Py_ssize_t. 
I suggest checking the passed SQL string size and raising OverflowError if the 
SQL string is larger than 2^31-1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Note, PR 26206 does not include statement creation in 
_pysqlite_connection_begin (Modules/_sqlite/connection.c). That needs further 
refactoring, so I'll add that in a separate PR if PR 26206 is accepted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
keywords: +patch
pull_requests: +24823
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26206

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44165] [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size

[Erlend E. Aasland]

New submission from Erlend E. Aasland :

The signature of sqlite3_prepare_v2 is as follows:
int sqlite3_prepare_v2(
  sqlite3 *db,/* Database handle */
  const char *zSql,   /* SQL statement, UTF-8 encoded */
  int nByte,  /* Maximum length of zSql in bytes. */
  sqlite3_stmt **ppStmt,  /* OUT: Statement handle */
  const char **pzTail /* OUT: Pointer to unused portion of zSql */
);


Quoting from the SQLite docs[1]:
"If the caller knows that the supplied string is nul-terminated, then there is 
a small performance advantage to passing an nByte parameter that is the number 
of bytes in the input string including the nul-terminator."


sqlite3_prepare_v2 is used five places in the sqlite3 module. We can easily 
provide the string size in those places.



[1] https://sqlite.org/c3ref/prepare.html

--
components: Extension Modules
messages: 393856
nosy: berker.peksag, erlendaasland, serhiy.storchaka
priority: low
severity: normal
status: open
title: [sqlite3] sqlite3_prepare_v2 micro optimisation: pass string size
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com