[issue30271] Make sqlite3 statement cache optional
Aviv Palivoda added the comment: I think we can close this issue and open a different one that will disable the cache implicitly on `set_authorizer()`. -- stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue30271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30271] Make sqlite3 statement cache optional
Aviv Palivoda added the comment: I don't have any specific use case for making the statement cache optional. I expected that by changing the cache size to 0 there will be no statement cache. I think that this is a common assumption as can be seen in https://github.com/ghaering/pysqlite/issues/126#issue-346189937. rogerbinns did give a use case where we would like to disable the statement cache in https://github.com/ghaering/pysqlite/issues/126#issuecomment-410030910. I think that statement cache should be disable implicitly in that case as you suggest. The code change will be very similar and I do believe we should allow the user to disable the cache. I will be happy to open a new PR once this is merged that will disable the statement cache implicitly when calling `set_authorizer()` -- ___ Python tracker <https://bugs.python.org/issue30271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33376] [pysqlite] Duplicate rows can be returned after rolling back a transaction
Change by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33376> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda <pala...@gmail.com> added the comment: As I wrote in the PR: I think that the contains operation should not be supported for blobs. As blobs can be very large, looking for a subset of bytes inside them will be a very inefficient process in memory or in compute. I believe that this is a very good feature for the sqlite module. I know that there is no active core developer that is currently working on sqlite module but this patch is already waiting 2 years. Could I do anything to help this patch merged? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27645] Supporting native backup facility of SQLite
Aviv Palivoda <pala...@gmail.com> added the comment: The problem is that change https://www.sqlite.org/src/info/169b5505498c0a7e was part of sqlite version 3.8.8 I opened a PR with a fix. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue27645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27645] Supporting native backup facility of SQLite
Change by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +5892 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue27645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Pinging. As I mentioned in the PR I need a little help with __contains__. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30593] sqlite3 executescript does not respect isolation_level?
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30593> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
Aviv Palivoda added the comment: I opened a separate PR for the deprecation of the Cache and Statement. > I would think you could replace those entries with calls to wrapper functions > that issue the deprecation and then call the real function. I made wrapper deprecation classes that issue the deprecation. I am not sure if there is a easier way... -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1668 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
Aviv Palivoda added the comment: I am not sure how to raise the deprecation waning in this case. We use both the Cache and Statement objects as part of the sqlite3 module internal flow. How can I only warn the user when he creates this classes directly and not when the sqlite3 module uses them? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
Aviv Palivoda added the comment: I have 3 argument for the motivation for this change: 1. Cleaner API - Both objects are internal implementation details. I think that we can look on exposing them as a bug. 2. Easier development - When you remove these objects from the API you can change them without any concern. I for one think that I can make the sqlite3 module faster by changing them from Python objects to simple C structs. 3. Documentation - Currently both objects are part of the API. Thus they should be documented but how would you document the Statement class? Do we really want to have a generic Cache class in the sqlite3 module? I have less experience with cpython then you. Do you think that maybe the Cache class should be moved to more appropriate place (maybe collections) and be used by others? > At a minimum there would need to be a deprecation period, Is there a place that document the deprecation process? I will update the patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30270] Remove sqlite3.Cache display method
Aviv Palivoda added the comment: >From looking at the git blame this is there since 2006. Maybe this should be >dependent on issue #30262. Maybe at first there should be a deprecation >warning and then removing this later. I understand that this might break someone's code but this is an undocumented method of an undocumented class in the sqlite3 module. I don't think that there will be a lot of people that use the sqlite3.Cache object. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30270> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30271] Make sqlite3 statement cache optional
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1560 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30271] Make sqlite3 statement cache optional
New submission from Aviv Palivoda: Currently the minimum size of the statement cache is 10. I suggest that it will be any value above 1 or no cache at all if the size is set to 0. -- components: Extension Modules messages: 293006 nosy: berker.peksag, ghaering, palaviv priority: normal severity: normal status: open title: Make sqlite3 statement cache optional type: enhancement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30271> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30270] Remove sqlite3.Cache display method
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1559 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30270> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30270] Remove sqlite3.Cache display method
New submission from Aviv Palivoda: The display method is there for debugging and should not be in the released code. -- components: Extension Modules messages: 293005 nosy: berker.peksag, ghaering, palaviv priority: normal severity: normal status: open title: Remove sqlite3.Cache display method type: enhancement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30270> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1538 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30262] Don't expose sqlite3 Cache and Statement
New submission from Aviv Palivoda: Both the Cache and Statement objects are internally used and should not be exposed to the user from the sqlite3 module. They are not mentioned in the documentation as well. -- components: Extension Modules messages: 292936 nosy: berker.peksag, ghaering, palaviv priority: normal severity: normal status: open title: Don't expose sqlite3 Cache and Statement type: behavior versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30126] CheckTraceCallbackContent of test_sqlite3 fails on OS X Tiger
Aviv Palivoda added the comment: > It's even more strange. The test started to fail since this build, Sun Apr 9 > 10:10:15 2017: This was merged on Apr 9 (commit 0e6cb2ea624570ed08c354f1ed1f595dab4192d6). >From a quick look in the internet some other people had a similar problem: http://stackoverflow.com/questions/28973887/how-to-fix-sqlite3-libversion-mismatch It seems there might be mismatch between sqlite3_libversion() and SQLITE_VERSION. I am not familiar with the buildbots but can you run a test that check that sqlite3_libversion() == SQLITE_VERSION? -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30126> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29021] Custom functions in sqlite receive None on invalid UTF-8
Aviv Palivoda added the comment: In my patch I use sqlite3_create_function_v2 which was added in sqlite 3.7.3 (2010-10-08). There were a few problems after adding sqlite3_stmt_readonly in 284676cf2ac8 as can be seen in issue #29355. sqlite3_stmt_readonly was added in 3.7.4 (2010-12-07) so I guess using sqlite3_create_function_v2 will cause the same problems. Do you think I should find another way to fix this issue? I can follow all the user functions and free on my own but that will make the code more complex. On the other hand I can add this fix in ifdef and the behavior will be different when using sqlite in versions smaller then 3.7.3. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16379] SQLite error code not exposed to python
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1248 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue16379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +1249 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29725] sqlite3.Cursor doesn't properly document "arraysize"
Aviv Palivoda added the comment:
Hi Cheryl, the arraysize value can be set by doing:
>>> cursor.array = 5
For example I can do the following
>>> import sqlite3
>>> s = sqlite3.connect(":memory:")
>>> s.execute("create table a(a,b)")
>>> s.execute("insert into a(a,b) values (1,2)")
>>> s.execute("insert into a(a,b) values (3,4)")
>>> c = s.cursor() # Array size is 1
>>> c.execute("select * from a")
>>> c.fetchmany()
[(1, 2)]
>>> c.fetchmany()
[(3, 4)]
>>> c.arraysize = 2 # Change array size
>>> c.fetchmany()
[(1, 2), (3, 4)]
As you can see the arraysize set the number of results the fetchmany will
return.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29725>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29725] sqlite3.Cursor doesn't properly document "arraysize"
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29725> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29903] struct.Struct Addition
Aviv Palivoda added the comment:
I have two use cases for this feature:
1.
struct a {
int a;
#ifdef VER2
unsigned int b;
#endif
}
Now I may do:
>>> ver1 = Struct("i")
>>> ver2 = ver1 + Struct("I")
2.
struct a {
int a;
union inner {
int b;
unsigned int c;
} u;
}
As you can see with this feature I may do:
>>> start = Struct("i")
>>> union_b = Struct("i")
>>> union_c = Struct("I")
>>> version_a = start + union_b
>>> version_b = start + union_c
If you have a big struct with many options in the union this save's copying the
initial format.
> As for the concrete implementation, it looks to me that Struct('2L') +
> Struct('25B') results to Struct('2L5B').
I will fix the case when there is no format provided.
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29903>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29903] struct.Struct Addition
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +723 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29903> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29903] struct.Struct Addition
New submission from Aviv Palivoda:
I would like to suggest that the struct.Struct class will support addition. For
example you will be able to do:
>>> s1 = Struct(">L")
>>> s2 = Struct(">B")
>>> s3 = s1 + s2
>>> s3.format
b">LB"
--
components: Extension Modules
messages: 290486
nosy: mark.dickinson, meador.inge, palaviv
priority: normal
severity: normal
status: open
title: struct.Struct Addition
type: enhancement
versions: Python 3.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29903>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26187] sqlite3 trace callback prints duplicate line
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +384 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26187> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Aviv Palivoda added the comment: I changed the patch to use SQLITE_VERSION_NUMBER and it looks way better. Thanks Serhiy -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Aviv Palivoda added the comment: from https://www.sqlite.org/changes.html: sqlite3_open_v2 - 2007-09-04 (3.5.0) alpha sqlite3_prepare_v2 - 2007-01-04 (3.3.9) sqlite3_close_v2 - 2012-09-03 (3.7.14) In issue 29355 Ma Lin says that RHEL6 comes with SQLite 3.6.x. I think that removing that removing the check for sqlite3_prepare_v2 and sqlite3_open_v2 is fine but sqlite3_close_v2 need to stay. > Don't forget about Windows where autotools are not used. What should I do for this to work on windows? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Aviv Palivoda added the comment: I opened a PR. This actually is a bugfix in addition to an enhancement as it solves issue 26187 as well. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Just to make sure when you say "sequence protocol" you thin about the doing blob[4:6] = "ab"? I actually think this is a nice feature. The Blob and the mmap object has a lot in common so I think that making the same API will be best. I think that adding the sequence protocol in addition to the file protocol is best. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +306 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27645] Supporting native backup facility of SQLite
Aviv Palivoda added the comment: I actually looked at the patch and have a few comments: 1. You need to put Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS before the sqlite3 calls (especially the sleep). 2. I think that the `pysqlite_connection_backup` function will look a lot better if you will have a cleanup/error label. I am not a core developer but I think you should open the PR as it will be easier for the CR. -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: I am not a DB expert but from a quick search I couldn't find a similar API. I did find a few API's in other programming languages to sqlite blob: 1. https://jgallagher.github.io/rusqlite/rusqlite/blob/index.html 2. https://godoc.org/github.com/mxk/go-sqlite/sqlite3 3. http://www.ch-werner.de/javasqlite/SQLite/Blob.html It seems like most of them give the same API as apsw. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: I opened a PR in github. I tagged some other developers if that will not start a discussion on the API I will post in the python-ideas. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Changes by Aviv Palivoda <pala...@gmail.com>: -- pull_requests: +244 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Uploading patch after fixes from berker CR. The `blob_open` API can can have the following options: 1. The table, column and row must be mandatory parameters. 2. The read/write permissions can have the following options: a. No default (mandatory parameter). b. default read-only c. default write-only 3. The dbname can be without a default of "main" and then it will be a mandatory parameter. I don't think that there is enough differences between the possible API's to justify sending a message to the mailing lists. -- Added file: http://bugs.python.org/file46549/blob4.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Thanks for the CR Serhiy. Attached is a new patch after the fixes from the CR. What other developers should I ask? The interface is file like and is the same as apsw. -- Added file: http://bugs.python.org/file46531/blob3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Pinging again. I think this would be a great enhancement to the sqlite module. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29121] sqlite3 Controlling Transactions documentation not updated
Aviv Palivoda added the comment: I would just like to note that I think that the correct solution is to do an implicit commit before: 1. VACUUM 2. ATTACH 3. DETACH 4. BEGIN -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29121> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29228] sqlite3 OperationalError on changing into WAL transaction mode
Aviv Palivoda added the comment: There is also a issue with the docs here. Look at issue 29121. -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: Removed opening a transaction on select. I will argue for that in issue 9924 after this is resolved. -- Added file: http://bugs.python.org/file46397/sqlite-ddl-dml-3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9924] sqlite3 SELECT does not BEGIN a transaction, but should according to spec
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9924> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: Uploading a new patch with fixes from Ma Lin comments. Two points: 1. Should we add the VACUUM with a explicit commit? Maybe there should be an implicit commit before VACUUM? 2. Should a SELECT start a transaction? I think it should according to PEP 249. There is a open issue on the case (#9924). Should we just change this on this patch? -- Added file: http://bugs.python.org/file46354/sqlite-ddl-dml-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29021] Custom functions in sqlite receive None on invalid UTF-8
Aviv Palivoda added the comment: After looking more into the _pysqlite_set_result function fail in Ingo example I think this is the expected behavior. The PyUnicode_AsUTF8 fails but this is expected as the value is an invalid UTF-8. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29006] 2.7.13 _sqlite more prone to "database table is locked"
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29006> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29021] Custom functions in sqlite receive None on invalid UTF-8
Aviv Palivoda added the comment: Actually had a small mistake in the patch I uploaded. Uploading a fixed one. -- Added file: http://bugs.python.org/file46166/29021-fixed.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29021] Custom functions in sqlite receive None on invalid UTF-8
Aviv Palivoda added the comment: I actually was wrong and there is no problem with the type that is returned from the text_factory as it passes directly as argument for the create_function function. Attached is a patch that change create_function and create_aggregate to use the text_factory when a TEXT data type is received. I am now using sqlite3_create_function_v2 but this is fine because we use sqlite3_stmt_readonly and it is from a newer version. As for Ingo example code it still don't work with this current fix but this is due to a different problem. Now the _pysqlite_set_result function fail when we do PyUnicode_AsUTF8 on the result from py_identity. As this is a different problem I will fix this in a different patch. -- keywords: +patch Added file: http://bugs.python.org/file46164/29021.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29021> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29021] Custom functions in sqlite receive None on invalid UTF-8
Aviv Palivoda added the comment:
The problem is in _pysqlite_build_py_params function at connection.c. In case
we have text we do the following code:
case SQLITE_TEXT:
val_str = (const char*)sqlite3_value_text(cur_value);
cur_py_value = PyUnicode_FromString(val_str);
/* TODO: have a way to show errors here */
if (!cur_py_value) {
PyErr_Clear();
Py_INCREF(Py_None);
cur_py_value = Py_None;
}
break;
As you can see we call PyUnicode_FromString instead of text_factory.
I started making a patch to fix this by passing the text_factory to
_pysqlite_build_py_params function but I currently have a problem with setting
the result to the sqlite. User text_factory may return any type of object and I
can't see how to handle that...
--
nosy: +palaviv
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29021>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: Yes. If a transaction is open you will need to explicitly commit before doing vacuum. I am not sure if that was intentional or not. From quick look in the sqlite source code there are few things that cannot happen from a transaction: 1. VACUUM 2. ATTACH 3. DETACH 4. BEGIN 5. Few PRAGMAs (temp_store, synchronous) All of the above worked with implicit commit before commit 284676cf2ac8 but now has to call commit explicitly. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: Adding the vacuum test can't actually happen because the change in commit 284676cf2ac8 changed the API and we don't commit before non DML statements. I opened a issue that will clarify the docs (#29121). -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29121] sqlite3 Controlling Transactions documentation not updated
New submission from Aviv Palivoda: commit 284676cf2ac8 changed the sqlite3 module so it will no longer implicitly commit an open transaction before DDL statements. The docs have been updated but there is still an incorrect paragraph that has not been removed. Attached is a patch that remove the old paragraph. -- files: sqlite-transaction-doc.patch keywords: patch messages: 284396 nosy: berker.peksag, ghaering, palaviv, serhiy.storchaka priority: normal severity: normal status: open title: sqlite3 Controlling Transactions documentation not updated versions: Python 3.6, Python 3.7 Added file: http://bugs.python.org/file46101/sqlite-transaction-doc.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29121> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: Issue #29003 seems to be related to this one. I think that they can be solved the same way as done in Serhiy patch but I would like to suggest a different approach. I suggest changing the check for DDL statement with a check for DML statement. We actually should start a transaction only for DML statements (https://docs.python.org/3/library/sqlite3.html#controlling-transactions) so all other statements (DDL, DCL and TCL) should not start a transaction. The attached patch solve both this issue and issue #29003. -- Added file: http://bugs.python.org/file46079/sqlite-ddl-dml.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: I think that adding Serhiy patch in addition to the documentation improvement in issue #8145 would be the best way to proceed. This will insure no regression problems. We can add a warning when someone try to use BEGIN, ROLLBACK, SAVEPOINT, and RELEASE without changing the isolation_level to None. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'
Aviv Palivoda added the comment: As we talk here about stdin and stdout which we don't want to close I think this issue is irrelevant to python2. In any case the patch in issue #13824 cover that problem. I actually write a lot of small scripts and if I need to open the file in binary mode I don't use FileType because of this issue. Any way I think this discussion is irrelevant because it does not seem like any core developer is currently working on argparse. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14156> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'
Aviv Palivoda added the comment: Hi paul thanks for looking into this. First are you sure this is a bug in python 2? If so I will happily port this patch once it is reviewed. As for use cases you may look at issue #26488. Although the patch was rejected you can see that I first used argparse.FileType and moved it because of this issue. I can't prove this patch is free of backward's compatibility issue's but there are tests now which should help to avoid problem. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14156> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14156] argparse.FileType for '-' doesn't work for a mode of 'rb'
Aviv Palivoda added the comment: Pinging as mentioned in the devguide. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14156> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8145] Documentation about sqlite3 isolation_level
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8145> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: There are already a few test's that start a transaction explicitly: In dbapi.SqliteOnConflictTests CheckOnConflictRollbackWithExplicitTransaction, CheckOnConflictAbortRaisesWithExplicitTransactions which set ``isolation_level = None``. On the other hand in transaction.TransactionalDDL.CheckTransactionalDDL the test do not set the isolation_level to None but start a transaction explicitly. The test pass because sqlite3_stmt_readonly return true for "begin" and "begin deferred". I think that if we say that in order to start a transaction implicitly the isolation_level needs to be None CheckTransactionalDDL should be changed. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment: I searched in the sqlite tickets from something related to this issue and I couldn't find anything. I wanted to check the sqlite-users mailing list to see if anyone had already reported it but there is a problem in gmane archives. I tried to find a good way to solve this problem but I couldn't. I am adding a patch that should solve this problem but it is for sure not a good way :(. -- keywords: +patch Added file: http://bugs.python.org/file45639/28518.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28518> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28729] Exception from sqlite3 adapter masked by sqlite3.InterfaceError
Changes by Aviv Palivoda <pala...@gmail.com>: -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28729> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28518] execute("begin immediate") throwing OperationalError
Aviv Palivoda added the comment:
I looked into this error and I think the problem is the sqlite3_stmt_readonly
check in _pysqlite_query_execute (cursor.c line 517):
if (self->connection->begin_statement &&
!sqlite3_stmt_readonly(self->statement->st) && !self->statement->is_ddl) {
if (sqlite3_get_autocommit(self->connection->db)) {
result = _pysqlite_connection_begin(self->connection);
if (!result) {
goto error;
}
Py_DECREF(result);
}
}
As you can see we check if the current statement is not readonly and if so we
start a transaction. The problem is that sqlite3_stmt_readonly return false for
the "begin immediate" statement. When this happens we open a transaction with
_pysqlite_connection_begin and then executing the "begin immediate".
I think this is a error in sqlite as the documentation says:
"ransaction control statements such as BEGIN, COMMIT, ROLLBACK, SAVEPOINT, and
RELEASE cause sqlite3_stmt_readonly() to return true,"
(https://www.sqlite.org/c3ref/stmt_readonly.html)
--
nosy: +palaviv
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28518>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20463] sqlite dumpiter dumps invalid script when virtual tables are used
Aviv Palivoda added the comment:
Attached is patch with a fix for this issue. This is not as comprehensive as
the solution in apsw but I think this should cover most of the cases. The
result for Ronny dump after this fix is:
BEGIN TRANSACTION;
PRAGMA writable_schema=ON;
INSERT INTO
sqlite_master(type,name,tbl_name,rootpage,sql)VALUES('table','test','test',0,'CREATE
VIRTUAL TABLE test using fts4(example)');
CREATE TABLE 'test_content'(docid INTEGER PRIMARY KEY, 'c0example');
CREATE TABLE 'test_docsize'(docid INTEGER PRIMARY KEY, size BLOB);
CREATE TABLE 'test_segdir'(level INTEGER,idx INTEGER,start_block
INTEGER,leaves_end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level,
idx));
CREATE TABLE 'test_segments'(blockid INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'test_stat'(id INTEGER PRIMARY KEY, value BLOB);
PRAGMA writable_schema=OFF;
COMMIT;
--
keywords: +patch
nosy: +palaviv
Added file: http://bugs.python.org/file45049/20463.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue20463>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16379] SQLite error code not exposed to python
Changes by Aviv Palivoda <pala...@gmail.com>: Added file: http://bugs.python.org/file44331/16379-3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue16379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27881] Fix possible bugs when setting sqlite3.Connection.isolation_level
Aviv Palivoda added the comment: The only change I see that can happen is that we return upper case isolation level when the user used a lower case. I think that it is worth to slow down the getter for the code simplification. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27881> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27881] Fix possible bugs when setting sqlite3.Connection.isolation_level
Aviv Palivoda added the comment: What do you think about removing the isolation_level member from the pysqlite_Connection. It is only there to be for the pysqlite_connection_get_isolation_level and that could be easily calculated from the begin_statement. -- Added file: http://bugs.python.org/file44328/issue27881_v5.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27881> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27881] Fix possible bugs when setting sqlite3.Connection.isolation_level
Aviv Palivoda added the comment:
Xiang what do you think about changing the isolation_levels list to
begin_statements list:
static const char * const begin_statements[] = {"BEGIN", "BEGIN DEFERRED",
"BEGIN IMMEDIATE", "BEGIN EXCLUSIVE", NULL};
This change will allow you to remove all the strings concatenations and the
malloc/free.
--
nosy: +palaviv
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27881>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
Aviv Palivoda added the comment: > Are the changes in the other ticket needed to implement new tests? > Or is it possible to include tests here? It is not possible to add any tests to this issue before exposing the error code. I will implement new tests when issue 16379 will be resolved. > Aviv the patch makes in itself. I think that this issue should be dependent on issue 16379 as the behavior of the sqlite module will no be changed after this patch until issue 16379 is resolved. In addition there will be some "rebase" issues that will need to be solved. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16379] SQLite error code not exposed to python
Aviv Palivoda added the comment: Attached is a new patch with the encoding problem fixed. -- Added file: http://bugs.python.org/file44306/16379-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue16379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24139] Use sqlite3 extended error codes
Aviv Palivoda added the comment: Attached is a patch to enable the extended error codes. This patch should be dependent on issue 16379. Without returning the sqlite error code in the exception the extended error code does not reveal any information not currently available. As you can see in https://github.com/mackyle/sqlite/blob/ebb27fe5bd5045d924d99cdd7dec9b7064c24768/src/main.c#L1318 there are messages only for the non-extended error codes and that is what is exposed now. -- keywords: +patch nosy: +palaviv Added file: http://bugs.python.org/file44280/24139.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16379] SQLite error code not exposed to python
Aviv Palivoda added the comment: Attached is a patch based on Daniel last patch with the following changes: * There is no errorcode mapping. * The exception object has a error_name attribute. I think this two changes should solve the API problems raised by Ezio and Gerhard about the error code mapping. > I propose to also set the SQLite extended error code if this is implemented. I think that this should be done in a separate patch. I will start working on the extended error code and will upload a patch to issue 24139. -- nosy: +palaviv Added file: https://bugs.python.org/file44213/16379.patch ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue16379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: The use case that made me think about this feature was when I was working on a Windows PC and needed to calculate an md5 of a file. I agree that in a unix environment there are existing tools but on windows you usually don't have them. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: Hi, is there anything more I need to do on this patch? If not do you think this can be added in 3.6? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Thanks for the review Serhiy. Attached is the updated patch after the changes. -- Added file: http://bugs.python.org/file44142/blob2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24905] Allow incremental I/O to blobs in sqlite3
Aviv Palivoda added the comment: Pinging as mentioned in the devguide. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24905> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24887] Sqlite3 has no option to provide open flags
Aviv Palivoda added the comment: IMO this issue can be closed as the URI filename interface can be used instead of the flags. The URI interface parameters can override the flags given as specified in: https://www.sqlite.org/c3ref/open.html -- nosy: +palaviv ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24887> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26187] sqlite3 trace callback prints duplicate line
Aviv Palivoda added the comment: This issue will be resolved when we change the sqlite3_prepare to sqlite3_prepare_v2. So there should be a dependency on issue 9303. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26187> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9303] Migrate sqlite3 module to _v2 API to enhance performance
Aviv Palivoda added the comment: I made a new patch to fix this issue. I left a fallback to the old API as Jim suggested. In addition to the changes in Robin`s patch I changed sqlite3_close to sqlite3_close_v2 if available. This solves issue 26187 as well. -- nosy: +palaviv Added file: http://bugs.python.org/file42663/9303.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9303> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26860] os.walk and os.fwalk yield namedtuple instead of tuple
Aviv Palivoda added the comment: Thanks for the response Ethan I think that I will leave the tests as they are in the current patch. > No doubt, there are exceptions to the rule in the standard library which is > less consistent than we might like: "stat_result". That said, stat_result > is a structseq and many C type names are old or violate the rules (list vs > List, etc). New named tuples should follow PEP 8 can use CapWords > convention unless there is a strong reason not to in a particular case. I actually thought we should keep on consistency with other "result" like objects. I can see your point about new named tuples that should follow PEP 8 and DirEntry is an example of new "result" class that follow PEP8. What names do you suggest? Maybe DirInfo and FDirInfo? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26860] os.walk and os.fwalk yield namedtuple instead of tuple
Aviv Palivoda added the comment:
In regard to Raymond`s points I agree with Serhiy`s comments.
As for Serhiy`s doubts:
> 3. Using namedtuple is slower and consumes more memory than using tuple. Even
> for FS-related operation like os.walk() this can matter. A lot of code is
> optimized for exact tuples, with namedtuple this optimization is lost.
I did some testing on my own PC:
./python -m timeit -s "from os import walk" "for x in walk('Lib'): pass"
Regular tuple: 7.53 msec
Named tuple: 7.66 msec
> 4. New names (dirpath, dirnames, filenames) are questionable. Why not use
> undersores (dir_names)? "dir" in dirpath refers to the current proceeded
> directory, but "dir" in dirnames refers to it's subdirectories. Currently you
> are free to use short names (root, dirs, files) from examples or what you
> prefer, but with namedtuple you are sticked with standard names forever.
> There are no names that satisfy everybody.
I agree that there will be no names that will satisfy everybody but I think the
names that are currently in the documentation are the most trivial choice.
As for points 1,2,5 this feature doesn`t break any of the old walk API.
One more point I would like input on is the testing. I can remove the walk
method from the WalkTests, FwalkTests classes and use the new named tuple
attributes in the tests. Do you think its better or should we keep the tests
with the old API (access using indexes)?
--
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26860>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26860] os.walk and os.fwalk yield namedtuple instead of tuple
New submission from Aviv Palivoda: I am suggesting that os.walk and os.fwalk will yield a namedtuple instead of the regular tuple they currently yield. The use case for this change can be seen in the next example: def walk_wrapper(walk_it): for dir_entry in walk_it: if dir_entry[0] == "aaa": yield dir_entry Because walk_it can be either os.walk or os.fwalk I need to access dir_entry via index. My change will allow me to change this function to: def walk_wrapper(walk_it): for dir_entry in walk_it: if dir_entry.dirpath == "aaa": yield dir_entry Witch is more clear and readable. -- components: Library (Lib) files: os-walk-result-namedtuple.patch keywords: patch messages: 264285 nosy: loewis, palaviv priority: normal severity: normal status: open title: os.walk and os.fwalk yield namedtuple instead of tuple type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file42612/os-walk-result-namedtuple.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26860> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26781] os.walk max_depth
Aviv Palivoda added the comment: I am not sure about the wide use of this feature but I do think this is a nice feature to add to os.walk. I can see how you can implement is_too_deep by counting the number of separators in the path. However I don't think it is trivial to do that. In addition doing the same when topdown is False will not work. As for the other limitations you suggested I think they are less trivial then limiting the recursion depth. As for the "symlink infinite loop" I agree that it will be better to do that by tracking the paths we have already visited but avoiding infinite loop is just a bonus that you get from this feature. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26781> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26781] os.walk max_depth
Changes by Aviv Palivoda <pala...@gmail.com>: Added file: http://bugs.python.org/file42494/os-walk-max-depth-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26781> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26783] test_os.WalkTests.test_walk_topdown don't test fwalk and Bytes
New submission from Aviv Palivoda: test_walk_topdown call os.walk directly instead of using self.walk. This test currently run 3 time's while checking the same thing. The test should use self.walk to check fwalk and Bytes as well. -- components: Tests files: os-test-walk-topdown-use-self-walk.patch keywords: patch messages: 263558 nosy: loewis, palaviv priority: normal severity: normal status: open title: test_os.WalkTests.test_walk_topdown don't test fwalk and Bytes versions: Python 3.6 Added file: http://bugs.python.org/file42492/os-test-walk-topdown-use-self-walk.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26783> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26781] os.walk max_depth
New submission from Aviv Palivoda: I am suggesting to add max_depth argument to os.walk. I think this is very useful for two cases. The trivial one is when someone wants to walk on a directory tree up to specific depth. The second one is when you follow symlinks and wish to avoid infinite loop. The patch add the max_depth both to os.walk and os.fwalk. -- components: Library (Lib) files: os-walk-max-depth.patch keywords: patch messages: 263556 nosy: loewis, palaviv priority: normal severity: normal status: open title: os.walk max_depth type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file42490/os-walk-max-depth.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26781> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14265] Fully qualified test name in failure output
Aviv Palivoda added the comment: Included is a patch with the suggested change. I changed the TestCase.__str__ method to: return "%s (%s.%s)" % (self._testMethodName, strclass(self.__class__), self._testMethodName) instead of return "%s (%s)" % (self._testMethodName, strclass(self.__class__)) So now a failed test look like this: == ERROR: test_error_handling (Lib.test.test_logging.StreamHandlerTest.test_error_handling) -- Traceback (most recent call last): File "/home/aviv/dev/cpython/Lib/test/test_logging.py", line 641, in test_error_handling self.assertIs(h.error_record, r) AttributeError: 'TestStreamHandler' object has no attribute 'error_record' -- -- keywords: +patch nosy: +palaviv versions: +Python 3.6 -Python 3.3 Added file: http://bugs.python.org/file42434/14265.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14265> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26705] logging.Handler.handleError should be called from logging.Handler.handle
Aviv Palivoda added the comment: I see the backwards compatibility issue. I have two suggestion's how to improve the code without breaking backwards compatibility: 1. Add a new Handler class named SafeHandler that will implement handle in the way suggested in the previous patch. All the stdlib handler's will inherit from SafeHandler and in the documentation we will suggest using this handler. I am adding a patch (logging-SafeGandle.patch) with this suggestion. 2. Add new module-level attribute handleException that will deprecate raiseException. When raiseException is set or when handleException is at the default value the current behavior will remain. You can set handleException to the following values: a. print - print exception to stderr b. propagate - propagate exception c. ignore - swallow the exception The current behavior has a few inconsistencies with raiseException. For example when raiseException is True the handleError method don't propagate the excpetion as will be expected. This patch will solve this problem in addition to the one named in the previous message. I am adding a patch with this suggested behavior (logging-handleException). Both patch's are in initial steps and are just to show more clearly what i am suggesting. There are currently no tests or documentation for either patch. Both patch's pass all logging test's without any changes to prove the backwards compatibility. -- Added file: http://bugs.python.org/file42425/logging-SafeHandle.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26705> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26705] logging.Handler.handleError should be called from logging.Handler.handle
Changes by Aviv Palivoda <pala...@gmail.com>: Added file: http://bugs.python.org/file42426/logging-handleException.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26705> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26705] logging.Handler.handleError should be called from logging.Handler.handle
New submission from Aviv Palivoda: Currently all the stdlib logging handlers (except BufferingHandler) emit method have the following structure: def emit(self, record): try: // do the emit except Exception: self.handleError(record) I suggest changing this so that the handle method will do the exception handling of the emit: def handle(self, record): rv = self.filter(record) if rv: self.acquire() try: self.emit(record) except Exception: self.handleError(record) finally: self.release() return rv Now the emit() method can be override without the need to handle it's own exceptions. I think this is more clear with the current documentation as well. For example in the handleError function it says that "This method should be called from handlers when an exception is encountered during an emit() call". In addition in the only example that implement the emit() function https://docs.python.org/3/howto/logging-cookbook.html#speaking-logging-messages there is no error handling at all. -- components: Library (Lib) files: logging-handle-error.patch keywords: patch messages: 262962 nosy: palaviv, vinay.sajip priority: normal severity: normal status: open title: logging.Handler.handleError should be called from logging.Handler.handle type: behavior versions: Python 3.6 Added file: http://bugs.python.org/file42381/logging-handle-error.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26705> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17436] hashlib: add a method to hash the content of a file
Aviv Palivoda added the comment: > * hashobj.readfile(filename: str) > * hashobj.readfileobj(file) where file is an object with a read() method > which returns bytes strings I changed the API to the one Victor suggested. > For readfile() it might make more sense to implement it directly in C and let > OpenSSL's BIO layer handle IO internally. It's more efficient and you can > release the GIL around the whole operation. The readfile method use the openSSL BIO and releases the GIL around the all operation. > I suggest to look at copyfile() and copyfileobj() functions of the shutil > module. For example, copyfileobj() has an optional parameter for the buffer > size. You should probably uses that to avoid complex heuristic to guess the > optimal buffer size. Added a block_size optional argument to the readfileobj(). > In readfile(), you know that it's a regular file which exists on the file > system. So you can directly uses _Py_fstat() to get st_blksize Currently using constant block size in readfile(). From the discussion in issue 26488 I am not sure if this should be changed. -- Added file: http://bugs.python.org/file42356/17436-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue17436> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: Adding new patch after CR changes. -- Added file: http://bugs.python.org/file42355/hashlib-script-mod-md5sum-style-5.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: Publishing another patch after SilentGhost and Victor CR. I also changed the block size to 256 KB. If someone can remove the dependency on issue 14156 (I don't think I have permissions). -- Added file: http://bugs.python.org/file42347/hashlib-script-mod-md5sum-style-4.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17436] hashlib: add a method to hash the content of a file
Aviv Palivoda added the comment: While working on issue 26488 I found a real need for this feature. I added a new method to the hash object named fromfile(). The function update the hash object with the content of the file like object it receives. I only added the feature to hash algorithm provided by OpenSSL. If there will be good reviews on this I will do the work of adding this to all hash algorithms. -- keywords: +patch nosy: +palaviv versions: +Python 3.6 -Python 3.4 Added file: http://bugs.python.org/file42343/17436.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue17436> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: I am adding a new patch with changes from Martin CR (Thanks for the review) and support in the "check" option. I also changed to examples in the Documentation to use sha256 instead of md5 as Christian asked. I left one example with sha1 so when someone read it he will see that other algorithms are supported. As for the multi-threading feature I checked on my PC and I never reach 100% CPU when calculating a single hash so I think leaving this feature out is better. -- Added file: http://bugs.python.org/file42340/hashlib-script-mod-md5sum-style-3.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Changes by Aviv Palivoda <pala...@gmail.com>: Added file: http://bugs.python.org/file42337/hashlib-script-mod-md5sum-style-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: Thanks for the review SilentGhost. I am including the patch after the changes from your CR comments. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: I am adding a new patch with a API compatible to GNU md5sum: $ python -m hashlib md5 /bin/sh $ d985d0ea551c1253c2305140c583d11f /bin/sh I will soon add the feature's requested by Victor: 1) The check option. 2) Running the hash calculation for different files in different threads. If anyone would like any other feature he thinks will be helpful please post it. If we choose to use this API we should remove the dependency on issue 14156. -- Added file: http://bugs.python.org/file42333/hashlib-script-mod-md5sum-style.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26488] hashlib command line interface
Aviv Palivoda added the comment: Sorry on the late response for some reason i don't receive email notification from the tracker for the past few days. 1) Thanks for the review SilentGhost the patch attached include your CR suggestions. 2) Raymond I have fixed the problem with ctrl+D. I tried writing a test to simulate this problem but i don't seem to be able to simulated the terminal behavior on ctrl+D. 3) Removed the block_size option as suggested by Raymond and Victor and using os.stat().st_blksize as suggested by Victor. 4) I changed the CLI to support all available algorithms in hashlib. I am not sure if this is too many choices to show in the --help message. 5) About removing the use of argparse.FileType i would prefer resolving issue 14156 but if you think this is problematic i will do the change. 6) What do you think about changing the API to be more like md5sum? a) Allowing * in the file name to calcualte on multiple files. b) Adding the check option. c) printing file name in output. -- Added file: http://bugs.python.org/file42329/hashlib-script-mod-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26488> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
