[issue34773] sqlite3 module inconsistently returning only some rows from a table
New submission from Shankar :
I am running a Python based system on an Android phone. The database is in
Sqlite3. It has been running well for approximately five years now, with
various upgraded phones, changes to the system, etc.
However, in the last week or so, there has been a problem in the system that is
very peculiar. One of the tables on the system is called "Invoices" and it
currently has approximately 21,500 records in it. However, reading the
database from Python on the phone, using the sqlite3 module, will suddenly
return only around 2,400 records. If I copy the database over to a PC and open
it in Python via Linux, the same thing will happen. If I then open it with the
command line `sqlite3` tool, the table will read correctly. After that, it
will start working correctly in Python as well. If I copy the same database
back to the phone, it will work correctly there as well - for approximately
three or four hours (i.e., given the usual frequency of my program, about 90 -
120 reads / writes). Then the problem will return.
I have changed phones in case this was a problem in the phone's memory, but
that didn't help. I have run `vacuum` on the sqlite3 database in question as
well, to no avail. There do not appear to be any other obvious errors in the
database.
What could be the reason for this behaviour?
Below I've posted some of the code that I use to read the database. Have cut
out some extraneous stuff so you may see variables that are not defined etc.
But I'm fairly sure it's not the code, as this same code has been running for
years with no trouble.
def sqlite_exec(sqlcommand, dbname, inserttable = "", insertstuff = None,
returndict = 0, override_stop = False, returncheck = False, nojournal = False,
onlyjournal = False):
#...
if sqlcommand == "insert":
# Substitute single quotes with double quotes in input text to
avoid sqlite syntax errors
actual_command = "INSERT INTO {0} ({1}) VALUES
({2});".format(inserttable, ", ".join(insertstuff.keys()), ", ".join(["'" +
re.sub("'",'"',valuetext) + "'" for valuetext in insertstuff.values()]))
else:
actual_command = sqlcommand
conn = sqlite3.connect(dbname,timeout = 40.0,isolation_level=None,
detect_types=sqlite3.PARSE_DECLTYPES)
if returndict:
# Using the sqlite module documentation example; this happens to be
better suited for our purposes than the sqlite.Row object
def dict_factory(cursor, row):
d = dict((col[0],row[idx]) for idx,col in
enumerate(cursor.description))
return d
conn.row_factory = dict_factory
sqliteobj = conn.cursor()
# ...
if not onlyjournal:
try:
sqliteobj.execute(actual_command)
# except...
return sqliteobj
--
components: Extension Modules
messages: 326132
nosy: shankargopal
priority: normal
severity: normal
status: open
title: sqlite3 module inconsistently returning only some rows from a table
type: behavior
versions: Python 2.7
___
Python tracker
<https://bugs.python.org/issue34773>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: Can I take this issue? -- nosy: +shankarj67 ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: Thank you for that!!! Can you please provide some pointers to what exactly I have to do? -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: This is my first contribution to Python. This is why I am looking for some guidance, just point me to some resource, and then I will look into it thoroughly. I am reading this https://www.python.org/dev/peps/pep-0572/ for better understanding. -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: Thank you for helping me out brandtbucher. As per my understanding, I am looking into the code (https://github.com/python/cpython/blob/master/Doc/reference/expressions.rst) and finding where the code doesn't follow the pattern described in (https://www.python.org/dev/peps/pep-0572/.) Please let me know if I am correct so that I will quickly go through everything and send you a pull request as soon as possible. -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: I am also checking your commit for this: (https://github.com/python/cpython/commit/8bae21962bab2fac7630982abd73676b89930902) and see that you are changing the "expression: to "assignment_expression". Do I have to fill what assignment expression does with some examples? -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: I got it that I have to fill the "Assignment expression" section in the code. I am working on it and send you the for review once I am done. -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: I have created my draft with an example but I am confused about where exactly do I have to add the code and push it. I have cloned these two repositories in my system and setup everything: https://github.com/python/cpython https://github.com/python/devguide 1. Please tell me where I should be adding my code. Do I have to add code here https://github.com/python/cpython/blob/master/Doc/reference/expressions.rst and push it? 2. I am working on the master branch only. Is there any specific branch I have to select? -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39702] PEP 614: Relaxing Grammar Restrictions On Decorators
Change by SHANKAR JHA : -- nosy: +shankarj67 nosy_count: 2.0 -> 3.0 pull_requests: +18208 pull_request: https://github.com/python/cpython/pull/18851 ___ Python tracker <https://bugs.python.org/issue39702> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
SHANKAR JHA added the comment: I have added the pull request: https://github.com/python/cpython/pull/18851. Please check it out and let me know if I need to change anything. Thank you everyone for your guidance. -- ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39868] Stale Python Language Reference docs (no walrus).
Change by SHANKAR JHA : -- keywords: +patch pull_requests: +18213 stage: -> patch review pull_request: https://github.com/python/cpython/pull/18851 ___ Python tracker <https://bugs.python.org/issue39868> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36200] display index on Index
Change by Aditya Shankar : -- nosy: Aditya Shankar priority: normal severity: normal status: open title: display index on Index ___ Python tracker <https://bugs.python.org/issue36200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36200] display index on IndexError
Change by Aditya Shankar : -- components: +Interpreter Core title: display index on Index -> display index on IndexError type: -> enhancement versions: +Python 3.8 ___ Python tracker <https://bugs.python.org/issue36200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36200] display index on IndexError
New submission from Aditya Shankar : considering a list of 5 elements, if the 6th element is asked, the Interpreter would raise IndexError: list index out of range, I think It'd be better if it actually said what the invalid index is Improvement benefits: *quicker debugging of faulty python code *person debugging does not need to edit and restart the program to fix issue, sometimes at least -- ___ Python tracker <https://bugs.python.org/issue36200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36200] display index on IndexError
Aditya Shankar added the comment: closed as this is a duplicate for https://bugs.python.org/issue18162 -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue36200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36628] Enhancement: i-Strings
New submission from Aditya Shankar :
Problem: multiline strings are a pain to represent (other than of-course in
docstrings), representing a multiline string inside a function looks something
like this -
def foo():
# some code
...
...
# some code
text = """abc meta alpha chronos
dudes uptomes this text
is nonsense"""
return somethingwith(text)
or
def foo():
# some code
...
...
# some code
text = "\n".join(["abc meta alpha chronos",
"dudes uptomes this text",
"is nonsense"])
return somethingwith(text)
an enhancement would be -
def foo():
# some code
...
...
# some code
text = i"""
abc meta alpha chronos
dudes uptomes this text
is nonsense
"""
return somethingwith(text)
i.e. all initial spaces are not considered as a part of the string in each ine
for example while throwing an exception -
def foo(bad_param):
...
try:
some_function_on(bad_param)
except someException:
throw(fi"""
you cant do that because, and I'm gonna explain
this in a paragraph of text with this {variable}
because it explains things more clearly, also
here is the {bad_param}
""")
...
which is far neater than -
def foo(bad_param):
...
try:
some_function_on(bad_param)
except someException:
throw(f"""you cant do that because, and I'm gonna explain
this in a paragraph of text with this {variable}
because it explains things more clearly, also
here is the {bad_param}""")
...
pros:
- represented code is closer to output text
- implementation should not be too hard
--
components: Interpreter Core
messages: 340208
nosy: Aditya Shankar
priority: normal
severity: normal
status: open
title: Enhancement: i-Strings
type: enhancement
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue36628>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36628] Enhancement: i-Strings
Change by Aditya Shankar : -- resolution: postponed -> duplicate ___ Python tracker <https://bugs.python.org/issue36628> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux
New submission from Shankar Unni: I know that an earlier request to use nanosleep() has been rejected as "wontfix", but I'm filing this one for a different reason. Today, timemodule.c:floatsleep() calls select() on platforms that support it. On Linux, select() with a timeout has an unfortunate property that it is very sensitive to clock jumps, because it computes a sleep end time based on the current kernel timestamp. If the system clock is yanked back (by ntpd, or other processes), then the process can end up sleeping for a very long time. (E.g. if the clock is yanked back by half an hour while we are in the middle of, say, a sleep(10), then the process will sleep until "original_kernel_clock+10", which will turn into a half-hour sleep. Yes, systems shouldn't jerk their clocks around, but we can't often control this sort of thing on end-user environments. Using clock_nanosleep(CLOCK_MONOTONIC, 0, , NULL) makes the sleep a much more reliable thing, and mostly insensitive to such jumps. (It'll still be affected by any adjtime(), but that's OK in this case). -- components: Library (Lib) messages: 216799 nosy: shankarunni priority: normal severity: normal status: open title: time.sleep (floatsleep()) should use clock_nanosleep() on Linux type: behavior ___ Python tracker <http://bugs.python.org/issue21302> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux
Shankar Unni added the comment: I'm working on a patch, but I noticed a similar issue in Condition.wait(), which also keeps re-evaluating the "remaining sleep time" based on the current kernel clock, with similar effects. I'll try to address both issues, or we could open a separate bug for the latter.. -- ___ Python tracker <http://bugs.python.org/issue21302> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux
Shankar Unni added the comment: > If you want to modify time.sleep(), you must be careful of the portability: > Windows, Linux, but also Mac OS X, FreeBSD, Solaris, etc. Oh, I totally agree. What I'm trying to do is to define another autoconf flag (HAVE_CLOCK_NANOSLEEP), that does a feature test and enable that flag, and just use that if available. Now that's a good point that if we have clock_nanosleep() on another platform (non-Linux) and it does the wrong thing, then I might have to add further discrimination. For now, one sticking point that I've stumbled across is that clock_nanosleep() requires "-lrt". Complicates the autoconf check a bit. -- ___ Python tracker <http://bugs.python.org/issue21302> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44549] BZip 1.0.6 Critical Vulnerability
New submission from siddhartha shankar mahato : Python (3.9.5 and 3.9.6 are using Bzip2 1.0.6 which has a known critical vulnerability. CVE-2019-12900 (BDSA-2019-1844) 9.8 Critical NVD CVE-2016-3189 (BDSA-2019-2036). Please upgrade the same to a stable version. -- components: Windows messages: 396853 nosy: paul.moore, s.s.mahato, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: BZip 1.0.6 Critical Vulnerability type: crash versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue44549> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
