[issue16096] Get rid of dangerous integer overflow tricks
Serhiy Storchaka added the comment: I withdraw my patches for 2.7 and 3.2 due to the fact that they have no visible effect on supported platforms. Patches for 3.3+ already committed, therefore I close this issue. -- resolution: - fixed stage: - committed/rejected status: open - closed versions: +Python 3.3, Python 3.4 -Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Serhiy Storchaka added the comment:
Here are updated to current codebase patches for 2.7 and 3.2. It seems that
all the rest of overflows are hypothetical bugs and do not appear on the
current supported platforms. Fix them is not necessary (rather for purity). If
no one can see visible bugs, I'll close this issue soon.
--
Added file: http://bugs.python.org/file28732/size_overflow-2.7_2.patch
Added file: http://bugs.python.org/file28733/size_overflow-3.2_2.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16096
___diff -r f2353e74b335 Modules/_randommodule.c
--- a/Modules/_randommodule.c Tue Jan 08 23:12:00 2013 +0200
+++ b/Modules/_randommodule.c Wed Jan 09 19:00:27 2013 +0200
@@ -283,7 +283,8 @@
n = newn;
if (keyused = keymax) {
unsigned long bigger = keymax 1;
-if ((bigger 1) != keymax) {
+if ((bigger 1) != keymax ||
+bigger PY_SSIZE_T_MAX / sizeof(*key)) {
PyErr_NoMemory();
goto Done;
}
diff -r f2353e74b335 Modules/arraymodule.c
--- a/Modules/arraymodule.c Tue Jan 08 23:12:00 2013 +0200
+++ b/Modules/arraymodule.c Wed Jan 09 19:00:27 2013 +0200
@@ -423,11 +423,11 @@
return NULL;
}
-nbytes = size * descr-itemsize;
/* Check for overflow */
-if (nbytes / descr-itemsize != (size_t)size) {
+if (size PY_SSIZE_T_MAX / descr-itemsize) {
return PyErr_NoMemory();
}
+nbytes = size * descr-itemsize;
op = (arrayobject *) type-tp_alloc(type, 0);
if (op == NULL) {
return NULL;
@@ -1205,13 +1205,10 @@
char *item = self-ob_item;
Py_ssize_t itemsize = self-ob_descr-itemsize;
size_t nread;
-Py_ssize_t newlength;
size_t newbytes;
-/* Be careful here about overflow */
-if ((newlength = Py_SIZE(self) + n) = 0 ||
-(newbytes = newlength * itemsize) / itemsize !=
-(size_t)newlength)
+if (n (PY_SSIZE_T_MAX - Py_SIZE(self)) / itemsize)
goto nomem;
+newbytes = (Py_SIZE(self) + n) * itemsize;
PyMem_RESIZE(item, char, newbytes);
if (item == NULL) {
nomem:
diff -r f2353e74b335 Modules/audioop.c
--- a/Modules/audioop.c Tue Jan 08 23:12:00 2013 +0200
+++ b/Modules/audioop.c Wed Jan 09 19:00:27 2013 +0200
@@ -1094,8 +1094,7 @@
PyErr_SetString(AudioopError, # of channels should be = 1);
return NULL;
}
-bytes_per_frame = size * nchannels;
-if (bytes_per_frame / nchannels != size) {
+if (size INT_MAX / nchannels) {
/* This overflow test is rigorously correct because
both multiplicands are = 1. Use the argument names
from the docs for the error msg. */
@@ -1103,6 +1102,7 @@
width * nchannels too big for a C int);
return NULL;
}
+bytes_per_frame = size * nchannels;
if (weightA 1 || weightB 0) {
PyErr_SetString(AudioopError,
weightA should be = 1, weightB should be = 0);
diff -r f2353e74b335 Modules/cPickle.c
--- a/Modules/cPickle.c Tue Jan 08 23:12:00 2013 +0200
+++ b/Modules/cPickle.c Wed Jan 09 19:00:27 2013 +0200
@@ -218,14 +218,12 @@
size_t nbytes;
PyObject **tmp;
+if (self-size == 0 || self-size (INT_MAX 1))
+goto nomemory;
bigger = self-size 1;
-if (bigger = 0)/* was 0, or new value overflows */
-goto nomemory;
-if ((int)(size_t)bigger != bigger)
+if ((size_t)bigger PY_SSIZE_T_MAX / sizeof(PyObject *))
goto nomemory;
nbytes = (size_t)bigger * sizeof(PyObject *);
-if (nbytes / sizeof(PyObject *) != (size_t)bigger)
-goto nomemory;
tmp = realloc(self-data, nbytes);
if (tmp == NULL)
goto nomemory;
diff -r f2353e74b335 Objects/bytearrayobject.c
--- a/Objects/bytearrayobject.c Tue Jan 08 23:12:00 2013 +0200
+++ b/Objects/bytearrayobject.c Wed Jan 09 19:00:27 2013 +0200
@@ -357,9 +357,9 @@
if (count 0)
count = 0;
mysize = Py_SIZE(self);
+if (count != 0 mysize PY_SSIZE_T_MAX / count)
+return PyErr_NoMemory();
size = mysize * count;
-if (count != 0 size / count != mysize)
-return PyErr_NoMemory();
result = (PyByteArrayObject *)PyByteArray_FromStringAndSize(NULL, size);
if (result != NULL size != 0) {
if (mysize == 1)
@@ -382,9 +382,9 @@
if (count 0)
count = 0;
mysize = Py_SIZE(self);
+if (count != 0 mysize PY_SSIZE_T_MAX / count)
+return PyErr_NoMemory();
size = mysize * count;
-if (count != 0 size / count != mysize)
-return PyErr_NoMemory();
if (size self-ob_alloc) {
Py_SIZE(self) = size;
self-ob_bytes[Py_SIZE(self)] = '\0'; /* Trailing null byte */
@@ -1568,7 +1568,7 @@
{
char *self_s,
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Mark Dickinson dicki...@gmail.com: -- assignee: mark.dickinson - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: Yes, reopening issue 14700 sounds good to me. I'm not against fixing these issues in the bugfix branches, but we need to do it carefully (which unfortunately probably also means slowly). I think that for the bugfix branches, each fix should be accompanied by a test that exercises the original bug. I'd also suggest having a separate issue for each bug, for ease of review. I'd probably also prioritise those bugs that can be triggered without having huge structures in memory: e.g., the issue 14700 bug seems more important to fix than the PyTuple_New bug. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Mark Dickinson dicki...@gmail.com: -- versions: -Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: It's maybe safer (and simpler) to not touch such code in Python older than 3.4. So far, I've been fixing these overflow bugs only in the development branches, unless they can be shown to cause actual bugs. That said, I think it's probably okay to apply these for 3.3 as well as 3.4, especially since the 3.3 patch is smaller than the others. I'll review and apply. -- assignee: - mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Serhiy Storchaka added the comment: especially since the 3.3 patch is smaller than the others. It's becouse 3.3 already contains some fixes which was not be backported to older versions. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: It's becouse 3.3 already contains some fixes which was not be backported to older versions. Yes, exactly! That's what I meant when I said: So far, I've been fixing these overflow bugs only in the development branches There were lots of integer overflow occurrences like these found by John Regehr in issue 9530. I chose to fix those only in the current development branch, which was 3.3 at the time. Since we've made an effort to clean up 3.3 in that respect, I think it's worth finishing that job off by applying your patch both to 3.3 and 3.4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Serhiy Storchaka added the comment: unless they can be shown to cause actual bugs. See issue14700. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: Serhiy, I don't understand what you're getting at. Can you explain? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Roundup Robot added the comment: New changeset 152d85b2da3a by Mark Dickinson in branch '3.3': Issue #16096: Fix several occurrences of potential signed integer overflow. Thanks Serhiy Storchaka. http://hg.python.org/cpython/rev/152d85b2da3a New changeset faae99459b43 by Mark Dickinson in branch 'default': Issue #16096: Merge fixes from 3.3. http://hg.python.org/cpython/rev/faae99459b43 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: Applied the 3.3 patch to 3.3 and default, with some minor changes: - revert the Objects/longobject.c changes, since they don't depend on signed overflow - fix the second change in Objects/tupleobject.c so that the overflow check happens before the multiplication rather than after. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Mark Dickinson added the comment: Whoops. I take it back about the Objects/longobject.c bit. Fixing ... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Roundup Robot added the comment: New changeset 906ae6485cb8 by Mark Dickinson in branch '3.3': Issue #16096: Fix signed overflow in Objects/longobject.c. Thanks Serhiy Storchaka. http://hg.python.org/cpython/rev/906ae6485cb8 New changeset b728aac3bdb3 by Mark Dickinson in branch 'default': Issue #16096: port fix from 3.3 http://hg.python.org/cpython/rev/b728aac3bdb3 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Serhiy Storchaka added the comment: In issue14700 were fixed two actual bugs. The fix was not be backported to older versions (and this changes included in patches for this issue). I think it is better to reopen issue14700 for backporting fixes to 2.7 and 3.2? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Andrew Svetlov added the comment: The patches looks good for me, but I like to double check before commit. Let's wait for a week for other reviewers. -- nosy: +asvetlov ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
STINNER Victor added the comment: It's maybe safer (and simpler) to not touch such code in Python older than 3.4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
New submission from Serhiy Storchaka: In several places such dungerous code used to check the integer overflow: size = n * itemsize; if (size / itemsize != n) raise exception... Because these values are signed, this results in undefined behavior. The proposed patches replace similar unsafe code to safe one. Note that the patches for the different versions are substantially different. -- components: Extension Modules, Interpreter Core files: size_overflow-3.3.patch keywords: patch messages: 171657 nosy: mark.dickinson, storchaka priority: normal severity: normal status: open title: Get rid of dangerous integer overflow tricks type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4 Added file: http://bugs.python.org/file27360/size_overflow-3.3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Serhiy Storchaka storch...@gmail.com: Added file: http://bugs.python.org/file27361/size_overflow-3.2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16096] Get rid of dangerous integer overflow tricks
Changes by Serhiy Storchaka storch...@gmail.com: Added file: http://bugs.python.org/file27362/size_overflow-2.7.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16096 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
