[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

It would be more helpful to split the patch on three independent patches for 
three modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24047] str.startswith and str.endswith should accept multiple arguments.

[Ethan Furman]

Ethan Furman added the comment:

We can already do

--> some_string.starts_with(('innie','minnie', 'minie', 'moe'))

Your proposal appears to be equivalent to:

--> 'test'.startswith(('a', 'b', 'c'))

How often do you check to see if a string starts with only a single character?

-1

tuple() is the correct solution to this problem.

--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24048] remove_module() needs to save/restore exception state

[Davis Herring]

New submission from Davis Herring:

import.c's remove_module() is always called with an exception set and can 
invoke arbitrary code via deallocation; if that code calls PyErr_Clear() (or is 
sensitive to PyErr_Occurred()) it will lose (or be damaged by) the preexisting 
exception.

--
components: Interpreter Core
messages: 241914
nosy: herring
priority: normal
severity: normal
status: open
title: remove_module() needs to save/restore exception state
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The committer should be careful, and manually make "hg mv" and apply the patch 
with the patch command, not "hg import". We shouldn't lost the history.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24047] str.startswith and str.endswith should accept multiple arguments.

[Elizabeth Myers]

New submission from Elizabeth Myers:

str.startswith and str.endswith should accept multiple arguments when passing 
in strings. This makes it easier to check if the first character of a string is 
one of a given option, versus this awkward construction:

>>> f = 'abc'
>>> 'test'.startswith(tuple(f))
False

With my proposal, this could be shortened to 'test'.startswith(*f) for easy 
testing.

This also makes it easier to check if a string begins with one of any 
combination of matches.

--
components: Interpreter Core, Unicode
messages: 241912
nosy: Elizacat, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: str.startswith and str.endswith should accept multiple arguments.
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24046] Incomplete build on AIX

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +David.Edelsohn, alex, christian.heimes, dstufft, giampaolo.rodola, 
janssen, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23227] Generator's finally block not run if close() called before first iteration

[Stephen Drake]

Stephen Drake added the comment:

Ok, I can accept that.  I think my mistake was to assume that because a 
generator has a close() method, I could treat it as a lightweight wrapper for 
another closeable object.

But it's better to regard a generator function that wraps an iterable as 
something more akin to map() or filter(), and use a class if it's necessary to 
wrap a file such that close() is passed through.

I happened to take a fresh look at this just the other day and it also occurred 
to me that the kind of composition I was trying to do can work if it's 
generators all the way down:

def open_lines(name, mode='rt', buffering=-1):
with open(name, mode, buffering) as f:
for line in f:
yield line

def logged_lines(f):
try:
for line in f:
logging.warning(line.strip())
yield line
finally:
f.close()

lines = open_lines('yyy', 'r')
if verbose:
lines = logged_lines(lines)
try:
for line in lines:
print(line)
finally:
lines.close()

So a generator can transparently wrap a plain iterable or another generator, 
but not closeable objects in general.  There's nothing really wrong with that, 
so I'm happy for this issue to be closed as invalid.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Ronald Oussoren]

Ronald Oussoren added the comment:

The test for type({}) is indeed wrong and should check for self._dict_type 
instead. However, there needs to be a test before that change is made. 

--
On the road, hence brief. 

Op 24 apr. 2015 om 01:40 heeft Ned Deily  het volgende 
geschreven:

> 
> Ned Deily added the comment:
> 
> Sorry, I don't understand your suggested (4).  If someone cares to provide a 
> suggested patch (with appropriate tests), we could review it.  Otherwise, I 
> would be inclined to close this issue as not an issue.  Ronald, any opinion?
> 
> --
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24036] GB2312 codec is using a wrong covert table

[Ma Lin]

Ma Lin added the comment:

Today, I investigated these popular programming languages, all are the latest 
version.

iconv-1.14  wrong version
php-5.6.8   wrong version (php is using iconv)
ActivePerl-5.20.2wrong version
GoLang-1.4.2 no GB2312, only has GBK/GB18030 
(golang.org/x/text/encoding)
Java 1.7.0_79-b15  wrong version (java.nio.charset)
.Net 2013   rignt version

It seems Python should stay at the wrong version.
Very sorry for waste your time.

Appendix A:
/* The right version's table should be:
 *
 *  gb2312 gbk
 * A1A4U+00B7 MIDDLE DOT  U+00B7 MIDDLE DOT
 * A1AAU+2014 EM DASH   U+2014 EM DASH
 * A844 undefinedU+2015 HORIZONTAL BAR
 */
 
Appendix B:
Advice for final user:
1, Use GBK as much as possible.
2, Be careful when you do interactive operation between GB2312 and GBK/GB18030.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24046] Incomplete build on AIX

[aixtools]

New submission from aixtools:

Actually, I have been building and using my builds of Python, when needed for 
./configure requirements for a long time. In short, it is quite nice that make 
"completes" even when there are missing and/or failed modules.

I have just resolved the problem with ctypes not building (see 
https://bugs.python.org/issue6006) and that got me started to research others.

Failed to build these modules:
_elementtree   _sqlite3   _ssl
bz2pyexpat


While there are several - I am looking first at ssl.

My first attempt comes up with some failed defines - probably because the 
latest openssl provided by IBM is openssl-1.0.0 and openssl-1.0.1 is needed.

Rather than wait for that to happen I decided to experiment with LibreSSL. If 
you are not familiar with LibreSSL - I shall be quick - openbsd (who also 
maintains openssh) has been cutting out insecure and/or superfluous code.

One of the more insecure (because it can be a predictable source of enthropy) 
is RAND_egd() - so it is unavoidable that this occurs:

ld: 0711-317 ERROR: Undefined symbol: .RAND_egd

After patching _ssl.c to this:
--- _ssl.c.orig 2014-06-30 02:05:42 +
+++ _ssl.c  2015-04-24 02:47:00 +
@@ -1604,6 +1604,7 @@
 static PyObject *
 PySSL_RAND_egd(PyObject *self, PyObject *arg)
 {
+#ifndef LIBRESSL_VERSION_NUMBER
 int bytes;
 
 if (!PyString_Check(arg))
@@ -1618,6 +1619,12 @@
 return NULL;
 }
 return PyInt_FromLong(bytes);
+#else
+PyErr_SetString(PySSLErrorObject,
+"external EGD connection not allowed when using 
LibreSSL:"
+"no data to seed the PRNG via PySSL_RAND_egd");
+return NULL;
+#endif
 }
 
 PyDoc_STRVAR(PySSL_RAND_egd_doc,

The end result is:
Failed to build these modules:
_elementtree   _sqlite3   bz2 
pyexpat 

In short, you can get ahead of the curve by depreciating/removing 
PySSL_RAND_egd() because any code that uses it may be receiving predictable 
input and thereafter everything may be predictable.

If you do not believe openbsd (or me) - just read the code. It calls anything 
configured (handy when /dev/urandom was hard to find anno 1999) but these days 
a backdoor waiting to be opened.

p.s. As I get time I shall continue with the other modules that do not build - 
just let me know if you prefer that I continue posting in this "issue", or make 
new one(s) for each module as I find a solution.

--
components: Extension Modules
messages: 241908
nosy: aixto...@gmail.com
priority: normal
severity: normal
status: open
title: Incomplete build on AIX
type: compile error
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6006] ffi.c compile failures on AIX 5.3 with xlc

[aixtools]

aixtools added the comment:

Although this is closed - it is not (yet) resolved, really.

ctypes does not compile on AIX 5.3, or AIX 6.1.

Applying the patch posted (https://bugs.python.org/file17098/pyffi.patch) does 
get rid of most of the problems. However, to completely "fix" it I also copied 
as follows:

cp  -rp ../sourceforge/libffi/libffi-3.2.1/src/powerpc 
./Python-2.7.8/Modules/_ctypes/libffi/src/powerpc
make clean
make

There are still other issues - BUT - with this ctypes completes as a module of 
python.

--
nosy: +aixto...@gmail.com
versions: +Python 2.7 -Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

[Christie]

Christie added the comment:

Hey @berker.peksag, I've added a new patch in response to your review!

> You should create the patch with "hg diff --git" to preserve moving. And then 
> apply it with "hg import", not the patch command. Unfortunately Rietveld 
> don't like patches in git format.

Ick, I'm guessing it's okay to just use plain old "hg diff" then, if Rietveld 
doesn't like the patches in git format.

--
Added file: 
http://bugs.python.org/file39189/iss9517_move_script_helpers_review1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23863] Fix EINTR Socket Module issues in 2.7

[Gregory P. Smith]

Gregory P. Smith added the comment:

diverging discussion: Go re-read the documentation on os.times().  It is 
plural, it isn't just CPU time.  (on POSIX os.times()[4] is likely to be the 
system uptime in seconds as a float... it cannot be changed like the absolute 
clock can, it is a relative monotonic clock).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Ethan Furman]

Ethan Furman added the comment:

Linux is not the only O/S that Python runs on.

There should be no difference between int and long.

Possible doc fix being tracked in issue24045.

Gareth, please ignore my comments about adding guards on the return value -- it 
is up to the O/S to use or adjust whatever Python returns.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24045] Behavior of large returncodes (sys.exit(nn))

[Ethan Furman]

New submission from Ethan Furman:

Not sure if this is a bug, or just One of Those Things:

sys.exit(large_value) can wrap around if the value is too large, but this is 
O/S dependent.

linux (ubuntu 14.04)

  $ python
  Python 2.7.8 (default, Oct 20 2014, 15:05:29) 
  [GCC 4.9.1] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  --> import sys
  --> sys.exit(256)
  $ echo $?
  0

  $ python
  Python 2.7.8 (default, Oct 20 2014, 15:05:29) 
  [GCC 4.9.1] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  --> import sys
  --> sys.exit(257)
  $ echo $?
  1

M$ (Windows 7)

  > python
  Python 2.7... 
  --> import sys
  --> sys.exit(65535)
  > echo %errorlevel%
  65535

  > python
  Python 2.7...
  --> import sys
  --> sys.exit(10)
  > echo %errorlevel%
  10

Perhaps a minor doc update that talks about return codes and why they might not 
be exactly what was given to Python?

--
assignee: docs@python
messages: 241903
nosy: docs@python, ethan.furman
priority: normal
severity: normal
status: open
title: Behavior of large returncodes  (sys.exit(nn))
versions: Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Passing anything other than one of the os.EX_* constants to sys.exit() is a bad 
idea.  In most cases you can get away with 0 and ±1.  Anything beyond 8 bit 
signed range is a gamble.  Passing a computed integer value is even more 
problematic.  With the current behavior, you at least get some diagnostic when 
a computed long finds its way to sys.axit().  With the proposed patch these 
errors will pass silently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24035] When Caps Locked, + alpha-character still displayed as uppercase

[eryksun]

eryksun added the comment:

Are you using pyreadline? It's commonly used with IPython. Pyreadline hooks 
PyOS_ReadlineFunctionPointer to provide readline functionality in the REPL. 
Since it accesses the console input via ReadConsoleInputW events instead of 
using ReadConsoleW, it wouldn't surprise me if it's mishandling the keyboard 
state.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Tal Einat]

Tal Einat added the comment:

Here's a new version of Georg's patch which applies to the current default 
branch. On my OSX 10.8, CPython compiles fine and all tests pass, though 
test_devpoll and test_epoll were skipped.

I haven't made additional changes, so there is still AC-related work to be done 
here. For example, there are legacy converters used throughout the code, which 
should be replaced.

Changes I had to make to get this working:

1)
Updated the class declarations in selectmodule.c with the additional parameters.

2)
The doc-string for signal.sigwaitinfo was changed since Georg made his patch, 
and its first line was now too long. I shortened the first line from:

Wait synchronously for a signal until one of the signals in *sigset* is 
delivered.

to:

Wait synchronously until one of the signals in *sigset* is delivered.

3)
Converted the new sys.is_finalizing function.

4)
Reverted the signal.set_wakeup_fd function, since it has since been changed, 
and now depending on #ifdef MS_WINDOWS it interprets the argument as either 'O' 
or 'i'. I'm not sure what to do with this one, so I reverted it back to not use 
AC for now.

5)
I haven't converted the new signal.default_int_handler function, since it seems 
to accept any number of arguments, and ignores them. Is there a way to do this 
with AC?

6)
Moved the definitions of object structs and PyTypeObject types to the top of 
selectmodule.c, so that they are defined when Modules/clinic/selectmodule.c.h 
is imported.

--
Added file: http://bugs.python.org/file39188/modules_issue20182_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Behdad Esfahbod]

Behdad Esfahbod added the comment:

The items on the stack are created in two ways: [], and self._dict_type().

Currently the code assumes that self._dict_type() returns an object that passes 
isinstance(..., type({})).  I suggested the following two ways to improve this 
check:

  - Replace with: isinstance(..., self._dict_type)

  - Replace with: not isinstance(..., type([]))

Feel free to close.  I reported because I saw room for improvement.  I don't 
/need/ the change myself.

Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Ned Deily]

Ned Deily added the comment:

Sorry, I don't understand your suggested (4).  If someone cares to provide a 
suggested patch (with appropriate tests), we could review it.  Otherwise, I 
would be inclined to close this issue as not an issue.  Ronald, any opinion?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24044] NULL pointer dereference in listsort() with key function

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Behdad Esfahbod]

Behdad Esfahbod added the comment:

I don't have a valid use-case in mind.  I was reading the code and noticed this 
discrepancy.

(4) replace the isinstance(self.stack[-1], type({})) with not 
isinstance(self.stack[-1], type([])).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Ryan Gonzalez]

Ryan Gonzalez added the comment:

>
> "errors should not pass silently"
>
> The "fix" makes the problem worse.
>
> Why would anyone want to pass a long integer to exit?
>
> I bet the user who discovered this problem had something like 0L or 1L
> coming from a lazily written C extension.

Or the person is using a Python code generator that makes everything a long
for portability reasons! Hy does this.

int and long are supposed to be indistinguishable. Quote by Guido from the
time when I figured out the re module's group function doesn't take longs:

...but any place where an int is accepted but a long is not, is a bug. The
language has been on a long trip to making the two types interchangeable
and this seems one of the last remnants.

Notice the word *interchangeable*.

Also, I find that the current behavior is even more confusing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Ned Deily]

Ned Deily added the comment:

The documentation does not explicitly state whether or not dict_type values 
have to be instances / subclasses of dict.  Can you give a code example, 
preferably something that could be added to Lib/test/test_plistlib.py, of a use 
case for something that is not a subclass of dict?   Some possible courses of 
action here: (1) Document that dict_type must be an instance or subclass of 
dict; (2) Change plistlib as you suggest to allow non-subclasses of dict; (3) 
Change plistlib to test for a subclass of collections.abc.MutableMapping; (4) 
other?

--
nosy: +ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

"errors should not pass silently"

The "fix" makes the problem worse.

Why would anyone want to pass a long integer to exit?

I bet the user who discovered this problem had something like 0L or 1L coming 
from a lazily written C extension.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24007] Write PyArg_Parse* format in a line with a function

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24007] Write PyArg_Parse* format in a line with a function

[Roundup Robot]

Roundup Robot added the comment:

New changeset 151cab576cab by Serhiy Storchaka in branch 'default':
Issue #24007: Argument Clinic now writes the format of PyArg_Parse*() at the
https://hg.python.org/cpython/rev/151cab576cab

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Ethan Furman]

Ethan Furman added the comment:

+1 for the fix.

Alexander, create a new issue for the problem of converting non-zero values to 
zero.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Ethan Furman]

Changes by Ethan Furman :


--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20042] Python Launcher, Windows, fails on scripts w/ non-latin names

[Mark Lawrence]

Mark Lawrence added the comment:

The bug reported in msg225529 has been fixed, but there's another one a few 
lines up https://hg.python.org/cpython/file/bd656916586f/PC/launcher.c#l265 as 
there's only one % but two parameters.  Although IIRC we'd get away with this 
the way C works, shouldn't we fix it anyway?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24044] NULL pointer dereference in listsort() with key function

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24038] Missing cleanup in list.sort() with key function

[Benjamin Peterson]

Changes by Benjamin Peterson :


--
resolution:  -> duplicate
status: open -> closed
superseder:  -> NULL pointer dereference in listsort() with key function

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24044] NULL pointer dereference in listsort() with key function

[Roundup Robot]

Roundup Robot added the comment:

New changeset 91096d27c802 by Benjamin Peterson in branch '3.2':
properly handle malloc failure (closes #24044)
https://hg.python.org/cpython/rev/91096d27c802

New changeset 0d8f15053f42 by Benjamin Peterson in branch '3.3':
merge 3.2 (#24044)
https://hg.python.org/cpython/rev/0d8f15053f42

New changeset 80485b8e43cd by Benjamin Peterson in branch '3.4':
merge 3.3 (#24044)
https://hg.python.org/cpython/rev/80485b8e43cd

New changeset bd656916586f by Benjamin Peterson in branch 'default':
merge 3.4 (#24044)
https://hg.python.org/cpython/rev/bd656916586f

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24044] NULL pointer dereference in listsort() with key function

[Benjamin Peterson]

New submission from Benjamin Peterson:

Found by Christian Heimes:

Coverity has found a flaw in Objects/listobject.c:listsort() that
eventually leads to a NULL pointer dereference. Because NULL pointer
dereferences can lead to exploits or DoS vulnerabilities I'm reporting
the error on PSRT first. The error is on a code path that can be
triggered by a remote attacker, although not that easily. All Python 3
versions are affected, Python 2.7 looks save.

The problematic code line is
https://hg.python.org/cpython/file/bc1a178b3bc8/Objects/listobject.c#l19
65
. The code fails to restore self->ob_item to saved_ob_item when
PyMem_MALLOC() fails. Subsequent access to the same list object will
dereference self->ob_item (which is still NULL) and cause a segfault.

A remote attack might be able to trigger the segfault with a large
data set. All it takes is an application that sorts this large data
set with list.sort() and a custom key function. When Python runs out
of memory just in the right spot ... CRASH.

Additionally there is another bug, too. list.sort() doesn't set an
exception when PyMem_MALLOC() fails. A fix for both issues is simple
and straight forward:

diff -r bc1a178b3bc8 Objects/listobject.c
- --- a/Objects/listobject.c  Sat Apr 18 05:54:02 2015 +0200
+++ b/Objects/listobject.c  Sat Apr 18 06:29:02 2015 +0200
@@ -1961,8 +1961,10 @@
 keys = &ms.temparray[saved_ob_size+1];
 else {
 keys = PyMem_MALLOC(sizeof(PyObject *) * saved_ob_size);
- -if (keys == NULL)
- -return NULL;
+if (keys == NULL) {
+PyErr_NoMemory();
+goto keyfunc_fail;
+}
 }

 for (i = 0; i < saved_ob_size ; i++) {

--
components: Interpreter Core
messages: 241889
nosy: benjamin.peterson, christian.heimes
priority: high
severity: normal
status: open
title: NULL pointer dereference in listsort() with key function
type: crash
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> The key issue here is not to report success for nonzero values.

This is different issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Ryan Gonzalez]

Ryan Gonzalez added the comment:

Now we're getting away from the original issue. This wasn't created to handle 
edge cases for sys.exit; is was created to make it accept long values under 
Python 2.

--
nosy: +Ryan.Gonzalez

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

The key issue here is not to report success for nonzero values.

I consider the following a bug:

$ python3
Python 3.4.2 (default, Oct 30 2014, 08:51:12)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.exit(256)
$ echo $?
0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are _PyInt_AsInt() and _PyLong_AsInt().

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24043] Implement mac_romanian and mac_croatian encodings

[Ned Deily]

Changes by Ned Deily :


--
nosy: +lemburg, loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14376] sys.exit documents argument as "integer" but actually requires "subtype of int"

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I am -1 on the patch.  (int)PyLong_AsLong(value) can silently convert non-zero 
error code to zero.  

I would leave 2.7 as is and limit allowable values to a range supported by 
the platform.

Note that ANSI C standard only specifies two values: EXIT_SUCCESS and 
EXIT_FAILURE, that may be passed to exit().

--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24041] Implement Mac East Asian encodings properly

[Ned Deily]

Changes by Ned Deily :


--
nosy: +hyeshik.chang, lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24037] Argument Clinic: add the boolint converter

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I consider this as transitional style, for simpler converting to Argument 
Clinic without changing the behaviour. In future it can be replaced with bool.

But the bool converter has a downside. It is harder to extend it. We can't add 
a support of say a tuple, or callable, or string, or ternary logic value 
without potential breaking some code. I propose first convert all semantic 
boolean ints to the boolint converter, and then, after fundamental analysis, 
replace boolint with bool if there is no any sense to extend concrete parameter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24037] Argument Clinic: add the boolint converter

[Larry Hastings]

Larry Hastings added the comment:

I think this is silly.

Python has a well-understood concept of "truth":
https://docs.python.org/3/library/stdtypes.html#truth-value-testing

I assert that the reason people used the "i" format unit for what are really 
boolean values is because a) the "p" format unit wasn't added until very 
recently, and b) they were lazy and it was easy to copy-and-paste from other 
code.

Rather than perpetuate these misguided hacks, when converting code to Argument 
Clinic we should convert them to properly support truth as defined in Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24042] Convert os._getfullpathname() and os._isdir() to Argument Clinic

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I have no Windows.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24042] Convert os._getfullpathname() and os._isdir() to Argument Clinic

[Larry Hastings]

Larry Hastings added the comment:

Why isn't the patch tested?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24043] Implement mac_romanian and mac_croatian encodings

[Behdad Esfahbod]

New submission from Behdad Esfahbod:

They are used in OpenType fonts, but not implemented by Python at this time.  
Here's are the Unicode mappings for them:

http://unicode.org/Public/MAPPINGS/VENDORS/APPLE/ROMANIAN.TXT
http://unicode.org/Public/MAPPINGS/VENDORS/APPLE/CROATIAN.TXT

Thanks.

--
components: Library (Lib)
messages: 241879
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: Implement mac_romanian and mac_croatian encodings
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24042] Convert os._getfullpathname() and os._isdir() to Argument Clinic

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +patch
Added file: http://bugs.python.org/file39187/clinic_os_getfullpathname.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24009] Get rid of rare format units in PyArg_Parse*

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +Convert os._getfullpathname() and os._isdir() to Argument Clinic

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24042] Convert os._getfullpathname() and os._isdir() to Argument Clinic

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch converts os._getfullpathname() and os._isdir() to Argument 
Clinic. This simplifies a code and as a sided effect adds a check for embedded 
null character in os._getfullpathname(). The patch is not tested and can 
contain bugs.

--
components: Argument Clinic, Windows
messages: 241878
nosy: larry, loewis, serhiy.storchaka, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: patch review
status: open
title: Convert os._getfullpathname() and os._isdir() to Argument Clinic
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24041] Implement Mac East Asian encodings properly

[Behdad Esfahbod]

Behdad Esfahbod added the comment:

Also, I'm not sure about the 'x_' prefix.  It's not kept for the other mac 
encodings.  There's a useful table here:

https://github.com/behdad/fonttools/issues/236

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24041] Implement Mac East Asian encodings properly

[Behdad Esfahbod]

New submission from Behdad Esfahbod:

encodings.aliases has this in it's tail, even master today [0]

# temporary mac CJK aliases, will be replaced by proper codecs in 3.1
'x_mac_japanese'  : 'shift_jis',
'x_mac_korean': 'euc_kr',
'x_mac_simp_chinese'  : 'gb2312',
'x_mac_trad_chinese'  : 'big5',

A full implementation is appreciated.

[0] https://github.com/python/cpython/blob/master/Lib/encodings/aliases.py#L539

--
components: Library (Lib)
messages: 241876
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: Implement Mac East Asian encodings properly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24040] plistlib assumes dict_type is descendent of dict

[Behdad Esfahbod]

New submission from Behdad Esfahbod:

Please replace instances of type({}) in plistlib.py with self._dict_type.

--
components: Library (Lib)
messages: 241875
nosy: Behdad.Esfahbod
priority: normal
severity: normal
status: open
title: plistlib assumes dict_type is descendent of dict
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10733] plistlib rejects strings containing control characters

[Behdad Esfahbod]

Behdad Esfahbod added the comment:

> Replacing all control characters by entities before trying to parse the Plist 
> XML would likely be the best way forward. 

That wouldn't work.  Control characters are disallowed in XML's character set, 
so they are invalid even if input as entities.

Unfortunately this causes a lot of trouble for clients [0], because it means 
that XML cannot represent the full Unicode repertoire.  I'm curious about 
alternates.  Perhaps the expat module can be extended to allow recovering from 
this if the client chooses to...

[0] eg. https://github.com/behdad/fonttools/issues/249

--
nosy: +Behdad.Esfahbod

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11101] plistlib has no graceful way of handing None values

[Behdad Esfahbod]

Behdad Esfahbod added the comment:

plistlib's internal implementation already supports sort_keys and skipkeys, 
those just are not wired to load() and loads().

--
nosy: +Behdad.Esfahbod

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You only need include generated file before defining PyMethodDef arrays and 
PyTypeObject instances. There shouldn't be problems if only one class is 
defined in a file, otherwise you can move these static initializations together 
to the end of the file.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24009] Get rid of rare format units in PyArg_Parse*

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: 
http://bugs.python.org/file39185/issue24009_textio_decoder_getstate.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24009] Get rid of rare format units in PyArg_Parse*

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: 
http://bugs.python.org/file39186/issue24009_textio_decoder_getstate.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24039] Minimize option doesn't work on Search Dialog box for idle

[Prince]

Changes by Prince :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24039] Minimize option doesn't work on Search Dialog box for idle

[Prince]

New submission from Prince:

I tried to search for a keyword in idle with help of search dialog box, it 
worked just fine however when I tried to minimize the box it didn't minimize. 
All other options like expand and close works as expected.

Steps To Reproduce:
1) open idle
2) press cntrl+f to open search dialog box.
3) press minimize button on the box.

expected - It should minimize
Actual - it doesn't
 
Tested on windows7-64bit

--
components: IDLE
messages: 241871
nosy: prince09cs
priority: normal
severity: normal
status: open
title: Minimize option doesn't work on Search Dialog box for idle
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24009] Get rid of rare format units in PyArg_Parse*

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In textio.c, the decoder always should return bytes, not arbitrary read-only 
buffer (this is required in other parts of the code). So "y#" can be replaced 
with "O" with PyBytes_GET_SIZE.

--
keywords: +patch
Added file: 
http://bugs.python.org/file39185/issue24009_textio_decoder_getstate.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24025] str(bytes_obj) should raise an error

[Guido van Rossum]

Guido van Rossum added the comment:

It would be unacceptable if print(b) were to raise an exception. The reason the 
transitional period is long is just that people are still porting Python 2 code.

--
assignee:  -> gvanrossum
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Tal Einat]

Tal Einat added the comment:

I'm working on a new version of Georg's patch which applies to the current 
default branch.

I'm having trouble with AC output going into a separate file, since this 
requires a lot of things to be declared before the #include of the .c.h file. 
Should I move all of those definitions to the top of the file, even if they are 
currently spread all over it? Should I just add forward declarations? And if 
the latter, how do I forward-declare a struct typedef?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24025] str(bytes_obj) should raise an error

[Antoine Pitrou]

Antoine Pitrou added the comment:

> I'm not sure what the "transitional period" refers to, though.

The Python 2 -> Python 3 migration.

> It's 8 years later now and doesn't look like str(bytes_object) will
go away a source of subtle bugs anytime soon

str(bytes_object) is perfectly reasonable when logging stuff, for example.

Recommend closing.

--
nosy: +gvanrossum, pitrou
resolution:  -> rejected
status: open -> pending
superseder:  -> py3k-pep3137: issue warnings / errors on str(bytes()) and 
similar operations

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23974] random.randrange() biased output

[Christopher Gurnee]

Christopher Gurnee added the comment:

> If you have to care about security, you shouldn't use the random module
> at all. random.SystemRandom() merely uses a CPRNG as entropy source. But
> It also manipulates numbers in ways that may or may not be safe.

I must respectfully disagree with this. The current docs say:

> Use os.urandom() or SystemRandom if you require a cryptographically
> secure pseudo-random number generator.

That's a pretty strong statement, and IMO it would lead most to believe that 
SystemRandom along with *all* of its member functions is safe to use for 
cryptographic purposes[1] (assuming of course that os.urandom() is also a safe 
CSPRNG).

As a compromise, perhaps SystemRandom could provide its own randrange() with 
the #9025 fix, while keeping random.randrange() unmodified to preserve the 
implied same-sequence rule.


[1] I don't mean to imply that this bias bug necessarily is a cryptographic 
safety issue--it seems unlikely to me that it is one, however not being a 
cryptographer myself, I'd rather not draw any conclusions either way, and 
instead I'd prefer to err on the side of safety.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24038] Missing cleanup in list.sort() with key function

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24033] Update _test_multiprocessing.py to use script helpers

[Davin Potts]

Changes by Davin Potts :


--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24038] Missing cleanup in list.sort() with key function

[Christian Heimes]

New submission from Christian Heimes:

One error path in list.sort() doesn't do proper cleanup. It neither sets an 
exception nor resets the internal pointers to its former values. In case of 
failed malloc() it leaves the list object in an invalid state.

--
components: Interpreter Core
files: list_keyfunc.patch
keywords: patch
messages: 241865
nosy: christian.heimes
priority: normal
severity: normal
stage: patch review
status: open
title: Missing cleanup in list.sort() with key function
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file39184/list_keyfunc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24036] GB2312 codec is using a wrong covert table

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +lemburg, loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20182] Derby #13: Convert 50 sites to Argument Clinic across 5 files

[Tal Einat]

Tal Einat added the comment:

Note: Despite not appearing in any of these patches, the zlib module seems to 
have already been converted.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23974] random.randrange() biased output

[Christian Heimes]

Christian Heimes added the comment:

IMO it's not a security issue at all. If you have to care about security, you 
shouldn't use the random module at all. random.SystemRandom() merely uses a 
CPRNG as entropy source. But It also manipulates numbers in ways that may or 
may not be safe.

Only os.getrandom() returns unmodified and unbiased random numbers -- iff the 
operating system provides a proper CPRNG.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23496] Steps for Android Native Build of Python 3.4.2

[Cyd Haselton]

Cyd Haselton added the comment:

Finally got readline to work with this build, but since the fixes are to the 
readline source tree I'm not sure if they should go here.

With that plus whatever fixes to patches I suggested previously, I think this 
build works. What would be the next steps for this...running tests?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23699] Add a macro to ease writing rich comparisons

[Petr Viktorin]

Petr Viktorin added the comment:

ping

Anything I can do to help move this forward?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24037] Argument Clinic: add the boolint converter

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The 'i' format unit is often used for semantically boolean values. When the 
parameter has a default value, in Argument clinic you should specify it twice, 
as Python and C values:

closefd: int(c_default="1") = True

or

keepends: int(py_default="False") = 0

Proposed patch adds the boolint converter which makes a conversion for you.

closefd: boolint = True
keepends: boolint = False

--
components: Argument Clinic
files: clinic_boolint_converter.patch
keywords: patch
messages: 241860
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Argument Clinic: add the boolint converter
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file39183/clinic_boolint_converter.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24036] GB2312 codec is using a wrong covert table

[Ma Lin]

Ma Lin added the comment:

"Since MIDDLE DOT is a common character, this bug is annoying."
Sorry, it's MIDDLE DOT, not DASH.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24036] GB2312 codec is using a wrong covert table

[Ma Lin]

New submission from Ma Lin:

While I was trying to optimize GB2312/GBK/GB18030-2000 codecs (they are three 
encodings that widely used in China), I found there is a bug.

The three encodings, their relation should be: GB2312 ⊂ GBK ⊂ GB18030-2000.
However, in Python's implement: GB2312 ⊄ GBK ⊂ GB18030-2000.
GBK should be backward compatible with GB2312, but in Python's implement, it's 
not.


I digged into, I found the Python's GB2312 codec is using a wrong convert table.
In this file /Modules/cjkcodecs/_codecs_cn.c , there is a comment block, I 
paste it here: 

/* GBK and GB2312 map differently in few code points that are listed below:
 *
 *  gb2312  gbk
 * A1A4 U+30FB KATAKANA MIDDLE DOT  U+00B7 MIDDLE DOT
 * A1AA U+2015 HORIZONTAL BAR   U+2014 EM DASH
 * A844 undefined   U+2015 HORIZONTAL BAR
 */
 
 In fact the second column (GB2312 column) is wrong, this column should be 
deleted.
 
 The four involved unicode codepoints are:
 U+30FB  ・ KATAKANA MIDDLE DOT
 U+00B7  ·  MIDDLE DOT
 U+2015  ―  HORIZONTAL BAR
 U+2014  —  EM DASH

So, GB2312 codec decodes b'0xA1, 0xA4' to U+30FB.
U+30FB is a Japanese symbol, but looks quite similar to U+00B7.

I searched "GB2312 Unicode Table" with Google, there are right verson and wrong 
version on the Internet, unfortunately we are using the wrong verson.

libiconv-1.14 is also using the wrong version.


Hold an example of bad behavior. 

Using GBK encoder, encode U+30FB to bytes, UnicodeEncodeError exception 
occurred, becase U+30FB is not in GBK.

In Simplified Chinese version of Microsoft Windows, console's default encoding 
is GBK[1].
If using GB2312 decoder to decode b'0xA1, 0xA4', then print U+30FB to console, 
UnicodeEncodeError raised.
Since DASH is a common character, this bug is annoying.


If we fix this, I don't know how many disasters will happen.
However, if we don't fix this, it's a bug.

I already made a patch, but I think we need a discussion, should we fix this?

---
Annotate:
[1] In fact console's default encoding is cp936, cp936 almost same as GBK, but 
not entirely same. Using GBK in here is not a problem.

--
components: Unicode
files: fixgb2312.patch
keywords: patch
messages: 241858
nosy: Ma Lin, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: GB2312 codec is using a wrong covert table
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file39182/fixgb2312.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24031] Add git support to make patchcheck

[Christian Heimes]

Christian Heimes added the comment:

Thanks!

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24031] Add git support to make patchcheck

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0f9c43fb189d by Christian Heimes in branch '3.4':
Issue #24031: make patchcheck now supports git checkouts, too.
https://hg.python.org/cpython/rev/0f9c43fb189d

New changeset d1b706e57fbe by Christian Heimes in branch 'default':
Issue #24031: make patchcheck now supports git checkouts, too.
https://hg.python.org/cpython/rev/d1b706e57fbe

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23967] Make inspect.signature expression evaluation more powerful

[Larry Hastings]

Larry Hastings added the comment:

It's only used for signatures in builtins.  Any possible security hole here is 
uninteresting because the evil hacker already got to run arbitrary C code in 
the module init.

Because it's only used for signatures in builtins, we shouldn't encounter a 
function with a mutable default value like {} or [] which gets mutated later.  
Builtins don't have those.

In case you're wondering about the "trusted" parameter, that was suggested by 
Nick Coghlan at the PyCon sprints.  He's thinking that other callers may use 
_signature_fromstr() in the future, and he wanted the API to make it clear that 
future uses may be on non-trustworthy sources.

And, finally, consider that the original version already calls eval(). 
Admittedly it uses eval() in a way that should be much harder to exploit.  But 
it's not an enormous difference between the two calls.

I don't really think we need to post to python-dev about this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23863] Fix EINTR Socket Module issues in 2.7

[STINNER Victor]

STINNER Victor added the comment:

> On POSIX systems using os.times()[4] rather than an absolute time.time(),

This is just wrong. os.times() doesn't use seconds, but an "CPU time" which is 
unrelated. socket.settimeout() uses seconds.

By the way, as I wrote before: using the system clock here may introduce bugs 
when the system clock goes forward to backward, but Python 2.7 doesn't have a 
monotonic clock :-( That's another reason why I didn't want to fix Python 2.7 
or 3.4.


> (For UDP it might be a valid thing to do but it still has an odd smell to it)

I'm concerned by backward compatibility. Trivial changes was made in Python 2.7 
and then came back as regressions because it broke several applications. See 
recent discussions on this topic on python-dev. Handling EINTR and recompute 
the timeout is not a trivial changes, some applications rely heavily on the 
current behaviour of sockets. Network servers using Twisted, eventlet and other 
libraries.

When we say "threads", also think about greenthreads which are used in eventlet 
to handle sockets.


> Some things in 2.7 related to EINTR have already been fixed in the past few 
> years such as data loss in readline() and IIRC writelines().  Correctness 
> isn't a feature.

readline()/writelines() don't have a timeout, so it was easier to handle EINTR 
there.


> Do you consider it an API change to prevent an error that nobody relies on or 
> even expects (as PEP 475 notes) and virtually no code is written to properly 
> handle when it _does_ happen?

For your information, some people already contacted me because their 
application behaves differently on Python 3.5 (with the PEP 475). Well, it was 
easy to modify their code to not rely on EINTR, but it means that "Applications 
relying on the fact that system calls are interrupted with InterruptedError 
will hang"  exists in the wild.

While it's acceptable to change such thing in Python 3.5, a new major version, 
I asked if it's acceptable for a minor version.

By the way, Python 3.5 is not released yet, so we can expect more feedback on 
PEP 475 changes when Python 3.5 will be released (other regressions in 
applications?).


> Please don't WONTFIX this issue, I intend to get safe fixes in.

I suggest WONTFIX for Python 2.7, this issue is *already* fixed in Python 3.5. 
Come on, in 2015 it's time to upgrade to Python 3!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23967] Make inspect.signature expression evaluation more powerful

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Using complex expressions is deceitful. In Python functions the default value 
is evaluated only once, at function creation time, but inspect.signature will 
evaluate it every time. For example foo(x={}) and foo(x=dict()) means the same 
in function declaration, but different in signature.

It could also affect security, because allow arbitrary code execution at the 
place where it was not allowed before.

I think this issue should be discussed on Python-Dev. I'm not sure that it is 
pythonic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23713] intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception

[Roundup Robot]

Roundup Robot added the comment:

New changeset 0ac30526c208 by Serhiy Storchaka in branch '2.7':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0ac30526c208

New changeset 0eb5968c15ad by Serhiy Storchaka in branch '3.4':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0eb5968c15ad

New changeset f60f65507d8e by Serhiy Storchaka in branch 'default':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/f60f65507d8e

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23713] intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23967] Make inspect.signature expression evaluation more powerful

[Larry Hastings]

Larry Hastings added the comment:

Cleaned up the patch some more--the code was stupid in a couple places.  I 
think it's ready to go in.

--
Added file: 
http://bugs.python.org/file39181/larry.improved.signature.expressions.3.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23713] intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thanks for your explanation and sorry for the delay Davin. Yes, it makes sense.

--
assignee: davin -> serhiy.storchaka
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24035] When Caps Locked, + alpha-character still displayed as uppercase

[Tim Golden]

Tim Golden added the comment:

Perhaps unsurprisingly, I can't reproduce this on Python 2.7.9 32-bit running 
on Win7 Home Premium. 

Python doesn't handle CapsLock/Shift interaction directly: it just gets what it 
gets from the underlying OS or framework. So I'm at a loss to know what might 
cause the effect you're seeing in so particular a way as to affect one version 
of Python in one i/o framework but not another.

Can you reproduce the situation on a different machine?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You should create the patch with "hg diff --git" to preserve moving. And then 
apply it with "hg import", not the patch command. Unfortunately Rietveld don't 
like patches in git format.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com