[issue27135] nested list produced with multiplication is linked to the same list

[Benjamin Peterson]

Benjamin Peterson added the comment:

That's just how Python works. 
https://docs.python.org/3/faq/programming.html#how-do-i-create-a-multidimensional-list

--
nosy: +benjamin.peterson
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27133] python 3.5.1 will not compile because libffi module uses wrong CFLAGS

[Dennis Clarke]

Dennis Clarke added the comment:

On 05/26/2016 06:01 PM, Zachary Ware wrote:
>
> Zachary Ware added the comment:
>
> Would you be interested in submitting a patch?

Right now I am trying to get a clean build of libffi outside of the 
python tree and then will use the --use-system-libffi option to get 
around this mess. Of course, that sort of means that the libffi sources 
build and test clean and that, of course, is its own struggle.

see  https://sourceware.org/ml/libffi-discuss/2016/msg00024.html

and  https://sourceware.org/ml/libffi-discuss/2016/msg00025.html

I'll keep you posted on progress here.

Dennis

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27135] nested list produced with multiplication is linked to the same list

[Matthew Tanous]

New submission from Matthew Tanous:

If I produce a list in this fashion:

l = [[x] * n] * n

I would expect that I would obtain a matrix-like structure.  Instead, I get a 
list of the *same* list, such that the statement:

l[x][y] = z

would change, in essence, every value in "column" y.  This is different from 
the case where the list contains a string or integer value, where the new list 
points to separate instances of the internal values.

In my view, this is strange and counter-intuitive behavior.  If I want to 
create a matrix-like set to None to start, instead of using:

 mat = [[None] * N] * N

I have to use:

 mat = [[None] * N for i in range(N)]

If there is any possibility for it to be changed, I think that would improve 
things considerably.  In almost no cases, in my opinion, would I want a list of 
lists that are forced to be the same.

--
components: Interpreter Core
messages: 266475
nosy: Matthew Tanous
priority: normal
severity: normal
status: open
title: nested list produced with multiplication is linked to the same list
versions: Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27134] allow str(bytes) raises an exception to be controlled programmatically

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Isn't this a duplicate of issue18373?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Xiang Zhang]

Xiang Zhang added the comment:

Quick and careless scanning at night lead me to a wrong result, Sorry.

> You would need to compress just under 4 GiB of data that requires 5 MB or 
> more when compressed (i.e. not all the same bytes, or maybe try level=0).

With enough memory, compressing with level 0 does raise a error while the 
default level didn't. 

Except for overflow fix, does zlib have to support large data in one operation? 
For example, it's OK that zlib.compress does not support data beyond 4GB since 
we can split data in application and then call zlib.compress on each part and 
finally concatenate the results.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27132] New assert method that checks an error message for a list of strings

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This issue looks similar to issue24922, and I think it should be closed for 
same reasons.

--
nosy: +berker.peksag, ezio.melotti, michael.foord, rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26834] Add truncated SHA512/224 and SHA512/256

[Martin Panter]

Martin Panter added the comment:

Would it be appropriate to mention these hashes higher up in the documentation, 
e.g. where it lists the names of the always-present constructors?

I haven’t had a close look at the C code. It looks like you are also adding 
better multithreading support for SHA-512 objects (? related to Issue 4821).

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27133] python 3.5.1 will not compile because libffi module uses wrong CFLAGS

[Dennis Clarke]

Dennis Clarke added the comment:

On 05/26/2016 06:01 PM, Zachary Ware wrote:
>
> Zachary Ware added the comment:
>
> Would you be interested in submitting a patch?

Sure, of course. There are a number of problems in the Makefile(s) for a 
system not using gcc and where CFLAGS and LD_foo is pretty important. 
Certainly where the RPATH and RUNPATH in the resultant ELF output 
binaries really really matters. So yes, sure. If I can get it sorted out.

> The whole ctypes package and the bundled libffi in particular are
 > fairly unloved.

Put me in that ground also :-\

> As a workaround, if you have libffi installed on your system

I try to avoid it actually.

> you can use the '--with-system-ffi' flag to Python's configure
 > script to direct the ctypes build to use the installed libffi.

If there were such a thing as system libffi then I would give that a try 
but for now I am on my own and will need to everything from sources.

> Also note that unless you specifically need ctypes (or are
 > building for someone who might)

Well, strictly speaking, I don't.  However someone else may and I have 
to roll this out to a pile of systems eventually.  For now it is just 
internal on my build servers.

> it may not be the end of the world for ctypes to not be available;

Is there a configure option to disable them?  I should look.

> the overall build should not fail just due to a ctypes build failure.

cool ... let's hope for the best.

Dennis

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10276] zlib crc32/adler32 buffer length truncation (64-bit)

[Martin Panter]

Changes by Martin Panter :


--
Removed message: http://bugs.python.org/msg135065

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10276] zlib crc32/adler32 buffer length truncation (64-bit)

[Martin Panter]

Changes by Martin Panter :


--
Removed message: http://bugs.python.org/msg135066

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23306] Within zipfile, use of zlib.crc32 raises OverflowError at argument-parsing time on large strings

[Martin Panter]

Martin Panter added the comment:

Apparently crc32() was fixed in Python 3 via Issue 10276.

See also Issue 27130 about 64-bit support more generally in zlib.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18373] implement sys.get/setbyteswarningflag()

[Daniel Holth]

Daniel Holth added the comment:

Superceded by http://bugs.python.org/issue27134 , a simpler solution providing 
a with StrBytesRaises(): context manager.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Martin Panter]

Martin Panter added the comment:

Sorry Issue 10276 regarding crc32() and adler32() was only fixed for Python 3. 
Issue 23306 is open about crc32() in Python 2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27134] allow str(bytes) raises an exception to be controlled programmatically

[Daniel Holth]

New submission from Daniel Holth:

When I discovered str(b'bytes') in my Python 3 program was causing errors to be 
serialized to disk, I was unhappy. It turns out there is a command line option 
to turn it off, but the vulnerable serialization code is not going to be able 
to set that argument; or the one-argument-per-shebang limit we have in Linux 
was already used for something else.

Instead, provide a threadlocal variable that causes str(bytes) to raise. A 
context manager makes it simple to use for just a portion of your code:

with string.StrBytesRaises():
   no_str_bytes_here()

If not set or False then Python behaves as before.

--
components: Unicode
files: strbytes.patch
keywords: patch
messages: 266465
nosy: dholth, ezio.melotti, haypo
priority: normal
severity: normal
status: open
title: allow str(bytes) raises an exception to be controlled programmatically
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file43024/strbytes.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Martin Panter]

Martin Panter added the comment:

This is similar, but different to the other bug. The other bug was only about 
output limits for incrementally decompressed data. Klamann’s bug is about the 
actual size of input (and possibly also output) buffers.

The gzip.compress() implementation uses zlib.compressobj.compress(), which does 
not accept 2 or 4 GiB input either.

The underlying zlib library uses “unsigned int” for the size of input and 
output chunks. It has to be called multiple times to handle 4 GiB. In both 
Python 2 and 3, the one-shot compress() function only does a single call to 
zlib. This explains why Python 3 cannot take 4 GiB.

Python 2 uses an “int” for the input buffer size, hence the 2 GiB limit.

I tend to think of these cases as bugs, which could be fixed in 3.5 and 2.7. 
Sometimes others also treat adding 64-bit support as a bug fix, e.g. 
file.read() on Python 2 (Issue 21932). But other times it is handled as a new 
feature for the next Python version, e.g. os.read() was fixed in 3.5, but not 
2.7 (Issue 21932), random.getrandbits() proposed for 3.6 only (Issue 27072).

This kind of bug is apparently already fixed for crc32() and adler32() in 
Python 2 and 3; see Issue 10276.

This line from zlib.compress() also worries me:

zst.avail_out = length + length/1000 + 12 + 1; /* unsigned ints */

I suspect it may overflow, but I don’t have enough memory to verify. You would 
need to compress just under 4 GiB of data that requires 5 MB or more when 
compressed (i.e. not all the same bytes, or maybe try level=0).

Also, the logic for expanding the output buffer in each of zlib.decompress(), 
compressobj.compress(), decompressobj.decompress(), compressobj.flush(), and 
decompressobj.flush() looks faulty when it hits UINT_MAX. I suspect it may 
overwrite unallocated memory or do other funny stuff, but again I don’t have 
enough memory to verify. What happens when you decompress more than 4 GiB when 
the compressed input is less than 4 GiB?

Code fixes that I think could be made:

1. Avoid the output buffer size overflow in the zlib.compress() function

2. Rewrite zlib.compress() to call deflate() in a loop, one iteration for each 
4 GiB input or output chunk

3. Allow the zlib.decompress() function to expand the output buffer beyond 4 GiB

4. Rewrite zlib.decompress() to pass 4 GiB input chunks to inflate()

5. Allow the compressobj.compress() method to expand the output buffer beyond 4 
GiB

6. Rewrite compressobj.compress() to pass 4 GiB input chunks to deflate()

7. Allow the decompressobj.decompress() method to expand the output buffer 
beyond 4 GiB

8. Rewrite decompressobj.decompress() to pass 4 GiB input chunks to inflate(), 
and to save 4 GiB in decompressobj.unconsumed_tail and unused_data

9. Change the two flush() methods to abort if they allocate UINT_MAX bytes, 
rather than pointing into unallocated memory (I don’t think this could happen 
in real usage, but the code shares the same problem as above.)

--
components: +Extension Modules -Library (Lib)
stage:  -> needs patch
versions: +Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27043] Describe what ‘inspect.cleandoc’ does to synopsis line.

[Ben Finney]

Ben Finney added the comment:

On Fri, May 27, 2016, at 02:58, Nathan Harold wrote:
> Here's my shot at a revision (corresponding patch attached):

Thanks for that. The only improvement I'd ask for is to more clearly
separate
the description into two paragraphs: overall description of the effect
of the
function; then, detailed description of the specific transformations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27043] Describe what ‘inspect.cleandoc’ does to synopsis line.

[Martin Panter]

Martin Panter added the comment:

Patch looks good to me, thanks Nathan

--
nosy: +martin.panter
stage:  -> patch review
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27132] New assert method that checks an error message for a list of strings

[Martin Panter]

Martin Panter added the comment:

Maybe a generic superset test would be good enough:

def assertSuperset(self, superset, subset):
# Expand for friendlier failure handling
self.assertTrue(all(e in superset for e in subset))

self.assertSuperset("Error message", ("mess", "or me"))

Or is that too obscure, treating a string as a set of substrings?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25591] refactor imaplib tests

[Maciej Szulik]

Maciej Szulik added the comment:

David I'm uploading the patch with all the review comments addressed. The 
changes from just the review can be seen here: 
https://bitbucket.org/soltysh/cpython/commits/91708e0c6e7c5230bd69b1135b82ef47b6dc4d43?at=default
 (please ignore the changes to test_logincapa_withclient_certfile, these are 
not part of the final patch).

--
Added file: http://bugs.python.org/file43023/new_imap_tests_v4.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27133] python 3.5.1 will not compile because libffi module uses wrong CFLAGS

[Zachary Ware]

Zachary Ware added the comment:

Would you be interested in submitting a patch?  The whole ctypes package and 
the bundled libffi in particular are fairly unloved.

As a workaround, if you have libffi installed on your system, you can use the 
'--with-system-ffi' flag to Python's configure script to direct the ctypes 
build to use the installed libffi.

Also note that unless you specifically need ctypes (or are building for someone 
who might), it may not be the end of the world for ctypes to not be available; 
the overall build should not fail just due to a ctypes build failure.

--
nosy: +amaury.forgeotdarc, belopolsky, meador.inge, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27133] python 3.5.1 will not compile because libffi module uses wrong CFLAGS

[Dennis Clarke]

New submission from Dennis Clarke:

While compiling from sources I see in the process :

.
.
.
creating build/temp.solaris-2.10-sun4v.64bit-3.5/libffi
checking build system type... sparc-sun-solaris2.10
checking host system type... sparc-sun-solaris2.10
checking target system type... sparc-sun-solaris2.10
checking for gsed... /usr/local/bin/gsed
checking for a BSD-compatible install... 
/usr/local/build/python-3.5.1_SunOS5.10_sparcv9.002/Modules/_ctypes/libffi/install-sh
 -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... 
/usr/local/build/python-3.5.1_SunOS5.10_sparcv9.002/Modules/_ctypes/libffi/install-sh
 -c -d
checking for gawk... /usr/local/bin/gawk
checking whether /usr/local/bin/gmake sets $(MAKE)... yes
checking whether /usr/local/bin/gmake supports nested variables... yes
checking for gcc... /opt/solarisstudio12.4/bin/cc
checking whether the C compiler works... no
configure: error: in 
`/usr/local/build/python-3.5.1_SunOS5.10_sparcv9.002/build/temp.solaris-2.10-sun4v.64bit-3.5/libffi':
configure: error: C compiler cannot create executables
See `config.log' for more details


The reason for this error is that the build process down in the libffi 
directory seems to spawn its own "configure" stage wherein the CFLAGS are lost, 
forgotten or simply not used and therefore a trivial compile fails because of :

configure:3873: /opt/solarisstudio12.4/bin/cc  -I/usr/local/include 
-I/usr/local/ssl/include -D_TS_ERRNO -D_POSIX_PTHREAD_SEMANTICS 
-D_LARGEFILE64_SOURCE  conftest.c  >&5
ld: fatal: file /opt/solarisstudio12.4/lib/compilers/crti.o: wrong ELF class: 
ELFCLASS32
ld: fatal: file processing errors. No output written to a.out
configure:3877: $? = 2
configure:3915: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "libffi"
| #define PACKAGE_TARNAME "libffi"
| #define PACKAGE_VERSION "3.1"
| #define PACKAGE_STRING "libffi 3.1"
| #define PACKAGE_BUGREPORT "http://github.com/atgreen/libffi/issues;
| #define PACKAGE_URL ""
| #define PACKAGE "libffi"
| #define VERSION "3.1"
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:3920: error: in 
`/usr/local/build/python-3.5.1_SunOS5.10_sparcv9.002/build/temp.solaris-2.10-sun4v.64bit-3.5/libffi':
configure:3922: error: C compiler cannot create executables
See `config.log' for more details


The configure process MUST use the CFLAGS that were passed to the original 
configure stage and reside in the Makefile.  The absence of the flag -m64 
causes a 32-bit compile here and thus the compile fails.

--
components: Build
messages: 266458
nosy: blastwave
priority: normal
severity: normal
status: open
title: python 3.5.1 will not compile because libffi module uses wrong CFLAGS
type: compile error
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27132] New assert method that checks an error message for a list of strings

[Maciej Szulik]

Maciej Szulik added the comment:

You could, but then you end up writing nasty regex-es to do that. The idea here 
is to create a native function that will verify a number of words in whatever 
order.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26553] Write HTTP in uppercase

[Nathan Harold]

Changes by Nathan Harold :


Added file: http://bugs.python.org/file43022/3_fix_2.7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27132] New assert method that checks an error message for a list of strings

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You can use assertRaisesRegex() for this.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26553] Write HTTP in uppercase

[Nathan Harold]

Changes by Nathan Harold :


Added file: http://bugs.python.org/file43021/3_fix_3.5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26553] Write HTTP in uppercase

[Nathan Harold]

Nathan Harold added the comment:

The 2_fix* patches were unfortunately slightly broken because an unrelated 
revision to xmlrpc.client.rst changed one of the lines the patch files were 
using as an anchor (by adding a period to the end).

I've uploaded the 3_fix* series, which addresses this.  The actual changes in 
these patches are identical to those in their precursors: capitalization of 
HTTP, HTTPS, FTP, and URL in a few places.

--
nosy: +nharold
Added file: http://bugs.python.org/file43020/3_fix_3.6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27132] New assert method that checks an error message for a list of strings

[Maciej Szulik]

New submission from Maciej Szulik:

To quote David from 
http://bugs.python.org/review/25591/diff/16398/Lib/test/test_imaplib.py:

"I've been thinking I'd like a new assert method that checks an error message 
for a list of strings in any order, but I haven't opened an issue for it :)"

--
components: Tests
messages: 266454
nosy: maciej.szulik, r.david.murray
priority: normal
severity: normal
status: open
title: New assert method that checks an error message for a list of strings
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27127] Never have GET_ITER not followed by FOR_ITER

[Dino Viehland]

Dino Viehland added the comment:

I like how this makes the loop opcodes more regular - it should make things 
like Pyjion (http://www.github.com/Microsoft/Pyjion) which translate bytecode 
into native code.  We already have some code (currently commented out) checking 
that GET_ITER is followed by FOR_ITER to do some optimizations around loops.  

Another issue we ran into was that this leaves the iterable on the stack while 
the loop is running...  I'm just curious if your larger change alter that in 
anyway?

--
nosy: +dino.viehland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27012] Rename the posix module to _os

[ppperry]

Changes by ppperry :


--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27131] Unit test random shuffle

[Raymond Hettinger]

Raymond Hettinger added the comment:

Okay, this looks fine.  I'll apply it within a few days.

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Klamann]

Klamann added the comment:

> But you can only get that feature with Python3.5+.

Well, I have Python 3.5.1 installed and the problem still persists. I'm not 
sure that 25626 ist the same problem - in the comments they say this was not an 
issue in Python 3.4 or 2.x, but this is clearly the case here.

Another thing I've noticed: Contrary to my previous statement, 
zlib.decompress() doesn't work on archives that are larger than 4GB (I was 
mislead by the fact that my 1GB archive contained a 6GB file).

When I use gzip.compress() on more than 2^32 bytes, the same OverflowError 
occurs as with zlib.compress(). But when I use gzip.decompress(), I can extract 
archives that are larger than 4GB.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27114] SSLContext._load_windows_store_certs fails with PermissionError

[Steve Dower]

Changes by Steve Dower :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27114] SSLContext._load_windows_store_certs fails with PermissionError

[Steve Dower]

Steve Dower added the comment:

One slight change to the patch for 2.7 - has to catch OSError.

Just realised that I forgot to mention the reason we shouldn't just let the 
exception propagate out is that we then never load certificates specified by 
the SSL_CERT_FILE variable. Handling the exception allows a workaround.

If the exception occurs and no other certificates have been provided, it seems 
certain that a later operation is going to fail with a much more useful message 
(i.e. unverifiable connection, or whatever it says). There are also libraries 
(I think urllib is the one that was causing me actual trouble) that keep 
retrying the call when it fails, and all of those would need to be updated to 
handle this error.

The docs don't specify potential exceptions, so I see only good by not raising 
an exception here.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27114] SSLContext._load_windows_store_certs fails with PermissionError

[Roundup Robot]

Roundup Robot added the comment:

New changeset 29f163db229e by Steve Dower in branch '3.5':
Issue #27114: Fix SSLContext._load_windows_store_certs fails with 
PermissionError
https://hg.python.org/cpython/rev/29f163db229e

New changeset eaee5aed6fbc by Steve Dower in branch 'default':
Issue #27114: Fix SSLContext._load_windows_store_certs fails with 
PermissionError
https://hg.python.org/cpython/rev/eaee5aed6fbc

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27114] SSLContext._load_windows_store_certs fails with PermissionError

[Roundup Robot]

Roundup Robot added the comment:

New changeset 40f3f2b27112 by Steve Dower in branch '2.7':
Issue #27114: Fix SSLContext._load_windows_store_certs fails with 
PermissionError
https://hg.python.org/cpython/rev/40f3f2b27112

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch contains the change of Lib/asyncio/coroutines.py. This is the only 
change in Python code besides the dis module.

I can keep f_lasti to be twice the number of instructions, but this will 
complicate the patch. The simplest way perhaps is to convert this read-only 
attribute to the property that multiplies internal f_lasti by 2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Brett Cannon]

Brett Cannon added the comment:

So is avoiding changing f_lasti just to minimize breakage of tools? Aren't they 
going to have to update to support the wordcode changes anyway?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Brett Cannon]

Changes by Brett Cannon :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26526] In parsermodule.c, replace over 2KLOC of hand-crafted validation code, with a DFA

[Brett Cannon]

Changes by Brett Cannon :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27131] Unit test random shuffle

[jonathan.kross]

jonathan.kross added the comment:

Changed the test to assertCountEqual which will check that all the inputs are 
present and equal in the output.

--
Added file: http://bugs.python.org/file43019/test_random_shuffle.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1222585] C++ compilation support for distutils

[B. Clausius]

Changes by B. Clausius :


--
nosy: +barcc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27043] Describe what ‘inspect.cleandoc’ does to synopsis line.

[Nathan Harold]

Nathan Harold added the comment:

Here's my shot at a revision (corresponding patch attached):

   Clean up indentation from docstrings that are indented to line up with blocks
   of code.  All leading whitespace is removed from the first line.  Any leading
   whitespace that can be uniformly removed from the second line onwards is
   removed.  Empty lines at the beginning and end are subsequently removed.  
Also,
   all tabs are expanded to spaces.

This version also makes it explicit that nothing happens to trailing 
whitespace, should a line happen to have any.  This actually seems to be a 
difference between this function and the sample implementation in the PEP.

One odd side effect of that difference is that a line consisting only of 
whitespace may actually not be removed from the beginning or end of the string, 
if the preceding whitespace removals aren't sufficient to leave the line 
completely empty.  For instance:

>>> inspect.cleandoc('   A \n  B \n   \n  ')
'A \nB \n '

Two spaces are removed from each of the second, third, and fourth lines.  The 
fourth line is then empty and is removed; the third line still contains one 
space and is kept.

On the other hand, as defined in the PEP, the extra space on the third line is 
removed as trailing whitespace, so that the third line is then removed entirely:

>>> trim('   A \n  B \n   \n  ')
'A\nB'

Most likely this difference rarely affects anything.  In any case, mentioning 
it here because it influenced exactly how I revised the docs.

--
keywords: +patch
nosy: +nharold
Added file: http://bugs.python.org/file43018/cleandoc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21084] IDLE can't deal with characters above the range (U+0000-U+FFFF)

[Terry J. Reedy]

Terry J. Reedy added the comment:

Tk Text (and other widgets, but Text is the main issue) has two display 
problems: astral chars and long lines (over a thousand chars, say).  These 
problems can manifest in various places: file names, shell input (keyboard or 
clipboard), shell output, editor input (keyboard, clipboard, or file).  IDLE 
needs to take more control over what is displayed to work around both problems.

Tk Text also has a display feature: substring tagging.  I have been heistant to 
simple replace astral chars with their \U000h expansion because of the 
aliasing problem: in shell output, for instance, the user would not know if the 
program wrote 1 char or 10.  It would also be impossible to know if a reverse 
transformation might be needed.  Tagging astral expansions would solve both 
problems.

import re

astral = re.compile(r'([^\x00-\u])')
s = 'X\U0001Y\U0002\U0003Z'
for i, ss in enumerate(re.split(astral, s)):
if not i%2:
print(ss, end='')
else:
print(r'\\U%08x' % ord(ss), end='')
# prints
X\\U0001Y\\U0002\\U0003Z

Now replace print with test.insert, with an 'astral' tag for the second.  tk 
will not double '\'s.  Astral tag could switch, for instance, to underline 
version of current font.  This should work with any color scheme.

[Separate but related issue: augment Format or context menu with functions to 
convert between literal char, escape string, and name representation (using 
unicodedatabase).]

--
nosy: +terry.reedy
resolution: duplicate -> 
versions: +Python 3.6 -Python 2.7, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27131] Unit test random shuffle

[Raymond Hettinger]

New submission from Raymond Hettinger:

This tests that a specific shuffle result is obtained (which is an 
implementation specific detail subject to change).  An alternate possible test 
would just check to see that all the inputs are present in the output.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26526] In parsermodule.c, replace over 2KLOC of hand-crafted validation code, with a DFA

[Fred L. Drake, Jr.]

Fred L. Drake, Jr. added the comment:

I see your message to python-dev, and apologize for taking so long to get to 
this.

I do intend to read through your changes, and hope to be able to make time 
while I'm at PyCon this coming week.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5996] abstract class instantiable when subclassing dict

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27010] email library could "recover" from bad mime boundary like (some?) email clients do

[R. David Murray]

R. David Murray added the comment:

Thanks for the patch.  I'll take a look at this during the PyCon sprints.

--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27131] Unit test random shuffle

[jonathan.kross]

Changes by jonathan.kross :


--
components: Tests
files: test_random_shuffle.patch
keywords: patch
nosy: jonathan.kross
priority: normal
severity: normal
status: open
title: Unit test random shuffle
versions: Python 3.6
Added file: http://bugs.python.org/file43017/test_random_shuffle.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27126] Apple-supplied libsqlite3 on OS X is not fork safe; can cause crashes

[Evan Jones]

Evan Jones added the comment:

To be clear: My reproduction scripts crash both Python 2.7.10 and Python 3.5.1 
when you:

1. Download the source bundle from python.org.
2. Run ./configure; make
3. Use the built binary (because ./configure picks up the system version of 
libsqlite.dylib)

I did some more digging: The underlying root cause is Mac OS X's 
libdispatch.dylib. A ton of system APIs (like this proxy one, or GUI libraries, 
etc), use it. It seems the proxy settings API use it to manage inter-process 
communication. libdispatch has code that explicitly "poisons" the process if it 
forks. I think this is because it internally spawns threads, so the forked 
child state is unreliable, and they figure it is better for it to crash than to 
fail randomly. This is the classic "don't mix threads and fork" issue, its just 
that the threads are hidden inside a bunch of system APIs.

One fix for this particular bug would be for _scproxy to fork and use IPC to 
read the settings, which I think was mentioned in Issue13829. I think it would 
not also be crazy to ship the amalgamated sqlite3 with Python, to avoid an 
accidental dependency on sqlite3.

Finally: it might make sense to have 'forkserver' be the default mode for 
multiprocessing on Mac OS X, since there are other things that cause this same 
problem (Tkinter is reported on the internet).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27010] email library could "recover" from bad mime boundary like (some?) email clients do

[Andrea De Pasquale]

Andrea De Pasquale added the comment:

How about the following patch? If it's different from what you had in mind, 
please let me know.

--
keywords: +patch
Added file: http://bugs.python.org/file43016/issue27010-notuniqueboundary.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26816] Make concurrent.futures.Executor an abc

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5996] abstract class instantiable when subclassing dict

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Xiang Zhang]

Xiang Zhang added the comment:

This behaviour seems to have been fixed in issue25626. But you can only get 
that feature with Python3.5+.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Klamann]

New submission from Klamann:

zlib fails to compress files larger than 4gb due to some 32bit issues.

I've tested this in Python 3.4.3 and 3.5.1:

> python3 -c "import zlib; zlib.compress(b'a' * (2**32 - 1))"
> python3 -c "import zlib; zlib.compress(b'a' * (2**32))"
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: Size does not fit in an unsigned int

For Python 2.7, the issue starts at 2^31 byte (due to signed 32bit integers):

> python2 -c "import zlib; zlib.compress(b'a' * (2**31 - 1))"
> python2 -c "import zlib; zlib.compress(b'a' * (2**31))"
Traceback (most recent call last):
  File "", line 1, in 
OverflowError: size does not fit in an int

Decompressing files larger than 4GB works just fine.

--
components: Library (Lib)
messages: 266436
nosy: Klamann
priority: normal
severity: normal
status: open
title: zlib: OverflowError while trying to compress 2^32 bytes or more
versions: Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[STINNER Victor]

STINNER Victor added the comment:

> Changes f_lasti, tb_lasti etc to count code units instead of bytes.

I asked Demur to not break f_lasti. I don't understand if this change
breaks applications using f_lasti or not.

For example, asyncio/coroutines.py uses:

if caller.f_code.co_code[caller.f_lasti] != _YIELD_FROM:
value = value[0]

Does this code still work with your change?

Maybe this code is already broken by wordcode, but it doesn't really
matter since it should only be used on the exact version 3.4.0. The
code works around a bug in Python 3.4.0, fixed in Python 3.4.1 (issue
#21209).

Other known users of f_lasti are development tools like debuggers
(pdb), profilers, code coverage, etc. We should check these tools.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27097] ceval: Wordcode follow up, explicit unsigned short read

[STINNER Victor]

STINNER Victor added the comment:

Please focus this issue on documenting changes, and open new issues
for further optimizations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()

[STINNER Victor]

STINNER Victor added the comment:

Updated bench_fast-2.py result with Python compiled with PGO+LTO, with 
benchmark.py fixed to compute average + standard deviation. Only getattr() 
really seems slower:

--+---+--
Tests |  original |  
fastcall
--+---+--
filter| 75.8 us +- 0.1 us (*) | 78.1 us +- 
0.1 us
map   | 72.6 us +- 0.1 us (*) | 71.4 us +- 
0.0 us
sorted(list, key=lambda x: x) | 83.7 us +- 0.1 us (*) | 82.3 us +- 
0.3 us
sorted(list)  | 14.9 us +- 0.0 us (*) | 14.7 us +- 
0.0 us
b=MyBytes(); bytes(b) |199 ns +- 2 ns (*) |194 ns 
+- 1 ns
namedtuple.attr   |   830 ns +- 20 ns (*) | 1.09 us +- 0.01 us 
(+31%)
object.__setattr__(obj, "x", 1)   |133 ns +- 0 ns (*) |134 ns 
+- 1 ns
object.__getattribute__(obj, "x") |117 ns +- 0 ns (*) |115 ns 
+- 1 ns
getattr(1, "real")| 93.2 ns +- 0.9 ns (*) |  76.9 ns +- 0.7 ns 
(-17%)
bounded_pymethod(1, 2)| 73.4 ns +- 0.6 ns (*) | 70.7 ns +- 
0.4 ns
unbound_pymethod(obj, 1, 2)   | 74.5 ns +- 0.2 ns (*) | 71.8 ns +- 
0.6 ns
func()| 60.2 ns +- 0.4 ns (*) | 59.3 ns +- 
0.1 ns
func(1, 2, 3) | 74.6 ns +- 0.4 ns (*) | 72.2 ns +- 
0.3 ns
--+---+--
Total |249 us (*) |
248 us
--+---+--

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18383] test_warnings modifies warnings.filters when running with "-W default"

[STINNER Victor]

STINNER Victor added the comment:

Note: My issue #26742 has been marked as a duplicate of this one.

I confirm that "./python -Wd -m test -j0 test_warnings" doesn't complain "1 
test altered the execution environment: test_warnings" anymore. Thanks for the 
fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Issue27129.

--
components:  -Interpreter Core
dependencies: +Wordcode, part 2
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27129] Wordcode, part 2

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

This is the second stage of converting to wordcode (issue26647).

Proposed patch makes bytecodecode consisting of code units (16-bit words) 
instead of bytes. It includes following changes:

* Changes meaning of jump offsets. They counts not bytes, but code units. This 
extends the range addressed by short commands (from 256 bytes to 256 words) and 
simplifies ceval.c.

* Changes f_lasti, tb_lasti etc to count code units instead of bytes.

* Changes disassembler to show addresses in code units, not bytes.

* Refactores the code.

These changes break compatibility (already broken by switching to 16-bit 
bytecode). The first one breaks compatibility with compiled bytecode and needs 
incrementing the magic number.

--
components: Interpreter Core
files: wordcode2.patch
keywords: patch
messages: 266431
nosy: haypo, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Wordcode, part 2
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file43015/wordcode2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()

[STINNER Victor]

STINNER Victor added the comment:

default-May26-13-36-33.log: CPython benchmark suite run using stable config.

Faster (15):
- regex_effbot: 1.26x faster
- telco: 1.08x faster
- unpack_sequence: 1.07x faster
- mako_v2: 1.05x faster
- meteor_contest: 1.05x faster
- chaos: 1.04x faster
- nbody: 1.04x faster
- call_method_slots: 1.04x faster
- etree_iterparse: 1.04x faster
- etree_parse: 1.04x faster
- call_method: 1.03x faster
- raytrace: 1.03x faster
- nqueens: 1.03x faster
- call_method_unknown: 1.03x faster
- formatted_logging: 1.02x faster

Slower (8):
- etree_generate: 1.05x slower
- etree_process: 1.03x slower
- call_simple: 1.03x slower
- chameleon_v2: 1.02x slower
- pathlib: 1.02x slower
- float: 1.02x slower
- silent_logging: 1.02x slower
- json_load: 1.02x slower

--
Added file: http://bugs.python.org/file43014/default-May26-13-36-33.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27097] ceval: Wordcode follow up, explicit unsigned short read

[Demur Rumed]

Demur Rumed added the comment:

PREDICT could be optimized by having a HAS_ARG check on the constant op to 
decide whether we assign oparg = OPARG(word)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18383] test_warnings modifies warnings.filters when running with "-W default"

[Martin Panter]

Martin Panter added the comment:

I eliminated initial_len from my final version.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[STINNER Victor]

STINNER Victor added the comment:

Serhiy: please open a new issue for your change. While it's related, it's
different enough to deserve its own issue.

By the way , please don't include generated importlib .h file in your
patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file43013/wordcode.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: http://bugs.python.org/file43012/wordcode.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is large patch (not including generated Python/importlib.h and 
Python/importlib_external.h).

--
stage: needs patch -> patch review
Added file: http://bugs.python.org/file43012/wordcode.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()

[STINNER Victor]

STINNER Victor added the comment:

Quick & dirty microbenchmark: I ran bench_fast-2.py of the issue #26814. It 
looks like everything is slower :-p In fact, I already noticed this issue and I 
think that it is fixed with better compilation option: use "./configure 
--with-lto" and "make profile-opt". See my article:
https://haypo.github.io/journey-to-stable-benchmark-deadcode.html

--+-+---
Tests |original |   fastcall
--+-+---
filter| 76.2 us (*) |  116 us (+52%)
map   | 73.6 us (*) |  102 us (+38%)
sorted(list, key=lambda x: x) |   82 us (*) |  121 us (+48%)
sorted(list)  | 14.7 us (*) | 17.3 us (+18%)
b=MyBytes(); bytes(b) |  182 ns (*) |  243 ns (+33%)
namedtuple.attr   |  802 ns (*) | 1.44 us (+80%)
object.__setattr__(obj, "x", 1)   |  133 ns (*) |  166 ns (+25%)
object.__getattribute__(obj, "x") |  116 ns (*) |  142 ns (+22%)
getattr(1, "real")|   76 ns (*) |   95 ns (+25%)
bounded_pymethod(1, 2)|   72 ns (*) |  102 ns (+42%)
unbound_pymethod(obj, 1, 2)   |   71 ns (*) |   99 ns (+38%)
func()|   57 ns (*) |   81 ns (+41%)
func(1, 2, 3) |   72 ns (*) |  100 ns (+39%)
--+-+---
Total |  248 us (*) |  358 us (+44%)
--+-+---

At least, we have a starting point ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18383] test_warnings modifies warnings.filters when running with "-W default"

[Roundup Robot]

Roundup Robot added the comment:

New changeset f57f4e33ba5e by Martin Panter in branch '3.5':
Issue #18383: Avoid adding duplicate filters when warnings is reloaded
https://hg.python.org/cpython/rev/f57f4e33ba5e

New changeset 90bb91be6f3b by Martin Panter in branch 'default':
Issue #18383: Merge warnings fix from 3.5
https://hg.python.org/cpython/rev/90bb91be6f3b

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27128] Add _PyObject_FastCall()

[STINNER Victor]

New submission from STINNER Victor:

Since the issue #26814 proved that avoiding the creation of temporary tuples to 
call Python and C functions makes Python faster (between 2% and 29% depending 
on the benchmark), I extracted a first "minimal" patch to start merging this 
work.

The first patch adds new functions:

* PyObject_CallNoArg(func) and PyObject_CallArg1(func, arg): public functions
* _PyObject_FastCall(func, args, nargs, kwargs): private function

I hesitate between the C types "int" and "Py_ssize_t" for nargs. I read once 
that using "int" can cause performance issues on a loop using "i++" and 
"data[i]" because the compiler has to handle integer overflow of the int type.

The "int" type is also annoying on Windows 64-bit, it causes compiler warnings 
on downcast like PyTuple_GET_SIZE(co->co_argcount) stored into a C int.


_PyObject_FastCall() avoids the creation of tuple for:

* All Python functions (PyFunction_Check)
* C functions using METH_NOARGS or METH_O

The patch removes the "cache tuple" optimization from property_descr_get(), it 
uses PyObject_CallArg1() instead. It means that the optimization is (currently) 
missed in some cases compared to the current code, but the code is safer and 
simpler.


The patch adds Python/pystack.c which currently only contains 
_PyStack_AsTuple(), but will contain more code later.


I tried to write the smallest patch, but I started to use PyObject_CallNoArg() 
and PyObject_CallArg1() when the code already created a tuple at each call: 
PyObject_CallObject(), call_function_tail() and PyEval_CallObjectWithKeywords().


In the patch, keywords are not used in fast calls. But they will be used later. 
I prefer to start directly with keywords than changing the calling convention 
once again later.

--

Later, I will propose other patches to:

* add METH_FASTCALL calling convention for C functions
* modify Argument Clinic to use METH_FASTCALL

So the fast call will be taken in more cases.

--

The long term plan is to slowly use the new FASTCALL calling convention 
"everywhere". The tricky point are tp_new, tp_init and tp_call attributes of 
type objects. In the issue #26814, I wrote a patch adding Py_TPFLAGS_FASTNEW, 
Py_TPFLAGS_FASTINIT and Py_TPFLAGS_FASTCALL flags to use the FASTCALL calling 
convention for tp_new, tp_init and tp_call. The problem is that calling 
directly these methods looks common. If we can the calling convention of these 
methods, it will break the C API, I propose to discuss that later ;-)

An alternative is to add a tp_fastcall method to PyTypeObject and use a wrapper 
for tp_call for backward compatibility. This option has also drawbacks. Again, 
I propose to discuss this later, and first start to focus on the changes that 
don't break anything ;-)

--
files: fastcall.patch
keywords: patch
messages: 266422
nosy: haypo, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
title: Add _PyObject_FastCall()
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file43011/fastcall.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25782] CPython hangs on error __context__ set to the error itself

[Rotem Yaari]

Changes by Rotem Yaari :


--
nosy: +Rotem Yaari

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27106] configparser.__all__ is incomplete

[Martin Panter]

Martin Panter added the comment:

My personal opinion is to include all public APIs. Names that are omitted from 
__all__ may not come up in pydoc, and it is surprising when I use “import * ” 
in the interactive interpreter to play with a module and there is something 
missing.

To mitigate the risk of breaking code, I have been maintaining a list of the 
modules affected at 
, 
which warns that extra symbols will be imported in 3.6.

On the other hand, there are other cases where people wanted to exclude APIs 
from __all__; I pointed out two at 
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think we should make yet few related changes:

* Change meaning of jump offsets. They should count not bytes, but code units 
(16-bit words). This will extend the range addressed by short commands (from 
256 bytes to 256 words) and simplify ceval.c.
* Change f_lasti, tb_lasti etc to count code units instead of bytes.
* Change disassembler to show addresses in code units, not bytes.

These changes break compatibility (already broken by switching to 16-bit 
bytecode). The first one breaks compatibility with compiled bytecode and needs 
incrementing the magic number. That is why I think we should do this in this 
issue.

What is better, provide one large patch or separate simpler patches for every 
stage?

--
components: +Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27124] binascii.a2b_hex raises binascii.Error and ValueError, not TypeError

[Martin Panter]

Martin Panter added the comment:

The TypeError → binascii.Error changeover was made in revision eb45f85c4c79, to 
this function, along with various decoding functions in the base64 module. The 
base64 documenation was only slowly updated in a series of bug fixes, and it 
looks like unhexlify() is the only one left.

At the top of the binascii documentation, it says that the decoding functions 
accept ASCII strings. The general check for non-ASCII code points (raising 
ValueError) would be done before the specific checks for hexadecimal digits and 
having the right length.

Luiz’s patch looks okay to me.

--
nosy: +martin.panter
stage:  -> patch review
versions: +Python 3.5, Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27076] Doc and comment spelling fixes

[Martin Panter]

Martin Panter added the comment:

I pushed all the changes, including the MSI files, except for “heterogeneous”.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] ceval: use Wordcode, 16-bit bytecode

[STINNER Victor]

STINNER Victor added the comment:

Hi, I ran the CPython benchmark suite (my fork modified to be more stable) on 
ed4eec682199 (patched) vs 7a7f54fe0698 (base). The patched version contains 
wordcode (issue #26647) + 16-bit fetch for opcode and oparg (issue #27097).

The speedup is quite nice. Attached default-May26-03-05-10.log contains the 
full output.

Faster (27):
- unpack_sequence: 1.11x faster
- simple_logging: 1.11x faster
- silent_logging: 1.10x faster
- formatted_logging: 1.09x faster
- raytrace: 1.08x faster
- chaos: 1.08x faster
- etree_process: 1.08x faster
- call_simple: 1.07x faster
- mako_v2: 1.07x faster
- tornado_http: 1.07x faster
- nqueens: 1.07x faster
- regex_compile: 1.06x faster
- pathlib: 1.06x faster
- 2to3: 1.06x faster
- richards: 1.05x faster
- spectral_norm: 1.05x faster
- etree_generate: 1.05x faster
- chameleon_v2: 1.04x faster
- pickle_list: 1.03x faster
- pickle_dict: 1.03x faster
- regex_v8: 1.03x faster
- go: 1.03x faster
- call_method: 1.03x faster
- django_v3: 1.03x faster
- telco: 1.02x faster
- json_load: 1.02x faster
- call_method_unknown: 1.02x faster

Slower (1):
- fannkuch: 1.07x slower

Not significat (14):
- unpickle_list
- startup_nosite
- regex_effbot
- pidigits
- normal_startup
- nbody
- meteor_contest
- json_dump_v2
- float
- fastunpickle
- fastpickle
- etree_parse
- etree_iterparse
- call_method_slots

--
Added file: http://bugs.python.org/file43010/default-May26-03-05-10.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5486] doc copyedits

[Roundup Robot]

Roundup Robot added the comment:

New changeset 00f379f33a3e by Georg Brandl in branch '2.7':
#5486: typos.
https://hg.python.org/cpython/rev/00f379f33a3e

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9911] doc copyedits

[Roundup Robot]

Roundup Robot added the comment:

New changeset 90b49a29fe0c by Georg Brandl in branch '2.7':
#9911: doc copyedits.
https://hg.python.org/cpython/rev/90b49a29fe0c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27076] Doc and comment spelling fixes

[Roundup Robot]

Roundup Robot added the comment:

New changeset a3a64b85883f by Martin Panter in branch '3.5':
Issue #27076: Doc, comment and tests spelling fixes
https://hg.python.org/cpython/rev/a3a64b85883f

New changeset 5d254703bd84 by Martin Panter in branch 'default':
Issue #27076: Merge spelling from 3.5
https://hg.python.org/cpython/rev/5d254703bd84

New changeset d3d8fade by Martin Panter in branch 'default':
Issue #27076: More doc and comment spelling fixes for 3.6, by Ville Skyttä
https://hg.python.org/cpython/rev/d3d8fade

New changeset 1e80e53ce20d by Martin Panter in branch '2.7':
Issue #27076: Doc, comment and test function name spelling fixes
https://hg.python.org/cpython/rev/1e80e53ce20d

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21528] Fix a number of typos in the documentation

[Roundup Robot]

Roundup Robot added the comment:

New changeset 33f6e360826e by Donald Stufft in branch '2.7':
Fix Issue #21528 - Fix documentation typos
https://hg.python.org/cpython/rev/33f6e360826e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24225] Idlelib: changing file names

[Terry J. Reedy]

Terry J. Reedy added the comment:

The only external uses I know of currently are the imports of 
percolator/colorizer and textview in turtledemo.  There is currently a coloring 
glitch in 3.5 which continues in 3.6 after the renames.  See #27117.  I won't 
be surprised if making the colorizer easier to test (by adding an option to 
explicity pass a color scheme, for instance) makes it easier to use.

I will keep other uses in mind if informed about them.

My careful use tests also pinned down an existing and bizarre glitch in 
tkinter.simpledialog, reported in #27115.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com