[issue30671] dict: improve lookup function

[Tim Peters]

Tim Peters added the comment:

Dmitry, I suggest you spend more of the time you give to thinking to writing 
code instead ;-)  But, really, that's the easiest & surest way to discover for 
yourself where your analysis is going off in the weeds.

For example, issue 28201 was both simpler and worse than your thoughts about 
it:  x and y collide initially if and only if x = y mod 2**k.  If they do 
collide, then _regardless_ of the value of N it's necessarily the case that N*x 
+ x + 1 = N*y + y + 1 mod 2**k too.  That is, the second probes _always_ 
collided (not just half the time) in the case of a first-probe collision, and 
this would be so regardless of whether we were multiplying by 5, 4, 1, 0, or 
31459265358.  That was an unfortunate "off by 1" kind of mistake in the way the 
code was written.  It had nothing to do with, e.g., zero divisors.

After the obvious fix was applied, there's very little that can be said in 
reality.  Because, after the fix, higher-order bits of the hash codes - which 
had nothing at all to do with the initial x = y mod 2**k collision - are added 
in to the second probe values.  There's no good reason I can see to calculate 
what happens if those never-before-considered bits _happen_ to be all zeroes.  
They might be - but so what?  They might be any pair of values in the cross 
product of range(2**5) with itself.  There's nothing especially interesting 
about the (0, 0) pair.

That's why you're getting pushback:  your analysis hasn't made sense to me, and 
the things you're calculating still don't appear to have much of anything to do 
with how collisions are actually resolved.  To the contrary, so long as you're 
ignoring the higher-order hash code bits (about which we can infer _nothing_ 
from that the first probe collided), you're ignoring the _heart_ of the 
collision resolution strategy.

Some other things writing code would make immediately obvious:

- Hashes of strings have nothing to do with pointers or memory addresses.  They 
have solely to do with the characters the string contains, and all values in 
range(256) show up as the last byte of string hashes.

- While hashes of pointers do, the internal `_Py_HashPointer()` rotates 
addresses right by 4 bits so that the predictable trailing zero bits have no 
effect on the first probe.  For example,

>>> a = object()
>>> id(a)# the address
1583819629360
>>> hex(_)
'0x170c301a330'
>>> hash(a)  # the address is rotated so `0x3` is the last nibble
98988726835
>>> hex(_)
'0x170c301a33'

Because of all that, I'm going to close this.  But if you have an idea that 
actually speeds things, please post a patch and reopen it!  While it's true 
that I don't expect to see an actual improvement, clocks usually beat arguments 
;-)

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup functions

[INADA Naoki]

INADA Naoki added the comment:

pr-2237 only changes lookdict_index() function.
The function is used from only dict_popitem().  So it's not performance 
critical part.

And microbench doesn't show significant performance changes:

$ ./python.default -m perf timeit -q -l 2000 -s 'd=dict.fromkeys(str(i) for i 
in range(2000))' -- 'd.popitem()'
Mean +- std dev: 96.1 ns +- 1.3 ns

$ ./python.patched -m perf timeit -q -l 2000 -s 'd=dict.fromkeys(str(i) for i 
in range(2000))' -- 'd.popitem()'
Mean +- std dev: 96.4 ns +- 1.3 ns

--
title: dict: simplify lookup function -> dict: simplify lookup functions

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[INADA Naoki]

Changes by INADA Naoki :


--
pull_requests: +2323

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30700] fileinput inplace clobbers file without leaving backup on decode errors

[switchnode]

New submission from switchnode:

Consider the script:

$ cat nop.py
#!/usr/bin/env python
import fileinput
srt = fileinput.input(inplace=True)
print(srt.readline(), end='')
for line in srt:
print(line, end='')

Called on text files, it will do nothing.

$ ls -alh test.*
-rw-r--r-- 1 501 utmp 1.3G Jun 18 22:17 test.mp4
-rw-r--r-- 1 501 utmp  71K Jun 18  2017 test.srt
$ ./nop.py test.srt
$ ls -alh test.*
-rw-r--r-- 1 501 utmp 1.3G Jun 18 22:17 test.mp4
-rw-r--r-- 1 501 utmp  71K Jun 18  2017 test.srt

However, if the user accidentally supplies the filename of a video instead of 
the associated srt...

$ ./nop.py test.mp4
Traceback (most recent call last):
  File "./nop.py", line 4, in 
print(srt.readline(), end='')
  File "/usr/lib/python3.6/fileinput.py", line 299, in readline
line = self._readline()
  File "/usr/lib/python3.6/fileinput.py", line 364, in _readline
return self._readline()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 43: 
invalid start byte
$ ls -alh test.*
-rw-r--r-- 1 501 utmp0 Jun 18  2017 test.mp4
-rw-r--r-- 1 501 utmp  71K Jun 18  2017 test.srt
$ ls -alh * | grep 'bak'
$

Oops! It is gone.

I'm not sure why this happens. (Without the context-manager syntax, I would 
expect the program to end by excepting, fail to close the FileInput, and leave 
the backup file behind—certainly that would be the merciful option.)

--
messages: 296304
nosy: switchnode
priority: normal
severity: normal
status: open
title: fileinput inplace clobbers file without leaving backup on decode errors
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29887] test_normalization doesn't work

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +2321

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29887] test_normalization doesn't work

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +2322

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue28647 and issue30404.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30671] dict: improve lookup function

[Dmitry Rubanovich]

Dmitry Rubanovich added the comment:

A few notes on my motivation and other comments I made(not necessarily in the 
order of significance):

1. I started looking at this because of the fix in Issue28201.  It effectively 
improves on the difference between 

print_collision_counts(previous_lookdict_perturb)

and 

print_collision_counts(py3_6_1_lookdict_perturb)

because the before-28201 code was, in effect, multiplying by 6 instead of 5 
and, therefore, always had a 1/2 chance of secondary hashes colliding.

2. What I demonstrated was that the fix, as it stands, didn't do as much magic 
as we would hope.  Because it produced a scheme in which **first** 
**secondary** hashes would collide at a rate as high as 1/3.

3. I somewhat regret extending the script to demonstrate what happens to 2nd, 
3rd secondary hashes because it created the perception that I was trying to 
demonstrate a simulation.  I wasn't.  I was trying to demonstrate a 
calculation.  Well, calculations for each dictionary size up to 2**20.  

4. I detect apathy towards the fix, but I get the sense that it's mostly due to 
the fact that I initially didn't realize that I should have squarely limited 
the issue to the 1st secondary hash.  And after I made the initial error in the 
calculation (as pointed out by Inada Naoki) which would have produced an 
unstable calculation of the 2nd, 3rd, 4th, etc. secondary hashes (as pointed 
out by Tim), there is a lot of push back.  

But the fix I propose DOES improve on ***1st secondary hash*** because it 
eliminates any possibility of collision.  That is to say, no 2 1st secondary 
hashes will be the same for any 2 different primary hashes.  

And in the latest iteration of this proposal I did limit the scope of the 
enhancement to only 1st secondary hash.  As it happens, this collision actually 
does not occur for dictionaries of sizes 8,16,32.  They only begin to be 
possible for dictionaries of sizes 64 or greater.  The latest script 
demonstrates it.

5.  Sorry, Tim, but I should have reserved my statements to only the 
calculation I was certain about.  The further comments about what happens to 
later secondary hashes was a speculation on my part and this calculation was 
not revealing much about it.  If you want to consider what happens with later 
secondary indexes (as you do in hashsim.py), that's a different problem.  To 
reiterate, my intention was to further improve on the fix in 28201.


On the size of perturb:

6.  Since hashes of strings and other objects are pointers, and can safely be 
assumed to be pointers addressing the heap, they are aligned on the boundary 2 
* sizeof(void*).  That's 16 in 64-bit builds.  So the last 4 bits are 0.  And 
(depending on how much heap is used) the higher bits of the pointers are also 
similar in applications which don't use a lot of memory.  

For example, in an application in which 16MiB of heap is used, only bits 
35(=63-4-24) through 59(63-4) are truly random.  The rest of the bits can be 
assumed to not change, but they are not reached until 5th iteration of the 
loop, so this represents an already mildly-degenerate case in this use case (in 
which most dictionary keys are strings).

I haven't given much thought to which bits will have the most entropy in 
general, but that's going too far off topic.

--
type: enhancement -> performance
Added file: http://bugs.python.org/file46961/1st_secondary_collision_counts.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30604] co_extra_freefuncs is stored thread locally and can lead to crashes

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

I'm setting the stage to 'backport needed', but it really is a 'porting needed' 
stage :)

The two PRs merged PRs here were made against 3.6.

--
nosy: +Mariatta
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Eryk Sun]

Eryk Sun added the comment:

Yes, I think this issue should be closed. But for the record I'd like to note a 
not uncommon case in which listdir() raise FileNotFoundError on Windows.

According to MS-FSA [1], if a request to open a directory resolves to a file, 
the operation should fail with STATUS_NOT_A_DIRECTORY (see 2.1.5.1, phase 7). 
That's the scenario I was discussing in previous messages. However, I neglected 
to discuss what happens when an intermediate path component is not a directory. 
In theory this case should also fail with STATUS_NOT_A_DIRECTORY (see 2.1.5.1, 
phase 6). However, in practice MS file systems instead return 
STATUS_OBJECT_PATH_NOT_FOUND, which becomes ERROR_PATH_NOT_FOUND, for which 
Python raises FileNotFoundError.

Walking the path to find the reason for the failure shouldn't be attempted 
because it's subject to race conditions -- e.g. the file that caused the 
failure may no longer exist. 

[1]: https://msdn.microsoft.com/en-us/library/ff469536.aspx

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[STINNER Victor]

STINNER Victor added the comment:

Nice. Do you know my https://pypi.python.org/pypi/pyfailmalloc project? It
helped me to identify and fix dozens of code which didn't handle properly
memory allocation failures. Seach for "pyfailmalloc" in the bug tracker
(closed issues) ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29970] Severe open file leakage running asyncio SSL server

[Nikolay Kim]

Nikolay Kim added the comment:

I see. this is server specific problem. as a temp solution I'd use proxy for 
ssl termination.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29970] Severe open file leakage running asyncio SSL server

[kyuupichan]

kyuupichan added the comment:

@Nikolay Kim

As I note in the original submission, 480 was tested and does NOT solve this 
issue.  Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30699] Misleading class names in datetime.tzinfo usage examples

[Robert Tasarz]

New submission from Robert Tasarz:

https://docs.python.org/3/library/datetime.html#datetime-objects
gives in an example two classes named GMT1 and GMT2 subclassing tzinfo, defined 
with dst(…) methods returning one hour timedelta for summer periods. This is in 
conflict with naming, as GMT timezone is not defined with any daylight saving 
time, so any timezone offsetted from it shouldn't be as well. I would suggest 
renaming them to something like BrusselsTZ and HelsinkiTZ (with a comment that 
this is simplified implementation, that doesn't account for exact hour when dst 
switch is made, and maybe mentioning pytz) and tzname(…) to return ('CET', 
'CEST')[bool(self.dst(dt))] and ('EET', 'EEST')[bool(self.dst(dt))] accordingly.

Also https://docs.python.org/3/library/datetime.html#time-objects
has an example with GMT1 class, though this time dst(…) unconditionally returns 
offset 0 properly, tzname(…) gives 'Europe/Prague' – region that uses dst. IMHO 
returning any of: "CET", "GMT+1", "Africa/Tunis" would be better here.

--
assignee: docs@python
components: Documentation
messages: 296296
nosy: Robert.Tasarz, docs@python
priority: normal
severity: normal
status: open
title: Misleading class names in datetime.tzinfo usage examples
type: enhancement
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30698] asyncio sslproto do not shutdown ssl layer cleanly

[Grzegorz Grzywacz]

Changes by Grzegorz Grzywacz :


--
pull_requests: +2320

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24465] Make shutil.make_archive have deterministic sorting

[Martin Panter]

Changes by Martin Panter :


--
dependencies: +tarfile add uses random order
title: Make tarfile have deterministic sorting -> Make shutil.make_archive have 
deterministic sorting

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30698] asyncio sslproto do not shutdown ssl layer cleanly

[Grzegorz Grzywacz]

New submission from Grzegorz Grzywacz:

Asyncio on shutdown do not send shutdown confirmation to the other side. 
_SSLPipe after doing unwrap is calling shutdown callback where transportis 
closed and quit ssldata wont be sent.

--
components: asyncio
messages: 296295
nosy: grzgrzgrz3, yselivanov
priority: normal
severity: normal
status: open
title: asyncio sslproto do not shutdown ssl layer cleanly
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29970] Severe open file leakage running asyncio SSL server

[Nikolay Kim]

Nikolay Kim added the comment:

question is, should asyncio handle timeouts or leave it to caller?

https://github.com/python/cpython/pull/480 fixes leak during handshake.

--
nosy: +fafhrd91

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29406] asyncio SSL contexts leak sockets after calling close with certain Apache servers

[Nikolay Kim]

Changes by Nikolay Kim :


--
pull_requests: +2319

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30609] Python 3.6.1 fails to generate 256 colors on Cygwin based 64-bit Windows 10

[Richard S. Gordon]

Richard S. Gordon added the comment:

FYI: Here is an update to my subsequent bug report to the Cygwin project team. 
You might find my answers to their questions useful in the future.

> Begin forwarded message:
> 
> From: "Richard S. Gordon" >
> Subject: Re: Python 3.6.1 fails to generate 256 colors, with xterm-256color, 
> on Cygwin based 64-bit Windows 10.
> Date: June 12, 2017 at 3:21:22 AM EDT
> To: cyg...@cygwin.com 
> Cc: rigo...@comcast.net , 
> brian.ing...@systematicsw.ab.ca 
> Reply-To: "Richard S. Gordon"  >
> 
> Hello Cygwin & Brian Inglis,
> 
> I  have  not yet received the subject e-mail but did see a copy in the
> Cygwin  Archive.  I’ve  reproduced  it below to facilitate my reply. I
> used   a   different  mailer  to  generate  plain  text,  without  pdf
> attachment.
> 
> My HTML formatted email was rejected.
> 
> Re: Python 3.6.1 fails to generate 256 colors, with xterm-256color, on Cygwin 
> based 64-bit Windows 10.
> 
> From: Brian Inglis 
> To: cygwin at cygwin dot com
> Date: Sun, 11 Jun 2017 09:44:09 -0600
> Subject: Re: Python 3.6.1 fails to generate 256 colors, with xterm-256color, 
> on Cygwin based 64-bit Windows 10.
> Authentication-results: sourceware.org ; auth=none
> References: <86daff59-6ea8-4288-9d7d-e3262988b...@comcast.net 
> >
> Reply-to: Brian dot Inglis at SystematicSw dot ab dot ca
> *** IMPORTANT ***
> Each test application is a small part of a software repository which contains:
> 
> Development Sandboxes (not requiring installation and registration with a 
> specific Python 2x or Python3x release).
> Site Packages (requiring installation and registration with a specific Python 
> 2x or Python3x release).
> My toolkit uses the Python curses API to interrogate the built-in features. 
> It overrides the built-in curses color palette only if curses allows the 
> change.
> 
> In order to verify or troubleshoot my Python 3.6.1 failure, it is necessary 
> to clone a copy of my toolkit repository on to your computer from its GitHub 
> repository.
> The errors you got when you tried to run one of my failing test applications 
> are the result of trying to run it without its associated built-in toolkit 
> libraries.
> 
> You can place the repository copy into any convenient location on your 
> system. If you work within one of its Developer Sandboxes, instead of 
> installing
> any of its Site Packages, you will be able to delete the entire repository 
> rather than those components which were installed and registered with 
> individual
> Python 2x or Python 3x releases.
> 
> Each Developer Sandbox automatically finds and uses its associated libraries.
> On 2017-06-11 08:18, Richard S. Gordon wrote:
> See how to make decent Cygwin problem reports at the link below my sig.
> 
>> 3. Python 3.6.1 generates 256 colors (65536-color pairs), with 
>> xterm-256color, on Cygwin based 64-bit Windows 10. However, the 
>> generated colors appear to be corrupted by overloading text
>> attribute with specified foreground and background colors.
> Could you please give some examples of what you expect to see and why,
> and what you actually see?

> NOTES:
> On left is 32-bit Python 3.6.1 which supports only 16 colors (per limitation 
> of 32-bit processor)
> On right is 64-bit Python 3.6.1 which supports 140 colors (per emulation of 
> 68 WxPython + 72 extra); besides wrong colors, notice the spurious underlines.
> Sample 32-bit and 64-bit Python 3.6.1.pdf
> 
> Which Windows console are you running the test in: mintty, cmd, …?
> Cygwin’s MINTTY, typically configured for 80 columns x 43-50 rows.
> 
> What are the results when you run it in another console?
> None available
> 
> Are you running a Windows Insider or some developer build?
> No
> That recently had a keyboard problem that was fixed a few days later.
> 
>> 6. Cygwin Problem Reporter's Test Case: This Cygwin problem can be 
>> demonstrated by running the Problem Reporter's 
>> test_tsWxColorPalette.py in Python 3x (developer-sandbox) which can 
>> be found in https://github.com/rigordo959/tsWxGTUI_PyVx_Repository 
>> 
> Could you please provide a direct link to a Simple Test Case program,
> again with examples of what you expect to see and what you actually see?
> I had to dig to find where you hid your test program.
> See Important Notes at beginning of my reply.
> 
> If the program requires certain Cygwin python modules installed to run,
> please state which Cygwin packages need to be installed.
> The console, ncursesw, pty, or tty person trying to reproduce and
> diagnose your problem may not be a python guy.
> My Toolkit and its Test Applications use only Python 3.6.1 and its associated 
> standard 

[issue27321] Email parser creates a message object that can't be flattened

[Johannes Löthberg]

Johannes Löthberg added the comment:

Ping?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22028] Python 3.4.1 Installer ended prematurely (Windows msi)

[Tim Golden]

Tim Golden added the comment:

[Housekeeping] Closing this as fixed. The mimetypes fix is in; the 3.4 
installer is well out of support. If any other issues arise on current 
installers they should be raised as new issues.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8631] subprocess.Popen.communicate(...) hangs on Windows

[Tim Golden]

Tim Golden added the comment:

I can't reproduce either on 2.7 or on 3.5 with any of the examples shown. 
Closing again as not-a-bug.

--
resolution:  -> not a bug
status: open -> closed
versions:  -Python 2.7, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29739] zipfile raises wrong exception for some incorrect passwords

[Jack Cushman]

Jack Cushman added the comment:

Agreed that no one should be using zip encryption. :) I hit this issue working 
on an academic exercise. I'm fine with closing this. 

(I do think the exception types in zipfile set up a trap for the programmer 
here: Python throws a different exception type 1/256 of the time for the same 
error case, which is undocumented and hard to discover by trial and error. It's 
pretty unlikely that anyone would write a correct `try-except` block for zip 
decryption on their first few attempts, and someone who needs to catch bad 
passwords but not bad zip files will most likely ship broken code. But just 
catching RuntimeError and BadZipfile gets you pretty close, so I don't think 
it's a big deal.)

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30671] dict: improve lookup function

[Tim Peters]

Tim Peters added the comment:

I don't see a reason to keep this open, but I haven't been able to follow the 
OP's line of argument.  My best _guess_ is that they're chasing illusions based 
on not understanding (or grossly undervaluing) that the primary point of the 
perturb logic is to incrementally fold in hash code bits that _didn't_ have any 
effect at all on the initial table index.  It's true that for a hash table with 
2**p slots, the current scheme could be improved in several ways _if_ hash 
codes only had p bits.  To judge from the Python code they posted, they either 
(erroneously) assumed that was the case, or offhandedly dismissed the potential 
value of folding in hash code bits beyond the p'th most significant.

But, since I've been unable to follow the argument, I could be wrong about all 
that.  So if they don't clarify within a few days, I'll close this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29430] zipfile: invalid link

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The link already is fixed.

--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30360] getpass.getpass() does not accept stdin from an Expect script

[Terry J. Reedy]

Terry J. Reedy added the comment:

As suggested, and with no further response, closing.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29366] os.listdir has inconsistent behavior when run on a non-directory

[Terry J. Reedy]

Terry J. Reedy added the comment:

Eryk, does your last comment suggest that we close this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25918] AssertionError in lib2to3 on 2.7.11 Windows

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> not a bug
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25303] Add option to py_compile to compile for syntax checking without writing bytecode

[Terry J. Reedy]

Terry J. Reedy added the comment:

Closing, as suggested above.

--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20663] Introduce exception argument to iter

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> rejected
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19956] inspect.getsource(obj.foo) fails when foo is an injected method constructed from another method

[Terry J. Reedy]

Terry J. Reedy added the comment:

Any review of the revised patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18967] Find a less conflict prone approach to Misc/NEWS

[Terry J. Reedy]

Terry J. Reedy added the comment:

Has this been superseded by core_workflow issues?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17773] test_pydoc fails with the installed testsuite (2.7)

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12420] distutils tests fail if PATH is not defined

[Terry J. Reedy]

Terry J. Reedy added the comment:

test_distutils now passes on 3.6, so this is out-of-date for 3.x.

On installed 2.7.13 (released Dec 2016) 
ERROR: test_get_outputs (distutils.tests.test_build_ext.BuildExtTestCase)
FAIL: test_customize_compiler_before_get_config_vars 
(distutils.tests.test_sysconfig.SysconfigTestCase)
the error is ValueError: [u'path'] instead of ValueError: ['path'].

Nick suggested leaving 2.7 alone, I don't care about it either, and I believe 
Victor Stinner has worked on its tests since last December.

--
assignee: merwok -> 
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6171] IDLE - TreeWidget draw and double-click (Ubuntu)

[Terry J. Reedy]

Terry J. Reedy added the comment:

(In 3.6, TreeWidget.py became tree.py.)
Even though IDLE trees now appear to work correctly, including on Ubuntu, I am 
concerned about possible memory leaks, which are usually detected by repeatedly 
running a module's test file.  Test_tree does not leak now, but it does not do 
much. In particular, it does not expand and contract items and move selections. 
 TreeNode.draw has this note: "This leaks bindings until canvas is deleted", 
and I wonder if the same could be true in .drawtext.

Tree.py currently has these notes at the top:
# - keep track of object ids to allow more careful cleaning
# - optimize tree redraw after expand of subnode
Currently, self.text_id is saved, but not used.  Reusing canvas items should be 
more efficient.

I won't edit tree.py immediately, as I intend to try using ttk.Treeview as a 
replacement.  But we might instead update tree.py with ttk components.  In the 
meanwhile, I will leave this open.

--
assignee:  -> terry.reedy
versions: +Python 3.6, Python 3.7 -Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30693] tarfile add uses random order

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch for similar issue with the glob module was rejected recently since it 
is easy to sort the result of glob.glob() (see issue30461). This issue looks 
similar, but there are differences. On one side, the command line tar utility 
doesn't have the option for sorting file names and seems don't sort them by 
default (I didn't checked). It is possible to use external sorting with the 
tarfile module as with the tar utility (generate the list of all files and 
directories, sort it, and pass every item to TarFile.add with the option 
recursive=False). But on other side, this is not so easy as for glob.glob(). 
And the overhead of the sorting is expected to be smaller than for glob.glob(). 
This may be considered as additional arguments for approving the patch.

If this approach will be approved, it should be applied also to the ZIP 
archives.

FYI the order of archived files can affect the compression ratio of the 
compressed tar archive. For example the 7-Zip archiver sorts files by 
extensions, this increases the chance that files of the same type (text, 
multimedia, spreadsheet, executables, etc) are grouped together and use the 
common dictionary for global compression. This isn't directly related to this 
issue, just a material for possible future enhancement.

--
nosy: +lars.gustaebel, rhettinger, serhiy.storchaka
stage:  -> patch review
versions:  -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30645] imp.py: load_package() appends to its own loop variable

[Alexandru Ardelean]

Alexandru Ardelean added the comment:

Admittedly, that PR refers to `imp.load_module()`
But, `imp.load_modules()` also calls `imp.load_package()`

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30645] imp.py: load_package() appends to its own loop variable

[Alexandru Ardelean]

Alexandru Ardelean added the comment:

FWIW, I took a look over the virtualenv repo.
There's an issue raised there
https://github.com/pypa/virtualenv/issues/955

And a PR to address it 
https://github.com/pypa/virtualenv/pull/947

Though, it seems like they still want to keep `imp.load_package()` for a while, 
in order to keep compatibility with Python 2.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29616] input() after restarting causes bug

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> works for me
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30645] imp.py: load_package() appends to its own loop variable

[Alexandru Ardelean]

Changes by Alexandru Ardelean :


--
pull_requests: +2318

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30657] Unsafe arithmetic in PyString_DecodeEscape

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30657] Unsafe arithmetic in PyString_DecodeEscape

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset c3c9db89273fabc62ea1b48389d9a3000c1c03ae by Serhiy Storchaka (Jay 
Bosamiya) in branch '2.7':
[2.7] bpo-30657: Check & prevent integer overflow in PyString_DecodeEscape 
(#2174)
https://github.com/python/cpython/commit/c3c9db89273fabc62ea1b48389d9a3000c1c03ae


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think the allocators must just return NULL, without setting the error.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29755] python3 gettext.lgettext sometimes returns bytes, not string

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I agree with everything @serhiy.storchaka said, including the questionable 
utility of the l* methods in Python 3. ;)

Thanks also for updating the documentation.  Reading the existing docs over 
now, it's shocking how imprecise "the translation is returned in the preferred 
system encoding" is.

I have some suggestion about the PR, so I'll comment over there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30697] segfault in PyErr_NormalizeException() after memory exhaustion

[Xavier de Gaye]

Xavier de Gaye added the comment:

Problem b) is IMO a clear demonstration that using tstate->recursion_depth and 
the PyExc_RecursionErrorInst singleton is not the correct way to control the 
recursive calls to PyErr_NormalizeException() since the problem here is memory 
exhaustion, not recursion. One could instead abort with a Fatal Error message 
printing the type of the last exception, when the depth of the recursivity of 
PyErr_NormalizeException() exceeds let's say 128 (knowing that anyway the stack 
is protected in the functions that attempt to instantiate those exceptions). 
The normalization of an exception that fails with an exception whose 
normalization fails with an ... and this, 128 times in a row, surely this can 
be considered as a fatal error, no ?

PR 2035 eliminates the tail recursive call in PyErr_NormalizeException() and 
transforms it into a loop. This loop obviously does not involve the stack 
anymore. This is another argument that shows  that tstate->recursion_depth and 
the PyExc_RecursionErrorInst singleton which are related to the stack should 
not be used to control the recursivity of PyErr_NormalizeException() or the 
iterations of this loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29657] os.symlink: FileExistsError shows wrong message

[Larry Hastings]

Larry Hastings added the comment:

"Special cases aren't special enough to break the rules."  I want the error 
message to mirror the API, which it currently does.  If we swapped them, the 
error message would now contradict the API.  So no, I don't support swapping 
"src" and "dst" in the error message only when the error pertains to os.symlink.

If literally every time the two-filename version of OSError is used inside 
Python, the two filenames are "src" and "dst", then we could consider making it 
slightly more explicit, e.g.

FileExistsError: [Errno 17] File exists: src='a', dst='a_link'

I think I'd want the source code to reflect this (e.g. thinking about "src" and 
"dst" rather than "filename" and "filename2").

Would OP et al consider this change to the error message an improvement, or is 
it not interesting?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30697] segfault in PyErr_NormalizeException() after memory exhaustion

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30696] infinite loop in PyRun_InteractiveLoopFlags()

[Xavier de Gaye]

Changes by Xavier de Gaye :


--
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30697] segfault in PyErr_NormalizeException() after memory exhaustion

[Xavier de Gaye]

New submission from Xavier de Gaye:

Nosying reviewers of PR 1981 of issue 22898.

The memerr.py script segfaults with the following gdb backtrace:

#0  0x00550268 in PyErr_NormalizeException 
(exc=exc@entry=0x7fffdee8, 
val=val@entry=0x7fffdef0, tb=tb@entry=0x7fffdef8) at 
Python/errors.c:315
#1  0x0055045f in PyErr_NormalizeException 
(exc=exc@entry=0x7fffdee8, 
val=val@entry=0x7fffdef0, tb=tb@entry=0x7fffdef8) at 
Python/errors.c:319
#2  0x0055045f in PyErr_NormalizeException 
(exc=exc@entry=0x7fffdee8, 
val=val@entry=0x7fffdef0, tb=tb@entry=0x7fffdef8) at 
Python/errors.c:319
#3  0x0055045f in PyErr_NormalizeException 
(exc=exc@entry=0x7fffdee8, 
val=val@entry=0x7fffdef0, tb=tb@entry=0x7fffdef8) at 
Python/errors.c:319
...

To be able to run this patch, one needs to apply the nomemory_allocator.patch 
from issue 30695 and the infinite_loop.patch from issue 30696.

This raises two different problems:

a) The segfault itself that occurs upon setting the PyExc_RecursionErrorInst 
singleton. Oh! That had already been pointed out in msg231933 in issue 22898 at 
case 4).

b) When the size of the Python frames stack is greater than the size of the 
list of preallocated MemoryError instances, this list becomes exhausted and 
PyErr_NormalizeException() enters an infinite recursion which is stopped:
* by the PyExc_RecursionErrorInst singleton when a) is fixed
* by a Fatal Error (abort) when applying PR 1981 in its current state 
(there is no stack overflow as expected even in the absence of any guard before 
the recursive call to PyErr_NormalizeException())
The user is presented in both cases with an error hinting at a recursion 
problem instead of a problem with memory exhaustion. This is bug b).

--
components: Interpreter Core
files: memerr.py
messages: 296272
nosy: brett.cannon, haypo, pitrou, serhiy.storchaka, xdegaye
priority: normal
severity: normal
status: open
title: segfault in PyErr_NormalizeException() after memory exhaustion
type: crash
versions: Python 3.7
Added file: http://bugs.python.org/file46960/memerr.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20823] Clarify copyreg.pickle() documentation

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you update your patch Peter, and since CPython development is moved to 
GitHub, create a pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12680] cPickle.loads is not thread safe due to non-thread-safe imports

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> not a bug
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29616] input() after restarting causes bug

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29657] os.symlink: FileExistsError shows wrong message

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See issue20517 for the discussion about current implementation. Are there any 
ideas about clearer error messages?

Added Larry as the author of the original idea and implementation.

--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16055] incorrect error text for int(base=1000, x='1')

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +easy (C)
stage: test needed -> needs patch
type: behavior -> enhancement
versions: +Python 3.7 -Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29739] zipfile raises wrong exception for some incorrect passwords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Actually this feature helps to do the brute-force attack. 255 of 256 passwords 
can be rejected by testing only the header. Old ZIP files encryption is very 
weak, it isn't used in serious applications.

--
nosy: +r.david.murray
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29755] python3 gettext.lgettext sometimes returns bytes, not string

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In Python 2 both gettext() and lgettext() are purposed to return 8-bit strings. 
The difference between them is only that gettext() encodes the translation back 
to the encoding of the translation file if the output encoding is not 
explicitly specified, while lgettext() encodes it to the preferred locale 
encoding. ugettext() returns Unicode strings.

In Python 3 ugettext() is renamed to gettext() and always returns Unicode 
strings. lgettext() should return a byte string, as in Python 2. The problem is 
that if the translation is not found, the untranslated message usually is 
returned, which is a Unicode string in Python 3. It should be encoded to a byte 
string, so that lgettext() always returns the same type -- bytes.

PR 2266 fixes lgettext() and related functions, updates the documentation, and 
adds tests.

Frankly, the usefulness of lgettext() in Python 3 looks questionable to me. 
gettext() can be used instead, with explicit encoding the result to the desired 
charset.

--
nosy: +barry
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29755] python3 gettext.lgettext sometimes returns bytes, not string

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +2317

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30696] infinite loop in PyRun_InteractiveLoopFlags()

[Xavier de Gaye]

New submission from Xavier de Gaye:

To reproduce the problem, apply the nomemory_allocator.patch from issue 30695 
and run the following two statements that must be interrupted with ^C:
$  ./python -q
>>> import _testcapi
>>> _testcapi.set_nomemory_allocator()
sys.excepthook is missing
^Cpython: Objects/call.c:785: PyEval_CallObjectWithKeywords: Assertion 
`!PyErr_Occurred()' failed.
Aborted (core dumped)

The attached patch fixes this problem.
No PR for the moment: this patch and nomemory_allocator.patch are needed to 
demonstrate other issues.

--
components: Interpreter Core
files: infinite_loop.patch
keywords: patch
messages: 296267
nosy: haypo, xdegaye
priority: normal
severity: normal
status: open
title: infinite loop in PyRun_InteractiveLoopFlags()
type: behavior
versions: Python 3.7
Added file: http://bugs.python.org/file46959/infinite_loop.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30695] add a nomemory_allocator to the _testcapi module

[Xavier de Gaye]

New submission from Xavier de Gaye:

Add the set_nomemory_allocator() function to _testcapi that sets a no memory 
allocator.

--
components: Tests
files: nomemory_allocator.patch
keywords: patch
messages: 296266
nosy: haypo, xdegaye
priority: normal
severity: normal
status: open
title: add a nomemory_allocator to the _testcapi module
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file46958/nomemory_allocator.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30671] dict: improve lookup function

[Raymond Hettinger]

Raymond Hettinger added the comment:

Can this be closed now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30691] WeakSet is not pickleable

[Raymond Hettinger]

Raymond Hettinger added the comment:

> If you pickle also hard refereces to the items,
>  a WeakSet can be pickled and unpickled as expected (after 
> fixing __reduce__).

That seems contrary to the original intent of a WeakSet.  I recommend not going 
down the path of adding pickle support.

If copying is also broken, then it is reasonable to either fix it or remove it 
depending on whether a fix is straight-forward.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29702] Error 0x80070003: Failed to launch elevated child process

[Eryk Sun]

Changes by Eryk Sun :


--
resolution: out of date -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30664] Change unittest's _SubTest to not sort its params when printing test failures

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Note that the order of parameters of nested subtests is from inner to outer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com