[issue30213] ZipFile from 'a'ppend-mode file generates invalid zip

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This may be related to issue18876 or issue20082.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30213] ZipFile from 'a'ppend-mode file generates invalid zip

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The problem is that seek() doesn't work properly with files opened in append 
mode.

with open('file', 'ab') as f:
f.write(b'abcd')
f.seek(0)
f.write(b'efgh')
f.flush()

with open('file', 'rb') as f:
print(f.read())

The result is b'abcdefgh' instead of expected b'efgh'.

--
resolution: duplicate -> 
stage: resolved -> 
status: closed -> open
superseder: Regression in zipfile writing in 2.7.13 -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32207] IDLE: run's tk update adds context traceback on callback error

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

try:
seq, request = rpc.request_queue.get(block=True, timeout=0.05)
except queue.Empty:
request = None
if request is None:
handle_tk_events()
continue

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32208] Improve semaphore documentation

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
title: Improve semaphore docmentation -> Improve semaphore documentation

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32140] IDLE debugger fails with non-trivial __new__ super call

[ppperry]

ppperry  added the comment:

Simplified reproducer for same bug without any imports:

class BadRepr:
def __repr__(self):
1/0
def broken():
x=BadRepr()
x=x #filler line for debugger

In this case, the problematic BadRepr object in the "broken" function explictly 
has a broken __repr__, but the uninitialized Fraction object in the "__new__" 
function of the Fraction class has a similarly broken __repr__

--
nosy: +ppperry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32209] Crash in set_traverse Within the Garbage Collector's collect_generations()

[Connor W Fitzgerald]

New submission from Connor W Fitzgerald :

This is my first time submitting a bug report, so forgive me if I miss on 
important information. I am writing code that makes heavy use of sets (8GB+ of 
them). It segfaults semi-randomly during the processing of them. I've used 
versions 3.5.3, 3.6.3, and 3.7.0rc2. I debugged the process and came across the 
error in object/setobject.c:742. I don't really know what information to 
provide here but this is the best I can do.

Error:
Unhandled exception thrown: read access violation.
entry->**key** was 0x8000.

Stack Trace:
python37.dll!set_traverse(PySetObject * so, int(*)(_object *, void *) visit, 
void * arg) Line 742
python37.dll!subtract_refs(_gc_head * containers) Line 295
python37.dll!collect(int generation, __int64 * n_collected, __int64 * 
n_uncollectable, int nofail) Line 862
python37.dll!collect_with_callback(int generation) Line 1027
python37.dll!collect_generations() Line 1049
python37.dll!PyType_GenericAlloc(_typeobject * type, __int64 nitems) Line 969
[Inline Frame] python37.dll!make_new_set(_typeobject *) Line 1055
python37.dll!set_new(_typeobject * type, _object * args, _object * kwds) Line 
1127
python37.dll!type_call(_typeobject * type, _object * args, _object * kwds) Line 
928
python37.dll!_PyObject_FastCallKeywords(_object * callable, _object * * stack, 
__int64 nargs, _object * kwnames) Line 199
python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * 
kwnames) Line 4705
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3182
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object 
* locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * 
kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, 
_object * kwdefs, _object * closure, _object * name, _object * qualname) Line 
4011
python37.dll!_PyFunction_FastCallDict(_object * func, _object * * args, __int64 
nargs, _object * kwargs) Line 376
[Inline Frame] python37.dll!PyEval_CallObjectWithKeywords(_object *) Line 819
python37.dll!defdict_missing(defdictobject * dd, _object * key) Line 1993
python37.dll!_PyMethodDef_RawFastCallDict(PyMethodDef * method, _object * self, 
_object * * args, __int64 nargs, _object * kwargs) Line 496
[Inline Frame] python37.dll!_PyCFunction_FastCallDict(_object *) Line 580
python37.dll!_PyObject_FastCallDict(_object * callable, _object * * args, 
__int64 nargs, _object * kwargs) Line 101
[Inline Frame] python37.dll!object_vacall(_object * callable, char *) Line 1194
python37.dll!PyObject_CallFunctionObjArgs(_object * callable, ...) Line 1259
python37.dll!dict_subscript(PyDictObject * mp, _object * key) Line 1984
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 1316
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object 
* locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * 
kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, 
_object * kwdefs, _object * closure, _object * name, _object * qualname) Line 
4011
[Inline Frame] python37.dll!_PyFunction_FastCallKeywords(_object * stack, 
_object * *) Line 433
python37.dll!call_function(_object * * * pp_stack, __int64 oparg, _object * 
kwnames) Line 4703
python37.dll!_PyEval_EvalFrameDefault(_frame * f, int throwflag) Line 3182
[Inline Frame] python37.dll!PyEval_EvalFrameEx(_frame *) Line 549
python37.dll!_PyEval_EvalCodeWithName(_object * _co, _object * globals, _object 
* locals, _object * * args, __int64 argcount, _object * * kwnames, _object * * 
kwargs, __int64 kwcount, int kwstep, _object * * defs, __int64 defcount, 
_object * kwdefs, _object * closure, _object * name, _object * qualname) Line 
4011
python37.dll!PyEval_EvalCodeEx(_object * _co, _object * globals, _object * 
locals, _object * * args, int argcount, _object * * kws, int kwcount, _object * 
* defs, int defcount, _object * kwdefs, _object * closure) Line 4045
python37.dll!PyEval_EvalCode(_object * co, _object * globals, _object * locals) 
Line 532
python37.dll!run_mod(_mod * mod, _object * filename, _object * globals, _object 
* locals, PyCompilerFlags * flags, _arena * arena) Line 987
python37.dll!PyRun_FileExFlags(_iobuf * fp, const char * filename_str, int 
start, _object * globals, _object * locals, int closeit, PyCompilerFlags * 
flags) Line 939
python37.dll!PyRun_SimpleFileExFlags(_iobuf * fp, const char * filename, int 
closeit, PyCompilerFlags * flags) Line 402
python37.dll!PyRun_AnyFileExFlags(_iobuf * fp, const char * filename, int 
closeit, PyCompilerFlags * flags) Line 84
python37.dll!run_file(_iobuf * fp, const wchar_t * filename, PyCompilerFlags * 
p_cf) Line 340
python37.dll!Py_Main(int argc, wchar_t * * argv) Line 894
[Inline Frame] python.exe!invoke_main() Line 79

[issue32208] Improve semaphore docmentation

[Garrett Berg]

Garrett Berg  added the comment:

It may be desirable to change:

> Once awoken, decrement the counter by 1.

to

> Once awoke, decrement the counter (which is guaranteed to be positive) by 1.

Although this could be considered obvious since the counter can never go below 
zero.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32208] Improve semaphore docmentation

[Garrett Berg]

Garrett Berg  added the comment:

Also, is "The order in which threads are awoken should not be relied on" 
actually true? Would it not be up to the OS specific scheduler? For instance, 
wouldn't the POSIX scheduler use a priority queue of some kind?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32208] Improve semaphore docmentation

[Garrett Berg]

New submission from Garrett Berg :

The definition of threading.Semaphore is confusing (for all versions of python 
docs)

https://docs.python.org/2/library/threading.html#semaphore-objects

 acquire([blocking])

It currently states the following:

> When invoked without arguments: if the internal counter is larger than zero 
> on entry, decrement it by one and return immediately. If it is zero on entry, 
> block, waiting until some other thread has called release() to make it larger 
> than zero. This is done with proper interlocking so that if multiple 
> acquire() calls are blocked, release() will wake exactly one of them up. The 
> implementation may pick one at random, so the order in which blocked threads 
> are awakened should not be relied on. There is no return value in this case.

However, after testing it myself I found that is missing a crutial detail. 
Let's step through the docs:

> If the internal counter is larger than zero on entry, decrement it by one and 
> return immediately.

This is exactly accurate and should stay the same

> If it is zero on entry, block, waiting until some other thread has called 
> release() to make it larger than zero. This is done with proper interlocking 
> so that if multiple acquire() calls are blocked, release() will wake exactly 
> one of them up. The implementation may pick one at random, so the order in 
> which blocked threads are awakened should not be relied on. There is no 
> return value in this case.

This is extremely confusing and I would like to rewrite it as follows:

> If it is zero on entry block until awoken by a call to ``release()``. Once 
> awoken, decrement the counter by 1. Exactly one thread will be awoken by a 
> call to ``release()``. The order in which threads are awoken should not be 
> relied on. ``None`` is returned in this case.

The major thing that was missing was that the **counter is decremented after 
the thread is awoken**. For instance, this code *generally* passes assertions:

```
#!/usr/bin/python2
import time
from threading import Thread, Semaphore

s = Semaphore(1)

def doit():
s.acquire()
print("did it")

th1 = Thread(target=doit)
th1.start()

th2 = Thread(target=doit)
th2.start()

time.sleep(0.2)

assert not th1.is_alive()
assert th2.is_alive()

s.release()
assert s._value == 1, "The value is increased to 1 MOMENTARILY"
start = time.time()
while sum([th2.is_alive(), th3.is_alive()]) > 1:
assert time.time() - start < 0.5
time.sleep(0.1)

assert s._value == 0, "when an aquire is awoken, THEN _value is decremented"
```

Obviously this behavior should not be relied on, but it gives a picture of what 
seems to be happening under the hood.

I came across this while trying to work through "The Little Book of 
Semaphores". After reading these docs I mistakingly thought that they didn't 
match Djestra's original semaphore since the values could not be negative. I 
now realize that while they may not match that implementation under the hood, 
they match it perfectly in practice since if you (for instance) ``acquire()`` 5 
times and then ``release()`` 5 times the value of Semaphore._value will be the 
same when all is said and done.

--
components: Library (Lib)
messages: 307536
nosy: Garrett Berg
priority: normal
severity: normal
status: open
title: Improve semaphore docmentation
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30862] parent logger should also check the level

[TaoQingyun]

TaoQingyun <845767...@qq.com> added the comment:

Maybe I misunderstand. At least the basicConfig should also set the level of 
the handler that it created.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32207] IDLE: run's tk update adds extra traceback on callback error

[Terry J. Reedy]

New submission from Terry J. Reedy :

import tkinter as tk
root = tk.Tk()

def bad(): print(a, 'bad')
def good(): print('good')

root.after(1000, bad)
root.after(1500, good)
root.mainloop()


# Correctly gives this traceback and output:

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Programs\Python37\lib\tkinter\__init__.py", line 1699, in __call__
return self.func(*args)
  File "C:\Programs\Python37\lib\tkinter\__init__.py", line 745, in callit
func(*args)
  File "F:\Python\a\tem2.py", line 13, in bad
def bad(): print(a, 'bad')
NameError: name 'a' is not defined
good
>>> 


Remove or comment-out the blocking 'root.mainloop()' call and run the result 
from an IDLE editor.  The callbacks are still called because after user code is 
run, run.py calls tcl.update in a loop nominally 20 x per second.  This allows 
developers to interact with a 'live' gui by entering statements in the shell at 
'>>>' prompts.  The result is this.
-
>>> Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Programs\Python37\lib\idlelib\run.py", line 137, in main
seq, request = rpc.request_queue.get(block=True, timeout=0.05)
  File "C:\Programs\Python37\lib\queue.py", line 169, in get
raise Empty
queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

--

The relevant code in run.py was written before callback chaining.

try:
seq, request = rpc.request_queue.get(block=True, timeout=0.05)
except queue.Empty:
handle_tk_events()
continue

Experiments with normal exceptions in a shell suggest that wrapping 
handle_tk_events in try:except and re-raising any exception 'from None' should 
work.
try:
handle_tk_events()
except BaseException as e:
raise e from None

However, it appears that callback errors resulting from handle_tk_events() are 
not being caught here.  (print('message') and 1/0 before 'raise' have no 
visible effect.)  I will investigate more later.

--
assignee: terry.reedy
components: IDLE
messages: 307534
nosy: terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: IDLE: run's tk update adds extra traceback on callback error
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32207] IDLE: run's tk update adds context traceback on callback error

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: IDLE: run's tk update adds extra traceback on callback error -> IDLE: 
run's tk update adds context traceback on callback error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32182] Infinite recursion in email.message.as_string()

[R. David Murray]

R. David Murray  added the comment:

I checked, and your example works correctly with the patch applied.  I've 
committed the PR.  Let me know if you find any other issues (there are several 
open issues with exactly how headers are wrapped, but I don't think there are 
any open where serialization fails completely.)

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type: crash -> behavior
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31831] EmailMessage.add_attachment(filename="long or spécial") crashes or produces invalid output

[R. David Murray]

R. David Murray  added the comment:

The PR has been committed.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type: crash -> behavior
versions:  -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30788] email.policy.SMTP.fold() issue for long filenames with spaces

[R. David Murray]

R. David Murray  added the comment:

The PR has been committed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27240] 'UnstructuredTokenList' object has no attribute '_fold_as_ew'

[R. David Murray]

R. David Murray  added the comment:


New changeset a87ba60fe56ae2ebe80ab9ada6d280a6a1f3d552 by R. David Murray (Miss 
Islington (bot)) in branch '3.6':
bpo-27240 Rewrite the email header folding algorithm. (GH-3488) (#4693)
https://github.com/python/cpython/commit/a87ba60fe56ae2ebe80ab9ada6d280a6a1f3d552


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30213] ZipFile from 'a'ppend-mode file generates invalid zip

[BoppreH]

BoppreH  added the comment:

I'm not sure this is a duplicate of issue29094. That issue includes random data 
at the start of the file, while this issue uses the 'ab' mode solely for a 
creating the file if it doesn't exist (and thus is either empty or already a 
valid zip file). It's not clear to me why 'wb' should work but not 'ab' if the 
file was empty/missing to begin with.

[Windows 10, Python 3.6.3, 64 bit] still has the same problem.

Here's a more complete test case, starting with no existing files:

from zipfile import ZipFile

# Append mode: v
with open('file.zip', 'ab') as f:
with ZipFile(f, 'a') as zip:
zip.writestr('file.txt', 'contents')
with open('file.zip', 'rb') as f:
with ZipFile(f, 'r') as zip:
print(zip.read('file.txt'))
# Fails with "zipfile.BadZipFile: Bad magic number for file 
header"

# Write mode:  v
with open('file.zip', 'wb') as f:
with ZipFile(f, 'a') as zip:
zip.writestr('file.txt', 'contents')
with open('file.zip', 'rb') as f:
with ZipFile(f, 'r') as zip:
print(zip.read('file.txt'))
# Works.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27240] 'UnstructuredTokenList' object has no attribute '_fold_as_ew'

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4605

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27240] 'UnstructuredTokenList' object has no attribute '_fold_as_ew'

[R. David Murray]

R. David Murray  added the comment:


New changeset 85d5c18c9d83a1d54eecc4c2ad4dce63194107c6 by R. David Murray in 
branch 'master':
bpo-27240 Rewrite the email header folding algorithm. (#3488)
https://github.com/python/cpython/commit/85d5c18c9d83a1d54eecc4c2ad4dce63194107c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thank you for reporting this.  Would you like to submit a PR?

--
nosy: +davin, pitrou
stage:  -> needs patch
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31619] Strange error when convert hexadecimal with underscores to int

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31619] Strange error when convert hexadecimal with underscores to int

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 30a6bc842945e3e9c9c7db887ab495c428ec7074 by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
bpo-31619: Fixed integer overflow in converting huge strings to int. (GH-3884) 
(#4690)
https://github.com/python/cpython/commit/30a6bc842945e3e9c9c7db887ab495c428ec7074


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28503] [Patch] '_crypt' module: fix implicit declaration of crypt(), use crypt_r() where available

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4604
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28503] [Patch] '_crypt' module: fix implicit declaration of crypt(), use crypt_r() where available

[Ed Schouten]

Ed Schouten  added the comment:

Ah, you folks switched to Git + Github in the mean time. Nice!

I've just sent this pull request: https://github.com/python/cpython/pull/4691

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25711] Rewrite zipimport from scratch

[Decorater]

Decorater  added the comment:

Alright, just wanted to make sure because I did not want to have something 
break when loading up the entire standard library from an zip file with it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31619] Strange error when convert hexadecimal with underscores to int

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4603

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31619] Strange error when convert hexadecimal with underscores to int

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 29ba688034fc4eef0693b86002cf7bee55d692af by Serhiy Storchaka in 
branch 'master':
bpo-31619: Fixed integer overflow in converting huge strings to int. (#3884)
https://github.com/python/cpython/commit/29ba688034fc4eef0693b86002cf7bee55d692af


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32137] Stack overflow in repr of deeply nested dicts

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4602

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32137] Stack overflow in repr of deeply nested dicts

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 1fb72d2ad243c965d4432b4e93884064001a2607 by Serhiy Storchaka in 
branch 'master':
bpo-32137: The repr of deeply nested dict now raises a RecursionError (#4570)
https://github.com/python/cpython/commit/1fb72d2ad243c965d4432b4e93884064001a2607


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19610] Give clear error messages for invalid types used for setup.py params (e.g. tuple for classifiers)

[Éric Araujo]

Éric Araujo  added the comment:

I am sorry this snowballed.  The intent in my messages here and in my PR review 
was to exchange a late, unfriendly traceback with a clear, early message, but 
only for package authors.  I forgot that a Python 3.7 could execute an invalid 
setup.py from an existing tarball, as Neil pointed with the pip install 
exifread example.  Even if these packages get PRs to build wheels, it’s still 
bad to break existing sdists.  +1 to reverting the patch, +1 to reverting the 
breakage and also fixing the original problem with a warning and explicit 
conversion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32200] Full docs build of 3.6 and 3.7 failing since 2017-10-15

[Ned Deily]

Ned Deily  added the comment:

OK, I'm going to close this.  We can discuss further steps elsewhere.  Thanks 
for reporting the problem, Carol, and thanks for the quick response, Julien!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type: behavior -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31392] Upgrade installers to OpenSSL 1.1.0g and 1.0.2m

[Ned Deily]

Ned Deily  added the comment:

1.1.0g and 1.0.2m are out as of 2017-11-02 so both Windows and Mac installer 
builds for 3.7 and 3.6 should be updated.  (I'll take care of the Mac ones.)

--
assignee: ned.deily -> 
stage: patch review -> needs patch
title: Upgrade installers to OpenSSL 1.1.0f -> Upgrade installers to OpenSSL 
1.1.0g and 1.0.2m
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Eric N. Vander Weele]

Change by Eric N. Vander Weele :


--
nosy: +ericvw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32028] Syntactically wrong suggestions by the new custom print statement error message

[Sanyam Khurana]

Change by Sanyam Khurana :


--
keywords: +patch
pull_requests: +4601
stage: test needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29519] weakref spewing exceptions during finalization when combined with multiprocessing

[Warwick Chapman]

Warwick Chapman  added the comment:

Ditto.

On Sun, 03 Dec 2017 at 16:53, INADA Naoki  wrote:

>
> INADA Naoki  added the comment:
>
> It seems it is merged in 2.7.14.
> But I can't find this in changelog of 2.7.14.
>
>
> https://raw.githubusercontent.com/python/cpython/c707893f9cee870bba8364b3a06eb9cfa3b80e58/Misc/NEWS
>
> --
>
> ___
> Python tracker 
> 
> ___
>
-- 
-- Warwick Bruce Chapman | 083 7797 094 | http://wa.rwick.com

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Yury Selivanov]

Yury Selivanov  added the comment:

In general, implementing coroutines using 'yield' expressions (not 'yield 
from'!) is slower than async/await, because the former approach needs a 
trampoline to manage the stack, whereas CPython itself handles that for 'yield 
from' and 'await'.  I suspect that any difference in performance is not related 
to 'async/await' vs 'yield' performance.

The attached benchmarks compare two completely different frameworks: asyncio 
and asynq.  Both have different implementations of Task and Future and event 
loops primitives.  Perhaps both of them schedule IO events and callbacks 
differently as well.

asyncio could be slower because all tasks' callbacks must be scheduled through 
the event loop, whereas some frameworks like Twisted schedule them right away, 
which makes them faster in some specific micro-benchmarks.  Or there might be 
an issue with 'asyncio.gather()', which is stressed heavily in the attached 
benchmarks.

What I can say for sure, is that Python implementation of async/await has 
nothing to do with the results you observe.

I suggest to take a look at 'asyncio.gather', maybe we can make it faster.  
Please open a new issue if you find any ways to make it faster.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29519] weakref spewing exceptions during finalization when combined with multiprocessing

[INADA Naoki]

INADA Naoki  added the comment:

It seems it is merged in 2.7.14.
But I can't find this in changelog of 2.7.14.

https://raw.githubusercontent.com/python/cpython/c707893f9cee870bba8364b3a06eb9cfa3b80e58/Misc/NEWS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[jfbu]

jfbu  added the comment:

Related https://github.com/sphinx-doc/sphinx/issues/4272

It is stated there that using babel-french in place of polyglossia-french 
avoids the "Improper discretionary list" xetex problem starting with xetex 
0.2 (i.e. TeXLive 2015) whereas with polyglossia-french the earliest xetex 
version I could test with success is 0.6 (TL2016). But starting with 
TL2016, polyglossia-french as the issue 
https://github.com/sphinx-doc/sphinx/issues/4272

With TeXLive 2014, using babel-french does not avoid the "Improper 
discretionary list" xetex problem. I don't know how this maps to Debian 
packaging. One needs xetex 0.2 at minimum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19610] Give clear error messages for invalid types used for setup.py params (e.g. tuple for classifiers)

[Nick Coghlan]

Nick Coghlan  added the comment:

Prohibiting strings and bytes on the grounds of "Yes they're iterable, but are 
more likely intended as atomic here, so treat them as ambiguous and refuse to 
guess" would be fine. (Although I'll also note the classifier case will already 
fail on upload, since they won't be valid classifiers)

The only part I'm not OK with is upgrading a historically unenforced type 
restriction that only sometimes causes problems into an eagerly enforced one 
that breaks currently working code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32205] test.pythoninfo does not print the cross-built sysconfig data

[Xavier de Gaye]

Change by Xavier de Gaye :


--
keywords: +patch
pull_requests: +4600
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32206] Run modules with pdb

[Mario Corchero]

New submission from Mario Corchero :

Since PEP 338 we can run python modules as a script via `python -m module_name` 
but there is no way to run pdb on those (AFAIK).

The proposal is to add a new argument "-m" to the pdb module to allow users to 
run `python -m pdb -m my_module_name`

This is especially useful when working on cli tools that use entrypoints in 
setup.py, as there is no other way to run them.


I have a WIP here: 
https://github.com/mariocj89/cpython/commit/f47d1b779333657d7d87b21db841d5d3ad6cfa5c

Haven't sent the PR as I am still polishing some issues. If it sounds like a 
good idea I'll clean it up and open the PR.

--
components: Library (Lib)
messages: 307513
nosy: mariocj89
priority: normal
severity: normal
status: open
title: Run modules with pdb
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32176] Zero argument super is broken in 3.6 for methods with a hacked __class__ cell

[Nick Coghlan]

Nick Coghlan  added the comment:

Thanks for the issue report! The fix will be released in 3.6.4 and 3.7.0a3 
(both of which are expected to be later this month).

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32176] Zero argument super is broken in 3.6 for methods with a hacked __class__ cell

[Nick Coghlan]

Nick Coghlan  added the comment:


New changeset c8f32aae0aa173e122cf4c0592caec620d0d1de9 by Nick Coghlan in 
branch '3.6':
[3.6] bpo-32176: Set CO_NOFREE in the code object constructor (GH-4684)
https://github.com/python/cpython/commit/c8f32aae0aa173e122cf4c0592caec620d0d1de9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32205] test.pythoninfo does not print the cross-built sysconfig data

[Xavier de Gaye]

New submission from Xavier de Gaye :

When cross-compiling the only useful information printed by pythoninfo is the 
cross-built sysconfig data since everything else is related to the native 
python process used during the cross-build.

Currently pythoninfo prints the native sysconfig data instead.

--
components: Cross-Build
messages: 307510
nosy: Alex.Willmer, vstinner, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: test.pythoninfo does not print the cross-built sysconfig data
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] Improve MAC address calculation and fix test_uuid.py

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

I missed it too.  Serhiy, do you want to add that fix to your already open PR 
4677?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32196] Rewrite plistlib with functional style

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I don't have time to perform a review right now, I'm trying to get PEP 447 
through review and that takes most of my available time at the moment.

I'm not convinced that the speedup of plistlib is relevant for real-world code, 
plist files are intended as simple configuration files and tend to contain 
little data and should be read/written only sporadically.

That said some people appear to abuse plistlib to process other files which are 
probably NSKeyedArchiver archives, and those can be a lot larger. But I'm 
opposed to explicitly supporting that use case, because the format of 
NSKeyedArchiver files is completely undocumented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[Julien Palard]

Julien Palard  added the comment:

For me, french compile correctly with current state of cpython's conf.py and 
texlive-xetex 2017.20171128-1, if it helps.

I tried (not enough) and fail to test locally with a texlive-xetex from 2013 (I 
should try with a VM maybe, Debian won't let me install packages from 2013 on 
my sid...).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[jfbu]

jfbu  added the comment:

On-going discussion at http://tug.org/pipermail/xetex/2017-December/027212.html 
has brought new element that polyglossia's French module is broken with xetex 
since TeXLive2016. We had only one problem, we now have two on our hands.

Possibly Sphinx could be default use babel + French, not polyglossia + French, 
as the former is maintained but apparently less so the latter.

I tested that TeXLive 2015 (fully updated) and test document showing the 
https://github.com/sphinx-doc/sphinx/issues/3546 problem now compiles fine if 
using 

latex_elements = {
'babel': r'\usepackage{babel}',
}

in conf.py file, to override polyglossia which is default for Sphinx with 
xelatex.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32147] improve performance of binascii.unhexlify() by using conversion table

[Antoine Pitrou]

Antoine Pitrou  added the comment:

(platform is Ubuntu 16.04, 64-bit, on a Core i5-2500K CPU)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32147] improve performance of binascii.unhexlify() by using conversion table

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here are the results here:

- Before:
$ ./python -m timeit -s "from binascii import unhexlify; b = b'aa'*2**20" 
"unhexlify(b)"
50 loops, best of 5: 4.37 msec per loop

- After:
$ ./python -m timeit -s "from binascii import unhexlify; b = b'aa'*2**20" 
"unhexlify(b)"
200 loops, best of 5: 1.16 msec per loop

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] Improve MAC address calculation and fix test_uuid.py

[Xavier de Gaye]

Xavier de Gaye  added the comment:

When I am the author of a regression (as I have been too many times 
unfortunately), I prefer to be the one that reverts the changes and I would not 
like someone else to do it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[Julien Palard]

Julien Palard  added the comment:

XeTeX 0.6 was released in march 2016, so it's not even in Ubuntu 16.04 
Xenial Xerus. On docs.iad1.psf.io we're having Ubuntu 14.04 (an LTS ending 
around february 2019).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

~3-5% difference is a random variance.

Add 1/0 in this method and repeat the benchmark.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] Improve MAC address calculation and fix test_uuid.py

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Ah, good catch! I missed this. Do you mind to create a PR Xavier?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] Improve MAC address calculation and fix test_uuid.py

[Xavier de Gaye]

Xavier de Gaye  added the comment:

1. Before the regression made by 23df2d1304ece169d7e0dfc843dfb8026b413d9f, on 
the first invocation of getnode(), the returned value is cached in '_node' (a 
global variable) and getnode() is idempotent.

2. After 23df2d1304ece169d7e0dfc843dfb8026b413d9f, the returned value is not 
cached in '_node' when it is obtained through _random_getnode() and getnode() 
returns different values each time in that case.

Not sure how you can miss that point :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Liran Nuna]

Liran Nuna  added the comment:

The PR is merely a finding I ran into while benchmarking. In my executions I 
saw a consistent ~3-5% performance increase.

At my company we are attempting to migrate from asynq to asyncio but 
performance suffers (our servers response times nearly double). I personally 
ran profiles and benchmarks and noticed this tiny bottleneck which I knew how 
to fix, so I submitted a PR to the python project in hopes it helps.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17611] Move unwinding of stack for "pseudo exceptions" from interpreter to compiler.

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I would point out that Serhiy posted some comments in the followup PR which 
aren't addressed by Neil's followup PR.  They relate to tracing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I don't see any difference after applying PR 4186.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19610] Give clear error messages for invalid types used for setup.py params (e.g. tuple for classifiers)

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 03/12/2017 à 05:57, Neil Schemenauer a écrit :
> 
> I like Nick's idea of calling list() to fix the argument.

I don't think that's a good idea.  Suppose someone is passing a string
by mistake:

setup(...,
  classifiers='Programming Language :: Python :: 3 :: Only',
  )

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Liran Nuna]

Liran Nuna  added the comment:

Added comparable benchmark in asynq

--
Added file: https://bugs.python.org/file47314/batch_asynq.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32204] async/await performance is very low

[Liran Nuna]

New submission from Liran Nuna :

The performance of async/await is very low when compared to similar code that 
implements similar functionality via iterators, such as Quora's asynq library 
(https://github.com/quora/asynq/tree/master/asynq).

Based on my benchmarks, asynq is almost twice as fast as async/await. 

I have found some performance hanging fruit when benchmarking (See attached 
GitHub PR).


$ time python batch_asyncio.py 

real0m5.851s
user0m5.760s
sys 0m0.088s
$ time python batch_asynq.py 

real0m2.999s
user0m2.900s
sys 0m0.076s

--
components: asyncio
files: batch_asyncio.py
messages: 307492
nosy: Liran Nuna, yselivanov
priority: normal
pull_requests: 4599
severity: normal
status: open
title: async/await performance is very low
type: performance
versions: Python 3.6
Added file: https://bugs.python.org/file47313/batch_asyncio.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[jfbu]

jfbu  added the comment:

I can confirm the "Improper discretionary list" error from xetex build is a 
xetex bug which is present at XeTeX 0.2 and absent at XeTeX 0.6 and 
presumably all more recent releases.

It was seen at https://github.com/sphinx-doc/sphinx/issues/3546 and reported to 
XeTeX mailing list at http://tug.org/pipermail/xetex/2017-March/027056.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32200] Full docs build of 3.6 and 3.7 failing since 2017-10-15

[jfbu]

jfbu  added the comment:

For info, the xetex problem "Improper discretionary list" is presumably the one 
seen at  https://github.com/sphinx-doc/sphinx/issues/3546. I asked on xetex 
mailing list at http://tug.org/pipermail/xetex/2017-March/027056.html to which 
xetex bug it was related but it appears I got no reply. Hence I don't know the 
precise xetex release which fixed it, but it is ok with `XeTeX 0.6`. No 
action was taken on Sphinx side as this appeared to be a XeTeX bug only.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[Julien Palard]

Julien Palard  added the comment:

Due to issue 32200, we switched on xelatex, it however did **not** fixed the 
french builds as expected, maybe because we're using an old version of xelatex.

The issue:

! Improper discretionary list.
 }

l.359 ...{PyObject} \PYG{o}{*}\PYG{n}{t}\PYG{p}{;}
  
? 
! Emergency stop. 

The version of texlive-xetex on the build server: 2013.20140215-1ubuntu0.1

The version I tried: 2017.20171128-1

I'm not having the 2013 version on Debian. I'm trying now with 
2014.20141024-2+deb8u1, and I'm trying to understand the error message too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32200] Full docs build of 3.6 and 3.7 failing since 2017-10-15

[Julien Palard]

Julien Palard  added the comment:

Japanese is failing expectedly (may work with platex but not xelatex), but I 
hoped french to build. But the build server is giving:

! Improper discretionary list.
 }

l.359 ...{PyObject} \PYG{o}{*}\PYG{n}{t}\PYG{p}{;}
  
? 
! Emergency stop.

I suspect it's because we're using an old version of xetex 
(2013.20140215-1ubuntu0.1) and I tested with a recent one (2017.20171128-1). 
I'm trying to reproduce the issue locally, and to understand the message.

In any cases, issue 32200 was about build failing since the introduction of the 
dotless i, which is now fixed, french and japanese problems are tracked here: 
https://bugs.python.org/issue31589.

I'd close issue 32200.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com