[issue28510] PyUnicodeDecodeError_GetObject always return bytes

[Xiang Zhang]

Xiang Zhang added the comment:

LGTM.

Actually I just read the codecs error handles codes last day but didn't think 
of this. :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28509] Key-sharing dictionaries can inrease the memory consumption

[Xiang Zhang]

Changes by Xiang Zhang :


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28509] Key-sharing dictionaries can inrease the memory consumption

[INADA Naoki]

INADA Naoki added the comment:

And I feel current target size of dict_merge is bit larger.

When inserting new item:

* ma_used = dk_size*2 / 3 when right before increasing keys
* ma_used = dk_size/ 3 when right after increasing keys

On the other hand, current dict_merge creates:

* ma_used = dk_size / 2 when all keys in two dict is distinct
* ma_used = dk_size / 4 when all keys in two dict is same

If changing it to dictresize(mp, (mp->ma_used + other->ma_used)*3/2),

* ma_used = dk_size*2 / 3 when all keys in two dict is distinct
* ma_used = dk_size/ 3 when all keys in two dict is same

I think this is more consistent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28509] Key-sharing dictionaries can inrease the memory consumption

[INADA Naoki]

INADA Naoki added the comment:

> 0 (128, 60)
> 1 (128, 60)
> 2 (128, 60)
> 3 (128, 60)
> 4 (128, 60)
> 5 (128, 60)

Minimum dict keysize is 8, and it's capacity is 5.

> 6 (196, 196)

Dict is resized. And since __dict__.update() caused the resizing,
both are normal (combined) dict.

> 7 (196, 196)
> 8 (196, 344)

dict.update() cause faster increasing keysize.

Adding to dict cause resizing when number of items reaches 2/3 of keysize.
On the other hand, dict.update() targets 1/2 of keysize is filled.

In this case, keysize is 16 and 16 * 2 // 3 = 10. Since 8 < 10, adding
item to key
doesn't increase it's size. Since 8 >= (16 / 2), dict.update() creates
dict having keysize == 32.

(note: My patch in http://bugs.python.org/issue28147 changes >= to >.
So keysize == 16 when
number of items == 8).

But, while checking this, I found another bug in dict_merge.

/* Do one big resize at the start, rather than
 * incrementally resizing as we insert new items.  Expect
 * that there will be no (or few) overlapping keys.
 */
if (mp->ma_keys->dk_usable * 3 < other->ma_used * 2)
if (dictresize(mp, (mp->ma_used + other->ma_used)*2) != 0)
   return -1;

dk_usable means "how many new items can be inserted without resizing".
So this if statement should be:

if (mp->ma_keys->dk_usable < other->ma_used)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28506] Multiprocessing Pool starmap - struct.error: 'i' format requires -2e10<=n<=2e10

[Justin Ting]

Justin Ting added the comment:

Actually, on further inspection, I seem to be having a slightly different 
problem with the same error that I initially described now.

Even after modifying my code so that each python forked off to another process 
was only given the following arguments:
args = [(None, models_shape, False, None, [start, end], 
'data/qp_red_features.npy') for start, end in jobs] 

where models_shape, start, and end are only single integers, the same error 
still comes up as a result. Within each process, I'm reading in a (relatively 
small, only 12MB) .npy ndarray and taking the [start:end] slice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[klappnase]

klappnase added the comment:

@Miguel
About tk_busy_configure():
I strongly suggest that my version of tk_busy_configure() is better, since it 
behaves exactly like the usual configure() methods in Tkinter (which includes 
e.g. Canvas.itemconfigure() or Text.tag_configure()), i.e. you can not only 
change the values of the configuration options, but also query available 
options and their default values. I think *this* follows the usual Tkinter 
conventions.
As you pointed out in your last post, I could not use _configure() directly for 
this, but had to "duplicate" the code with a slight modification, because of 
the order of arguments for tk_busy_configure().
The reason that this construct is not used for pack_configure() is simply that 
widget.pack_configure() does not return anything if called without arguments 
(in fact it appears to be the very same as widget.pack()).

About busy_status():
the construct I use here is to work around the broken _getboolean() which does 
not always return True or False, but often returns None instead of False (if 
wantobjects==True). Therefore I used (unlike most Tkinter methods) 
tkapp.getboolean() directly, however quite a while ago I noticed that this was 
broken, too, and sometimes returned 0 or 1 instead of True or False. However I 
admit that I did not test right now if this is still the case, in fact I just 
copy'n'pasted it from code I already use and that works perfectly well (BTW, I 
suggested the same method to fix these issue with _getboolean() here some time 
ago, and of course I agree that it would be preferrable to fix _getboolean() 
and possibly tkapp.getboolean() in the first place).

@Serhiy
Thanks for the link, I did not find the source tree on the new python web site 
(I admit I did not look too hard :) .
I now added a patch against the default branch'es __init__.py . Doc strings are 
(still :) included. I also changed the function names according to Miguel's 
suggestion.
I'll see if I find the time to add a proper test class one of these days (you 
guys make it really hard for outsiders to help out ;)

--
Added file: http://bugs.python.org/file45193/tk_busy.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Raymond Hettinger]

Raymond Hettinger added the comment:

>>> from sys import getsizeof
>>> class A:
def __init__(self, a, b, c, d, e, f):
self.a = a
self.b = b
self.c = c
self.d = d
self.e = e
self.f = f

>>> a = A(10, 20, 30, 40, 50, 60)
>>> b = A(10, 20, 30, 40, 50, 60)
>>> c = A(10, 20, 30, 40, 50, 60)
>>> d = A(10, 20, 30, 40, 50, 60)
>>> [getsizeof(vars(inst)) for inst in [a, b, c, d]]
[152, 152, 152, 152]
>>> [getsizeof(dict(vars(inst))) for inst in [a, b, c, d]]
[368, 368, 368, 368]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Raymond Hettinger]

Raymond Hettinger added the comment:

> Hmm, seems no dict here is shared-key dict.

Yes.  That seems to be the case.  Apparently, doing an update() to the inst 
dict cause it to recombine.

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Hmm, seems no dict here is shared-key dict.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28281] Remove year limits from calendar

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

The patch should include an update to documentation.

1. We should probably explain that python -mcalendar does not reproduce the 
output of UNIX cal.  For example, on Mac OS (and various Linux variants): 

$ cal 9 1752
   September 1752
Su Mo Tu We Th Fr Sa
   1  2 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30


but 

$ python3 -mcalendar 1752 9
   September 1752
Mo Tu We Th Fr Sa Su
 1  2  3
 4  5  6  7  8  9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30

2. We should explain that while the calendar module relies on datetime, it 
implements an infinite calendar with a period of 400 years.

3. A reference should be made to ISO 8601 for our treatment of nonpositive 
years.

Given ISO 8601 and the simplicity of this change, I don't think Raymond will 
insist that we continue imposing datetime-like limits, but I would like to give 
him a chance to renew his objection once the no-limits calendar is documented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Raymond Hettinger]

Raymond Hettinger added the comment:

> Isn't this already implemented?

No.

>>> class A:
pass

>>> d = dict.fromkeys('abcdefghi')
>>> a = A()
>>> a.__dict__.update(d)
>>> b = A()
>>> b.__dict__.update(d)
>>> import sys
>>> [sys.getsizeof(m) for m in [d, vars(a), vars(b)]]
[368, 648, 648]
>>> c = A()
>>> c.__dict__.update(d)
>>> [sys.getsizeof(m) for m in [d, vars(a), vars(b), vars(c)]]
[368, 648, 648, 648]

There is no benefit reported for key-sharing.  Even if you make a thousand of 
these instances, the size reported is the same.  Here is the relevant code:

_PyDict_SizeOf(PyDictObject *mp)
{
Py_ssize_t size, usable, res;

size = DK_SIZE(mp->ma_keys);
usable = USABLE_FRACTION(size);

res = _PyObject_SIZE(Py_TYPE(mp));
if (mp->ma_values)
res += usable * sizeof(PyObject*);
/* If the dictionary is split, the keys portion is accounted-for
   in the type object. */
if (mp->ma_keys->dk_refcnt == 1)
res += (sizeof(PyDictKeysObject)
- Py_MEMBER_SIZE(PyDictKeysObject, dk_indices)
+ DK_IXSIZE(mp->ma_keys) * size
+ sizeof(PyDictKeyEntry) * usable);
return res;
}

It looks like the fixed overhead is included for every instance of a 
split-dictionary.   Instead, it might make sense to take the fixed overhead and 
divide it by the number of instances sharing the keys (averaging the overhead 
across the multiple shared instances):

 res = _PyObject_SIZE(Py_TYPE(mp)) / num_instances;

Perhaps use ceiling division:

 res = -(- _PyObject_SIZE(Py_TYPE(mp)) / num_instances);

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[Miguel]

Miguel added the comment:

Misc._configure is only used when the first Tcl command is the name of the 
widget.

Very probably my proposal for tk_busy_configure is a better candidate because 
it follows the conventions used in tkinter (it's similar to pack_configure and 
place_configure):
 def tk_busy_configure(self, cnf=None, **kw):
self.tk.call(('tk', 'busy', 'configure', self._w) + self._options(cnf, 
kw))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28281] Remove year limits from calendar

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Alexander as core developer and the creator of this issue can merge the patch 
after making his review. But first we should make a decision whether it is 
worth to do at all. I'm +0 now. Raymond seems has objections.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28512] PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject() always set the offset attribute to None

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The purpose of PyErr_SyntaxLocationEx() function (added in 00e4ce31d404) was 
setting the offset attribute of raised syntax error to specified value. But 
this never worked as expected. The offset attribute is set to integer value in 
Python/errors.c:1067, but then replaced with None in Python/errors.c:1083.

--
components: Interpreter Core
messages: 279224
nosy: benjamin.peterson, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PyErr_SyntaxLocationEx() and PyErr_SyntaxLocationObject() always set the 
offset attribute to None
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[Miguel]

Miguel added the comment:

Thanks klappnase for your collaboration.

I dont understand this function:
def busy_status(self):
'''Returns the busy status of this window.
If the window presently can not receive user interactions,
True is returned, otherwise False.'''
return((self.tk.getboolean(self.tk.call(
'tk', 'busy', 'status', self._w)) and True) or False)

This pattern is not used in other functions that make use of 
self.tk.getboolean. These functions simply returns the value of 
self.tk.getboolean directly.

The code of your function busy_configure() is very similar to 
Misc._configure(). I think that you are duplicating code. Other functions 
related to configuration like pack_configure() and place_configure() simply use 
self._options(). For example:
def pack_configure(self, cnf={}, **kw):
self.tk.call(
  ('pack', 'configure', self._w)
  + self._options(cnf, kw))

def place_configure(self, cnf={}, **kw):
self.tk.call(
  ('place', 'configure', self._w)
  + self._options(cnf, kw))

I think that my proposal can do the job well. It follows the same pattern than 
the other functions:
def tk_busy_configure(self, cnf=None, **kw):
self.tk.call(('tk', 'busy', 'configure', self._w) + self._options(cnf, 
kw))

But I am not totally sure whether it's better to call directly Misc._configure 
or Misc._options in this situation.

Also if we follow the naming convention used in tkinter, it seems that we have 
to define first tk_busy_configure and then make this assignation:
 busy_configure = tk_busy_configure

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Chi Hsuan Yen]

Chi Hsuan Yen added the comment:

Thanks. Sorry for not reading devguide carefully.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Ned Deily]

Ned Deily added the comment:

https://docs.python.org/devguide/faq.html#how-do-i-regenerate-configure

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28504] Cleanup unicode_decode_call_errorhandler_wchar/writer

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM. Thank you Xiang.

I have extended your approach to all sources and have written other two 
patches: issues28510 and issue28511.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28511] Use the "U" format for parsing Unicode object arg in PyArg_Parse*

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch uses the "U" format in PyArg_Parse* functions instead of the 
"O!" format with _Type argument. This makes code cleaner, faster, and 
allows to remove additional PyUnicode_READY checks.

The patch is inspired by the patch of Xiang Zhang (issue28504).

--
components: Interpreter Core
files: parse_unicode_arg.patch
keywords: patch
messages: 279219
nosy: serhiy.storchaka, xiang.zhang
priority: normal
severity: normal
stage: patch review
status: open
title: Use the "U" format for parsing Unicode object arg in PyArg_Parse*
type: enhancement
versions: Python 3.7
Added file: http://bugs.python.org/file45192/parse_unicode_arg.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28510] PyUnicodeDecodeError_GetObject always return bytes

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Since PyUnicodeDecodeError_GetObject() always returns bytes object, following 
PyBytes_AsString() never fails and can be replaced with the PyBytes_AS_STRING() 
macro. This makes error handlers looking a littler clearer and a little faster.

The patch is inspired by the patch of Xiang Zhang (issue28504).

--
components: Interpreter Core
files: unicodedecodeerror_object_is_bytes.patch
keywords: patch
messages: 279218
nosy: serhiy.storchaka, xiang.zhang
priority: normal
severity: normal
stage: patch review
status: open
title: PyUnicodeDecodeError_GetObject always return bytes
type: enhancement
versions: Python 3.7
Added file: 
http://bugs.python.org/file45191/unicodedecodeerror_object_is_bytes.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28504] Cleanup unicode_decode_call_errorhandler_wchar/writer

[Roundup Robot]

Roundup Robot added the comment:

New changeset 204a43c452cc by Serhiy Storchaka in branch 'default':
Issue #28504: Cleanup unicode_decode_call_errorhandler_wchar/writer.
https://hg.python.org/cpython/rev/204a43c452cc

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you provide a patch against the default branch? See 
https://docs.python.org/devguide/ for help. The patch should include not just 
changes to the tkinter module, but tests for new methods (add new class in 
Lib/tkinter/test/test_tkinter/test_widgets.py). Unfortunately there is no 
appropriate place for documenting this feature in the module documentation, but 
new methods should have docstrings. Would be nice to add an entry in 
Doc/whatsnew/3.7.rst.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28509] Key-sharing dictionaries can inrease the memory consumption

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The size of small key-sharing dictionary (PEP 412) can be larger than the size 
of normal dictionary.

Python 3.6:

>>> def dictsizes(k):
... d = {'a%d'%i: None for i in range(k)}
... class C:
... def __init__(self):
... self.__dict__.update(d)
... return sys.getsizeof(d), sys.getsizeof(C().__dict__)
... 
>>> for i in range(20):
... print(i, dictsizes(i))
... 
0 (128, 60)
1 (128, 60)
2 (128, 60)
3 (128, 60)
4 (128, 60)
5 (128, 60)
6 (196, 196)
7 (196, 196)
8 (196, 344)
9 (196, 344)
10 (196, 344)
11 (344, 344)
12 (344, 344)
13 (344, 344)
14 (344, 344)
15 (344, 344)
16 (344, 628)
17 (344, 628)
18 (344, 628)
19 (344, 628)

Normal dictionaries of size 8-10 are more compact than corresponding 
key-sharing dictionaries.

Python 3.5:

>>> for i in range(20):
... print(i, dictsizes(i))
... 
0 (144, 48)
1 (144, 48)
2 (144, 48)
3 (144, 48)
4 (144, 240)
5 (144, 240)
6 (240, 240)
7 (240, 240)
8 (240, 432)
9 (240, 432)
10 (240, 432)
11 (240, 432)
12 (432, 432)
13 (432, 432)
14 (432, 432)
15 (432, 432)
16 (432, 816)
17 (432, 816)
18 (432, 816)
19 (432, 816)

In Python 3.5 normal dictionaries of size 4-5 and 8-11 are more compact than 
corresponding key-sharing dictionaries. And note that key-sharing dictionaries 
of size 0-3 consume more memory on 3.6 that on 3.5. I think we should use more 
thrifty strategy for allocating key-sharing dictionaries.

--
components: Interpreter Core
messages: 279215
nosy: Mark.Shannon, benjamin.peterson, inada.naoki, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Key-sharing dictionaries can inrease the memory consumption
type: resource usage

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28505] pip installation issues with default path on Windows

[Eryk Sun]

Eryk Sun added the comment:

Issues with pip should be reported to its GitHub site. But installing to the 
"Program Files" directory certainly isn't a problem for pip. You need to run it 
with administrator rights.

--
nosy: +eryksun
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Chi Hsuan Yen]

Chi Hsuan Yen added the comment:

Well, I remember someone asked me to include ./configure in patches. Maybe it's 
worth to add a comment in devguide.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Ned Deily]

Ned Deily added the comment:

That's why we suggest to not include configure changes in a patch, only 
configure.ac changes. It's the responsibility of the patch committer to ensure 
that generated files like configure are updated properly at commit time. But, 
yes, it would be better to not have spurious changes go in and out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Xiang Zhang]

Xiang Zhang added the comment:

> Isn't this already implemented?

Get the same question. dict.__sizeof__ can identify shared dicts.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[klappnase]

klappnase added the comment:

Ok, I investigated this a little further. First I noticed another bug with the 
code from my first post, the "self._w" must be omitted from the call to 
busy_current(), so the func should look like:

def busy_current(self, pattern=None):
return([self._nametowidget(x) for x in
self.tk.splitlist(self.tk.call(
   'tk', 'busy', 'current', pattern))])

I did then some more testing, the code now seems to work flawlessly both with 
Python-3.5.2 on windows 10 and with Python 2.7 on debian Jessie. So I finally 
prepared a patch (against the __init__.py file from the 3.5.2 windows 
installer). Following Miguel's suggestion I also added aliases prefixed with 
tk_ to the various busy_... methods.
Since I am not familiar with Python's "official" test mechanisms, for now I 
wrote a simple script that does a number of calls to the tk_busy_... functions 
to test if everything works as expected:

try:
import Tkinter
except:
import tkinter as Tkinter
#Tkinter.wantobjects = False

root = Tkinter.Tk()
f = Tkinter.Frame(root, name='f')
f.pack(fill='both', expand=1)
b=Tkinter.Button(f, name='b', text='hi', command=root.bell)
b.pack(padx=100, pady=100)

top = Tkinter.Toplevel(root, name='top')

def test1():
root.tk_busy()
def test2():
root.tk_busy_forget()
def test3():
root.tk_busy_hold(cursor='gumby')
def test4():
root.tk_busy_forget()
def test5():
root.tk_busy_hold()
top.tk_busy(cursor='gumby')
print(root.tk_busy_current())
print(root.tk_busy_current(pattern='*t*'))
def test6():
print(root.tk_busy_current())
def test7():
root.tk_busy_configure(cursor='gumby')
def test8():
print(root.tk_busy_configure())
print(root.tk_busy_configure('cursor'))
print(root.tk_busy_cget('cursor'))
def test9():
print(root.tk_busy_status())
def test10():
root.tk_busy_forget()
print(root.tk_busy_status())
print(root.tk_busy_current())

delay = 0
for test in (test1, test2, test3, test4, test5, test6, test7,
test8, test9, test10):
delay += 1000
root.after(delay, test)

root.mainloop()

--
keywords: +patch
Added file: http://bugs.python.org/file45190/tk_busy.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

>>> class C:
... def __init__(self):
... for i in range(682):
... setattr(self, 'a%d'%i, None)
... 
>>> sys.getsizeof(C().__dict__) / len(C().__dict__)
4.058651026392962

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Isn't this already implemented?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28508] Need way to expose incremental size of key sharing dicts

[Raymond Hettinger]

New submission from Raymond Hettinger:

In many Python programs much of the memory utilization is due to having many 
instances of the same object.  We have key-sharing dicts that reduce the cost 
by storing only in the incremental values.  It would be nice to have visibility 
to the savings.

One possible way to do this is to have sys.getsizeof(d) report only the 
incremental space.  That would let users make reasonable memory estimates in 
the form of n_instances * sizeof(vars(inst)).

--
components: Interpreter Core
messages: 279207
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Need way to expose incremental size of key sharing dicts
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Chi Hsuan Yen]

Chi Hsuan Yen added the comment:

Thanks for the fast response. My main point is not that would affect users who 
compile from tarballs. The point is that if I modify configure.ac on the 
default branch, I can't generate a patch that applies to the default branch 
cleanly; there are always --runstatedir related changes, so line numbers are 
incorrect. Seems Rietveld doesn't handle patches with wrong line numbers. An 
example is ncurses.patch of issue28190.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Ned Deily]

Ned Deily added the comment:

Don't worry about it. We will take care of it as necessary when we release.

--
resolution:  -> later
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28507] Regenerate ./configure on the default branch

[Chi Hsuan Yen]

New submission from Chi Hsuan Yen:

In 0ea088671bc2 and 3ce29b2452f0, --runstatedir was added to ./configure. Seems 
Benjamin uses a slightly different version of autoconf than 2.69. As a result, 
modifying ./configure.ac leads to unrelated changes in ./configure. Later in 
17bd5239b886, this option is removed from 3.6 branch. The issue is still in the 
default branch. Can anyone apply 17bd5239b886 to the default branch?

Added authors of aforementioned commits.

--
components: Build
messages: 279204
nosy: Chi Hsuan Yen, benjamin.peterson, ned.deily
priority: normal
severity: normal
status: open
title: Regenerate ./configure on the default branch
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28506] Multiprocessing Pool starmap - struct.error: 'i' format requires -2e10<=n<=2e10

[Justin Ting]

Justin Ting added the comment:

Ah, should have picked that up, coding at 3:30am doesn't do wonders for
keeping a clear head.

Thanks Tim, I'll keep that in mind!

*Justin Ting*
*E* justingl...@gmail.com |  *M* +61 424 751 665 | *L*
*https://au.linkedin.com/in/justinyting
* | *G *https://github.com/jyting

On Sun, Oct 23, 2016 at 3:48 AM, Tim Peters  wrote:

>
> Tim Peters added the comment:
>
> This has nothing to do with the _values_ you're passing - it has to do
> with the length of the pickle string:
>
> def _send_bytes(self, buf):
> n = len(buf)
> # For wire compatibility with 3.2 and lower
> header = struct.pack("!i", n)  IT'S BLOWING UP HERE
> if n > 16384:
> ...
> self._send(header)
> self._send(buf)
>
> where the traceback shows it's called here:
>
> self._send_bytes(ForkingPickler.dumps(obj))
>
> Of course the less data you're passing, the smaller the pickle, and that's
> why it doesn't blow up if you pass subsets of the data.
>
> I'd suggest rethinking how you're sharing data, as pushing two-gigabyte
> pickle strings around is bound to be the least efficient way possible even
> if it didn't blow up ;-)
>
> --
> nosy: +tim.peters
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28506] Multiprocessing Pool starmap - struct.error: 'i' format requires -2e10<=n<=2e10

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This looks as a duplicate of issue17560.

--
nosy: +serhiy.storchaka
resolution:  -> duplicate
superseder:  -> problem using multiprocessing with really big objects?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28506] Multiprocessing Pool starmap - struct.error: 'i' format requires -2e10<=n<=2e10

[Tim Peters]

Tim Peters added the comment:

This has nothing to do with the _values_ you're passing - it has to do with the 
length of the pickle string:

def _send_bytes(self, buf):
n = len(buf)
# For wire compatibility with 3.2 and lower
header = struct.pack("!i", n)  IT'S BLOWING UP HERE
if n > 16384:
...
self._send(header)
self._send(buf)

where the traceback shows it's called here:

self._send_bytes(ForkingPickler.dumps(obj))

Of course the less data you're passing, the smaller the pickle, and that's why 
it doesn't blow up if you pass subsets of the data.

I'd suggest rethinking how you're sharing data, as pushing two-gigabyte pickle 
strings around is bound to be the least efficient way possible even if it 
didn't blow up ;-)

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28506] Multiprocessing Pool starmap - struct.error: 'i' format requires -2e10<=n<=2e10

[Justin Ting]

New submission from Justin Ting:

Multiprocessing is throwing this error when dealing with large amounts of data 
(all floating points an integers), but none of which exceeds the number 
boundaries in the error that it throws:

  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 268, in 
starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
  File "/root/anaconda3/lib/python3.5/multiprocessing/pool.py", line 385, in 
_handle_tasks
put(task)
  File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 206, 
in send
self._send_bytes(ForkingPickler.dumps(obj))
  File "/root/anaconda3/lib/python3.5/multiprocessing/connection.py", line 393, 
in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
> /root/anaconda3/lib/python3.5/multiprocessing/connection.py(393)_send_bytes()
-> header = struct.pack("!i", n)

It works fine on any number of subsets of this data, but not when put together.

--
components: Library (Lib)
messages: 279200
nosy: Justin Ting
priority: normal
severity: normal
status: open
title: Multiprocessing Pool starmap - struct.error: 'i' format requires 
-2e10<=n<=2e10
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28504] Cleanup unicode_decode_call_errorhandler_wchar/writer

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28505] pip installation issues with default path on Windows

[Wojtek Swiatek]

New submission from Wojtek Swiatek:

When installing Python 3.5 on Windows (I checked the 64bits installer but I 
believe the same issue will be with the 32bits), the default installation path 
when installing for "everyone" is now C:\ Program Files\Python35.

When subsequently installing packages via pip, the installation crashes midway 
with a cryptic error about "permission denied" (if I remember correctly). I 
checked with the cryptography module.

It is not an issue with access rights but with the whitespace in "Program 
Files". Reinstalling Python to C:\Python35 fixes the issue (pip install 
cryptography runs correctly).

--
components: Installation
messages: 279199
nosy: Wojtek Swiatek
priority: normal
severity: normal
status: open
title: pip installation issues with default path on Windows
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28504] Cleanup unicode_decode_call_errorhandler_wchar/writer

[Xiang Zhang]

New submission from Xiang Zhang:

The patch makes several cleanups to 
unicode_decode_call_errorhandler_wchar/writer:

1. Use U instead O! for argument parser, it ought to be more efficient and 
write less code.
2. In theory, if inputobj is not bytes, there needs to be a goto onError, or it 
could crash. But PyUnicodeDecodeError_GetObject is guaranteed to return bytes, 
so we can remove the incomplete branch.
3. On success, we don't need Xdecref.

--
components: Interpreter Core
files: unicode_decode_call_errorhandler_*.patch
keywords: patch
messages: 279198
nosy: haypo, serhiy.storchaka, xiang.zhang
priority: normal
severity: normal
stage: patch review
status: open
title: Cleanup unicode_decode_call_errorhandler_wchar/writer
type: enhancement
versions: Python 3.7
Added file: 
http://bugs.python.org/file45189/unicode_decode_call_errorhandler_*.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Guido van Rossum]

Guido van Rossum added the comment:

Honestly I think pydoc is already too verbose. It would be better if the class 
header looked more like what was written in the source code -- that is the most 
compact way to render it. I say open a separate issue since this issue is about 
functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Guido van Rossum]

Changes by Guido van Rossum :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Roundup Robot]

Roundup Robot added the comment:

New changeset dc030d15f80d by Guido van Rossum in branch '3.5':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of 
function annotations. Ivan L.
https://hg.python.org/cpython/rev/dc030d15f80d

New changeset 3937502c149d by Guido van Rossum in branch '3.6':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of 
function annotations. Ivan L. (3.5->3.6)
https://hg.python.org/cpython/rev/3937502c149d

New changeset 62127e60e7b0 by Guido van Rossum in branch 'default':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of 
function annotations. Ivan L. (3.6->3.7)
https://hg.python.org/cpython/rev/62127e60e7b0

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28498] tk busy command

[Miguel]

Miguel added the comment:

Maybe it's better to add also these methods:
busy = tk_busy
busy_cget = tk_busy_cget
busy_configure = tk_busy_configure
busy_current = tk_busy_current
busy_forget = tk_busy_forget
busy_hold = tk_busy_hold
busy_status = tk_busy_status

Many other methods in tkinter module follow the same pattern. For example:

iconbitmap = wm_iconbitmap
iconify = wm_iconify
group = wm_group
geometry = wm_geometry
focusmodel = wm_focusmodel
frame = wm_frame

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Actually, for classes, it is probably worth adding a separate section "Generic 
type info" that will render information using __orig_bases__, __parameters__, 
and __args__. At the same time the "header" will be the same as now, listing 
runtime __bases__.

What do you think about this? Should I open a separate issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Guido van Rossum]

Guido van Rossum added the comment:

OK, sounds good then. I guess most of the work was in typing.py, not in 
inspect. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

For function annotations I did as originally proposed. In my previous comment I 
was talking about documentation for classes. For example:

class C(Generic[T], Mapping[int, str]): ...
pydoc.render_doc(C)

will show "class C(typing.Mapping)".

while for function annotations typing is indeed much more common so that 
pydoc.render_doc(foo) will show

foo(data: List[Any]) -> Iterator[Tuple[int, Any]]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Guido van Rossum]

Guido van Rossum added the comment:

Hm, I actually like the original proposal better. Perhaps 
collections.abc.Mapping is more common than typing.Mapping, but is it more 
common *in function annotations*? I don't think so.

Also, I like showing e.g. Iterator[Tuple[int, Any]] rather than just Iterator. 
This is documentation we're talking about, and the parameter types are very 
useful as documentation. (However, abbreviating List[Any] as List is fine, 
since they mean the same thing.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27989] incomplete signature with help function using typing

[Ivan Levkivskyi]

Ivan Levkivskyi added the comment:

Here is the patch according to the discussion (modifying inspect).

I didn't change the rendering of docs for classes (neither stripped 'typing.' 
nor changed __bases__ to __orig_bases__). First, collections.abc.X are widely 
used as base classes, so that plain Mapping could be confused with 
collections.abc.Mapping. Second, seeing the actual runtime type-erased bases 
suggests that one should use isinstance() and issubclass() with those (not 
with, e.g.,  Mapping[int, str], the latter will raise TypeError).

--
keywords: +patch
nosy: +yselivanov
Added file: http://bugs.python.org/file45188/typing-pydoc.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25953] re fails to identify invalid numeric group references in replacement strings

[SilentGhost]

SilentGhost added the comment:

I've modified addgroup to take a pos argument, this seem to introduce minimal 
disturbance.

--
Added file: http://bugs.python.org/file45187/25953_4.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1602] windows console doesn't print or input Unicode

[irdb]

Changes by irdb :


--
nosy: +irdb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28499] Logging module documentation needs a rework.

[Pierre Bousquie]

Pierre Bousquie added the comment:

Hi stephane,

I have tweeted Florian and he is still interested.

I think the doc has a lot of information but does not organize it efficiently.

My notes from the conference:
- Logging reccord attribute is at 16.6.7 (middle of the doc) and that's a must 
to fine tune the message.

- Configuration: the "interesting" (read: must read for use in production) is 
in advanced tutorial: configuration...
https://docs.python.org/3.7/howto/logging.html#configuring-logging
ndd in :
https://docs.python.org/3.7/library/logging.config.html#module-logging.config

- Lots of information are also in the "See also" section.

and I had a personal fail time with logrotate... wich will work with python 
logging only if it used with WatchFileHandler.
https://docs.python.org/3.7/library/logging.handlers.html#logging.handlers.WatchedFileHandler

This is in the doc but only in handles section.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28469] timeit: use powers of 2 in autorange(), instead of powers of 10

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Following patch takes numbers from the sequence 1, 2, 5, 10, 20, 50, ... It 
also updates the documentation and tests.

--
components: +Library (Lib)
stage:  -> patch review
Added file: http://bugs.python.org/file45186/timeit_autorange_numbers.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28503] [Patch] '_crypt' module: fix implicit declaration of crypt(), use crypt_r() where available

[Ed Schouten]

New submission from Ed Schouten:

The '_crypt' module provides a binding to the C crypt(3) function. It is used 
by the crypt.crypt() function.

Looking at the C code, there are a couple of things we can improve:

- Because crypt() only depends on primitive C types, we currently get away with 
calling it without it being declared. Ensure that we include , which 
is the POSIX header file declaring this.

- The disadvantage of crypt() is that it's thread-unsafe. Systems like Linux 
and recent versions of FreeBSD nowadays provide crypt_r() as a replacement. 
This function allows you to pass in a 'crypt_data' object that will hold the 
resulting string for you. Extend the code to use this function when available.

This patch is actually needed to make this module build on CloudABI 
(https://mail.python.org/pipermail/python-dev/2016-July/145708.html). 
CloudABI's C library doesn't provide any thread-unsafe functions, meaning that 
crypt_r() is the only way you can crypt passwords.

--
components: Extension Modules
files: crypt.diff
keywords: patch
messages: 279186
nosy: EdSchouten
priority: normal
severity: normal
status: open
title: [Patch] '_crypt' module: fix implicit declaration of crypt(), use 
crypt_r() where available
versions: Python 3.6
Added file: http://bugs.python.org/file45185/crypt.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20357] Mention buildbots in the core dev section of the devguide

[Berker Peksag]

Berker Peksag added the comment:

Moved to https://github.com/python/devguide/issues/70

--
nosy: +berker.peksag
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20847] asyncio docs should call out that network logging is a no-no

[STINNER Victor]

STINNER Victor added the comment:

+Logs for :mod:`asyncio` module should always point to a file on the local
+filesystem.  Using any kind of network logging will block the event loop.

Well... even writing to a local file can block :-/

By "network", what about the local UNIX socket used by syslog?

I guess that the safest option would be a new asyncio library running
all logs functions to a thread, or maybe using loop.run_in_executor().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com