Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 10e8b97d0fd7 by Antoine Pitrou in branch 'default':
Make the reference counting of dictkeys objects participate in refleak hunting
http://hg.python.org/cpython/rev/10e8b97d0fd7
--
Changes by Benjamin Peterson benja...@python.org:
--
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Mark Shannon m...@hotpy.org added the comment:
Change insertdict to follow normal (non-stealing) ref-counting behaviour which
fixes possible leakage.
Patch attached.
--
Added file: http://bugs.python.org/file25422/insertdict.patch
___
Python
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset c5fd332e5857 by Benjamin Peterson in branch 'default':
change insertdict to not steal references (#13903)
http://hg.python.org/cpython/rev/c5fd332e5857
--
___
Python
Mark Shannon m...@hotpy.org added the comment:
Decref cached-keys when type is deallocated.
Patch attached.
--
Added file: http://bugs.python.org/file25381/cached_keys.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset a3beae842f13 by Benjamin Peterson in branch 'default':
decref cached keys on type deallocation (#13903)
http://hg.python.org/cpython/rev/a3beae842f13
--
___
Python
Antoine Pitrou pit...@free.fr added the comment:
New changeset a3beae842f13 by Benjamin Peterson in branch 'default':
decref cached keys on type deallocation (#13903)
http://hg.python.org/cpython/rev/a3beae842f13
Is there any way to detect / avoid leaks on this separate refcounting
scheme?
Antoine Pitrou pit...@free.fr added the comment:
This patch integrates the dictkeys' refcounting into the refcount checking
framework. Seems to work ok, but it would be better if someone more acquainted
with the code could confirm it.
--
Added file:
Mark Shannon m...@hotpy.org added the comment:
Failing to maintain GC tracking in setdefault and copy (for split-tables)
Patch attached
--
Added file: http://bugs.python.org/file25339/gc_tracking.patch
___
Python tracker rep...@bugs.python.org
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 507a6703d6a3 by Benjamin Peterson in branch 'default':
fix dict gc tracking (#13903)
http://hg.python.org/cpython/rev/507a6703d6a3
--
___
Python tracker
Mark Shannon m...@hotpy.org added the comment:
Failed to differentiate between failure and error in make_split_table().
Patch attached
--
Added file: http://bugs.python.org/file25340/make_split_table_error.patch
___
Python tracker
Changes by Giampaolo Rodola' g.rod...@gmail.com:
--
nosy: -giampaolo.rodola
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset b044e0568be2 by Martin v. Loewis in branch 'default':
Account for shared keys in type's __sizeof__ (#13903).
http://hg.python.org/cpython/rev/b044e0568be2
--
___
Python
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 5d5b72a71898 by Benjamin Peterson in branch 'default':
distiguish between refusing to creating shared keys and error (#13903)
http://hg.python.org/cpython/rev/5d5b72a71898
--
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file25313/73423916a242.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Mark Shannon m...@hotpy.org added the comment:
I've updated the repository and uploaded a new patch in response to Benjamin's
review. (And the contributor form is in the post).
One remaining issue is the return value of __sizeof__().
If it is an int, then it cannot accurately reflect the
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 6e5855854a2e by Benjamin Peterson in branch 'default':
Implement PEP 412: Key-sharing dictionaries (closes #13903)
http://hg.python.org/cpython/rev/6e5855854a2e
--
nosy: +python-dev
resolution: - fixed
Benjamin Peterson benja...@python.org added the comment:
Okay. I committed the latest patch. Subtleties like __sizeof__ can be worked
out as people start using it.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
Yury Selivanov yseliva...@gmail.com added the comment:
Mark, did you add the test that your patch initially was failing with?
http://mail.python.org/pipermail/python-dev/2012-February/116605.html
--
___
Python tracker rep...@bugs.python.org
Mark Shannon m...@hotpy.org added the comment:
I fixed it back then, but didn't add the test.
It subsequently regressed.
Should know better.
Patch (with test this time) attached.
--
resolution: fixed -
status: closed - open
Added file:
Roundup Robot devn...@psf.upfronthosting.co.za added the comment:
New changeset 34b6998efd2c by Benjamin Peterson in branch 'default':
fix instance dicts with str subclasses (#13903)
http://hg.python.org/cpython/rev/34b6998efd2c
--
___
Python tracker
Martin v. Löwis mar...@v.loewis.de added the comment:
Mark, can you please submit a contributor form?
--
nosy: +loewis
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Yury Selivanov yseliva...@gmail.com:
--
nosy: +Yury.Selivanov
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Raymond Hettinger raymond.hettin...@gmail.com added the comment:
Moving the assigned to to nobody. I won't have a chance for a thorough
review for another ten days. Hopefully, someone else will have a chance to
review it before then.
--
assignee: rhettinger -
Mark Shannon m...@hotpy.org added the comment:
I don't really understand your objection to changing the method-cache size. As
I said, I can remove the change, but that will cause the performance regression
that Antoine noticed.
--
___
Python
Mark Shannon m...@hotpy.org added the comment:
How about changing the method-cache size in a separate patch?
On my machine, reducing the method-cache size from 2**10 to 2**9 results
in a very small improvement in performance (with the original dict).
That way we don't get a performance
Antoine Pitrou pit...@free.fr added the comment:
I'm not bothered by the regression in silent_logging,
as it is a micro benchmark with a very short running time.
I'm not concerned about the micro-benchmark itself but the fact that it
might hint at a wider problem.
Also, I don't get your
Mark Shannon m...@hotpy.org added the comment:
Antoine Pitrou wrote:
Antoine Pitrou pit...@free.fr added the comment:
I'm not bothered by the regression in silent_logging,
as it is a micro benchmark with a very short running time.
I'm not concerned about the micro-benchmark itself but
Antoine Pitrou pit...@free.fr added the comment:
Also, I don't get your remark about it running in a short time. Your
patch AFAICT doesn't need any warm up period to exhibit any
improvements.
What I mean is that the runtime is so short, no one would notice any
change, so who cares?
Raymond Hettinger raymond.hettin...@gmail.com added the comment:
[Antoine]
I also think that, apart from the dict storage changes,
your patch should strive not to change any other tunables.
I agree. Please keep the patch focused on the single task, the shared keys.
--
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file25096/372d0bca85ae.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Mark Shannon m...@hotpy.org added the comment:
I'm not bothered by the regression in silent_logging,
as it is a micro benchmark with a very short running time.
The regression in mako is, I think, caused by competition for the
data cache between the new dict implementation and the method-cache
Antoine Pitrou pit...@free.fr added the comment:
The latest patch has a significant (negative) effect on some benchmarks:
### silent_logging ###
Min: 0.057927 - 0.068228: 1.18x slower
Avg: 0.058218 - 0.068660: 1.18x slower
Significant (t=-36.06)
### mako ###
Min: 0.118240 - 0.140451: 1.19x
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24766/257e16e71654.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Jim Jewett jimjjew...@gmail.com added the comment:
As of Feb 28, 2012, the PEP mentions an additional
optimization of storing the values in an array indexed
by (effectively) key insertion order, rather than key
position. (Alternative Implementation)
It states that this would reduce memory
Jim Jewett jimjjew...@gmail.com added the comment:
On Fri, Mar 9, 2012 at 12:13 PM, Jim Jewett
So to get beneath 2/3 without lots of reallocation
probably requires knowing when the key set is likely
to be complete, and that is indeed more complex than
the current changes. (That said, you
Jim Jewett jimjjew...@gmail.com added the comment:
As of Feb 28, 2012, the PEP mentions an additional optimization of storing the
values in an array indexed by (effectively) key insertion order, rather than
key position. (Alternative Implementation)
It states that this would reduce memory
Mark Shannon m...@hotpy.org added the comment:
Jim Jewett wrote:
Jim Jewett jimjjew...@gmail.com added the comment:
As of Feb 28, 2012, the PEP mentions an additional optimization of storing
the values in an array indexed by (effectively) key insertion order, rather
than key position.
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24670/49b7e7e4a27c.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24510/691ce331f955.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24465/e50db1b7ad7b.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24465/e50db1b7ad7b.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24456/20702d1acf17.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
--
hgrepos: +112
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Python-bugs-list mailing list
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24453/a9138aba7896.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24360/6a21f3b35e20.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24453/a9138aba7896.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24454/1f703b2607af.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Antoine Pitrou pit...@free.fr added the comment:
Looking at your latest patch, I worry about any deletion
+(including pop popitem) causes a split table to become a combined table.
Why wouldn't you use a dummy pointer (such as ((PyObject *) 1)) to signal
deleted slots?
--
Mark Shannon m...@hotpy.org added the comment:
Antoine Pitrou wrote:
Antoine Pitrou pit...@free.fr added the comment:
Looking at your latest patch, I worry about any deletion
+(including pop popitem) causes a split table to become a combined table.
Why wouldn't you use a dummy pointer
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24454/1f703b2607af.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24455/bc286099ce9a.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Removed file: http://bugs.python.org/file24455/bc286099ce9a.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24456/20702d1acf17.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
--
hgrepos: -109
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Python-bugs-list mailing
Mark Shannon m...@hotpy.org added the comment:
Raymond Hettinger wrote:
Raymond Hettinger raymond.hettin...@gmail.com added the comment:
Changing dictionaries is a big deal. You're changing many pieces at once
(not a good idea) including changing tunable parameters that are well-studied
Antoine Pitrou pit...@free.fr added the comment:
As Victor I think the tunables could be changed separately, unless they truely
get in the way of the shared-keys optimization.
By the way, there will need a protection for the case where instances are
used as bags of key/value pairs
Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Python-bugs-list mailing list
Gregory P. Smith g...@krypto.org added the comment:
FYI - I strongly support this type of work to reduce memory use of the Python
interpreter! :)
Also, yes, constant changing should be separate from this change but are worth
occasionally re-measuring and justifying as common computer
Raymond Hettinger raymond.hettin...@gmail.com added the comment:
This would likely require a PEP before having a chance of being considered for
inclusion.
--
nosy: +rhettinger
___
Python tracker rep...@bugs.python.org
Mark Shannon m...@hotpy.org added the comment:
Does this really need a PEP?
There is no new language feature and no change to any API.
It is just saving some memory.
The only possible issue is changing dict repr() ordering,
but issue 13703 will do that anyway.
--
Changes by Raymond Hettinger raymond.hettin...@gmail.com:
--
assignee: - rhettinger
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Raymond Hettinger raymond.hettin...@gmail.com added the comment:
Changing dictionaries is a big deal. You're changing many pieces at once (not
a good idea) including changing tunable parameters that are well-studied (I
spent a month testing whether 5/8 was better idea that 2/3 for resizing or
New submission from Mark Shannon m...@hotpy.org:
The proposed dictionary implementation allows sharing of keys hashes between
dictionaries. This leads to substantial memory savings for object-oriented
programs. For non-OO programs the impact is negligible.
--
components: Interpreter
Changes by Georg Brandl ge...@python.org:
--
keywords: +patch
Added file: http://bugs.python.org/file24357/061f8573af54.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Mark Shannon m...@hotpy.org:
Added file: http://bugs.python.org/file24360/6a21f3b35e20.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Georg Brandl ge...@python.org:
Removed file: http://bugs.python.org/file24357/061f8573af54.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Changes by Antoine Pitrou pit...@free.fr:
--
nosy: +benjamin.peterson, pitrou
stage: - patch review
versions: +Python 3.3 -Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
Terry J. Reedy tjre...@udel.edu added the comment:
In the initial comment, 'Dummy' to 'Deleted' here but only here:
- Holds an active (key, value) pair. Active can transition to Dummy
+ Holds an active (key, value) pair. Active can transition to Deleted
Im Lib/test/test_pprint.py
Changes by Giampaolo Rodola' g.rod...@gmail.com:
--
nosy: +giampaolo.rodola
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Changes by Philip Jenvey pjen...@underboss.org:
--
nosy: +pjenvey
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Python-bugs-list
Changes by Gregory P. Smith g...@krypto.org:
--
nosy: +gregory.p.smith
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
STINNER Victor victor.stin...@haypocalc.com added the comment:
I see two unrelated parts in your patch:
- change dictionary structure in memory
- change many constants linked to optimization: PyDICT_MAXFREELIST: 80-40,
2/3-5/8, etc.
You may open a new issue for the second part, except if I
Changes by John O'Connor tehj...@gmail.com:
--
nosy: +jcon
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13903
___
___
Python-bugs-list mailing
74 matches
Mail list logo
