PMJI, but I wonder if some of these objects could come from
trivial re-instatiations instead of re-use of mutable
objects, e.g., fishing out one attribute to use together
with a new value as init values for an (unnecessarily) new obj.
obj = ObjClass(obj.someattr, chgval)
where
obj.chg = chgval
would have done the job without creating garbage. I suspect this
pattern can happen more subtly than above, especially if __new__
is defined to do something tricky with old instances.
Also, creating a new object can be a tempting way to feel sure about
its complete state, without having to write a custom (re)init method.
On 03/18/2014 10:37 AM Martin Koch wrote:
Thanks, Carl.
This bit of code certainly exhibits the surprising property that some runs
unpredictably stall for a very long time. Further, it seems that this stall
time can be made arbitrarily large by increasing the number of nodes
generated (== more data in the old generation == more stuff to traverse if
lots of garbage is generated and survives the young generation?). As a user
of an incremental garbage collector, I would expect that there are pauses
due to GC, but that these are predictable and small.
I tried running
PYPY_GC_NURSERY=2000M pypy ./mem.py 10000000
but that seemed to have no effect.
I'm looking forward to the results of the Software Transactional Memory,
btw :)
/Martin
On Tue, Mar 18, 2014 at 9:47 AM, Carl Friedrich Bolz<[email protected]> wrote:
On 17/03/14 20:04, Martin Koch wrote:
Well, it would appear that we have the problem because we're generating
a lot of garbage in the young generation, just like we're doing in the
example we've been studying here.
No, I think it's because your generating a lot of garbage in the *old*
generation. Meaning objects which survive one minor collection but then
die.
I'm unsure how we can avoid that in
our real implementation. Can we force gc of the young generation? Either
by gc.collect() or implcitly somehow (does the gc e.g. kick in across
function calls?).
That would make matters worse, because increasing the frequency of
minor collects means *more* objects get moved to the old generation
(where they cause problems). So indeed, maybe in your case making the
new generation bigger might help. This can be done using
PYPY_GC_NURSERY, I think (nursery is the space reserved for young
objects). The risk is that minor collections become unreasonably slow.
Anyway, if the example code you gave us also shows the problem I think
we should eventually look into it. It's not really fair to say "but
you're allocating too much!" to explain why the GC takes a lot of time.
Cheers,
Carl Friedrich
_______________________________________________
pypy-dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________
pypy-dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________
pypy-dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/pypy-dev
