Michael Bayer wrote:
matiskiva wrote:
Just to skip the newbie replies :) I attempted to go over the "rects"
collection and expunge all the items.
This direct approach didn't work.
expunging is not enough. did you expire the rects collection on each
Detection object, and also remove any other references which may exist to
any of those "rects" ? this is a python reference counting issue. you
can see just how many objects are not released by using gc.collect() and
then len(gc.get_objects()). note that the memory footprint of the
cPython interpreter doesn't shrink very much once grown to a particular
size so releasing will also not really help much with the total size of
the interpreter.
Mati
On Jan 31, 11:01 am, matiskiva <[email protected]> wrote:
Hello all,
I have an application that loads many "Detection" objects, each of
them having thousands of "Rect" objects, loaded with eager loading.
I am performing some calculation on the rects and saves the result,
along with the Detection objects.
But than I am stuck with all the Rect objects, taking up lots of
memory and I can't find a way to release them.
And for the code:
# there is some kind of RectDB mapping and definition...
mapper(RectDB....
# there is some kind of DetectionDB mapping and definition
mapper(DetectionDB,_detection_table,properties={
"rects":relation(RectDB),
....
# query the detections
detections = session.query(DetectionDB).filter(....).eagerload
("rects").all()
for d in detections:
# make calculation
rectsScore = calculateScore(d)
# save score
d.score = rectsScore
# release rects, they take up lots of space
releaseEagerLoading(d.rects)
# update the detections
...
Currently I have no idea how to implement releaseEagerLoading. The
result is very large (over 1GB) memory footprint. I tried the same
without using the rects (empty score calculation) and without the
eager loading and it was below 100MB.
I think I can guess why it happens, but the question is - how can I
release the eager loading?
Thank you in advance,
Mati
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Thank you very much.
I was unaware of the "expire" functionality. That is what I was looking for.
Regards,
Mati
This mail was sent via Mobileye Mail-SeCure system.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
