El dv 23 de 03 del 2007 a les 14:51 -0700, en/na Stefan Kuzminski va
escriure:
> Hi,
>
> In the code below, as it goes through the 100,000 loop appending rows, the
> memory being used ( as measured by top on a linux box ) steadily increases.
> How can I prevent this from happening? ( calling flush does not seem to help
> ) This would seem to present a limit on how much data I can append to a
> file? The behavior is with v1.4 and with the 2.0 beta.
>
> thanks!
> Stefan Kuzminski
>
>
> from tables import *
> from time import sleep
>
> num_cols = 1500
> fp = openFile( "foo", 'w' )
>
>
> Float64Col( shape=(num_cols,))
> table = fp.createTable( fp.root, 'title',
> { 'var1' : Float64Col( shape=(num_cols,)) }, '' )
> print 'appending...'
> row = range( num_cols )
> for i in range( 100000 ):
>
> table.append( [[row]])
> if i % 1000 == 0:
> print i
> table.flush()
> print 'done appending'
> sleep( 20 )
Stefan,
After struggling a bit with this, the problem seems to be the use of the
range() function. As you know, this function returns a list of elements,
and that takes memory. Curiously enough, this memory is not consumed in
one shot, but it grows steadily with time (sorry, but I don't know the
interpreter of python enough for guessing the reason of this).
Anyway, if you replace range() by xrange() everyting will go fine (i.e.
the 'leak' will misteriously disappear).
Cheers,
--
Francesc Altet | Be careful about using the following code --
Carabos Coop. V. | I've only proven that it works,
www.carabos.com | I haven't tested it. -- Donald Knuth
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
