A Tuesday 21 August 2007, David Worrall escrigué: > I lowered NODE_MAX_SLOTS from 256 to128 and that slowed the leak - > enough to get some sort of DB happening. > It eventually seg faulted, however. > I've noticed that sometimes the seg fault causes the (non-python) > heap to become corrupted, requiring a HW reboot before pytables > becomes useful again. > Who knows what memory it may be writing over!
I don't think there is a leak in PyTables itself; it is just that it takes a lot of memory to work with many nodes simultaneously. Perhaps reducing the NODE_MAX_SLOTS (to 64, for example) could help a bit more. Perhaps it would be a good idea to reduce the memory requirements of PyTables nodes (for example, avoid the creation of buffers when they are not needed), but this should take quite a few of thought. Indeed it seems that there is a problem with the MacOSX platform, but the fact that this is not reproducible on Linux is unfortunate. Anyone that is reading this can try David's script on a Windows machine? That could apport more clues on what's going on. > I don't know valgrind - It seems not to be available for OSX - has > anyone compiled/used it on this platform? Well, after looking at http://valgrind.org/info/platforms.html, it seems that MacOSX is not supported yet. Perhaps you can get some advice on MacOSX forums on which tools you can use for debugging memory problems. Hope that helps, > > ciao, > > David > > On 16/08/2007, at 3:49 AM, Francesc Altet wrote: > > A Wednesday 15 August 2007, Francesc Altet escrigué: > >> A Wednesday 15 August 2007, David Worrall escrigué: > >>> All that aside, > >>> 1. it is a worry that there is no exception generated, by python, > >>> at least, don't you agree? > >> > >> Yes, I do. > >> > >>> 2. Is there a way of purging memory after a flush or are we up > >>> against python's memory management? > >> > >> I think the thing to try first would be lowering NODE_MAX_SLOTS. > >> Have you tried this already? > > > > Now that I think, there a possibility to debug where the problem > > can be. > > Just install valgrind and run your script through it. Its output > > would > > bring some more information on the table. > > > > > > -- > > > >> 0,0< Francesc Altet http://www.carabos.com/ > > > > V V Cárabos Coop. V. Enjoy Data > > "-" > > > > ------------------------------------------------------------------- > >--- --- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a > > browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Pytables-users mailing list > > Pytables-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/pytables-users > > _________________________________________________ > experimental polymedia: www.avatar.com.au > Sonic Communications Research Group, > University of Canberra: www.canberra.edu.au/vc-forum/scrg > vip=Verbal Interactivity Project > > > > > --------------------------------------------------------------------- >---- This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a > browser. Download your FREE copy of Splunk now >> > http://get.splunk.com/ > _______________________________________________ > Pytables-users mailing list > Pytables-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pytables-users -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
