2010/8/15, Vineet Jain <vinjv...@gmail.com>: > I'm getting the following warning on an index. What does the message > mean and what do you recommend I do? > > /usr/local/lib/python2.6/dist-packages/tables/leaf.py:416: > PerformanceWarning: The Leaf ``/_i_data/fullSymbol/sorted`` is > exceeding the maximum recommended rowsize (104857600 bytes); > be ready to see PyTables asking for *lots* of memory and possibly slow > I/O. You may want to reduce the rowsize by trimming the value of > dimensions that are orthogonal (and preferably close) to the *main* > dimension of this leave. Alternatively, in case you have specified a > very small/large chunksize, you may want to increase/decrease it. > PerformanceWarning)
This is telling you that PyTables is not able to create an internal chunksize for column index less than 10 MB. 10 MB is way too much for most of modern CPU to efficiently operate with data on their internal caches, hence the performance warning. If performance is acceptable to you, then fine, but you should investigate why the warning is happening. In the context of table columns that can be indexed, this should only happen with very large datatypes (typically string types exceeding 10 MB, which should be pretty unusual). Mmh, based on your previous thread, I would say that the definition of the 'fullSymbol' column is something like StringCol(100), which is perfectly fine, so I cannot see what's going on there exactly. Could you provide a self-contained script that reproduces the warning? -- Francesc Alted ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
