Dinesh B Vadhia (el 2008-05-10 a les 10:10:29 -0700) va dir:: > I'm using the OS filesystem to store 32,000 images files. I'm now > going to move them into a datastore and the choices are pysqlite or > MySQL or PyTables. The number of images will grow rapidly (to the > millions and more) and hence performance is critical. Multiple images > will be accessed from the data strore at a time. There are no write > operations just read only. > > The data schema is: image index (on the image filename), image > filename, image (jpg initially but will be other formats in the > future). > > Any and all suggestions would be appreciated.
Well, I don't quite understand the data schema (are you describing a row
of three fields in a table), but you may have a look at the
``tables.nodes.filenode`` module, which contains a ``FileNode`` class
which offers a Python file-like interface to a PyTables dataset (a one-
dimensional ``EArray`` ) holding the bytes of the file. I should be
specially useful if you keep images stored with a file format like JPEG,
PNG and the like.
Also, I'd recommend not cramming all images under a single group to
avoid performance problems when opening the group, but to pack them in
groups of at most 4096 (see ``tables.parameters.MAX_GROUP_WIDTH``)
images per group.
Hope that helps,
::
Ivan Vilata i Balaguer >qo< http://www.carabos.com/
Cárabos Coop. V. V V Enjoy Data
""
signature.asc
Description: Digital signature
------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
